A fan intelligent inspection path optimization method and system based on deep learning

Abstract

The application relates to the technical field of unmanned aerial vehicle inspection, and particularly discloses a fan intelligent inspection path optimization method and system based on deep learning, which acquires multi-dimensional inspection time sequence data, inputs the multi-dimensional inspection time sequence data into a blade motion state deep learning prediction model, outputs blade motion state prediction values, generates an unmanned aerial vehicle inspection waypoint sequence based on the blade motion state prediction values, constructs a multi-target path optimization model according to the unmanned aerial vehicle inspection waypoint sequence, and outputs an optimal inspection path; the inspection task is performed according to the optimal inspection path, an image quality evaluation model is established, and an inspection effect evaluation coefficient is output; the inspection task priority is formulated based on the inspection effect evaluation coefficient, and whether the optimal inspection path meets the standard is evaluated. The application can effectively improve the automation degree, detection accuracy and operation safety of fan blade inspection under the condition that the fan does not stop running, reduce the artificial inspection cost, and improve the intelligent operation and maintenance level of wind power equipment.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, and more specifically, to a method and system for optimizing intelligent inspection paths for wind turbines based on deep learning. Background Technology

[0002] Wind turbines operate in complex natural environments for extended periods, making their blades susceptible to defects such as cracks, surface damage, corrosion, and structural anomalies. To ensure safe operation, regular blade inspections are necessary to detect potential faults and mitigate downtime risks. However, manual blade inspections suffer from low efficiency, high operational risks, and limited coverage. While drone inspections can improve efficiency when the turbine is shut down, they require stopping the turbine, impacting wind farm power generation. Furthermore, relying solely on current blade rotation speed or spatial position for path planning makes it difficult to accurately reflect future blade movement trends. This can lead to the drone arriving at the inspection location after the blades have deviated from the intended image area, reducing inspection efficiency and image quality.

[0003] Currently, most drone inspection path planning methods rely on preset rules or static optimization models for path calculation. When faced with changes in wind turbine operating conditions, they can only readjust the flight path based on the current state, failing to predict future blade movement changes in advance, resulting in a lag in drone path adjustment. When blade speed changes or environmental disturbances intensify, problems such as insufficient safe distance between the drone and blades, shooting angle deviations, and missed inspection areas can easily occur. Existing inspection processes typically rely solely on image clarity, shooting distance, or human experience to determine whether the acquired images meet inspection requirements, lacking an evaluation mechanism. When drone-acquired images have motion blur, blade occlusion, abnormal lighting, or unreasonable shooting angles, the defect detection model may fail to accurately identify blade defects, requiring replanning of the inspection task and increasing inspection time and energy consumption.

[0004] Therefore, it is necessary to provide a method and system for optimizing the intelligent inspection path of wind turbines based on deep learning to solve the above-mentioned technical problems. In order to solve the above problems, a technical solution is provided. Summary of the Invention

[0005] To overcome the aforementioned deficiencies in existing technologies, this invention provides a deep learning-based intelligent wind turbine inspection path optimization method and system. This method addresses the problem that existing wind turbine blade UAV inspection technologies have failed to achieve near real-time prediction and adaptive planning of blade motion, resulting in a mismatch between inspection timing and the actual blade motion state, making it difficult to balance capture rate, safety, and economy under dynamic operating conditions.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A deep learning-based intelligent inspection path optimization method for wind turbines includes the following steps: Multidimensional inspection data is acquired through data monitoring equipment, and multidimensional inspection time-series data is output after data preprocessing. By inputting multidimensional inspection time series data into the deep learning prediction model of blade motion state, the predicted value of blade motion state is output. A sequence of UAV inspection waypoints is generated based on the predicted values ​​of blade motion state. A multi-objective path optimization model is constructed based on the UAV cruise point sequence to output the optimal inspection path. The inspection task is carried out according to the optimal inspection path. The target blade image of each UAV inspection point is captured. The pre-obtained standard blade image is compared with the target blade image to obtain the image quality evaluation factor. An image quality evaluation model is established and the inspection effect evaluation coefficient is output. Based on the inspection effect evaluation coefficient, the inspection task priority is determined, the blade defect detection is performed on the target blade image, the corresponding UAV inspection waypoints are associated with the blade defect results, and the optimal inspection path is evaluated to determine whether it meets the standard.

[0007] As a further aspect of the present invention, the multidimensional inspection heterogeneous data includes wind turbine SCADA data, meteorological data, historical inspection data, wind turbine 3D model data, and UAV real-time status data.

[0008] As a further aspect of the present invention, multi-dimensional inspection time-series data is input into a deep learning prediction model for blade motion state, and predicted values ​​for blade motion state are output. The specific steps are as follows: By inputting multidimensional inspection time-series data into the temporal convolutional network (TCN), the first-order motion change characteristics during the blade motion process are extracted. By inputting short-term motion features into a gated recurrent unit (GRU) network, the long-term temporal correlation of blade motion states is learned, and the characteristics of the secondary motion change patterns of the blade are captured. By inputting the secondary motion change pattern characteristics into the attention mechanism module, the importance of the input features is adaptively calculated, and the features of future attention factors are identified based on the importance. By fusing the features of future factors of interest and inputting them into the prediction output layer, the predicted value of the blade motion state is output.

[0009] As a further aspect of the present invention, the predicted values ​​of blade motion state include blade spatial position, blade attitude angle, blade motion time, and blade target area.

[0010] As a further aspect of the present invention, a UAV inspection waypoint sequence is generated based on the predicted values ​​of blade motion state, and a multi-objective path optimization model is constructed based on the UAV cruise point sequence to output the optimal inspection path. The specific steps are as follows: Based on the predicted values ​​of blade motion status, UAV inspection waypoints are automatically generated, forming a UAV inspection waypoint sequence. A multi-objective path optimization model is constructed based on the UAV cruise point sequence to output the optimal inspection path. The multi-objective path optimization model includes inspection quality optimization objective, flight safety optimization objective, flight energy consumption optimization objective, and inspection time optimization objective.

[0011] As a further aspect of the present invention, each drone inspection waypoint includes the drone inspection location, the drone gimbal adjustment attitude, the drone's estimated arrival time, and the drone's shooting direction.

[0012] As a further aspect of the present invention, the generation of dynamic inspection waypoints from blade motion state prediction values ​​includes: calculating the corresponding UAV inspection position based on the blade spatial position corresponding to a future time node; calculating the UAV gimbal adjustment attitude based on the blade attitude angle prediction value; calculating the expected arrival time of the UAV based on the blade motion time prediction value; and generating the UAV shooting direction based on the blade target area.

[0013] As a further aspect of the present invention, the image quality evaluation factors include the sharpness of the leaf image, the integrity of the leaf image, and the suitability of the lighting of the leaf image.

[0014] As a further aspect of the present invention, the inspection task priority is determined based on the inspection effect evaluation coefficient, blade defect detection is performed on the target blade image, and the corresponding UAV inspection waypoints are associated with the blade defect results to evaluate whether the optimal inspection path meets the standard. The specific steps are as follows: Based on the inspection effectiveness evaluation coefficient of each drone patrol point, the inspection task sequence is determined by arranging the values ​​in descending order, and the inspection task priority is determined according to the inspection task sequence. A deep learning target detection model is used to detect blade defects in target blade images, and the detected defect type and location are associated with the corresponding UAV cruise point; If the image quality of a target blade at a certain waypoint does not meet the requirements or the confidence level of defect detection is low, the corresponding UAV cruise point needs to be re-photographed; The number of drone patrol points that need to be re-captured is counted. If the number exceeds the preset number, the optimal inspection path does not meet the standard; if the number does not exceed the preset number, the optimal inspection path meets the standard.

[0015] A deep learning-based intelligent wind turbine inspection path optimization system includes a multi-dimensional inspection data acquisition module, a motion state prediction module, an inspection path optimization module, an inspection effect evaluation module, and a blade defect detection module. The multidimensional inspection data acquisition module is used to acquire multidimensional inspection data through data monitoring equipment, and to perform data preprocessing to output multidimensional inspection time-series data; The motion state prediction module is used to input multidimensional inspection time series data into the blade motion state deep learning prediction model and output the blade motion state prediction value. The inspection path optimization module is used to generate a UAV inspection waypoint sequence based on the predicted value of blade motion state, and to build a multi-objective path optimization model based on the UAV cruise point sequence to output the optimal inspection path. The inspection effect evaluation module is used to execute inspection tasks according to the optimal inspection path, capture target blade images of each UAV inspection point, obtain pre-standard blade images and target blade images to obtain image quality evaluation factors, establish an image quality evaluation model, and output inspection effect evaluation coefficients. The blade defect detection module is used to prioritize inspection tasks based on the inspection effect evaluation coefficient, detect blade defects in target blade images, associate the blade defect results with corresponding UAV inspection waypoints, and evaluate whether the optimal inspection path meets the standards.

[0016] The technical effects and advantages of this invention, a deep learning-based intelligent wind turbine inspection path optimization method and system, are as follows: This invention acquires multi-dimensional inspection data during wind turbine operation and uses a deep learning prediction model to predict the blade motion state. This allows the UAV to plan its shooting position in advance based on the dynamic changes of the blades, avoiding shooting misalignment caused by blade rotation and improving the success rate of target area acquisition. By generating an inspection waypoint sequence based on the blade motion state prediction results and constructing a multi-objective path optimization model, it can comprehensively consider factors such as image acquisition quality, UAV flight stability, and energy consumption, outputting a more reasonable inspection path and improving UAV inspection efficiency and path reliability. By establishing an image quality evaluation model based on standard blade image comparison, it comprehensively evaluates the clarity, completeness, and lighting suitability of the images acquired by the UAV and outputs an inspection effect evaluation coefficient, realizing automatic judgment of the validity of inspection data and reducing the problem of false or missed defects due to insufficient image quality. By combining a deep learning-based blade defect detection model, the defect type, location, and severity are associated with the corresponding inspection waypoints. The optimal inspection path is then determined based on the number of waypoints that need to be reshot, thus achieving closed-loop feedback optimization of the inspection path and improving the reliability of the inspection results.

[0017] This invention enables precise prediction of blade motion status, intelligent planning of UAV paths, automatic evaluation of inspection image quality, and optimization of defect detection results under the condition of continuous operation of wind turbines. It effectively improves the automation level, detection accuracy, and operational safety of wind turbine blade inspection, reduces the cost of manual inspection, and enhances the level of intelligent operation and maintenance of wind power equipment. Attached Figure Description

[0018] Figure 1 A flowchart of a deep learning-based intelligent wind turbine inspection path optimization method provided in an embodiment of the present invention; Figure 2 This is a system block diagram of a deep learning-based intelligent wind turbine inspection path optimization system provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described technical solutions are only a part of this invention, and not all of it. All other technical solutions obtained by those skilled in the art based on the technical solutions of this invention without inventive effort are within the scope of protection of this invention.

[0020] like Figure 1 The diagram shown is a flowchart of a deep learning-based intelligent wind turbine inspection path optimization method provided by an embodiment of the present invention. Figure 1 The execution entity of the method shown can be a software and / or hardware device. The execution entity of this application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. User equipment can include, but is not limited to, computers, smartphones, personal digital assistants (PDAs), and the aforementioned electronic devices. Network equipment can include, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this. Steps S1 to S5 are detailed as follows: Step S1: Acquire multi-dimensional inspection data through data monitoring equipment, and perform data preprocessing to output multi-dimensional inspection time-series data. The multi-dimensional inspection heterogeneous data includes: wind turbine SCADA data, including time-series parameters such as blade speed, pitch angle, nacelle yaw angle, active power, and wind speed; meteorological data, including real-time wind speed, wind direction, temperature, and humidity; historical inspection data, including historical inspection paths, shooting locations, and image quality scores; wind turbine 3D model data, including static parameters such as blade geometric parameters, wind turbine coordinates, and blade numbers; and real-time UAV status data, including UAV position, attitude, speed, and battery level. Data preprocessing includes timestamp synchronization, spatial coordinate unification, abnormal data removal, missing data compensation, and normalization processing, and a continuous time-series input sequence is constructed through a sliding time window. Step S2: Input multidimensional inspection time series data into the deep learning prediction model of blade motion state and output the predicted value of blade motion state. Step S3: Generate a UAV inspection waypoint sequence based on the predicted blade motion state value, and construct a multi-objective path optimization model based on the UAV cruise point sequence to output the optimal inspection path. Step S4: Perform the inspection task according to the optimal inspection path, capture the target blade image of each UAV inspection point, compare the pre-obtained standard blade image with the target blade image to obtain the image quality assessment factor, establish the image quality assessment model, and output the inspection effect assessment coefficient. Step S5: Based on the inspection effect evaluation coefficient, determine the inspection task priority, perform blade defect detection on the target blade image, associate the corresponding UAV inspection waypoints according to the blade defect results, and evaluate whether the optimal inspection path meets the standard.

[0021] It should be noted that all data parameters involved in the calculation must be dimensionless beforehand.

[0022] Preferably, the multidimensional inspection time-series data is input into the deep learning prediction model of the blade motion state, and the predicted value of the blade motion state is output. The specific steps are as follows: By inputting multidimensional inspection time-series data into a temporal convolutional network (TCN), the first-order motion change characteristics during the blade motion process are extracted, namely the short-term motion change characteristics; the periodic change characteristics include the short-term fluctuation characteristics of blade rotation speed, the pitch angle adjustment characteristics, the motion change characteristics caused by wind speed disturbance, and the blade rotation periodic change characteristics. By inputting short-term motion features into a gated recurrent unit (GRU) network, the long-term temporal correlation of the blade motion state is learned, capturing the characteristics of the secondary motion change pattern of the blade, i.e., the long-term motion change pattern. By inputting long-term motion change patterns into the attention mechanism module, the importance of the input features is adaptively calculated, and features for future attention are identified based on their importance. The importance of the input features is compared with a preset importance threshold. If the importance of the input feature is greater than or equal to the preset importance threshold, the corresponding input feature is marked as a feature for future attention; if the importance of the input feature is less than the preset importance threshold, the corresponding input feature is marked as a candidate feature for attention. The specific formula for importance is as follows: ; In the formula: The significance of the i-th long-term motion change pattern feature in predicting future leaf motion is given by... This represents the characteristic of the i-th long-term motion change pattern. This represents the total number of long-term dynamic change patterns and characteristics. By fusing the features of future factors of interest and inputting them into the prediction output layer, the predicted values ​​of blade motion state are output. The predicted values ​​of blade motion state include blade spatial position, blade attitude angle, blade motion time, and blade target area.

[0023] In this embodiment of the invention, during actual wind farm operation, a large horizontal axis wind turbine is used as the inspection target. While the turbine is generating power normally, a SCADA monitoring system, environmental monitoring equipment, and a drone inspection system deployed on the turbine side are used to simultaneously collect multi-dimensional inspection time-series data during turbine operation. For example, within a continuous acquisition time window, wind turbine operating status data within the past 10 to 30 seconds are obtained. The blade speed, pitch angle, wind speed, and drone status parameters corresponding to each moment are arranged in chronological order to form an input time-series matrix. Each time point corresponds to a multidimensional state vector, which describes the change process of the blade's motion state within the current time period. By normalizing the above multidimensional inspection time series data, the influence of different parameters due to different units is eliminated, and the processed data is input into the deep learning prediction model of the blade's motion state.

[0024] First, the multidimensional inspection time-series data is input into the temporal convolutional network (TCN) to extract the short-term time-scale variation patterns during blade motion. Because wind turbine blades are affected by instantaneous wind speed changes, load adjustments, and pitch angle adjustments during actual operation, their motion state is not constant but fluctuates over a short period. Therefore, through the causal convolutional and dilated convolutional structures in the TCN network, features are extracted from the data over a continuous time range to obtain the primary motion variation patterns during blade motion.

[0025] Specifically, the TCN network performs convolutional calculations on the input time-series data, extracting short-term fluctuation features of blade rotational speed through convolutional kernels at different time scales. For example, when wind speed suddenly increases, the blades are driven by wind, and their rotational speed shows a short-term increasing trend. The TCN network can identify this rotational speed change trend based on data changes at multiple historical time points. Simultaneously, the TCN network extracts features from the pitch angle adjustment process. When the wind turbine control system adjusts the pitch angle according to wind speed changes, this adjustment behavior affects the blade's stress state and rotational speed. The TCN network can extract the correlation between pitch angle changes and blade motion changes.

[0026] Furthermore, the TCN network can also extract the characteristics of blade motion changes caused by wind speed disturbances and the characteristics of blade rotation period changes. For example, under the influence of periodic gusts, the blade rotation speed may exhibit a periodic variation pattern. The TCN network obtains the short-period motion change trend of the blade by performing convolutional analysis on multiple historical periodic data. After processing by the TCN network, the output is a first-level motion change pattern feature containing short-term motion patterns.

[0027] Subsequently, the first-level motion change characteristics output by the TCN network are input into the gated recurrent unit (GRU) network to learn the long-term temporal correlation of the blade motion state. Since the wind turbine blade motion is affected not only by short-term disturbances but also by long-term operating state changes, such as long-term wind speed trends, wind turbine operating conditions, and periodic rotation patterns, it is necessary to further analyze the correlation between different time stages through the GRU network.

[0028] Specifically, the GRU network filters historical motion information through internal update and reset gates, determining the proportion of information to retain based on the current input features and historical hidden states. For example, when the wind turbine is in a stable operating state, the GRU network retains more historical periodic motion information to learn the long-term stable rotational patterns of the blades; when a significant change in wind speed is detected, the GRU network reduces the influence of some historical information and increases the importance of the current change, thereby accurately capturing changes in blade motion trends. After processing by the GRU network, the secondary motion change pattern characteristics of the blades are obtained, namely, the long-term motion change pattern characteristics. This feature reflects the motion evolution of the blades over a long period of time, including the trend of blade rotation period changes, the long-term speed change trend, and the continuous influence of environmental factors on the blade motion state.

[0029] Long-term motion variation patterns are input into the attention mechanism module, which adaptively calculates the importance of different input features. Since different operating parameters have varying degrees of influence on future blade motion—for example, blade rotational speed may be the primary influencing factor under stable wind conditions, while the magnitude of wind speed change may be the primary factor under rapidly changing wind conditions—an attention mechanism is needed to automatically identify key factors affecting future motion prediction. Specifically, the attention mechanism module calculates the importance of each long-term motion variation pattern feature based on the correlation between it and the predicted future blade motion.

[0030] After calculating the importance of each long-term motion change pattern feature, this importance is compared with a preset importance threshold. When the importance of a certain long-term motion change pattern feature is greater than or equal to the preset importance threshold, this feature is marked as a feature of future focus. For example, if the model calculates that the wind speed change feature has a high importance, it is considered that the future blade motion state is mainly affected by wind speed changes, and this wind speed change feature is used as a key prediction basis.

[0031] When the importance of a certain long-term motion change pattern feature is less than a preset importance threshold, the feature is marked as a candidate feature of interest. These candidate features are not discarded directly, but rather retained as auxiliary information to avoid increasing prediction errors due to short-term state changes.

[0032] After feature selection, the features of future interest factors are fused and input into the prediction output layer to predict the blade motion state in the future time period. The prediction output includes predicted blade spatial position, predicted blade attitude angle, predicted blade motion time, and predicted blade target area.

[0033] Among them, the blade spatial position prediction value is used to describe the position of the key blade position in the wind turbine spatial coordinate system at a certain time point in the future, such as the blade tip position coordinates, the blade center area coordinates, etc.; the blade attitude angle prediction value is used to describe the future changes in the blade rotation attitude, including parameters such as blade deflection angle and rotation angle; the blade motion time prediction value is used to determine the estimated time for the blade to reach the target inspection area, providing a time basis for the UAV to arrive at the shooting position in advance; the blade target area is used to determine the blade area that needs to be focused on shooting in the future.

[0034] For example, when the model predicts that the tip of a blade will enter a preset inspection range in 5 seconds, the UAV control system calculates the flight distance and arrival time in advance based on the predicted blade spatial position and movement time, enabling the UAV to reach the optimal shooting position simultaneously when the blade enters the target area. At the same time, the UAV gimbal angle is adjusted based on the predicted blade attitude angle to maintain the camera equipment in the optimal shooting direction relative to the blade surface.

[0035] This invention realizes a complete process from multi-dimensional inspection time-series data acquisition, short-term motion pattern extraction, long-term motion trend learning, key influencing factor screening, to future blade motion state prediction. Compared with methods that only plan inspection paths based on the current blade rotation speed or current position, this embodiment can predict the future motion state of the blade in advance, enabling the UAV to actively adjust inspection waypoints according to the future blade motion trend, improving the accuracy and stability of UAV path planning and the quality of blade image acquisition during non-stop inspection.

[0036] Preferably, the constraints for determining the path based on the UAV inspection waypoint sequence are as follows: Constraints are set for each drone inspection waypoint, including: image quality constraint: ensure that the angle between the camera optical axis and the blade surface normal is within a reasonable range (e.g., ≤30°) to ensure image quality; safe distance constraint: the drone maintains a safe distance from the blade (e.g., ≥5m); field of view constraint: ensure that the target blade area falls completely within the camera's field of view.

[0037] As a further aspect of the present invention, a UAV inspection waypoint sequence is generated based on the predicted values ​​of blade motion state, and a multi-objective path optimization model is constructed based on the UAV cruise point sequence to output the optimal inspection path. The specific steps are as follows: Based on the predicted blade motion state, UAV inspection waypoints are automatically generated, forming a sequence of UAV inspection waypoints. Each UAV inspection waypoint includes the UAV inspection location, the UAV gimbal adjustment attitude, the UAV's estimated arrival time, and the UAV's shooting direction. Specifically, generating dynamic inspection waypoints based on the predicted blade motion state includes: calculating the corresponding UAV inspection location based on the blade's spatial location at a future time point; calculating the UAV gimbal adjustment attitude based on the predicted blade attitude angle; calculating the UAV's estimated arrival time based on the predicted blade motion time; and generating the UAV's shooting direction based on the target area of ​​the blade.

[0038] In this embodiment of the invention, during actual wind farm inspection, a horizontal axis wind turbine is used as the inspection object. When the turbine is in normal power generation mode, the blades continuously rotate around the hub, and different areas of the blades correspond to different spatial positions at different time points. The UAV inspection system first acquires the predicted values ​​of the blade motion state output by the deep learning prediction model of the blade motion state. For example, the blade motion state prediction model predicts the blade trajectory within the next 10 seconds. At the 5th second, the tip area of ​​the blade will enter the preset defect detection area, and at the 8th second, the middle area of ​​the blade will enter the optimal shooting range. Based on the above prediction results, the UAV control system needs to plan the flight position in advance so that the UAV can reach the target position at the corresponding time and maintain a suitable shooting posture to complete the blade image acquisition.

[0039] The system automatically generates a sequence of UAV inspection waypoints based on predicted blade motion states. Specifically, the UAV inspection position is calculated based on the spatial position of the blade at future time points. Since the UAV cannot directly hover near the blade surface for photography, a certain safety offset distance needs to be set based on the target blade's spatial position. Therefore, the corresponding spatial position of the UAV is calculated based on the predicted blade spatial coordinates and the preset shooting distance. For example, when the predicted future position of the blade tip is in three-dimensional coordinates, the system adds a 5m safety distance in the blade's normal direction according to the UAV's shooting distance requirements, calculating the optimal shooting position for the UAV. This ensures that the UAV can clearly acquire images of the blade while avoiding the risk of collision due to blade rotation.

[0040] The drone gimbal adjusts its attitude based on the predicted blade attitude angle. Since the surface normal direction of the wind turbine blades changes continuously during rotation, maintaining a fixed angle for the drone camera will cause the shooting area to shift. Therefore, based on the predicted blade attitude angle changes, the pitch and yaw angles that the drone gimbal needs to adjust in advance are calculated to ensure that the camera's optical axis always points towards the target blade area. For example, if a 15° rotation change in the blade is predicted in the future, the system adjusts the gimbal angle in advance to maintain a reasonable angle between the camera's optical axis and the blade surface normal, thereby reducing image distortion caused by shooting angle deviations.

[0041] The estimated arrival time of the UAV is calculated based on the predicted blade motion time. Since UAV inspection requires not only reaching a spatial location but also maintaining time synchronization with the blade's motion, the system calculates the time required for the UAV to reach the target waypoint based on the time it takes for the target area on the blade to reach the optimal shooting position, as well as the UAV's current spatial position and flight speed. For example, if the prediction shows that the target area on the blade will enter the optimal shooting position in 8 seconds, the system calculates the flight distance based on the UAV's current position and adjusts the flight speed so that the UAV can reach the corresponding inspection waypoint in the 8th second, achieving synchronization between blade motion and UAV shooting actions. Simultaneously, the system generates the UAV's shooting direction based on the target area. The system determines the shooting direction of the UAV camera based on the location of the target defect detection area and the changes in blade attitude, ensuring that the target area is fully within the camera's field of view.

[0042] Based on the above calculations, each UAV inspection waypoint includes: the UAV's 3D inspection location, the UAV gimbal's adjusted attitude, the UAV's estimated arrival time, and the UAV's shooting direction. Multiple dynamic inspection waypoints are arranged in chronological order to form a UAV inspection waypoint sequence.

[0043] A multi-objective path optimization model is constructed based on the UAV cruise point sequence to output the optimal inspection path. The multi-objective path optimization model includes inspection quality optimization objective, flight safety optimization objective, flight energy consumption optimization objective, and inspection time optimization objective.

[0044] The specific goal of inspection quality optimization is to maximize the overall shooting quality score of all drone patrol points. The formula for the overall shooting quality score is: ; In the formula: The overall image quality score for the i-th drone cruise point is... For geometric quality score, Let i be the three-dimensional spatial coordinates of the i-th waypoint. Let i be the target time corresponding to the i-th waypoint. for Predict confidence level at any time This represents the confidence sensitivity coefficient.

[0045] The specific objectives of flight safety optimization are: to minimize the total flight path length and attitude adjustment energy consumption of the UAV cruise point; The specific goal of flight energy consumption optimization is to minimize the risk of collision between the drone's cruise point and obstacles such as blades and towers; The specific goal of the inspection time optimization is to minimize the total time for all drone patrol points.

[0046] In this embodiment of the invention, a sequence of UAV cruise points is generated based on the predicted values ​​of blade motion states. Each cruise point includes information such as the UAV's spatial location, gimbal adjustment attitude, estimated arrival time, and shooting direction. Since the blade motion states, shooting angles, and prediction reliability vary for different cruise points, it is necessary to establish a cruise quality optimization objective to evaluate the image acquisition effect of each cruise point.

[0047] The goal of the inspection quality optimization is to maximize the overall image quality score of all UAV cruise points. The overall image quality score evaluates whether the UAV can obtain high-quality images that meet the requirements for blade defect detection at the current waypoint. This score comprehensively considers factors such as the waypoint's three-dimensional spatial location, the target shooting time, the blade motion prediction confidence level, and the shooting geometry. Specifically, the geometric quality score describes the positional relationship between the UAV and the blade, including the distance between the UAV and the blade, the angle between the camera's optical axis and the blade surface normal, and the coverage of the target blade area; the waypoint's three-dimensional spatial coordinates determine whether the UAV's current position is in the optimal shooting area; the target time evaluates whether the UAV arrives at the waypoint in sync with the blade's motion state; and the prediction confidence level measures the reliability of the predicted blade motion state. A high prediction confidence level indicates accurate prediction of the blade position at the current waypoint, which improves the corresponding image quality score; a low prediction confidence level reduces the evaluation weight of that waypoint through a confidence sensitivity coefficient to avoid invalid shots due to prediction errors.

[0048] A flight safety optimization objective is established to ensure the safe flight of UAVs without shutting down the wind turbines. This objective is primarily achieved by minimizing the total flight path length between UAV cruise points and the energy consumption for attitude adjustment. Specifically, the UAV flight path is optimized based on the spatial distance relationship between each inspection waypoint, reducing unnecessary waypoint transfer distances. Simultaneously, based on changes in the UAV gimbal attitude, the decrease in flight stability caused by frequent adjustments is mitigated, enabling the UAV to maintain a stable flight state.

[0049] A flight energy consumption optimization objective is established to reduce energy consumption during UAV inspection missions. This objective primarily considers the collision risk between the UAV and blades, towers, and other wind turbine structures. By optimizing the UAV's flight position and waypoint connection paths, the UAV can reduce detour distances and attitude adjustments while meeting safe distance constraints. For example, when a certain inspection waypoint has a high collision risk, the model automatically adjusts the waypoint position to maintain a safe distance between the UAV and the blades and towers, thereby reducing the additional energy consumption caused by obstacle avoidance.

[0050] An inspection time optimization target is established to reduce the time required for UAVs to complete all inspection tasks. By optimizing the order of waypoint visits, UAV flight speed, and waypoint dwell time, the UAV can quickly complete the inspection of key areas of the blades. For example, based on blade motion prediction results, priority is given to visiting blades that are about to enter the optimal shooting area, reducing the time lost waiting for the blades to move to the target position.

[0051] This invention integrates the above four optimization objectives. The multi-objective path optimization model jointly calculates inspection quality, flight safety, energy consumption, and inspection time to output the UAV inspection path with optimal overall performance. Unlike traditional planning methods that only rely on the shortest path or fixed waypoints, this invention introduces blade motion prediction results and multi-factor constraints, enabling the UAV to dynamically adjust the inspection path while the wind turbine is running without stopping. This improves the quality of blade image acquisition, reduces flight risks, and enhances overall inspection efficiency.

[0052] Preferably, a pre-obtained standard leaf image is compared with the target leaf image to obtain image quality assessment factors, an image quality assessment model is established, and the inspection effect assessment coefficient is output; wherein, the image quality assessment factors include the clarity of the leaf image, the integrity of the leaf image, and the suitability of the lighting of the leaf image; specifically, Obtain pre-processed standard leaf images and target leaf images, and obtain image quality assessment factors. Construct an image quality assessment model based on the image quality assessment factors, and output inspection effect assessment coefficients. The image quality assessment model can be a weighted model.

[0053] The sharpness of the leaf image is determined by extracting edge features from both a standard leaf image and a target leaf image, and then calculating the gradient change value of each pixel in the image using a gradient operator. The calculation formula is as follows: ; In the formula: For pixels The gradient change value, For pixels The horizontal gradient, For pixels The gradient in the vertical direction; The gradient change values ​​of all pixels in the target leaf image and the standard leaf image are summarized separately. The average gradient change value is calculated, and the ratio of the gradient change value of the target leaf image to that of the standard leaf image is used as the sharpness of the leaf image.

[0054] The integrity of the leaf image is determined by segmenting the leaf region into both the standard leaf image and the target leaf image, extracting the leaf contour to obtain the leaf region, and constructing a spatial matching model to output the integrity of the leaf image. The specific formula is as follows: ; In the formula: For the completeness of the leaf image, This refers to the leaf region in a standard leaf image. The leaf region in the target leaf image; The light suitability of leaf images is determined by extracting light features from both standard and target leaf images. These features include the mean image brightness and the degree of brightness variation. A light assessment factor is calculated based on these features, and the ratio of this ratio to the light assessment factor of the target leaf image is used as the light suitability of the leaf image. The specific formula for the light assessment factor is as follows: ; In the formula: For the light suitability of the leaf image, The weighting coefficients for the mean image brightness. The average brightness of the image. The weighting coefficients represent the degree of change in image brightness. This represents the degree of change in image brightness.

[0055] In this embodiment of the invention, after the UAV arrives at the corresponding inspection waypoint according to the planned inspection path, it acquires images of the target blades using an onboard high-definition camera. Because the wind turbine blades are continuously rotating and affected by factors such as changes in ambient wind speed, lighting conditions, UAV attitude adjustment errors, and shooting distance, the actual acquired images of the target blades may suffer from problems such as image blurring, missing blade areas, and abnormal brightness. Therefore, it is necessary to quantitatively evaluate the target blade images using an image quality assessment model.

[0056] The system acquires pre-stored standard blade images and target blade images currently collected by the UAV. The standard blade images are high-quality blade images acquired under ideal inspection conditions, featuring clear blade edge structures, complete blade areas, and stable lighting environments, and can serve as a reference benchmark for evaluating the quality of target blade images.

[0057] After acquiring standard and target leaf images, preprocessing operations were performed on both types of images, including image size unification, image coordinate correction, noise filtering, and leaf region extraction, to ensure that the standard and target leaf images have a consistent data basis. Preprocessing reduces the impact of differences in shooting angle, image resolution, and background interference on the quality evaluation results. Based on the preprocessed image data, three image quality evaluation factors were obtained: leaf image sharpness, leaf image integrity, and leaf image lighting suitability.

[0058] The sharpness of the blade images is evaluated. Blade image sharpness reflects whether the target blade image effectively preserves the surface texture, edge structure, and defect details of the blade. In actual inspection processes, when the drone's flight speed is too high or the blade rotation speed changes significantly, motion blur can easily occur, weakening the blade edge information. Therefore, edge features are extracted from both standard and target blade images, and gradient operators are used to calculate the gradient change value of each pixel in the image to obtain the degree of edge change corresponding to each pixel. Subsequently, the gradient change values ​​of all pixels in both the standard and target blade images are statistically analyzed, and the average gradient change value is calculated.

[0059] The mean gradient of the standard leaf image represents the intensity of the leaf edge features under ideal conditions, while the mean gradient of the target leaf image represents the actual edge feature intensity of the image acquired by the UAV. Furthermore, the ratio of the mean gradient change of the target leaf image to the mean gradient change of the standard leaf image is calculated to obtain the leaf image sharpness evaluation factor.

[0060] When the clarity of the target leaf image is close to that of the standard leaf image, it means that the current image can preserve the leaf texture information relatively completely, and the clarity evaluation factor is close to 1; when the target leaf image has obvious blurring, the gradient change value decreases, resulting in a decrease in the clarity evaluation factor.

[0061] Secondly, the integrity of the blade images is evaluated. Blade image integrity is used to determine whether the image captured by the drone completely covers the target blade area. In actual inspections, due to deviations in the drone's shooting angle, flight position errors, or changes in blade movement, some blade areas may not be within the camera's field of view, thus affecting the defect detection results.

[0062] Specifically, blade region segmentation is performed on both the standard blade image and the target blade image. The blade contour region is extracted using an image segmentation algorithm to obtain the standard blade region and the target blade region. The standard blade region represents the complete blade region that needs to be detected under ideal conditions, while the target blade region represents the blade region actually captured by the UAV. Subsequently, a spatial matching model is established to calculate the spatial overlap between the two blade regions, thus obtaining the completeness of the blade image.

[0063] The closer the integrity calculation result is to 1, the higher the degree of matching between the target blade area and the standard blade area, and the more complete the detection area is covered by the current image. When the integrity is low, it means that there are missing blades or occlusion in the target blade image, and the drone inspection waypoints or shooting direction need to be readjusted.

[0064] The suitability of illumination in blade images is evaluated. Since wind turbines are typically deployed in unobstructed areas, the inspection process is easily affected by changes in sunlight, cloud cover, and environmental reflections, leading to overly bright, underly dark, or uneven brightness distribution in target blade images. Therefore, illumination features are extracted from standard and target blade images to evaluate the current image illumination conditions. These features include the average image brightness and the degree of brightness variation. The average image brightness describes the overall exposure level, reflecting the overall brightness of the image; the degree of brightness variation describes the brightness differences between different areas of the image, reflecting the uniformity of illumination.

[0065] The illumination assessment factors corresponding to the standard leaf image and the target leaf image are calculated separately. The ratio of the illumination assessment factor of the target leaf image to the illumination assessment factor of the standard leaf image is then calculated to obtain the illumination suitability of the leaf image.

[0066] When the lighting conditions of the target leaf image are close to the standard acquisition environment, the lighting suitability is high; when the target leaf image has abnormal exposure or uneven brightness, the lighting suitability decreases. After obtaining the sharpness evaluation factor, integrity evaluation factor, and lighting suitability evaluation factor, an image quality evaluation model is constructed. The image quality evaluation model adopts a weighted fusion method, which comprehensively calculates multiple image quality evaluation factors according to preset weights, and outputs the inspection effect evaluation coefficient.

[0067] Using the above method, the inspection effect evaluation coefficient can comprehensively reflect whether the current UAV inspection image meets the defect detection requirements. When the inspection effect evaluation coefficient is greater than the preset threshold, the current target blade image is deemed to be of acceptable quality and can be directly input into the blade defect detection model; when the inspection effect evaluation coefficient is lower than the preset threshold, it is considered that the current image cannot support accurate defect identification, and it is necessary to readjust the UAV inspection waypoint, shooting attitude, or re-acquire images.

[0068] Therefore, this solution establishes a multi-dimensional image quality assessment model based on the comparison of standard blade images, thereby achieving automated and quantitative evaluation of the image quality of UAV inspections. Compared with the traditional method of judging image quality based on human experience, it can more accurately identify low-quality inspection images, improve the reliability of subsequent blade defect detection, and provide effective feedback for the dynamic adjustment of UAV inspection paths.

[0069] Preferably, the inspection task priority is determined based on the inspection effect evaluation coefficient, blade defect detection is performed on the target blade image, the corresponding UAV inspection waypoints are associated with the blade defect results, and the optimal inspection path is evaluated to determine whether it meets the standard. The specific steps are as follows: Based on the inspection effectiveness evaluation coefficient of each drone patrol point, the inspection task sequence is determined by arranging the values ​​in descending order, and the inspection task priority is determined according to the inspection task sequence. A deep learning target detection model is used to detect blade defects in target blade images, and the detected defect type, location, severity and other information are associated with the corresponding waypoints; If the image quality of a target blade at a certain waypoint does not meet the requirements or the confidence level of defect detection is low, the corresponding UAV cruise point needs to be re-photographed; The number of drone patrol points that need to be re-captured is counted. If the number exceeds the preset number, the optimal inspection path does not meet the standard; if the number does not exceed the preset number, the optimal inspection path meets the standard.

[0070] In this embodiment of the invention, the inspection effectiveness evaluation coefficients of all UAV patrol points are arranged in descending order of value to generate an inspection task sequence. For example, when a wind turbine inspection task includes multiple inspection points, if the inspection effectiveness evaluation coefficients for point A, point B, and point C are 0.95, 0.82, and 0.90 respectively, then the inspection tasks are sorted according to the evaluation coefficients to form the sequence A, C, B, and their corresponding inspection priorities are set accordingly. This method allows UAVs to prioritize inspection tasks with higher image acquisition quality and stronger detection reliability, and ensures the acquisition of inspection data in critical areas when UAV endurance is insufficient or environmental conditions change.

[0071] Furthermore, blade defect detection is performed on the target blade images acquired by each UAV. Specifically, the target blade images are input into a pre-trained deep learning target detection model. The model extracts surface texture features, edge features, and defect region features to automatically identify blade defects. The blade defect detection results include information such as defect type, defect spatial location, defect area, defect severity, and defect detection confidence. For example, if the target detection model detects a crack defect in a target blade region, it outputs the crack type, the coordinates of the blade region where the crack is located, and the corresponding detection confidence.

[0072] The system associates blade defect detection results with corresponding UAV inspection waypoints. Specifically, based on the target blade image acquisition time, image number, UAV spatial location, and inspection waypoint number, a correspondence is established between defect detection results and inspection waypoints, enabling the system to determine the location of the defect and the corresponding flight path. For example, when an abnormal defect is detected in a certain blade area, the system can directly locate the corresponding inspection waypoint, providing a basis for subsequent re-shooting or path adjustment.

[0073] The validity of each UAV inspection point is assessed. If the image quality of the target blade corresponding to a certain inspection point does not meet the preset requirements, it means that the current point cannot provide a valid image that meets the defect detection needs, and the UAV patrol point is marked as a point that needs to be retaken. For example, if the target blade image has insufficient clarity, the blade area is not completely covered, or the lighting conditions are abnormal, the current image quality is determined to be unqualified.

[0074] The defect detection confidence score output by the defect detection model is assessed. If the defect detection confidence score corresponding to a certain inspection waypoint is lower than a preset threshold, it indicates that the current target blade image cannot guarantee the accuracy of defect identification, and the corresponding UAV cruise point is marked as a waypoint that needs to be reshot. After completing the assessment of all inspection waypoints, the number of UAV cruise points that need to be reshot is counted and compared with the preset allowed number of reshots. If the number of UAV cruise points that need to be reshot exceeds the preset number, it indicates that although the current optimal inspection path meets the requirements of spatial distance, safety constraints, and energy consumption optimization, there are many invalid acquisition points in the actual inspection process, which cannot meet the blade defect detection requirements. Therefore, the current optimal inspection path is determined to be non-compliant with the standard.

[0075] At this point, the path evaluation results are fed back to the path optimization module. Based on the reasons for the inspection failure, the drone inspection waypoints are readjusted, including optimizing the drone's shooting position, adjusting the gimbal attitude, changing the inspection time, or replanning the flight path. If the number of drone cruise points that need to be reshot does not exceed the preset number, it means that the current inspection path can ensure that most inspection waypoints obtain effective detection data, meeting the requirements for image quality and defect detection. Therefore, the current optimal inspection path is determined to meet the standards.

[0076] The embodiments of the present invention implement a path evaluation mechanism based on actual inspection effect feedback, enabling the UAV inspection system to not only optimize the inspection path from the perspectives of theoretical path distance, safety and energy consumption, but also to verify the effectiveness of the path by combining the actual acquired image quality and defect detection results, thereby improving the accuracy, reliability and intelligence level of wind turbine non-stop inspection.

[0077] Example 2: In this embodiment of the invention, a deep learning-based intelligent wind turbine inspection path optimization method is applied to a real-world scenario. Multi-source data monitoring devices deployed on the wind turbine acquire multi-dimensional inspection data during wind turbine operation. Since different monitoring devices have different data types, sampling frequencies, and data formats, the acquired multi-source inspection data needs to be preprocessed to enable the subsequent prediction model to learn the blade motion change patterns based on data in a unified format, thereby improving the accuracy of the prediction results.

[0078] After obtaining multidimensional inspection time-series data, it is input into a deep learning prediction model for blade motion state to predict the blade motion state in future time periods. The multidimensional inspection time-series data is then input into a temporal convolutional network (TCN), which uses the temporal convolutional structure to extract short-term variation features during blade motion. Since blade motion state is affected by factors such as instantaneous wind speed changes, rotational speed fluctuations, and pitch angle adjustments, the TCN can extract short-term variation features from continuous time series, such as short-term fluctuations in blade rotational speed, motion changes caused by wind speed disturbances, and changes in blade rotation period.

[0079] The short-term motion variation characteristics output by the TCN are input into a gated recurrent unit network (GRU), and the recurrent structure is used to further learn the long-term temporal correlations in the blade motion process. By analyzing the historical operating state through GRU, the future motion trend of the blade can be obtained. For example, it can be determined whether the current wind speed change will lead to the future change of the blade's spatial position, and the trend of the blade attitude angle change over time can be predicted, thus obtaining the long-term motion variation characteristics.

[0080] Long-term motion change patterns are input into the attention mechanism module, which adaptively calculates the importance of different input features for predicting future blade motion. For example, when wind speed changes have a significant impact on future blade motion, the weight of wind speed-related features is increased; when pitch angle adjustments have a significant impact on motion state, the importance of pitch angle-related features is increased. The attention mechanism filters key factors affecting the future state of the blades, and features with higher importance are input into the prediction output layer as future focus factors.

[0081] Finally, the prediction model outputs predicted values ​​of the blade's motion state, including the blade's spatial position at future time points, predicted blade attitude angles, predicted blade motion time, and the target detection area of ​​the blade. For example, if the model predicts that a defective area on a blade will enter the designated detection range in 5 seconds, the UAV control system can adjust its flight position in advance so that the UAV can simultaneously complete the image capture when the blade moves to the target position.

[0082] Based on the blade motion state prediction results, a sequence of UAV inspection waypoints is further generated. Specifically, the UAV inspection position is calculated according to the spatial position of the blades corresponding to future time nodes, the UAV gimbal attitude is adjusted according to the predicted blade attitude angle, the expected arrival time of the UAV is calculated according to the predicted blade motion time, and the UAV shooting direction is determined according to the blade target detection area. Each inspection waypoint includes information such as the UAV spatial position, gimbal attitude adjustment, expected arrival time, and shooting direction.

[0083] After generating the UAV inspection waypoint sequence, a multi-objective path optimization model is constructed based on the positional relationships between the inspection waypoints to optimize the UAV flight path. The multi-objective path optimization model includes inspection quality optimization objectives, flight safety optimization objectives, flight energy consumption optimization objectives, and inspection time optimization objectives.

[0084] By jointly optimizing the above multiple objectives, the optimal inspection path is output, which meets the requirements of inspection quality, safety, energy consumption, and time.

[0085] Subsequently, the UAV performs inspection tasks according to the optimal inspection path, and collects images of the corresponding target blades at each UAV inspection waypoint. To ensure that the collected images meet the requirements for subsequent defect detection, pre-established standard blade images are acquired and compared with the target blade images to establish an image quality assessment model.

[0086] Specifically, the standard leaf image and the target leaf image are first preprocessed, including image size unification, coordinate correction, noise filtering, and leaf region extraction. Then, image quality evaluation factors are obtained from three aspects: sharpness, completeness, and suitability of illumination.

[0087] After obtaining the aforementioned image quality assessment factors, a weighted fusion-based image quality assessment model is established. This model comprehensively calculates sharpness, integrity, and lighting suitability according to preset weights, outputting an inspection effect evaluation coefficient. This coefficient is used to evaluate whether the images acquired by the current UAV inspection waypoints meet the defect detection requirements.

[0088] Inspection tasks are prioritized based on inspection effectiveness evaluation coefficients, and the optimal inspection path is evaluated in conjunction with blade defect detection results. Specifically, based on the inspection effectiveness evaluation coefficients corresponding to each UAV cruise point, the tasks are sorted from largest to smallest to form an inspection task sequence, and the inspection points with higher evaluation coefficients are set as priority tasks.

[0089] The target blade image is input into a deep learning target detection model to identify surface defects. The detection results include defect type, defect location, defect severity, and defect detection confidence. Based on the target blade image acquisition time, image number, and UAV waypoint number, the detection results are associated with the corresponding inspection waypoints, establishing a correspondence between defect location and flight path.

[0090] The validity of each inspection waypoint is assessed. If the image quality of the target blade corresponding to a waypoint does not meet the preset requirements, or the defect detection confidence level is lower than the set threshold, then the waypoint is marked as a waypoint that needs to be reshot. The total number of all UAV inspection waypoints that need to be reshot is counted and compared with the preset allowable number.

[0091] If the number of waypoints that need to be reshot exceeds the preset threshold, it means that although the current optimal inspection path meets the theoretical path optimization requirements, the actual inspection effect cannot meet the needs of blade defect detection. Therefore, it is determined that the current optimal inspection path does not meet the standard, and the path optimization module is returned to readjust the inspection waypoints and flight strategy.

[0092] If the number of waypoints that need to be reshot does not exceed the preset threshold, it means that the current inspection path can ensure that most inspection waypoints obtain effective detection data, and the current optimal inspection path is deemed to meet the standard.

[0093] The embodiments of the present invention can proactively adjust the inspection strategy according to the future movement state of the blades, improve the safety of UAV inspection, image acquisition quality and blade defect identification accuracy under the condition that the wind turbine does not stop running, and realize intelligent and highly reliable unmanned inspection of wind turbine blades.

[0094] Example 3: A deep learning-based intelligent wind turbine inspection path optimization system, comprising a multi-dimensional inspection data acquisition module, a motion state prediction module, an inspection path optimization module, an inspection effect evaluation module, and a blade defect detection module; the multi-dimensional inspection data acquisition module is connected to the motion state prediction module, the motion state prediction module is connected to the inspection path optimization module, the inspection path optimization module is connected to the inspection effect evaluation module, and the inspection effect evaluation module is connected to the blade defect detection module.

[0095] The multidimensional inspection data acquisition module is used to acquire multidimensional inspection data through data monitoring equipment, and to perform data preprocessing to output multidimensional inspection time-series data; The motion state prediction module is used to input multidimensional inspection time series data into the blade motion state deep learning prediction model and output the blade motion state prediction value. The inspection path optimization module is used to generate a UAV inspection waypoint sequence based on the predicted value of blade motion state, and to build a multi-objective path optimization model based on the UAV cruise point sequence to output the optimal inspection path. The inspection effect evaluation module is used to execute inspection tasks according to the optimal inspection path, capture target blade images of each UAV inspection point, obtain pre-standard blade images and target blade images to obtain image quality evaluation factors, establish an image quality evaluation model, and output inspection effect evaluation coefficients. The blade defect detection module is used to prioritize inspection tasks based on the inspection effect evaluation coefficient, detect blade defects in target blade images, associate the blade defect results with corresponding UAV inspection waypoints, and evaluate whether the optimal inspection path meets the standards.

[0096] like Figure 2 The diagram shown is a system block diagram of a deep learning-based intelligent wind turbine inspection path optimization system according to an embodiment of the present invention, which can be used to execute... Figure 1 The steps in the method embodiments shown are implemented in a similar manner and have similar technical effects, and will not be repeated here.

[0097] Through the above embodiments, this invention acquires multi-dimensional inspection data during wind turbine operation and uses a deep learning prediction model to predict the blade motion state. This allows for advance acquisition of the blade's future spatial position, attitude changes, and motion time information, enabling the UAV to plan its shooting position in advance based on the blade's dynamic changes. This avoids shooting misalignment caused by blade rotation and improves the success rate of target area acquisition. By generating inspection waypoint sequences based on the blade motion state prediction results and constructing a multi-objective path optimization model that includes inspection quality, flight safety, flight energy consumption, and inspection time, this model comprehensively considers factors such as image acquisition effect, UAV flight stability, and energy consumption, outputting a more reasonable inspection path. Compared with traditional single-distance optimization methods, this improves the efficiency and reliability of UAV inspections. By establishing an image quality evaluation model based on standard blade image comparison, the model comprehensively evaluates the clarity, completeness, and lighting suitability of the images acquired by the UAV and outputs an inspection effect evaluation coefficient. This achieves automatic judgment of the validity of inspection data and reduces the problem of false or missed defects due to insufficient image quality. By combining a deep learning-based blade defect detection model, the defect type, location, and severity are associated with the corresponding inspection waypoints. The optimal inspection path is then determined based on the number of waypoints that need to be reshot, thus achieving closed-loop feedback optimization of the inspection path and improving the reliability of the inspection results.

[0098] The embodiments of the present invention can achieve accurate prediction of blade motion status, intelligent planning of UAV paths, automatic evaluation of inspection image quality, and optimization of defect detection results under the condition of continuous operation of wind turbines. This effectively improves the automation level, detection accuracy, and operational safety of wind turbine blade inspection, reduces the cost of manual inspection, and improves the level of intelligent operation and maintenance of wind power equipment.

[0099] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0100] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for optimizing intelligent inspection paths for wind turbines based on deep learning, characterized in that, Includes the following steps: Multidimensional inspection data is acquired through data monitoring equipment, and multidimensional inspection time-series data is output after data preprocessing. By inputting multidimensional inspection time series data into the deep learning prediction model of blade motion state, the predicted value of blade motion state is output. A sequence of UAV inspection waypoints is generated based on the predicted values ​​of blade motion state. A multi-objective path optimization model is constructed based on the UAV cruise point sequence to output the optimal inspection path. The inspection task is carried out according to the optimal inspection path. The target blade image of each UAV inspection point is captured. The pre-obtained standard blade image is compared with the target blade image to obtain the image quality evaluation factor. An image quality evaluation model is established and the inspection effect evaluation coefficient is output. Based on the inspection effect evaluation coefficient, the inspection task priority is determined, the blade defect detection is performed on the target blade image, the corresponding UAV inspection waypoints are associated with the blade defect results, and the optimal inspection path is evaluated to determine whether it meets the standard.

2. The wind turbine intelligent inspection path optimization method based on deep learning according to claim 1, characterized in that, The multidimensional inspection heterogeneous data includes wind turbine SCADA data, meteorological data, historical inspection data, wind turbine 3D model data, and drone real-time status data.

3. The wind turbine intelligent inspection path optimization method based on deep learning according to claim 1, characterized in that, By inputting multidimensional inspection time-series data into a deep learning prediction model for blade motion state, the predicted values ​​of blade motion state are output. The specific steps are as follows: By inputting multidimensional inspection time-series data into the temporal convolutional network (TCN), the first-order motion change characteristics during the blade motion process are extracted. By inputting short-term motion features into a gated recurrent unit (GRU) network, the long-term temporal correlation of blade motion states is learned, and the characteristics of the secondary motion change patterns of the blade are captured. By inputting the secondary motion change pattern characteristics into the attention mechanism module, the importance of the input features is adaptively calculated, and the features of future attention factors are identified based on the importance. By fusing the features of future factors of interest and inputting them into the prediction output layer, the predicted value of the blade motion state is output.

4. The wind turbine intelligent inspection path optimization method based on deep learning according to claim 3, characterized in that, The predicted values ​​of blade motion state include blade spatial position, blade attitude angle, blade motion time, and blade target area.

5. The wind turbine intelligent inspection path optimization method based on deep learning according to claim 4, characterized in that, A sequence of UAV inspection waypoints is generated based on the predicted values ​​of blade motion state. A multi-objective path optimization model is then constructed based on the UAV cruise point sequence to output the optimal inspection path. The specific steps are as follows: Based on the predicted values ​​of blade motion status, UAV inspection waypoints are automatically generated, forming a UAV inspection waypoint sequence. A multi-target path optimization model is constructed based on the drone patrol point sequence, and the optimal inspection path is output. The multi-objective path optimization model includes inspection quality optimization objective, flight safety optimization objective, flight energy consumption optimization objective, and inspection time optimization objective.

6. The wind turbine intelligent inspection path optimization method based on deep learning according to claim 5, characterized in that, Each drone inspection waypoint includes the drone inspection location, the drone gimbal adjustment attitude, the drone's estimated arrival time, and the drone's shooting direction.

7. The wind turbine intelligent inspection path optimization method based on deep learning according to claim 6, characterized in that, The generation of dynamic inspection waypoints based on blade motion state predictions includes: calculating the corresponding UAV inspection position based on the blade's spatial position at future time points; calculating the UAV gimbal attitude adjustment based on the predicted blade attitude angle; calculating the UAV's estimated arrival time based on the predicted blade motion time; and generating the UAV's shooting direction based on the blade target area.

8. The wind turbine intelligent inspection path optimization method based on deep learning according to claim 1, characterized in that, Image quality assessment factors include the sharpness of the leaf image, the completeness of the leaf image, and the suitability of the lighting conditions of the leaf image.

9. The method for optimizing intelligent inspection paths for wind turbines based on deep learning according to claim 1, characterized in that, Based on the inspection effectiveness evaluation coefficient, inspection tasks are prioritized. Blade defects are detected in the target blade images. Corresponding UAV inspection waypoints are linked to the blade defect results. The optimal inspection path is evaluated to determine if it meets the standards. The specific steps are as follows: Based on the inspection effectiveness evaluation coefficient of each drone patrol point, the inspection task sequence is determined by arranging the values ​​in descending order, and the inspection task priority is determined according to the inspection task sequence. A deep learning target detection model is used to detect blade defects in target blade images, and the detected defect type and location are associated with the corresponding UAV cruise point; If the image quality of a target blade at a certain waypoint does not meet the requirements or the confidence level of defect detection is low, the corresponding UAV cruise point needs to be re-photographed; The number of drone patrol points that need to be re-captured is counted. If the number exceeds the preset number, the optimal inspection path does not meet the standard; if the number does not exceed the preset number, the optimal inspection path meets the standard.

10. A deep learning-based intelligent wind turbine inspection path optimization system, applied to the deep learning-based intelligent wind turbine inspection path optimization method as described in any one of claims 1-9, characterized in that, It includes a multi-dimensional inspection data acquisition module, a motion state prediction module, an inspection path optimization module, an inspection effect evaluation module, and a blade defect detection module; The multidimensional inspection data acquisition module is used to acquire multidimensional inspection data through data monitoring equipment, and to perform data preprocessing to output multidimensional inspection time-series data; The motion state prediction module is used to input multidimensional inspection time series data into the blade motion state deep learning prediction model and output the blade motion state prediction value. The inspection path optimization module is used to generate a UAV inspection waypoint sequence based on the predicted value of blade motion state, and to build a multi-objective path optimization model based on the UAV cruise point sequence to output the optimal inspection path. The inspection effect evaluation module is used to execute inspection tasks according to the optimal inspection path, capture target blade images of each UAV inspection point, obtain pre-standard blade images and target blade images to obtain image quality evaluation factors, establish an image quality evaluation model, and output inspection effect evaluation coefficients. The blade defect detection module is used to prioritize inspection tasks based on the inspection effect evaluation coefficient, detect blade defects in target blade images, associate the blade defect results with corresponding UAV inspection waypoints, and evaluate whether the optimal inspection path meets the standards.

Images