A stain following and drone disturbance detection method for offshore wind turbine blades

Abstract

The present invention provides a method for tracking and detecting stains on offshore wind turbine blades using a drone. The method comprises the following steps: using a drone-mounted camera to capture images of wind turbine blades and preprocessing the images to obtain a spot trajectory on the wind turbine blades; performing time-series processing on the spot trajectory of the wind turbine blades to extract the temporal and spatial characteristics of the spot trajectory, decoupling the interference source of the drone cleaning the offshore wind turbine blades, and obtaining a disturbance detection result; and adjusting the drone flight control strategy based on the disturbance detection result to maintain the stability of the drone when cleaning the spot on the wind turbine blades. The present invention uses multiple disturbance identification methods to perform time-series processing of the trajectory from visual recognition, without relying on the drone's own sensors for identification. Instead, the method identifies the disturbance type directly through the change in the spot's position. This change in the time-series signal has high recognizability and decoupling properties on the trajectory, effectively identifying disturbances while enabling the drone to follow the blade position.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power generation, and in particular to a method for detecting stains on offshore wind turbine blades and disturbances caused by unmanned aerial vehicles (UAVs). Background Art

[0002] The blades of offshore wind turbines are a core component of offshore wind farms and are crucial for converting wind energy into mechanical energy. However, the complex offshore environment, particularly factors such as sea breezes, salt spray, humidity, and bird activity, can easily accumulate contaminants such as bird droppings, salt deposits, and the attachment of marine organisms to the blade surfaces. This contaminant not only affects the blades' aerodynamic performance, reducing wind turbine efficiency, but can also accelerate corrosion of the blade material, shortening the turbine's lifespan.

[0003] Therefore, regular blade cleaning has become a crucial task in wind farm maintenance. Drone technology offers an automated and efficient solution for cleaning wind turbine blades. Drones equipped with high-pressure water jets or other cleaning tools can clean blades without stopping them, thus reducing downtime.

[0004] In actual applications, drone cleaning also faces multiple technical difficulties:

[0005] 1) Wind speeds are typically high at sea, which not only affects the flight stability of the drone but can also interfere with the precise positioning of the cleaning nozzles. In high wind conditions, it is difficult for the drone to maintain a fixed position, which can result in reduced cleaning accuracy.

[0006] 2) Fan blades continuously rotate during operation, making cleaning more difficult. Drones must be able to sense the blades' position and trajectory in real time to ensure precise spraying of cleaning fluid during their rotation. This requires high-precision tracking and positioning technology.

[0007] 3) Drones typically use high-pressure water guns or other high-pressure nozzles. However, the recoil generated by the nozzles can affect the drone's attitude control, causing it to deviate from its target during the cleaning process. Therefore, the drone needs to have good anti-interference capabilities and attitude stability to ensure a continuous and stable cleaning process.

[0008] Faced with these three types of interference, the spot location disturbances detected by the drone exhibit distinct characteristics. First, for sea breeze disturbances, the wind direction typically remains relatively constant over short periods of time. This causes the detected spot location trajectory to curve in the direction of the wind. For the rotation of wind turbine blades, the detected spot location trajectory, relative to the world coordinate system, follows an arc around the turbine rotor. The recoil generated by the onboard cleaning device during cleaning operations causes the spot coordinate trajectory to shift in the opposite direction of the water spray. In actual operating environments, all three of these interferences are present simultaneously, resulting in significant irregularities in the measurement estimates, largely due to the combined effects of the three interferences. During flight control, if the three interference sources can be decoupled and flight control anti-interference control tailored to the specific source of the disturbance, the computational resources required for flight stability control can be significantly reduced, adjustments can be made more reliably, and the drone's position can be maintained more quickly and stably. Therefore, interference detection is particularly important during offshore wind turbine blade maintenance.

[0009] However, much of the current research on UAV flight interference resistance is based on flight control compensation within the drone itself. For example, disturbance observers are primarily used to estimate and compensate for unknown disturbances or modeling errors in the system, but they are generally unable to directly identify the specific type of disturbance, such as "sea breeze disturbance" or "water jet recoil disturbance." Rather, they tend to estimate the magnitude and direction of the disturbance and perform real-time compensation within the system. In most cases, disturbance observers only estimate the sum of the disturbances, such as changes in the external force or torque of the entire system. The observer cannot directly distinguish whether a specific disturbance is caused by sea breeze, water jet recoil, etc. This is because the observer lacks the ability to decode external environmental information; it only infers an unknown disturbance signal based on the error between the system output and the internal state model. Summary of the Invention

[0010] In view of the shortcomings of the existing technology, the present invention provides a stain following and drone disturbance detection method for offshore wind turbine blades.

[0011] The technical solution of the present invention is: a method for spot tracking and drone disturbance detection for offshore wind turbine blades, comprising the following steps:

[0012] Step 1: Use the drone's onboard camera to capture wind turbine blade images at a constant frame rate;

[0013] Step 2: Preprocess the collected wind turbine blade image to obtain the three-dimensional coordinate trajectory P(t) = [x(t), y(t), z(t)] of the wind turbine blade stain in the world coordinate system; where [x(t), y(t), z(t)] represents the three-dimensional coordinate sequence that changes with time t;

[0014] Step 3: Perform time series processing on the stain trajectory P(t) = [x(t), y(t), z(t)] of the wind turbine blade to extract the temporal and spatial characteristics of the stain trajectory, decouple the interference source of the UAV offshore wind turbine blade cleaning, and obtain the disturbance detection results;

[0015] Step 4: Based on the disturbance detection results, adjust the drone flight control strategy to maintain the stability of the drone when cleaning the fan blade stains.

[0016] Preferably, in step 1, the collected wind turbine blade image includes a normal color image and a depth image; the normal color image and the depth image are used to identify the plane position (x, y) and depth position z of the stain, respectively.

[0017] Preferably, in step 2, the collected fan blade image is preprocessed to obtain a three-dimensional coordinate trajectory P(t)=[x(t), y(t), z(t)] of the fan blade stain relative to the world coordinate system, which specifically includes the following steps:

[0018] S21), dividing each collected ordinary color image into k blocks of S×S grids; wherein each grid is responsible for detecting the target falling within the grid in the image;

[0019] S22), for each grid, using a convolutional neural network to extract feature information of the grid to divide each image into k state feature quantities;

[0020] S23) According to the state transfer equation, the state feature quantity is converted into an intermediate state feature quantity, which is used to capture the spatial features in the image, namely:

[0021] h i+1 =Dh i +Bx i ;

[0022] Where h i represents the intermediate state characteristic of the i-th grid; h i+1 is the intermediate state feature quantity of the i+1th grid; D is the state transfer matrix; B is the transformation matrix of the state feature quantity; x i Represents the state characteristic of the i-th grid;

[0023] S24), based on the state estimation, an observation value y of the image is obtained, and the matching degree between the current state and the target category is calculated according to the observation value y; wherein the calculation formula of the observation value y is:

[0024] y=Ch;

[0025] Where C is the transformation matrix of the intermediate state feature; the observation value y represents the matching degree of each intermediate state feature h to the recognition target, which is used to determine the grid position of the object to be recognized in the image. The position of the recognition target in the image is determined by a multi-layer perceptron MLP. The input layer of the multi-layer perceptron MLP is a vector composed of the image observation value y and the image position of the state feature, and the output is the recognition bounding box, which includes the center coordinates (x, y) of the bounding box and the width and height (w, l) of the bounding box.

[0026] S25), aligning the depth image and the color image, obtaining the depth information of the area corresponding to the recognition bounding box to determine the three-dimensional coordinate position of the stain, and correcting the three-dimensional coordinate position of the stain;

[0027] S26) Repeat steps S21)-S25) to obtain the position trajectory of the fan blade stain.

[0028] Preferably, in step S25), the three-dimensional coordinate position of the stain is corrected, specifically comprising the following steps:

[0029] S251), select multiple images with small disturbance as reference positions; and record the position of the ideal stain i and its position relative to the ideal stain j distance Among them, the distance The calculation formula is:

[0030]

[0031] S252) Construct a reference relative coordinate matrix R, which records the relative position relationship of all ideal stains:

[0032]

[0033] S253) During the repeated collection and positioning process, the actual stain position is recorded. and its distance from the actual stain j position Calculate the relative position matrix between all actual stains detected And make a difference with the reference relative coordinate matrix R to obtain the deviation matrix ΔR;

[0034]

[0035] S254) Correct the stain coordinates using a rigid transformation to ensure that the identified stains conform to the positional relationship in the reference relative coordinate matrix. The rigid transformation is performed using the following formula:

[0036]

[0037] Where x′ i , y′ i 、z′ i is the position coordinate of the i-th stain after rigid transformation; x i 、y i 、z i is the position coordinate of the i-th stain before transformation; t is the translation vector, which is optimized by minimizing the coordinate deviation matrix ΔR;

[0038] S255), dynamically adjust the rotation matrix A and the translation vector t according to the deviation matrix ΔR, and gradually correct the positions of all stains until the relative coordinate error reaches a preset threshold; wherein the total error E is defined as the sum of the squares of the relative position deviations of all stains, that is:

[0039]

[0040] If the total error E is lower than the preset threshold, the correction is considered effective, and the corrected stain coordinates (x′1, y′1, z′1), (x′2, y′2, z′2), ... are output as the recorded position.

[0041] Preferably, step 3 specifically includes the following steps:

[0042] S31), normalizing, removing noise and interpolating the stain trajectory P(t)=[x(t), y(t), z(t)];

[0043] S32) Map the three-dimensional coordinates [x(t), y(t), z(t)] of each time step to the high-dimensional feature space through linear transformation to generate the high-dimensional feature representation f of the trajectory t ;

[0044] S33), the high-dimensional feature representation after embedding f t Perform position encoding to explicitly incorporate the sequential information of time steps into the feature representation to ensure that the subsequent self-attention mechanism can capture the temporal order;

[0045] S34), constructing a bidirectional dynamic self-attention mechanism to capture the dependency relationship at different time steps in the three-dimensional trajectory data of the embedded features through the bidirectional dynamic self-attention mechanism;

[0046] S35) Constructing a spatiotemporal graph convolutional network ST-GCN to capture the spatial correlation of trajectory data; treating each time step of the three-dimensional trajectory as a node in the spatiotemporal graph and the dependencies between time steps as edges in the spatiotemporal graph, and constructing a neighborhood for each time step node; and aggregating features from neighboring time steps through the spatiotemporal graph convolutional network ST-GCN:

[0047]

[0048] Where, f t (l+1) represents the node features on the time dimension t in the l+1th hidden layer; σ represents the activation function; d t is the degree of time step t; d j is the degree of time step j; W (l) is the weight matrix of the lth hidden layer; is the feature vector of node j in the lth hidden layer; is the set of neighboring time steps of time step t;

[0049] The spatiotemporal graph convolutional network ST-GCN updates the node representation of each time step layer by layer through multi-layer graph convolution operations, captures its global spatiotemporal dependencies, and obtains the spatiotemporal features of each time step;

[0050] S36) Fusing the temporal and spatial features output by the bidirectional dynamic self-attention mechanism and the spatiotemporal graph convolutional network ST-GCN;

[0051] S37) Input the fused spatiotemporal features into the classifier and use the Softmax activation function to predict the disturbance type and size, and output the final disturbance prediction classification result and the corresponding perturbation size Right now:

[0052]

[0053] Where softmax is a nonlinear activation function; W fc is the weight of the fully connected layer; x is the fused spatiotemporal features after fusion;

[0054] Among them, the disturbance size Including the direction and magnitude of the disturbance; the predicted disturbance classification results Including wind turbine blade rotation, sea breeze interference, and nozzle impact interference.

[0055] Preferably, in step S34), the bidirectional dynamic self-attention mechanism includes forward attention and backward attention; the forward attention and backward attention respectively calculate the dependency between the current time step and the past and future time steps to generate a forward attention weight matrix. The expression of the bidirectional dynamic self-attention mechanism is:

[0056]

[0057] Where Q = W Q f t ′, K=W K f′k 、V=W V f′ k are query matrix, key matrix and value matrix respectively; W Q 、W K 、W V is the learnable weight matrix, f′ k Represents the features of other time steps; softmax is a nonlinear activation function, which processes the inner product of Q and K to obtain the correlation score matrix between each feature. The inner product of the result and V can be used to obtain the attention map between the features; Represents the scaling factor. When the sequence length is long, scaling the inner product of Q and K is beneficial to smooth the output of softmax, thereby obtaining a more appropriate score matrix.

[0058] Preferably, in step S36), the fusion adopts a weighted addition operation to combine the two parts of features, learns the contributions of different feature sources through adaptive weights, and dynamically adjusts the importance of different time steps to obtain a more expressive feature fusion representation.

[0059] The beneficial effects of the present invention are:

[0060] 1. This invention uses multiple disturbance recognition methods from visual recognition trajectory time series processing. It does not rely on the drone's own sensors for recognition. It directly identifies the disturbance type by the position change of the stain. This change in the time series signal has high recognition and decoupling on the trajectory, effectively performing disturbance recognition and simultaneously completing the drone's tracking of the blade position.

[0061] 2. This invention uses the location and trajectory information of stains as input to detect external disturbances to the drone. It can identify the type of external disturbance and make more targeted adjustments accordingly to ensure tracking of stains during flight, achieving the goal of stable stain cleaning.

[0062] 3. The present invention uses image segmentation as spatial state information and combines it with the state transition model for positioning, thereby improving the efficiency of image processing;

[0063] 4. The present invention adopts the principle of relative position invariance to update and correct the stain position in real time during stain position positioning. It dynamically adjusts the correction parameters based on the results of disturbance detection to achieve more accurate position tracking.

[0064] 5. The present invention makes full use of the temporal and spatial characteristics of the trajectory during the time series processing, and can well detect the disturbance type, direction and size. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Schematic diagram of the process of the present invention;

[0066] Figure 2 A schematic diagram of stain location following the present invention;

[0067] Figure 3 Schematic diagram of the flow of the positioning estimation algorithm of the present invention;

[0068] Figure 4 A schematic diagram of the disturbance detection process of the present invention;

[0069] Figure 5 This is a framework diagram of the spatiotemporal graph convolutional network ST-GCN of the present invention. DETAILED DESCRIPTION

[0070] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0071] like Figure 1 As shown, this embodiment provides a method for spot tracking and drone disturbance detection for offshore wind turbine blades, comprising the following steps:

[0072] Step 1: Use the drone's onboard camera to capture wind turbine blade images at a constant frame rate;

[0073] The wind turbine blade images captured in this embodiment include a color image and a depth image; the color image and depth image are used to identify the plane position (x, y) and depth position (z) of the stain, respectively. The frame rate captured in this embodiment can be set to 15 fps, meaning 15 wind turbine blade images are captured every second.

[0074] Step 2: Preprocess the collected wind turbine blade image to obtain the three-dimensional coordinate trajectory P(t) = [x(t), y(t), z(t)] of the wind turbine blade stain in the world coordinate system; where [x(t), y(t), z(t)] represents the three-dimensional coordinate sequence that changes with time t; Figure 2 and 3 As shown, the specific steps include:

[0075] S21), dividing each collected ordinary color image into k blocks of S×S grids; wherein each grid is responsible for detecting the target falling within the grid in the image;

[0076] S22), for each grid, using a convolutional neural network to extract feature information of the grid to divide each image into k state feature quantities x;

[0077] S23) According to the state transfer equation, the state feature quantity x is converted into an intermediate state feature quantity, which is used to capture the spatial features in the image, namely:

[0078] h i+1 =Dh i+Bx i ;

[0079] Where h i represents the intermediate state characteristic of the i-th grid; h i+1 is the intermediate state feature quantity of the i+1th grid; D is the state transfer matrix; B is the transformation matrix of the state feature quantity; x i Represents the state characteristic of the i-th grid;

[0080] S24), based on the state estimation, an observation value y of the image is obtained, and the matching degree between the current state and the target category is calculated according to the observation value y; wherein the calculation formula of the observation value y is:

[0081] y=Ch;

[0082] Where C is the transformation matrix of the intermediate state feature; the observation value y represents the matching degree of each intermediate state feature h to the recognition target, which is used to determine the grid position of the target to be recognized in the image. The position of the recognized target in the image is determined by a multi-layer perceptron MLP. The input layer of the multi-layer perceptron MLP is a vector composed of the image observation value y and the image position of the state feature, and the output is the recognition bounding box, which includes the center coordinates (x, y) of the bounding box and the width and height (w, l) of the bounding box.

[0083] S25), aligning the depth image and the color image, obtaining the depth information of the area corresponding to the recognition bounding box, determining the three-dimensional coordinate position of the stain, and correcting the three-dimensional coordinate position of the stain;

[0084] S26) Repeat steps S21)-S25) to obtain the position trajectory of the fan blade stain.

[0085] Step 3: Perform time series processing on the trajectory of the wind turbine blade stain P(t) = [x(t), y(t), z(t)] to extract the time and space characteristics of the trajectory, extract the time and space characteristics of the stain trajectory, decouple the interference source of the UAV offshore wind turbine blade cleaning, and obtain the disturbance detection result; Figure 4 As shown, the specific steps include:

[0086] S31), normalizing, removing noise and interpolating the stain trajectory P(t)=[x(t), y(t), z(t)];

[0087] S32) Map the three-dimensional coordinates [x(t), y(t), z(t)] of each time step to the high-dimensional feature space through linear transformation to generate the high-dimensional feature representation f of the trajectory t ;

[0088] S33), the high-dimensional feature representation after embedding f tPerform position encoding to explicitly incorporate the sequential information of time steps into the feature representation to ensure that the subsequent self-attention mechanism can capture the temporal order;

[0089] S34), constructing a bidirectional dynamic self-attention mechanism to capture the dependency relationship at different time steps in the three-dimensional trajectory data of the embedded features through the bidirectional dynamic self-attention mechanism;

[0090] S35) Constructing a spatiotemporal graph convolutional network ST-GCN to capture the spatial correlation of trajectory data; treating each time step of the three-dimensional trajectory as a node in the spatiotemporal graph and the dependencies between time steps as edges in the spatiotemporal graph, and constructing a neighborhood for each time step node; and aggregating features from neighboring time steps through the spatiotemporal graph convolutional network ST-GCN:

[0091]

[0092] Where, f t (l+1) represents the node features on the time dimension t in the l+1th hidden layer; σ represents the activation function; d t is the degree of time step t; d j is the degree of time step j; W (l) is the weight matrix of the lth hidden layer; f j (l) is the feature vector of node j in the lth hidden layer; is the set of neighboring time steps of time step t;

[0093] The spatiotemporal graph convolutional network ST-GCN updates the node representation of each time step layer by layer through multi-layer graph convolution operations, captures its global spatiotemporal dependencies, and obtains the spatiotemporal features of each time step;

[0094] S36) Fusing the features output by the bidirectional dynamic self-attention mechanism and the spatiotemporal graph convolutional network ST-GCN;

[0095] S37) Input the fused spatiotemporal features into the classifier with regression task and use the Softmax activation function to predict the disturbance type and size, and output the final disturbance prediction classification result and the corresponding perturbation size Right now:

[0096]

[0097] Where softmax is a nonlinear activation function; W fc is the weight of the fully connected layer; is the fused spatiotemporal features after fusion;

[0098] Perturbation size Including the direction and magnitude of the disturbance; the predicted disturbance classification results Including wind turbine blade rotation, sea breeze interference, and nozzle impact interference.

[0099] Step 4: Based on the disturbance detection results, adjust the drone flight control strategy to maintain the stability of the drone when cleaning the fan blade stains.

[0100] As a preferred embodiment of this invention, in step S25), the three-dimensional coordinate position of the stain is corrected, which specifically includes the following steps:

[0101] S251), select multiple images with small disturbance as reference positions; and record the position of the ideal stain i and its position relative to the ideal stain j distance Among them, the distance The calculation formula is:

[0102]

[0103] S252) Construct a reference relative coordinate matrix R to record the relative position relationship of all ideal stains:

[0104]

[0105] S253) During repeated collection and positioning, record the stain location and its distance relative to the position of stain j Calculate the relative position matrix between all detected stains And compared with the reference relative coordinate matrix R, the deviation matrix ΔR is obtained;

[0106]

[0107] S254) Correct the stain coordinates using a rigid transformation to ensure that the identified stains conform to the positional relationship in the reference relative coordinate matrix. The rigid transformation is performed using the following formula:

[0108]

[0109] Where x′ i , y′ i 、z′ i is the position coordinate of the i-th stain after rigid transformation; x i 、y i 、z i is the position coordinate of the i-th stain before transformation; t is the translation vector, which is optimized by minimizing the coordinate deviation matrix ΔR;

[0110] S255), dynamically adjust the rotation matrix A and the translation vector t according to the deviation matrix ΔR, and gradually correct the positions of all stains until the relative coordinate error reaches a preset threshold; wherein the total error E is defined as the sum of the squares of the relative position deviations of all stains, that is:

[0111]

[0112] If the total error E is lower than the preset threshold, the correction is considered effective, and the corrected stain coordinates (x′1, y′1, z′1), (x′2, y′2, z′2), ... are output as the recorded position.

[0113] As a preferred embodiment of the present invention, in step S31), the trajectory of the fan blade stain P(t)=

[0114] [x(t), y(t), z(t)] is normalized, denoised, and interpolated; the details are as follows:

[0115] Normalize the coordinates of each stain point to the range [0,1], that is:

[0116]

[0117] in, represents the normalized coordinates of the fan blade stain; P(t) represents the real-time coordinates of the detected fan blade stain; P(t) max 、P(t) min They are the maximum and minimum values ​​of the coordinates of the stains on the fan blades.

[0118] Then the moving average method is used to smooth the trajectory, which can effectively remove high-frequency noise. The formula of the moving average method is:

[0119]

[0120] Where w is the size of the moving window, is the coordinate of the fan blade stain after smoothing at time t, and i represents the i-th time step.

[0121] If there are missing data points in the stain trajectory, use linear interpolation to fill them. Assuming that the stain point P(t) is missing, perform linear interpolation based on the positions of the two stain points before and after the stain point P(t), that is:

[0122]

[0123] Among them, P(t-1) and P(t+1) respectively represent the positions of the two stains before and after the stain P(t).

[0124] As a preferred embodiment of the present invention, in step S32), a high-dimensional feature representation f of the trajectory is generated by mapping it to a high-dimensional feature space through linear transformation. t , specifically:

[0125] The stain coordinate P(t) is regarded as a three-dimensional vector of length L, and the homology weight matrix W maps the trajectory to a high-dimensional vector, that is:

[0126] f t =P(t)W T +b

[0127] Where T represents the transpose operation; b represents the bias matrix.

[0128] As a preferred embodiment of this invention, in step S33), the embedded features are position-encoded, specifically:

[0129] Assume that the position encoding vector P t The dimension of is the same as the input feature dimension d after trajectory mapping, and then the sine and cosine functions are used to construct the position encoding; for each time step t:

[0130]

[0131] Where i is the dimension index of the feature, t is the time step, and d is the dimension of the input feature;

[0132] Then the position code P t Added to the input features, we get the new input feature representation:

[0133] f t ′=ft t +P t .

[0134] As a preferred embodiment of this invention, in step S34), the bidirectional dynamic self-attention mechanism includes forward attention and backward attention; the forward attention and backward attention respectively calculate the dependency between the current time step and the past and future time steps to generate a forward attention weight matrix. The expression of the bidirectional dynamic self-attention mechanism is:

[0135]

[0136] Where Q = W Q f t ′, K=W K f′ k 、V=W V f′ k are query matrix, key matrix and value matrix respectively; W Q 、W K 、W Vis the learnable weight matrix, f′ k Represents the features of other time steps; softmax is a nonlinear activation function, which processes the inner product of Q and K to obtain the correlation score matrix between each feature. The inner product of the result and V can be used to obtain the attention map between the features; Represents the scaling factor. When the sequence length is long, scaling the inner product of Q and K is beneficial to smooth the output of softmax, thereby obtaining a more appropriate score matrix.

[0137] As a preferred embodiment of this invention, in step S35), if Figure 5 As shown, the spatiotemporal graph convolutional network ST-GCN includes a graph convolutional network GCN and a CNN network; in this embodiment, a learnable weight matrix is ​​introduced and bitwise multiplied with the adjacency matrix to give important edges and nodes in the adjacency matrix a larger weight and suppress the weights of unimportant edges and nodes; the weighted adjacency matrix A and the input f′ are sent to the graph convolutional network GCN for calculation to realize the aggregation of spatial dimension information; then the CNN network is used to realize the aggregation of temporal dimension information, and the residual structure is introduced to calculate and obtain the residual.

[0138] As a preferred embodiment of this invention, in step S35), the graph convolutional network GCN includes an input layer, an output layer, and multiple hidden layers; the input layer receives the adjacency matrix A and the node feature matrix of the graph, normalizes the adjacency matrix A, and outputs the initial node feature representation of the graph;

[0139] Each hidden layer propagates node features according to the adjacency matrix and updates the node representation through the weight matrix and nonlinear activation function; multiple hidden layers can capture more complex local structure and feature information on the graph.

[0140] The output layer is used to generate the final node representation and perform the overall representation of the graph. The output layer generates a probability distribution for classification through an activation function.

[0141] As a preference, in step S36), the feature fusion adopts a weighted addition operation to combine the two parts of features, learn the contribution of different feature sources through adaptive weights, and dynamically adjust the importance of different time steps to obtain a more expressive feature fusion representation. The fused spatiotemporal features Expressed as:

[0142]

[0143] Where α1 and α2 represent the weights of features X1 and X2, respectively. X1 and X2 represent the output features of the bidirectional dynamic self-attention mechanism and the output features of ST-GCN, respectively.

[0144] Preferably, in step S4), the roll angle and pitch angle of the UAV are adjusted in real time according to the direction of the sea breeze interference in the disturbance detection result to offset the influence of the wind on the UAV attitude; and the roll angle is increased to counteract the lateral force in crosswind, and the pitch angle is increased to reduce wind resistance in headwind;

[0145] And according to the wind speed in the disturbance detection results, the thrust output of the quadcopter is adjusted in real time.

[0146] The above embodiments and descriptions are only for explaining the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, which shall fall within the scope of the invention to be protected.

Claims

1. A stain following and drone disturbance detection method for offshore wind turbine blades, characterized in that: The steps include: Step 1: Use the drone’s onboard camera to capture images of wind turbine blades; Step 2: Preprocess the collected wind turbine blade image to obtain the three-dimensional coordinate trajectory P(t) = [x(t), y(t), z(t)] of the wind turbine blade stain in the world coordinate system; where [x(t), y(t), z(t)] represents the three-dimensional coordinate sequence that changes with time t; Step 3: Perform time series processing on the stain trajectory P(t) = [x(t), y(t), z(t)] of the wind turbine blade to extract the temporal and spatial characteristics of the stain trajectory, decouple the interference source of the UAV offshore wind turbine blade cleaning, and obtain the disturbance detection results; Step 4: Based on the disturbance detection results, adjust the drone flight control strategy to maintain the stability of the drone when cleaning the fan blade stains.

2. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 1, characterized in that: In step S1), the collected wind turbine blade image includes a normal color image and a depth image; the normal color image and the depth image are used to identify the plane position (x, y) and depth position z of the stain respectively.

3. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 1, characterized in that: Obtaining the three-dimensional coordinate trajectory P(t) = [x(t), y(t), z(t)] of the fan blade stain specifically includes the following steps: S21), dividing each collected ordinary color image into k blocks of S×S grids; wherein each grid is used to detect the target falling within the grid in the image; S22), for each grid, using a convolutional neural network to extract feature information of the grid to divide each image into k state feature quantities; S23) According to the state transfer equation, the state feature quantity is converted into an intermediate state feature quantity, which is used to capture the spatial features in the image, namely: h i+1 =Dh i +Bx i ; Where h i represents the intermediate state characteristic of the i-th grid; h i+1 is the intermediate state feature quantity of the i+1th grid; D is the state transfer matrix; B is the transformation matrix of the state feature quantity; x i Represents the state characteristic of the i-th grid; S24), based on the state estimation, an observation value y of the image is obtained, and the matching degree between the current state and the target category is calculated according to the observation value y; wherein the calculation formula of the observation value y is: y=Ch; Where C is the transformation matrix of the intermediate state feature h; the observation value y represents the matching degree of each intermediate state feature h to the recognition target, which is used to determine the grid position of the target to be recognized in the image. The position of the recognized target in the image is determined by a multi-layer perceptron (MLP). The input layer of the multi-layer perceptron (MLP) is a vector composed of the image observation value y and the image position of the state feature, and the output is the recognition bounding box. S25), aligning the depth image and the color image, obtaining the depth information of the area corresponding to the recognition bounding box to determine the three-dimensional coordinate position of the stain, and correcting the three-dimensional coordinate position of the stain; S26) Repeat steps S21)-S25) to obtain the trajectory of the fan blade stain.

4. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 3 is characterized in that: In step S25), the three-dimensional coordinate position of the stain is corrected, which specifically includes the following steps: S251), select multiple images with small disturbance as reference positions; and record the position of the ideal stain i and its position relative to the ideal stain j distance Among them, the distance The calculation formula is: S252) Construct a reference relative coordinate matrix R, and record the relative position relationship of all ideal stains through the reference relative coordinate matrix R: S253) Record the actual stain position during repeated collection and positioning and its distance relative to the actual stain j position Calculate the relative position matrix between all actual stains detected And make a difference with the reference relative coordinate matrix R to obtain the deviation matrix ΔR; S254) Correct the stain coordinates using a rigid transformation to ensure that the identified stains conform to the positional relationship in the reference relative coordinate matrix. The rigid transformation is performed using the following formula: Where x i ′ 、y i ′ 、z i ′ is the position coordinate of the i-th stain after rigid transformation; x i 、y i 、z i is the position coordinate of the i-th stain before transformation; D is the state transfer matrix; t is the translation vector, which is optimized by minimizing the coordinate deviation matrix ΔR; S255), dynamically adjust the rotation matrix A and the translation vector t according to the deviation matrix ΔR, and gradually correct the positions of all stains until the relative coordinate error reaches a preset threshold; wherein the total error E is expressed as the sum of the squares of the relative position deviations of all stains, that is: If the total error E is lower than the preset threshold, the correction is considered effective and the corrected stain coordinates (x1 ′ ,y1 ′ ,z1 ′ ),(x2 ′ ,y2 ′ ,z2 ′ ),…as the record location.

5. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 1, characterized in that: Step S3) specifically includes the following steps: S31), normalizing, removing noise and interpolating the stain trajectory P(t)=[x(t), y(t), z(t)]; S32) Map the three-dimensional coordinates [x(t), y(t), z(t)] of each time step to the high-dimensional feature space through linear transformation to generate the high-dimensional feature representation f of the trajectory t ; S33), the high-dimensional feature representation after embedding f t Perform position encoding to explicitly incorporate the sequential information of time steps into the feature representation to ensure that the subsequent self-attention mechanism can capture the temporal order; S34), constructing a bidirectional dynamic self-attention mechanism to capture the dependency relationship at different time steps in the three-dimensional trajectory data of the embedded features through the bidirectional dynamic self-attention mechanism; S35), constructing a spatiotemporal graph convolutional network ST-GCN, and capturing the spatial correlation of trajectory data through the spatiotemporal graph convolutional network ST-GCN; By treating each time step of the 3D trajectory as a node in the space-time graph and the dependencies between time steps as edges in the space-time graph, a neighborhood is constructed for each time step node; and the features from the neighborhood time steps are aggregated through the space-time graph convolutional network ST-GCN, namely: Where, represents the node features on the time dimension t in the l+1th hidden layer; σ represents the activation function; d t is the degree of time step t; d j is the degree of time step j; W (l) is the weight matrix of the lth hidden layer; is the feature vector of node j in the lth hidden layer; is the set of neighboring time steps of time step t; S36), fusing the temporal and spatial features output by the bidirectional dynamic self-attention mechanism and the spatiotemporal graph convolutional network ST-GCN; S37) Input the fused spatiotemporal features into the classifier with regression task and use the Softmax activation function to predict the disturbance type and size, and output the final disturbance prediction classification result and the corresponding perturbation size Right now: Where softmax is a nonlinear activation function; W fc is the weight of the fully connected layer; is the fused spatiotemporal features after fusion.

6. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 5, characterized in that: In step S32), the high-dimensional feature representation f of the trajectory is generated by mapping it to the high-dimensional feature space through linear transformation t , specifically: The stain coordinate P(t) is regarded as a three-dimensional vector of length L, and the homology weight matrix W maps the trajectory to a high-dimensional vector, that is: f t =P(t)W T +b Where T represents the transpose operation; b represents the bias matrix.

7. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 5, characterized in that: In step S33), the embedded features are position-encoded, specifically: Assume that the position encoding vector P t The dimension of is the same as the input feature dimension d after trajectory mapping, and then the sine and cosine functions are used to construct the position encoding; for each time step t: Where i is the dimension index of the feature, t is the time step, and d is the dimension of the input feature; Then the position code P t Added to the input features, we get the new input feature representation: f t ′=f t +P t 。 8. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 5, characterized in that: The spatiotemporal graph convolutional network ST-GCN includes a graph convolutional network GCN and a CNN network. By introducing a learnable weight matrix and multiplying it bit by bit with the adjacency matrix, it is used to give important edges and nodes in the adjacency matrix a larger weight and suppress the weight of non-important edges and nodes. The weighted adjacency matrix A is combined with the input f ′ It is sent to the graph convolutional network GCN for calculation to realize the aggregation of spatial dimension information; then the CNN network is used to realize the aggregation of temporal dimension information, and the residual structure is introduced to calculate and obtain the residual.

9. The method for spot tracking and drone disturbance detection for offshore wind turbine blades according to claim 5, characterized in that: In step S36), the feature fusion adopts weighted addition operation to combine the two parts of features, learn the contribution of different feature sources through adaptive weights, and dynamically adjust the importance of different time steps to obtain a more expressive feature fusion representation. The fused spatiotemporal features Expressed as: Where α1 and α2 represent the weights of features X1 and X2, respectively. X1 and X2 represent the output features of the bidirectional dynamic self-attention mechanism and the output features of ST-GCN, respectively.

10. The method for stain tracking and drone disturbance detection for offshore wind turbine blades according to claim 1, characterized in that: In step S4), the roll angle and pitch angle of the UAV are adjusted in real time according to the direction of the sea breeze disturbance in the disturbance detection result; and the roll angle is increased to counteract the side force in crosswind, and the pitch angle is increased to reduce wind resistance in headwind; According to the wind speed in the disturbance detection results, the thrust output of the quadcopter is adjusted in real time.

Images