A method and related equipment for collaborative inspection of wind turbines

Abstract

This application discloses a wind turbine collaborative inspection method and related equipment. By dividing the inspection area into fan-shaped zones that rotate synchronously with the wind turbine's yaw angle, and combining comprehensive fitness to allocate optimal camera base stations to each zone, it ensures that the wind turbine is covered omnidirectionally and without blind spots under different yaw states, eliminating blind spots. Utilizing real-time data acquisition from multi-source sensors and an automated base station allocation mechanism, intelligent and automated inspection is achieved, eliminating the need for on-site human intervention and significantly reducing labor and time costs. Simultaneously, the allocation and reassignment mechanism based on comprehensive fitness and a priority decision tree ensures the real-time and continuous nature of the inspection, avoids subjective human error, significantly improves inspection efficiency and accuracy, and effectively compensates for the shortcomings of existing technologies.

Description

Technical Field

[0001] This application relates to the field of wind turbine operation and maintenance technology, and more specifically, to a wind turbine collaborative inspection method and related equipment. Background Technology

[0002] In the wind power industry, the stable operation of wind turbines is directly related to power generation efficiency and equipment safety. Therefore, efficient and comprehensive inspection of wind turbines is crucial. Currently, wind turbine inspections mainly rely on two methods: one is monitoring with a single or a small number of fixed cameras, and the other is manual inspection.

[0003] However, existing technologies have significant drawbacks. For fixed camera monitoring, the coverage is limited. Due to the fixed viewing angle and limited monitoring range, it is difficult to achieve comprehensive monitoring of wind turbines. Different parts of the wind turbine, such as the blades and the top of the tower, often cannot be simultaneously within the field of view of the same camera, resulting in some critical areas becoming monitoring blind spots. For example, minute cracks generated during blade rotation may be outside the camera's field of view and cannot be detected in time, potentially leading to serious equipment failures such as blade breakage.

[0004] Manual inspections are costly and inefficient. They require personnel to be on-site to inspect each part of the wind turbine, consuming significant manpower, resources, and time. Furthermore, the frequency of inspections is limited, making real-time, continuous monitoring difficult. Additionally, the results of manual inspections are susceptible to subjective factors; different inspectors may have varying standards and levels of detail, potentially leading to missed inspections or misjudgments, thus compromising the accuracy and reliability of the inspections.

[0005] These shortcomings make it difficult for existing technologies to meet the wind power industry's requirements for efficient and comprehensive wind turbine inspection, and there is an urgent need for a better wind turbine inspection solution. Summary of the Invention

[0006] This application provides a wind turbine collaborative inspection method and related equipment. By dividing a fan-shaped area that rotates synchronously with the yaw angle and allocating optimal camera base stations, combined with an automation mechanism and decision tree reassignment, it can eliminate blind spots, reduce costs, and improve efficiency and accuracy, effectively solving the shortcomings of existing technologies.

[0007] A method for collaborative inspection of wind turbines includes:

[0008] The wind turbine's pose feature data is collected in real time by multiple source sensors, and the yaw angle of the wind turbine is generated by fusion calculation.

[0009] Based on the yaw angle of the wind turbine, the polar coordinate space with the wind turbine as the center origin is divided into several sector-shaped inspection areas, and each sector-shaped inspection area rotates synchronously with the yaw angle of the wind turbine.

[0010] For each of the aforementioned fan-shaped inspection areas, the comprehensive suitability of candidate camera base stations within the area is calculated by combining satellite positioning data of the wind turbines and camera base stations. The comprehensive suitability is a fusion of horizontal viewing angle matching degree and vertical coverage degree.

[0011] Based on the comprehensive fitness score, the optimal camera base station is assigned to each of the aforementioned sector-shaped inspection areas to form an inspection camera base station group;

[0012] When there is task contention among the camera base stations in the inspection camera base station group, task reallocation is performed based on a multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation result.

[0013] The inspection camera base station group is used to perform collaborative filming on the wind turbine to achieve wind turbine inspection.

[0014] Optionally, the pose feature data includes at least the blade projection features acquired by a visual sensor, the spatial point cloud acquired by a lidar, and the attitude information output by the wind turbine nacelle gyroscope.

[0015] The fusion and calculation of the pose feature data to generate the wind turbine yaw angle includes:

[0016] Based on the blade projection characteristics, the spatial normal vector of the blade is calculated through projective geometric transformation, and the spatial tilt angle is obtained.

[0017] Based on the spatial point cloud, principal component analysis is used to fit the blade plane equation to generate the first yaw angle.

[0018] Based on the attitude information, the cabin attitude angles are mapped to the global coordinate system using a coordinate system transformation matrix to generate a second yaw angle.

[0019] Based on the confidence levels of each data source, the wind turbine yaw angle is generated by fusing the spatial tilt angle, the first yaw angle, and the second yaw angle through an adaptive weighted fusion model.

[0020] Optionally, the calculation process for the overall fitness of candidate camera base stations within the area includes:

[0021] Calculate the horizontal viewing angle deviation between the horizontal direction of the optical axis of the camera base station and the azimuth angle of the center of the fan-shaped inspection area, and determine the horizontal viewing angle matching degree of the candidate camera base station based on the horizontal viewing angle deviation;

[0022] Calculate the overlap ratio between the vertical field of view interval corresponding to the pitch angle of the camera base station and the height distribution interval of the wind turbine blades in the fan-shaped inspection area, and determine the vertical coverage of the candidate camera base station based on the overlap ratio;

[0023] The horizontal viewing angle matching degree and the vertical coverage degree are superimposed according to a preset weight to calculate the comprehensive suitability of the candidate camera base station.

[0024] Optionally, the formula for calculating the horizontal viewpoint matching degree is:

[0025]

[0026] The formula for calculating the vertical coverage is:

[0027]

[0028] in, For horizontal viewpoint matching degree, The effective adaptation coefficient for the horizontal field of view of the camera base station. The exponential decay factor for the angle deviation, This is due to horizontal viewing angle deviation. For vertical coverage, The standard deviation of the wind turbine blade height. For the overlap ratio integral, The vertical field of view corresponding to the elevation angle of the camera base station. The vertical coverage indication function for the base station indicates the height distribution range of the wind turbine blades within the fan-shaped inspection area.

[0029] Optionally, based on the comprehensive fitness score, an optimal camera base station is allocated to each of the sector-shaped inspection areas to form an inspection camera base station group, including:

[0030] Candidate camera base stations within each sector inspection area are sorted in descending order of comprehensive suitability, and the camera base station with the highest comprehensive suitability is selected as the initial allocation target.

[0031] During the allocation process, it is simultaneously verified whether the initial allocation object meets the single camera maximum load constraint, key sector coverage constraint, and energy consumption balance constraint.

[0032] If the initial allocation object does not meet any constraint, iterative adjustment is performed based on the comprehensive fitness ranking until all the sector inspection areas are covered and all constraints are met, thus forming an inspection camera base station group.

[0033] Optionally, the maximum load constraint for a single camera is:

[0034]

[0035] The key sector coverage constraint is:

[0036]

[0037] The energy consumption balance constraint is:

[0038]

[0039] in, To assign the inspection task of the i-th sector inspection area to the j-th camera base station, a binary decision variable is used, where N is the total number of sector inspection areas. The maximum task capacity of the j-th camera base station is... To make the number The high-risk sector-shaped inspection area task is assigned to the j-th base station using a binary decision variable. This is a collection of high-risk sector-shaped inspection areas. Let j be the remaining battery power of the j-th camera base station. Where M is the energy consumption deviation threshold, M is the total number of camera base stations, and K is the traversal camera base station coefficient.

[0040] Optionally, task reallocation is performed based on a multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation result, including:

[0041] Construct a multi-dimensional priority decision tree, where the priority dimensions of the decision tree are listed in descending order:

[0042] The first dimension is the task importance, which is calculated by multiplying the dynamic weight of the sector inspection area with the overall suitability of the camera base station. The dynamic weight is positively correlated with the historical defect detection rate of the area.

[0043] The second dimension is the urgency of the task, which is determined by the anomaly confidence level of the real-time data collected by the multi-source sensors.

[0044] The third dimension is spatial adaptability, which is quantified by the straight-line distance between the camera base station and the wind turbine. The spatial adaptability is positively correlated with the straight-line distance, which is calculated in real time based on the satellite positioning data.

[0045] The competitive tasks are prioritized based on the multi-dimensional priority decision tree, retaining high-priority tasks and including low-priority tasks in the allocation pool.

[0046] For tasks in the allocation pool, after excluding candidate camera base stations that have been occupied by other tasks, the overall fitness of the remaining camera base stations is recalculated and sorted in descending order. The camera base station with the highest overall fitness is selected for secondary allocation.

[0047] If there are no available camera base stations in the current area, the search is extended to adjacent areas, and the overall fitness of the extended camera base stations decreases with distance.

[0048] The inspection camera base station group is updated based on the secondary allocation results. If no usable camera base station is found after extended search, the task fragmentation mechanism is triggered.

[0049] Optionally, the task sharding mechanism execution process includes:

[0050] The task of segmented inspection is split into spatiotemporal dimensions to generate multiple segmented sub-tasks, including time-segmented sub-tasks executed by the same camera base station and spatial sub-tasks allocated to multiple camera base stations for collaborative coverage.

[0051] Calculate the overall suitability of each of the aforementioned sub-tasks to ensure that the sum of the coverage of each of the aforementioned sub-tasks is not less than the task to be inspected in the sub-task.

[0052] Optionally, if it is detected that there is no candidate camera base station in the fan-shaped inspection area, a virtual collaboration mechanism is executed for the empty inspection area without candidate camera base stations;

[0053] The virtual collaboration mechanism includes:

[0054] A prediction model is built based on historical yaw data of wind turbines, and the trend and confidence interval of the wind turbine yaw angle change within a preset time period are output.

[0055] Dispatch nearby mobile camera base stations and plan the optimal mobile path based on the yaw prediction results. The path planning must meet the constraints of minimizing energy consumption and time of arrival.

[0056] Before the physical base station is in place, the three-dimensional environmental map of the wind turbine constructed based on the visual sensor is combined with the existing camera base station data to generate virtual inspection data for the vacant inspection area.

[0057] Optionally, if a communication interruption camera base station is detected in the inspection camera base station group, a marking and replacement mechanism is activated;

[0058] The marker padding mechanism includes:

[0059] Mark the current task of the communication interruption camera base station;

[0060] If the interruption exceeds a preset time, the camera base station with the second highest overall suitability within the fan-shaped inspection area to which the communication interruption camera base station belongs will take over.

[0061] Optionally, the process of obtaining the blade projection features from the image data acquired by the vision sensor includes:

[0062] The image data acquired by the vision sensor is dehazed, denoised, and distortion corrected to extract the blade outline;

[0063] Identify the main axis of the blade and calculate the angle between the main axis of the blade and the axis of the horizontal axis of the image coordinate system. The angle corresponds to the angular features of the sector region.

[0064] Stable feature points are extracted from the surface texture features of the blade, and the two-dimensional coordinates and feature vectors of the stable feature points in the image coordinate system are recorded to form a stable feature point set.

[0065] The extracted blade contour, principal axis angle, and stable feature point set are integrated into blade projection features.

[0066] Optionally, the process of obtaining spatial point clouds from the raw point cloud data collected by the lidar includes:

[0067] The collected point cloud raw data is processed by removing noise points, eliminating redundant points, and calibrating coordinates.

[0068] Point cloud data from multiple frames are stitched and fused using point cloud registration technology.

[0069] A point cloud segmentation algorithm is used to segment the fused point cloud data to obtain a spatial point cloud containing three-dimensional coordinate information.

[0070] Optionally, the processing of the attitude information output by the wind turbine nacelle gyroscope includes:

[0071] The raw attitude data is filtered to eliminate high-frequency noise. The raw attitude data includes angular velocity and angular acceleration.

[0072] The filtered angular velocity data is solved into Euler angles that include yaw, pitch, and roll angles, where the yaw angle is used for azimuth calibration of the sector inspection area;

[0073] The Euler angles are synchronized using the timestamps of the satellite positioning data;

[0074] Calculate the mean value of the Euler angles within a continuous sampling period, and use the mean value as the output attitude information.

[0075] A wind turbine collaborative inspection device, comprising:

[0076] The fusion calculation module is used to collect the wind turbine's pose feature data in real time through multiple source sensors, and to generate the wind turbine's yaw angle through fusion calculation;

[0077] The region division module is used to divide the polar coordinate space with the wind turbine as the center origin into several sector-shaped inspection areas based on the wind turbine yaw angle. Each sector-shaped inspection area rotates synchronously with the wind turbine yaw angle.

[0078] The fitness calculation module is used to calculate the comprehensive fitness of candidate camera base stations in each of the fan-shaped inspection areas, based on the satellite positioning data of the wind turbine and camera base station. The comprehensive fitness is a fusion of horizontal view matching degree and vertical coverage degree.

[0079] The base station allocation module is used to allocate the optimal camera base station to each of the sector inspection areas based on the comprehensive suitability, so as to form an inspection camera base station group;

[0080] The task allocation module is used to perform task reallocation based on a multi-dimensional priority decision tree when there is task competition among the camera base stations in the inspection camera base station group, and to update the inspection camera base station group based on the reallocation result.

[0081] The inspection execution module is used to perform collaborative filming of the wind turbine using the inspection camera base station group to achieve wind turbine inspection.

[0082] A wind turbine collaborative inspection device includes a memory and a processor;

[0083] The memory is used to store programs;

[0084] The processor is used to execute the program to implement the various steps of the wind turbine collaborative inspection method as described in any of the above claims.

[0085] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the wind turbine collaborative inspection method as described in any of the preceding claims.

[0086] A computer program product includes a computer program that, when executed by a processor, performs the steps of the wind turbine collaborative inspection method as described in any of the preceding claims.

[0087] As can be seen from the above technical solutions, the wind turbine collaborative inspection method and related equipment provided in this application collect wind turbine pose feature data from multi-source sensors to generate a yaw angle. Based on the yaw angle, a fan-shaped inspection area that rotates synchronously with the wind turbine is divided. The fusion horizontal view matching degree and vertical coverage comprehensive suitability of candidate camera base stations in each area are calculated. The optimal camera base station is assigned to form an inspection group. If there is task competition, the task is reassigned through a multi-dimensional priority decision tree. Finally, the inspection group is used to perform collaborative shooting to achieve inspection.

[0088] This application specifically addresses the shortcomings of existing technologies, achieving significant beneficial effects. On one hand, by dividing the inspection area into fan-shaped zones that rotate synchronously with the wind turbine's yaw angle, and assigning optimal camera base stations to each zone based on comprehensive fitness, it ensures that each zone has a camera base station with the highest matching degree responsible for monitoring under different yaw states, eliminating blind spots and achieving comprehensive, all-around coverage of the wind turbine. On the other hand, the solution achieves intelligent and automated inspection processes through real-time data acquisition from multi-source sensors and an automated base station allocation mechanism, eliminating the need for on-site human intervention and significantly reducing labor and time costs. Simultaneously, the base station allocation and task reassignment mechanism based on comprehensive fitness and a priority decision tree ensures the real-time and continuous nature of the inspection, avoids subjective errors in manual inspection, significantly improves inspection efficiency and accuracy, and effectively compensates for the deficiencies of existing technologies. Attached Figure Description

[0089] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0090] Figure 1 This is a schematic diagram illustrating an application scenario of a wind turbine collaborative inspection method disclosed in an embodiment of this application;

[0091] Figure 2 This is a flowchart of a wind turbine collaborative inspection method disclosed in an embodiment of this application;

[0092] Figure 3 This is a schematic diagram of a wind turbine collaborative inspection device disclosed in an embodiment of this application;

[0093] Figure 4 This is a hardware structure block diagram of a wind turbine collaborative inspection device disclosed in an embodiment of this application. Detailed Implementation

[0094] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0095] This application can be used in a wide variety of general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0096] The following section introduces the solution proposed in this application. The technical solution is as follows, and details are provided below.

[0097] Figure 1 This diagram illustrates the application scenario of this application. The solution presented here is primarily applied to the daily operation and maintenance of large-scale wind farms, focusing on the dynamic inspection needs arising from wind turbine yaw caused by wind direction variations. In vast wind farms located in mountainous and plain areas, distributed camera base stations need to monitor multiple critical components of wind turbines, such as blades, towers, and nacelles. Due to the continuous yaw of the turbines, traditional fixed inspection methods are prone to coverage blind spots or task conflicts. In this case, this solution uses a collaborative inspection strategy and target allocation algorithm to dynamically address how to select suitable cameras for the inspected turbine at a given moment and how to rationally allocate camera task competition, adapting to the dynamic changes in turbine yaw and ensuring continuous and effective coverage of critical components.

[0098] In terms of effectiveness, the collaborative inspection strategy accurately matches the yaw attitude of wind turbines and dynamically adjusts the tasks of camera base stations, eliminating monitoring blind spots caused by yaw. The target allocation algorithm introduces multi-dimensional priority decision-making and constraint verification. Under conditions such as base station load, key area coverage, and energy consumption balance, it prioritizes the execution of high-value tasks (such as historical defect areas and real-time anomaly areas), scientifically resolves task competition conflicts, and improves inspection efficiency and rationality. At the same time, combined with mechanisms such as virtual collaboration and marker filling, it can quickly respond when base stations are empty or communication is interrupted, maintain the continuity of inspection, and significantly reduce the risk of missed wind turbine faults.

[0099] Figure 2 This is a flowchart of a wind turbine collaborative inspection method disclosed in an embodiment of this application.

[0100] like Figure 2 As shown, the method may include:

[0101] Step S1: Collect the wind turbine's pose feature data in real time through multi-source sensors, and calculate and generate the wind turbine's yaw angle.

[0102] Specifically, the multi-source sensors include gyroscopes, accelerometers, tilt sensors, and satellite positioning modules installed on key components such as the wind turbine tower, nacelle, and blades. Real-time data acquisition covers the wind turbine's real-time position coordinates, nacelle rotation angle, blade oscillation amplitude, and three-dimensional attitude parameters. The acquired data is fused using either a Kalman filter or a federated filter algorithm to eliminate errors and redundancy from different sensor data. The final calculated yaw angle accurately reflects the real-time steering state of the wind turbine nacelle, providing a reference parameter for subsequent area delineation.

[0103] Step S2: Based on the yaw angle of the wind turbine, the polar coordinate space with the wind turbine as the center origin is divided into several sector-shaped inspection areas, and each sector-shaped inspection area rotates synchronously with the yaw angle of the wind turbine.

[0104] Specifically, a three-dimensional polar coordinate space is established with the rotation center of the wind turbine nacelle as the origin and the central axis of the wind turbine tower as the polar axis. Based on the number and length of the wind turbine blades, the polar coordinate space is divided into basic sector-shaped areas matching the number of blades, with equal angle differences (e.g., 120° intervals for 3 blades). Auxiliary sector-shaped areas covering the top and bottom of the tower are added above and below the basic areas to ensure all critical components are within the inspection range. The angle range of each sector-shaped area is set according to the size of the corresponding component; for example, the angle of the sector-shaped area corresponding to a blade is slightly larger than the maximum sweep angle when the blade rotates. By acquiring the wind turbine yaw angle in real time, the starting angle parameters of each sector-shaped area are dynamically adjusted to ensure that all sector-shaped inspection areas remain synchronized with the rotation of the wind turbine nacelle, guaranteeing consistency between the area division and the actual attitude of the wind turbine.

[0105] Step S3: For each of the aforementioned fan-shaped inspection areas, the comprehensive suitability of candidate camera base stations within the area is calculated by combining the satellite positioning data of the wind turbine and camera base station. The comprehensive suitability is a fusion of horizontal viewing angle matching degree and vertical coverage degree.

[0106] Specifically, the real-time coordinates of the wind turbines and the fixed coordinates of each camera base station are obtained, and the relative positional relationship between the two is calculated. The horizontal viewing angle matching degree is calculated by the deviation of the angle between the horizontal rotation angle of the candidate camera base station lens and the center line of the fan-shaped area; the smaller the deviation, the higher the matching degree. The vertical coverage degree is calculated by the overlap ratio between the vertical viewing angle range of the candidate camera base station lens and the height range of the wind turbine components within the fan-shaped area; the higher the overlap ratio, the higher the coverage. A weighted summation algorithm is used to fuse the two into a comprehensive fitness score; the higher the score, the more suitable the candidate base station is for the area.

[0107] Step S4: Based on the comprehensive fitness score, allocate the optimal camera base station to each of the sector inspection areas to form an inspection camera base station group.

[0108] Specifically, for each sector-shaped inspection area, the base station with the highest overall suitability is selected from all candidate camera base stations within its coverage area as the preferred camera base station for that area. If multiple candidate base stations have the same overall suitability, the equipment performance parameters of each base station are further compared, and the base station with higher resolution, faster frame rate, and lower historical failure rate is selected first. The preferred base stations assigned to all sector areas are aggregated to form an initial inspection camera base station group, and corresponding shooting task parameters are assigned to each base station in the group, including shooting time period, image resolution, shooting frequency, and key areas of focus, to ensure that each base station can accurately cover the sector area it is responsible for.

[0109] Step S5: When there is task contention among the camera base stations in the inspection camera base station group, the task is reallocated based on the multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation result.

[0110] Specifically, task competition refers to a conflict where a single camera base station is simultaneously selected by two or more sector areas. The multi-dimensional priority decision tree contains three layers of decision nodes: the first layer is the importance level of the sector area (blade area > top of tower area > middle of tower area > bottom of tower area); the second layer is the urgency of the task (fault warning task > periodic inspection task > daily monitoring task); and the third layer is the historical matching success rate between the base station and the area (the higher the success rate, the higher the priority). The decision tree prioritizes each competing task, sorting them from highest to lowest score. High-priority tasks retain their original allocated base station resources, while low-priority tasks are reselected from the candidate base stations in that area based on their overall suitability. Based on the task redistribution results, conflicting base stations in the original inspection camera base station group are replaced, generating an updated inspection camera base station group, and the task parameters of each base station are updated synchronously.

[0111] Step S6: Use the inspection camera base station group to perform collaborative shooting of the wind turbine to realize wind turbine inspection.

[0112] Specifically, each camera base station in the inspection camera base station group synchronously activates its high-definition camera device during a preset time period according to the assigned task parameters. This continuously captures high-definition images or video data of the wind turbine components within its assigned fan-shaped area. During the recording process, each base station transmits the collected data to the central control system in real time via wireless communication modules. The control system uses a deep learning-based image recognition algorithm to process and analyze the received data, automatically identifying abnormal conditions such as blade surface cracks, tower corrosion, and loose bolts. When an abnormality is detected, the system automatically marks the location, type, and severity of the abnormality, generates an inspection report, and triggers an early warning mechanism. Simultaneously, the system compares and analyzes the current inspection data with historical data to assess the aging trend of wind turbine components, achieving comprehensive and intelligent inspection of the wind turbine.

[0113] As can be seen from the above technical solutions, the wind turbine collaborative inspection method and related equipment provided in this application collect wind turbine pose feature data from multi-source sensors to generate a yaw angle. Based on the yaw angle, a fan-shaped inspection area that rotates synchronously with the wind turbine is divided. The fusion horizontal view matching degree and vertical coverage comprehensive suitability of candidate camera base stations in each area are calculated. The optimal camera base station is assigned to form an inspection group. If there is task competition, the task is reassigned through a multi-dimensional priority decision tree. Finally, the inspection group is used to perform collaborative shooting to achieve inspection.

[0114] This application specifically addresses the shortcomings of existing technologies, achieving significant beneficial effects. On one hand, by dividing the inspection area into fan-shaped zones that rotate synchronously with the wind turbine's yaw angle, and assigning optimal camera base stations to each zone based on comprehensive fitness, it ensures that each zone has a camera base station with the highest matching degree responsible for monitoring under different yaw states, eliminating blind spots and achieving comprehensive, all-around coverage of the wind turbine. On the other hand, the solution achieves intelligent and automated inspection processes through real-time data acquisition from multi-source sensors and an automated base station allocation mechanism, eliminating the need for on-site human intervention and significantly reducing labor and time costs. Simultaneously, the base station allocation and task reassignment mechanism based on comprehensive fitness and a priority decision tree ensures the real-time and continuous nature of the inspection, avoids subjective errors in manual inspection, significantly improves inspection efficiency and accuracy, and effectively compensates for the deficiencies of existing technologies.

[0115] In some embodiments of this application, the pose feature data includes at least blade projection features acquired by a visual sensor, spatial point clouds acquired by a lidar, and attitude information output by the wind turbine nacelle gyroscope.

[0116] The process of fusing and calculating the pose feature data to generate the wind turbine yaw angle in step S1 is described, and may specifically include:

[0117] Step S11: Based on the blade projection characteristics, calculate the blade spatial normal vector through projective geometric transformation, and calculate the spatial tilt angle;

[0118] Step S12: Based on the spatial point cloud, use principal component analysis to fit the blade plane equation and generate the first yaw angle;

[0119] Step S13: Based on the attitude information, the cabin attitude angle is mapped to the global coordinate system through a coordinate system transformation matrix to generate the second yaw angle;

[0120] Step S14: Based on the confidence levels of each data source, the wind turbine yaw angle is generated by fusing the spatial tilt angle, the first yaw angle, and the second yaw angle through an adaptive weighted fusion model.

[0121] Specifically, the first step involves extracting blade edge contour points and feature corner points from blade images acquired by a vision sensor as projection features. Combining these with the camera's intrinsic matrix (focal length, principal point coordinates) and extrinsic matrix (relative position and attitude of the camera and wind turbine), an inverse perspective transformation is used to map the two-dimensional image points to three-dimensional space, resulting in a sparse three-dimensional point set on the blade surface. Based on this point set, the least squares method is used to fit the blade plane, solving for the plane equation and normal vector. The normal vector is then projected onto a plane perpendicular to the wind turbine's rotation axis, and the angle between the projected vector and the horizontal plane is calculated; this angle represents the spatial tilt angle reflecting the blade's spatial inclination.

[0122] The wind turbine point cloud data acquired by lidar is preprocessed by removing noise points through radius filtering and segmenting the blade point cloud into subsets using a region growing algorithm. Principal component analysis is performed on the blade point cloud to calculate the covariance matrix and solve for eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is the normal vector of the blade plane. Based on the normal vector, the azimuth angle of the blade plane in the global coordinate system is calculated. Combined with the coordinates of the wind turbine hub center, the plane azimuth angle is converted into a horizontal rotation angle with true north as the reference, which is used as the first yaw angle.

[0123] Obtain the attitude angles (including heading, pitch, and roll angles) in the body coordinate system from the gyroscope output of the wind turbine nacelle. Construct a transformation matrix from the body coordinate system to the global coordinate system. This matrix is ​​obtained by multiplying the pitch angle rotation matrix, roll angle rotation matrix, and heading angle rotation matrix sequentially. Map the heading angle output by the gyroscope to the global coordinate system using the transformation matrix to obtain the angle value reflecting the horizontal rotation state of the nacelle in the global coordinate system, which is the second yaw angle.

[0124] A data consistency test-based approach is used to evaluate the confidence level of each data source in real time: the deviation between the blade spatial tilt angle and the historical mean is calculated, with smaller deviations indicating higher confidence in the visual data; the sum of squared residuals from the point cloud plane fitting is statistically analyzed, with smaller residuals indicating higher confidence in the lidar data; and the drift rate of the gyroscope is monitored, with data confidence increasing when the drift rate is below a threshold. Weighting coefficients for each parameter are calculated based on the confidence level, with parameters having higher confidence levels assigned higher weights. A weighted summation is then used for fusion calculation to obtain the final wind turbine yaw angle, a result that combines the complementary advantages of multi-source data.

[0125] In some embodiments of this application, the calculation process for the comprehensive suitability of candidate camera base stations within the area described in step S3 is introduced, which may specifically include:

[0126] Step S31: Calculate the horizontal viewing angle deviation between the horizontal direction of the optical axis of the camera base station and the azimuth angle of the center of the fan-shaped inspection area, and determine the horizontal viewing angle matching degree of the candidate camera base station based on the horizontal viewing angle deviation.

[0127] Specifically, real-time location information of candidate camera base stations and wind turbines is obtained through satellite positioning. Based on this, the horizontal azimuth angle of the camera base station pointing towards the wind turbine is determined. Simultaneously, according to the wind turbine yaw angle and the division rules of the fan-shaped inspection area, the center azimuth angle of the current fan-shaped area is determined. This azimuth angle is dynamically updated according to the wind turbine yaw status. The absolute value of the difference between the two azimuth angles is calculated to obtain the horizontal viewing angle deviation. If the deviation exceeds 180 degrees, it is corrected by subtracting the deviation value from 360 degrees to ensure that the deviation is within the effective range of 0-180 degrees. The determination of the horizontal viewing angle matching degree needs to combine the deviation value with the horizontal field-of-view characteristics of the camera base station: when the deviation is small, the matching degree is high; as the deviation increases, the matching degree gradually decreases, thus quantitatively reflecting the degree of alignment of the camera base station with the fan-shaped area in the horizontal direction.

[0128] The formula for calculating the horizontal viewpoint matching degree is:

[0129]

[0130] Step S32: Calculate the overlap ratio between the vertical field of view interval corresponding to the pitch angle of the camera base station and the height distribution interval of the wind turbine blades in the fan-shaped inspection area, and determine the vertical coverage of the candidate camera base station based on the overlap ratio.

[0131] Specifically, the installation height of the camera base station, its horizontal distance from the wind turbine, and its current pitch angle are first obtained. Using geometric relationships, the upper and lower boundaries of the vertical field of view corresponding to this pitch angle are calculated, forming the vertical field of view interval. Simultaneously, equipment such as LiDAR is used to scan the wind turbine blades within the fan-shaped inspection area, extracting the blade height distribution range and determining the height distribution interval. The ratio of the overlapping length of the two intervals to the total length of the blade height distribution interval is calculated to obtain the overlap ratio. The determination of vertical coverage needs to consider both this overlap ratio and the dispersion of the blade height distribution. A higher overlap ratio and a more concentrated blade height distribution result in higher vertical coverage, thus quantitatively reflecting the coverage effect of the camera base station on the blades in the vertical direction.

[0132] The formula for calculating the vertical coverage is:

[0133]

[0134] in, For horizontal viewpoint matching degree, The effective adaptation coefficient for the horizontal field of view of the camera base station. The exponential decay factor for the angle deviation, This is due to horizontal viewing angle deviation. For vertical coverage, The standard deviation of the wind turbine blade height. For the overlap ratio integral, The vertical field of view corresponding to the elevation angle of the camera base station. The vertical coverage indication function for the base station indicates the height distribution range of the wind turbine blades within the fan-shaped inspection area.

[0135] Step S33: The horizontal viewing angle matching degree and the vertical coverage degree are superimposed according to a preset weight to calculate the comprehensive suitability of the candidate camera base station.

[0136] Specifically, based on the actual needs of the inspection scenario, the weights of horizontal viewing angle matching and vertical coverage are pre-set, with the sum of their weights being 1. For example, in scenarios emphasizing horizontal alignment accuracy, the weight of horizontal viewing angle matching can be increased; in scenarios emphasizing blade height coverage, the weight of vertical coverage can be increased. The overall suitability is obtained by multiplying the horizontal viewing angle matching and vertical coverage by their respective weights and then summing them. The higher this value, the better the overall suitability of the candidate camera base station in terms of both horizontal alignment and vertical coverage, providing an intuitive quantitative basis for subsequent base station allocation.

[0137] In some embodiments of this application, step S4, which involves allocating optimal camera base stations to each of the sector-shaped inspection areas based on the comprehensive fitness score to form an inspection camera base station group, is described. Specifically, it may include:

[0138] Step S41: Sort the candidate camera base stations in each sector inspection area in descending order of comprehensive suitability, and select the camera base station with the highest comprehensive suitability as the initial allocation object;

[0139] Step S42: During the allocation process, synchronously verify whether the initial allocation object meets the single camera maximum load constraint, key sector coverage constraint, and energy consumption balance constraint.

[0140] Step S43: If the initial allocation object does not meet any constraint condition, iterative adjustment is performed based on the comprehensive fitness ranking until all the sector inspection areas are covered and all constraints are met, forming an inspection camera base station group.

[0141] Specifically, for each sector-shaped inspection area, the overall suitability data of all candidate camera base stations within the coverage area is collected and sorted in descending order to form a candidate base station sequence. The camera base station ranked first in the sequence is determined as the initial allocation target for that area, ensuring that the base station with the best suitability is prioritized for inspection tasks. If there are no candidate base stations in an area or the overall suitability of all candidate base stations is below a preset threshold, it is marked as an area to be supplemented, and the issue will be resolved later by adjusting the coverage area of ​​surrounding base stations or adding temporary base stations.

[0142] When assigning initial objects to each sector area, the following constraints are checked in real time: Single camera maximum load constraint: The number of sector areas assigned to a single camera base station does not exceed the maximum inspection task volume supported by its hardware performance, avoiding shooting delays or data loss due to excessive load; Critical sector coverage constraint: For sector areas corresponding to critical areas such as wind turbine blade tips and tower connection parts, the assigned camera base stations must meet higher equipment accuracy requirements (such as resolution and frame rate) to ensure the monitoring quality of critical parts; Energy consumption balance constraint: By balancing the task volume, the energy consumption difference of each camera base station is controlled within a preset range, avoiding shortened equipment lifespan due to long-term high-load operation of some base stations.

[0143] When an initially assigned object violates constraints, an adjustment mechanism is activated: If the load constraint is violated, a base station with the second-highest overall suitability is selected from the already allocated areas of that base station as a replacement, and the excess area is reallocated; if the critical sector constraint is violated, a second-best base station that meets the accuracy requirements is selected for that critical sector, and the other allocated areas of the original base station are adjusted simultaneously; if the energy consumption balance constraint is violated, energy consumption balance is achieved by transferring some workload to low-load base stations. After each adjustment, all constraints are re-verified until all sector areas find assigned objects that meet the constraints. Finally, all selected camera base stations are aggregated to form a complete inspection camera base station group, and the responsible area and task parameters of each base station are recorded.

[0144] The maximum load constraint for a single camera is as follows:

[0145]

[0146] The key sector coverage constraint is:

[0147]

[0148] The energy consumption balance constraint is:

[0149]

[0150] in, To assign the inspection task of the i-th sector inspection area to the j-th camera base station, a binary decision variable is used, where N is the total number of sector inspection areas. The maximum task capacity of the j-th camera base station is... To make the number The high-risk sector-shaped inspection area task is assigned to the j-th base station using a binary decision variable. This is a collection of high-risk sector-shaped inspection areas. Let j be the remaining battery power of the j-th camera base station. Where M is the energy consumption deviation threshold, M is the total number of camera base stations, and K is the traversal camera base station coefficient.

[0151] In some embodiments of this application, the process of performing task reallocation based on a multi-dimensional priority decision tree in step S5 and updating the inspection camera base station group based on the reallocation result is described, which may specifically include:

[0152] Step S51: Construct a multi-dimensional priority decision tree. The priority dimensions of the decision tree are listed in descending order as follows:

[0153] The first dimension is task importance, which is calculated by multiplying the dynamic weight of the sector-shaped inspection area by the overall suitability of the camera base station. The dynamic weight is closely related to the historical defect detection rate of that sector-shaped area; the more times and the higher the frequency of defect detections in the past, the greater the dynamic weight value for that area. For example, if a sector-shaped area has detected blade cracks three times in the past six months, while another area has only detected them once, the former has a higher dynamic weight than the latter. Multiplying the dynamic weight by the overall suitability of the camera base station for that area yields the task importance. The higher this value, the more priority the inspection task for that area needs to be prioritized in the decision-making process.

[0154] The second dimension is task urgency, determined by the anomaly confidence level of real-time data collected by multi-source sensors. Multi-source sensors (such as vibration sensors, temperature sensors, and vision sensors) continuously monitor the wind turbine's operating status. When sensor data indicates a potential anomaly in a certain part of the wind turbine (such as vibration frequency exceeding a threshold, a sudden temperature rise, or suspected cracks appearing in the image), the system calculates the confidence level of that anomaly. The more obvious the anomaly characteristics and the higher the data consistency, the greater the anomaly confidence level, and the higher the corresponding inspection task urgency. For example, when both vibration and vision sensors simultaneously indicate a blade anomaly, the urgency level is higher than an anomaly alerted by a single sensor.

[0155] The third dimension is spatial adaptability, quantified by the straight-line distance between the camera base station and the wind turbine. Spatial adaptability is positively correlated with this straight-line distance. The straight-line distance is calculated in real-time based on satellite positioning data; that is, by obtaining the real-time coordinates of the camera base station and the fixed coordinates of the wind turbine, the distance between the two points is calculated using the formula for distance between the two points. The closer the distance, the less likely the camera base station is to be affected by environmental interference (such as atmospheric scattering or obstruction) when photographing the wind turbine, and the easier it is to ensure image quality, thus resulting in better spatial adaptability. In priority ranking, when task importance and urgency are equal, tasks with higher spatial adaptability are prioritized.

[0156] The three dimensions mentioned above form a hierarchical structure of the decision tree in the order of task importance, task urgency, and spatial adaptability. The priority judgment result of the upper level directly determines whether the lower level participates in the comparison, thus forming a complete multi-dimensional priority decision logic.

[0157] Step S52: Based on the multi-dimensional priority decision tree, prioritize the competing tasks, retain high-priority tasks, and include low-priority tasks in the allocation pool.

[0158] Specifically, when multiple tasks in a sector-shaped inspection area compete for the same camera base station, a pre-constructed multi-dimensional priority decision tree is invoked to compare the competing tasks sequentially according to task importance, task urgency, and spatial adaptability. First, the first dimension (task importance) is compared, with tasks having higher values ​​having higher priority. If the first dimension is the same, the second dimension (task urgency) is compared, with tasks having higher anomaly confidence having higher priority. If the first two dimensions are the same, the third dimension (spatial adaptability) is compared, with tasks closer in a straight line having higher priority. After determining the priority ranking through hierarchical comparison, a pre-set number of high-priority tasks at the top of the ranking are retained, allowing them to continue occupying the original camera base station resources, while the remaining low-priority tasks at the bottom of the ranking are removed from the current allocation queue and added to the waiting-to-be-allocated pool for reassignment.

[0159] Step S53: For the tasks in the allocation pool, after excluding candidate camera base stations that have been occupied by other tasks, recalculate the overall suitability of the remaining camera base stations and sort them in descending order. Select the camera base station with the highest overall suitability for secondary allocation.

[0160] Specifically, for each task in the allocation pool, all candidate camera base stations within its corresponding sector inspection area are first selected, excluding those already occupied by higher-priority tasks, retaining only currently available candidate base stations. For these remaining base stations, their overall fit (combining horizontal view matching and vertical coverage) with the sector area where the task is located is recalculated, and they are sorted from highest to lowest overall fit. The base station with the highest overall fit is selected as the secondary allocation target, and the task to be allocated is assigned to this base station, completing the initial secondary allocation process.

[0161] Step S54: If there are no available camera base stations in the current area, the search is extended to adjacent areas, and the overall fitness of the extended camera base stations is reduced by distance.

[0162] Specifically, when there are no available candidate camera base stations within the sector-shaped inspection area where the task to be assigned is located (i.e., the number of remaining base stations is 0), the area expansion mechanism is activated. Centered on the current area, the search range is gradually expanded to adjacent sector-shaped inspection areas, incorporating camera base stations within those adjacent areas as candidates. For these expanded base stations, their overall fitness score needs to be corrected for distance attenuation: the farther away from the current area, the greater the attenuation of the overall fitness score. For example, for every 10 meters increase in distance, the overall fitness score decreases by a preset ratio, thus balancing the adaptability and coverage effect of distant base stations.

[0163] Step S55: Update the inspection camera base station group according to the secondary allocation result. If there are still no available camera base stations after the extended search, the task fragmentation mechanism is triggered.

[0164] Specifically, the successfully reassigned camera base stations and their corresponding task information are integrated into the original inspection camera base station group, replacing conflicting task assignment relationships in the original group. This completes the dynamic update of the inspection group, ensuring that the matching relationship between base stations and tasks within the group meets current actual needs. If no assignable camera base station is found after the maximum extended search, the task fragmentation mechanism is automatically triggered: the task to be assigned is decomposed into multiple time fragments, and each fragment task is sequentially assigned to camera base stations in other idle time periods according to the time sequence, or temporarily added mobile inspection equipment is dispatched to complete the inspection in time periods to ensure that the task is ultimately executed.

[0165] Furthermore, the task sharding mechanism execution process includes:

[0166] The task of segmented inspection is split into spatiotemporal dimensions to generate multiple segmented sub-tasks, including time-segmented sub-tasks executed by the same camera base station and spatial sub-tasks allocated to multiple camera base stations for collaborative coverage.

[0167] Calculate the overall suitability of each of the aforementioned sub-tasks to ensure that the sum of the coverage of each of the aforementioned sub-tasks is not less than the task to be inspected in the sub-task.

[0168] Specifically, the first step is to analyze the spatiotemporal characteristics of the inspection tasks to be performed, determining the core coverage area, inspection accuracy requirements, and time constraints. Based on this, the original task is broken down into multiple consecutive time segments in the time dimension. Each segment corresponds to a time-segmented sub-task, and the time interval between each sub-task is determined according to the idle time periods of the camera base station. This ensures that the same camera base station can execute tasks sequentially in different time segments, and the total coverage duration does not exceed the maximum allowable delay of the original task. For example, if the original task requires the complete imaging of a certain sector area within one hour, it can be broken down into three 20-minute time-segmented sub-tasks, which are executed sequentially by the same base station in different idle windows.

[0169] In terms of spatial dimension, the coverage area of ​​the original task is divided into multiple continuous sub-regions based on geometric features. Each sub-region corresponds to a spatial segmentation sub-task. The boundaries of each sub-region are determined based on the wind turbine structural features (such as blade segmentation and tower height layering), ensuring that there is an overlapping transition zone between adjacent sub-regions to avoid missed detections. Each sub-region is assigned to a different camera base station, and each base station is responsible for the shooting task of its assigned area. Through collaborative work, complete coverage of the original area is achieved. For example, the blade area can be divided into three spatial segmentation sub-tasks along its length: root, middle, and tip, which are shot by three different base stations respectively.

[0170] Subsequently, for each sub-task, the overall suitability of the candidate camera base station is recalculated: for the time sub-task, the idle status of the base station in the corresponding time period needs to be considered in addition, and the equipment occupancy rate is included as a correction factor in the overall suitability calculation; for the spatial sub-task, the focus is on evaluating the horizontal viewing angle matching degree and vertical coverage of the base station to its sub-region, to ensure that the sub-region is completely within the effective field of view of the base station.

[0171] Finally, the coverage of all sharded subtasks is superimposed and verified. Spatial topology analysis confirms that the sum of the coverage areas of each subtask completely includes the coverage area of ​​the original task to be sharded, and the overlap ratio does not exceed a preset value. If there are coverage gaps, the boundaries of the subtasks are adjusted or the number of sharded subtasks is increased until the coverage requirements are met, ultimately forming an executable sharding task scheme.

[0172] In some embodiments of this application, considering that in practical applications there may be situations where some fan-shaped inspection areas have no candidate camera base stations or where communication is interrupted in the inspection camera base station group, a virtual collaboration mechanism and a marker filling mechanism are set up. The two mechanisms are described below.

[0173] Virtual collaboration mechanism:

[0174] If it is detected that there is no candidate camera base station in the fan-shaped inspection area, a virtual collaboration mechanism is executed for the empty inspection area without candidate camera base station.

[0175] The virtual collaboration mechanism is specifically as follows:

[0176] ① A prediction model is constructed based on historical yaw data of wind turbines, outputting the trend and confidence interval of wind turbine yaw angle changes within a preset future time period. By analyzing the yaw angle records of wind turbines over a period of time, a time series prediction algorithm is used to train the model, predicting the dynamic change curve of wind turbine yaw angle in future periods, and providing the prediction error range at each time point, thus providing attitude reference for mobile base station path planning.

[0177] ② Dispatch mobile camera base stations in the vicinity and plan the optimal movement path based on the yaw prediction results. The path planning must meet the constraints of minimizing energy consumption and arrival time. Select available equipment from the mobile base station cluster closest to the vacant area, and calculate the shortest movement path from the base station's current location to the coverage area of ​​the vacant area based on the predicted yaw angle change trend of the wind turbine. At the same time, optimize the path through an energy consumption model to ensure that the total energy consumption is minimized and the arrival time in the vacant area does not exceed a preset threshold, thus avoiding inspection delays.

[0178] ③ Before the physical base station is in place, the 3D environment map of the wind turbine, constructed based on the aforementioned visual sensors, is combined with existing camera base station data to generate virtual inspection data for the missing inspection areas. Using wind turbine images collected by surrounding visual sensors, a 3D reconstruction technique is used to construct an overall wind turbine environment map, extracting the geometric features of wind turbine components (such as a section of blade or a certain height of the tower) corresponding to the missing areas. Then, feature matching and interpolation processing are performed on the adjacent area data captured by existing camera base stations to generate virtual images or video data for the missing areas, temporarily filling the gaps in the inspection data and ensuring the continuity of wind turbine status monitoring. Marking and filling mechanism:

[0179] If a communication interruption camera base station is detected in the inspection camera base station group, a marking and replacement mechanism is activated;

[0180] The marker padding mechanism is specifically as follows:

[0181] ① Mark the current task of the communication interrupted camera base station. The communication signal strength of each camera base station is detected in real time by the base station status monitoring module. For example, when a base station fails to communicate handshake three times in a row or the signal is interrupted for more than 5 seconds, the system automatically marks the inspection task that the base station is performing (including the responsible sector area, shooting parameters, task progress and other information) and suspends the execution process of the task.

[0182] ② If the interruption exceeds a preset time, the second most suitable camera base within the fan-shaped inspection area to which the communication interrupted camera base belongs will take over. From the list of candidate camera bases in this fan-shaped area, the base station with the second highest overall suitability will be selected, and the interrupted inspection task will be automatically assigned to it, while simultaneously transmitting the original task's shooting parameters and progress data. After taking over, the second most suitable base station will continue shooting according to the original task requirements to ensure that the inspection task is not interrupted until the communication interrupted base station restores its connection or is replaced.

[0183] In some embodiments of this application, the process of acquiring data from visual sensors, lidar, and wind turbine nacelle gyroscopes and processing it to obtain corresponding feature information is described.

[0184] The process of obtaining blade projection features from image data acquired by a vision sensor includes:

[0185] ① The image data acquired by the vision sensor is dehazed, denoised, and distortion corrected to extract the blade outline. Image dehazing algorithms are used to eliminate fog interference, and Gaussian filtering is used to remove image noise. The image is then distorted using the camera intrinsic parameter matrix to restore the true shape of the blade. The edge contour of the blade is extracted using an edge detection algorithm to obtain the two-dimensional shape boundary of the blade.

[0186] ② Identify the main axis of the blade and calculate the angle between the main axis of the blade and the horizontal axis of the image coordinate system. The angle corresponds to the angular feature of the sector region. The main axis running through the blade is identified from the blade contour using Hough transform or skeleton extraction algorithm. By calculating the angle between the main axis and the horizontal axis of the image coordinate system, this angle is used as the orientation feature of the blade in the image to match the corresponding sector inspection area.

[0187] ③ Extract stable feature points from the leaf surface texture features, and record the two-dimensional coordinates and feature vectors of the stable feature points in the image coordinate system to form a stable feature point set. Use feature extraction algorithms such as SIFT or ORB to select feature points that remain stable under changes in illumination and angle from the leaf surface texture; record the two-dimensional coordinates of each feature point and the feature vector describing its texture characteristics to form a stable feature point set that can be used for matching.

[0188] ④ The extracted blade contour, principal axis angle, and stable feature point set are integrated into blade projection features. A data association algorithm is used to link and integrate these three types of information to form complete blade projection features, providing fundamental data for subsequent calculation of the blade's spatial normal vector and spatial tilt angle.

[0189] The process of acquiring spatial point clouds from raw point cloud data collected by lidar includes:

[0190] ① The collected point cloud raw data is processed to remove noise points, eliminate redundant points, and perform coordinate calibration. Statistical filtering algorithms are used to remove isolated noise points caused by measurement errors; voxel grid downsampling algorithms are used to reduce redundant points and decrease data volume; and the point cloud data is transformed from the device coordinate system to the global coordinate system based on the installation position parameters of the LiDAR to complete coordinate calibration.

[0191] ② The point cloud data after processing multiple frames is stitched and fused using point cloud registration technology. The ICP (Iterative Closest Point) algorithm is used to register multiple frames of point cloud data collected at different times, so that the point clouds of each frame are aligned in the global coordinate system; the fusion algorithm is used to merge the registered multiple frames of point clouds into complete point cloud data, eliminating the field-of-view limitations of single frame point clouds.

[0192] ③ A point cloud segmentation algorithm is used to segment the fused point cloud data to obtain a spatial point cloud containing three-dimensional coordinate information. Based on Euclidean distance or region growing algorithms, the point clouds belonging to the wind turbine (blades, towers, nacelles, etc.) in the fused point cloud are separated from the background environment point cloud; the wind turbine-related point clouds are retained, and the three-dimensional coordinate information of each point contained therein constitutes a spatial point cloud, which is used for subsequent processing such as fitting the blade plane equation.

[0193] The processing of attitude information output from the gyroscope in the wind turbine nacelle includes:

[0194] ① The original attitude data, including angular velocity and angular acceleration, is filtered to eliminate high-frequency noise. Kalman filtering or moving average filtering algorithms are used to process the angular velocity (rotational speed around three axes) and angular acceleration data output by the gyroscope, filtering out high-frequency noise introduced by factors such as equipment vibration, and obtaining smooth original data.

[0195] ② The filtered angular velocity data is converted into Euler angles including yaw, pitch, and roll angles, with the yaw angle used for azimuth calibration of the sector inspection area. The angular velocity data is converted into angle data through integration, and then Euler angles in the body coordinate system are calculated through coordinate transformation; the yaw angle reflects the rotation angle of the cabin around the vertical axis and is the core parameter for determining the azimuth of the sector inspection area.

[0196] ③ Synchronize the Euler angles with the timestamps of the satellite positioning data. Align the acquisition time of the Euler angle data with the timestamps of the satellite positioning data to ensure that the attitude information and position information are consistent in the time dimension, avoiding fusion errors caused by time deviations.

[0197] ④ Calculate the mean value of the Euler angles within a continuous sampling period, and use the mean value as the output attitude information. By averaging multiple consecutive sets of Euler angles within a certain time window, the influence of random errors is further reduced, resulting in stable attitude information and providing reliable input for the subsequent generation of the second yaw angle.

[0198] The following describes a wind turbine collaborative inspection device provided in the embodiments of this application. The wind turbine collaborative inspection device described below and the wind turbine collaborative inspection method described above can be referred to and correspond to each other.

[0199] See Figure 3 , Figure 3 This is a schematic diagram of a wind turbine collaborative inspection device disclosed in an embodiment of this application.

[0200] like Figure 3 As shown, the wind turbine collaborative inspection device may include:

[0201] The fusion calculation module 110 is used to collect the wind turbine's pose feature data in real time through multi-source sensors and generate the wind turbine's yaw angle through fusion calculation.

[0202] The region division module 120 is used to divide the polar coordinate space with the wind turbine as the center origin into several sector-shaped inspection areas based on the wind turbine yaw angle. Each sector-shaped inspection area rotates synchronously with the wind turbine yaw angle.

[0203] The fitness calculation module 130 is used to calculate the comprehensive fitness of candidate camera base stations in each of the fan-shaped inspection areas, based on satellite positioning data of the wind turbine and camera base station. The comprehensive fitness is a fusion of horizontal view matching degree and vertical coverage degree.

[0204] The base station allocation module 140 is used to allocate the optimal camera base station to each of the sector inspection areas according to the comprehensive fitness score, so as to form an inspection camera base station group;

[0205] The task allocation module 150 is used to perform task reallocation based on a multi-dimensional priority decision tree when there is task competition among the camera base stations in the inspection camera base station group, and update the inspection camera base station group based on the reallocation result.

[0206] The inspection execution module 160 is used to perform collaborative shooting of the wind turbine using the inspection camera base station group to realize wind turbine inspection.

[0207] As can be seen from the above technical solutions, the wind turbine collaborative inspection method and related equipment provided in this application collect wind turbine pose feature data from multi-source sensors to generate yaw angles. Based on the yaw angles, a fan-shaped inspection area that rotates synchronously with the wind turbine is divided. The fusion horizontal view matching degree and vertical coverage comprehensive suitability of candidate camera base stations in each area are calculated. The optimal camera base station is assigned to form an inspection group. If there is task competition, the task is reassigned through a multi-dimensional priority decision tree. Finally, the inspection group is used to perform collaborative shooting to achieve inspection.

[0208] This application specifically addresses the shortcomings of existing technologies, achieving significant beneficial effects. On one hand, by dividing the inspection area into fan-shaped zones that rotate synchronously with the wind turbine's yaw angle, and assigning optimal camera base stations to each zone based on comprehensive fitness, it ensures that each zone has a camera base station with the highest matching degree responsible for monitoring under different yaw states, eliminating blind spots and achieving comprehensive, all-around coverage of the wind turbine. On the other hand, the solution achieves intelligent and automated inspection processes through real-time data acquisition from multi-source sensors and an automated base station allocation mechanism, eliminating the need for on-site human intervention and significantly reducing labor and time costs. Simultaneously, the base station allocation and task reassignment mechanism based on comprehensive fitness and a priority decision tree ensures the real-time and continuous nature of the inspection, avoids subjective errors in manual inspection, significantly improves inspection efficiency and accuracy, and effectively compensates for the deficiencies of existing technologies.

[0209] The wind turbine collaborative inspection device provided in this application embodiment can be applied to wind turbine collaborative inspection equipment. Figure 4 The hardware structure block diagram of the wind turbine collaborative inspection equipment is shown. (Refer to...) Figure 4The hardware structure of the wind turbine collaborative inspection equipment may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0210] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0211] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0212] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0213] The memory stores a program, which the processor can call. The program is used for:

[0214] The wind turbine's pose feature data is collected in real time by multiple source sensors, and the yaw angle of the wind turbine is generated by fusion calculation.

[0215] Based on the yaw angle of the wind turbine, the polar coordinate space with the wind turbine as the center origin is divided into several sector-shaped inspection areas, and each sector-shaped inspection area rotates synchronously with the yaw angle of the wind turbine.

[0216] For each of the aforementioned fan-shaped inspection areas, the comprehensive suitability of candidate camera base stations within the area is calculated by combining satellite positioning data of the wind turbines and camera base stations. The comprehensive suitability is a fusion of horizontal viewing angle matching degree and vertical coverage degree.

[0217] Based on the comprehensive fitness score, the optimal camera base station is assigned to each of the aforementioned sector-shaped inspection areas to form an inspection camera base station group;

[0218] When there is task contention among the camera base stations in the inspection camera base station group, task reallocation is performed based on a multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation result.

[0219] The inspection camera base station group is used to perform coordinated filming of the wind turbine to achieve wind turbine inspection. Optionally, the refined and extended functions of the program can be referred to the description above.

[0220] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0221] The wind turbine's pose feature data is collected in real time by multiple source sensors, and the yaw angle of the wind turbine is generated by fusion calculation.

[0222] Based on the yaw angle of the wind turbine, the polar coordinate space with the wind turbine as the center origin is divided into several sector-shaped inspection areas, and each sector-shaped inspection area rotates synchronously with the yaw angle of the wind turbine.

[0223] For each of the aforementioned fan-shaped inspection areas, the comprehensive suitability of candidate camera base stations within the area is calculated by combining satellite positioning data of the wind turbines and camera base stations. The comprehensive suitability is a fusion of horizontal viewing angle matching degree and vertical coverage degree.

[0224] Based on the comprehensive fitness score, the optimal camera base station is assigned to each of the aforementioned sector-shaped inspection areas to form an inspection camera base station group;

[0225] When there is task contention among the camera base stations in the inspection camera base station group, task reallocation is performed based on a multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation result.

[0226] The inspection camera base station group is used to perform coordinated filming of the wind turbine to achieve wind turbine inspection. Optionally, the refined and extended functions of the program can be referred to the description above.

[0227] This application also provides a computer program product, including a computer program, wherein the computer program is executed by a processor using the following method:

[0228] The wind turbine's pose feature data is collected in real time by multiple source sensors, and the yaw angle of the wind turbine is generated by fusion calculation.

[0229] Based on the yaw angle of the wind turbine, the polar coordinate space with the wind turbine as the center origin is divided into several sector-shaped inspection areas, and each sector-shaped inspection area rotates synchronously with the yaw angle of the wind turbine.

[0230] For each of the aforementioned fan-shaped inspection areas, the comprehensive suitability of candidate camera base stations within the area is calculated by combining satellite positioning data of the wind turbines and camera base stations. The comprehensive suitability is a fusion of horizontal viewing angle matching degree and vertical coverage degree.

[0231] Based on the comprehensive fitness score, the optimal camera base station is assigned to each of the aforementioned sector-shaped inspection areas to form an inspection camera base station group;

[0232] When there is task contention among the camera base stations in the inspection camera base station group, task reallocation is performed based on a multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation result.

[0233] The inspection camera base station group is used to perform coordinated filming of the wind turbine to achieve wind turbine inspection. Optionally, the refined and extended functions of the program can be referred to the description above.

[0234] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0235] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0236] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for collaborative inspection of wind turbines, characterized in that, include: The wind turbine's pose feature data is collected in real time by multiple source sensors and the yaw angle of the wind turbine is generated by fusion calculation. The pose feature data includes blade projection features obtained by visual sensors, spatial point clouds obtained by lidar, and attitude information output by the gyroscope of the wind turbine nacelle. Based on the yaw angle of the wind turbine, the polar coordinate space with the wind turbine as the center origin is divided into several sector-shaped inspection areas, and each sector-shaped inspection area rotates synchronously with the yaw angle of the wind turbine. For each of the aforementioned fan-shaped inspection areas, the comprehensive suitability of candidate camera base stations within the area is calculated by combining satellite positioning data of the wind turbines and camera base stations. The comprehensive suitability integrates horizontal viewing angle matching degree and vertical coverage degree. Based on the comprehensive fitness score, the optimal camera base station is assigned to each of the aforementioned sector-shaped inspection areas to form an inspection camera base station group; When there is task contention among the camera base stations in the inspection camera base station group, task reallocation is performed based on a multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation result. The inspection camera base station group is used to perform coordinated filming of the wind turbine to achieve wind turbine inspection; The fusion and calculation of the pose feature data to generate the wind turbine yaw angle includes: Based on the blade projection characteristics, the spatial normal vector of the blade is calculated through projective geometric transformation, and the spatial tilt angle is obtained. Based on the spatial point cloud, principal component analysis is used to fit the blade plane equation to generate the first yaw angle. Based on the attitude information, the cabin attitude angles are mapped to the global coordinate system using a coordinate system transformation matrix to generate a second yaw angle. Based on the confidence levels of each data source, the wind turbine yaw angle is generated by fusing the spatial tilt angle, the first yaw angle, and the second yaw angle through an adaptive weighted fusion model.

2. The method according to claim 1, characterized in that, The calculation process for the overall suitability of candidate camera base stations within the area includes: Calculate the horizontal viewing angle deviation between the horizontal direction of the optical axis of the camera base station and the azimuth angle of the center of the fan-shaped inspection area, and determine the horizontal viewing angle matching degree of the candidate camera base station based on the horizontal viewing angle deviation; Calculate the overlap ratio between the vertical field of view interval corresponding to the pitch angle of the camera base station and the height distribution interval of the wind turbine blades in the fan-shaped inspection area, and determine the vertical coverage of the candidate camera base station based on the overlap ratio; The horizontal viewing angle matching degree and the vertical coverage degree are superimposed according to a preset weight to calculate the comprehensive suitability of the candidate camera base station.

3. The method according to claim 2, characterized in that, The formula for calculating the horizontal viewpoint matching degree is: The formula for calculating the vertical coverage is: in, For horizontal viewpoint matching degree, The effective adaptation coefficient for the horizontal field of view of the camera base station. The exponential decay factor for the angle deviation, This is due to horizontal viewing angle deviation. For vertical coverage, The standard deviation of the wind turbine blade height. For the overlap ratio integral, The vertical field of view corresponding to the elevation angle of the camera base station. The vertical coverage indication function for the base station indicates the height distribution range of the wind turbine blades within the fan-shaped inspection area.

4. The method according to claim 1, characterized in that, Based on the comprehensive fitness score, the optimal camera base station is allocated to each of the aforementioned sector-shaped inspection areas to form an inspection camera base station group, including: Candidate camera base stations within each sector inspection area are sorted in descending order of comprehensive suitability, and the camera base station with the highest comprehensive suitability is selected as the initial allocation target. During the allocation process, it is simultaneously verified whether the initial allocation object meets the single camera maximum load constraint, key sector coverage constraint, and energy consumption balance constraint. If the initial allocation object does not meet any constraint, iterative adjustment is performed based on the comprehensive fitness ranking until all the sector inspection areas are covered and all constraints are met, thus forming an inspection camera base station group.

5. The method according to claim 4, characterized in that, The maximum load constraint for a single camera is: The key sector coverage constraint is: The energy consumption balance constraint is: in, In order to make the first i The inspection task for the first sector-shaped inspection area is assigned to the first j A binary decision variable for a camera base station, N This represents the total number of fan-shaped inspection areas. For the first j The maximum task capacity of a single camera base station Number the high-risk sector-shaped inspection area. To make the number The high-risk sector-shaped inspection area task was assigned to the first j Binary decision variables for each base station This is a collection of high-risk sector-shaped inspection areas. For the first j The remaining battery power of each camera base station, Energy consumption deviation threshold M The total number of camera base stations, K To iterate through the camera base station coefficients.

6. The method according to claim 1, characterized in that, Task reallocation is performed based on a multi-dimensional priority decision tree, and the inspection camera base station group is updated based on the reallocation results, including: Construct a multi-dimensional priority decision tree, where the priority dimensions of the decision tree are listed in descending order: The first dimension is the task importance, which is calculated by multiplying the dynamic weight of the sector inspection area with the overall suitability of the camera base station. The dynamic weight is positively correlated with the historical defect detection rate of the area. The second dimension is the urgency of the task, which is determined by the anomaly confidence level of the real-time data collected by the multi-source sensors. The third dimension is spatial adaptability, which is quantified by the straight-line distance between the camera base station and the wind turbine. The spatial adaptability is positively correlated with the straight-line distance, which is calculated in real time based on the satellite positioning data. The competitive tasks are prioritized based on the multi-dimensional priority decision tree, retaining high-priority tasks and including low-priority tasks in the allocation pool. For tasks in the allocation pool, after excluding candidate camera base stations that have been occupied by other tasks, the overall suitability of the remaining camera base stations is recalculated and sorted in descending order. The camera base station with the highest overall suitability is selected for secondary allocation. If there are no available camera base stations in the current area, the search is extended to adjacent areas, and the overall fitness of the extended camera base stations decreases with distance. The inspection camera base station group is updated based on the secondary allocation results. If no usable camera base station is found after extended search, the task fragmentation mechanism is triggered.

7. The method according to claim 6, characterized in that, The task sharding mechanism execution process includes: The task of segmented inspection is split into spatiotemporal dimensions to generate multiple segmented sub-tasks, including time-segmented sub-tasks executed by the same camera base station and spatial sub-tasks allocated to multiple camera base stations for collaborative coverage. Calculate the overall suitability of each of the aforementioned sub-tasks to ensure that the sum of the coverage of each of the aforementioned sub-tasks is not less than the task to be inspected in the sub-task.

8. The method according to claim 1, characterized in that, If a sector-shaped inspection area is found to have no candidate camera base station, a virtual coordination mechanism is executed for the empty inspection area without candidate camera base station. The virtual collaboration mechanism includes: A prediction model is built based on historical yaw data of wind turbines, and the trend and confidence interval of wind turbine yaw angle change within a preset time period are output. Dispatch nearby mobile camera base stations and plan the optimal mobile path based on the yaw prediction results. The path planning must meet the constraints of minimizing energy consumption and time of arrival. Before the physical base station is in place, the three-dimensional environmental map of the wind turbine constructed based on the visual sensor is combined with the existing camera base station data to generate virtual inspection data for the vacant inspection area.

9. The method according to claim 1, characterized in that, If a communication interruption camera base station is detected in the inspection camera base station group, a marking and replacement mechanism is activated; The marker padding mechanism includes: Mark the current task of the communication interruption camera base station; If the interruption exceeds a preset time, the camera base station with the second highest overall suitability within the fan-shaped inspection area to which the communication interruption camera base station belongs will take over.

10. The method according to claim 1, characterized in that, The process of obtaining the blade projection features from the image data acquired by the vision sensor includes: The image data acquired by the vision sensor is dehazed, denoised, and distortion corrected to extract the blade outline; Identify the main axis of the blade and calculate the angle between the main axis of the blade and the axis of the horizontal axis of the image coordinate system. The angle corresponds to the angular features of the sector region. Stable feature points are extracted from the surface texture features of the blade, and the two-dimensional coordinates and feature vectors of the stable feature points in the image coordinate system are recorded to form a stable feature point set. The extracted blade contour, the included angle of the axis, and the set of stable feature points are integrated into a blade projection feature.

11. The method according to claim 1, characterized in that, The process of obtaining spatial point clouds from the raw point cloud data collected by the lidar includes: The collected point cloud raw data is processed by removing noise points, eliminating redundant points, and calibrating coordinates. Point cloud data from multiple frames are stitched and fused using point cloud registration technology. A point cloud segmentation algorithm is used to segment the fused point cloud data to obtain a spatial point cloud containing three-dimensional coordinate information.

12. The method according to claim 1, characterized in that, The processing of attitude information output by the gyroscope in the wind turbine nacelle includes: The raw attitude data is filtered to eliminate high-frequency noise. The raw attitude data includes angular velocity and angular acceleration. The filtered angular velocity data is solved into Euler angles that include yaw, pitch, and roll angles, where the yaw angle is used for azimuth calibration of the sector inspection area; The Euler angles are synchronized using the timestamps of the satellite positioning data; Calculate the mean value of the Euler angles within a continuous sampling period, and use the mean value as the output attitude information.

13. A wind turbine collaborative inspection device, characterized in that, include: The fusion calculation module is used to collect the wind turbine's pose feature data in real time through multi-source sensors, and generate the wind turbine's yaw angle through fusion calculation. The pose feature data includes blade projection features obtained by a visual sensor, spatial point cloud obtained by a lidar, and attitude information output by the wind turbine nacelle gyroscope. The region division module is used to divide the polar coordinate space with the wind turbine as the center origin into several sector-shaped inspection areas based on the wind turbine yaw angle. Each sector-shaped inspection area rotates synchronously with the wind turbine yaw angle. The fitness calculation module is used to calculate the comprehensive fitness of candidate camera base stations in each of the fan-shaped inspection areas, based on the satellite positioning data of the wind turbine and camera base station. The comprehensive fitness is a fusion of horizontal view matching degree and vertical coverage degree. The base station allocation module is used to allocate the optimal camera base station to each of the sector inspection areas based on the comprehensive suitability, so as to form an inspection camera base station group; The task allocation module is used to perform task reallocation based on a multi-dimensional priority decision tree when there is task competition among the camera base stations in the inspection camera base station group, and update the inspection camera base station group based on the reallocation result. The inspection execution module is used to perform collaborative filming of the wind turbine using the inspection camera base station group to achieve wind turbine inspection; The fusion and calculation of the pose feature data to generate the wind turbine yaw angle includes: Based on the blade projection characteristics, the spatial normal vector of the blade is calculated through projective geometric transformation, and the spatial tilt angle is obtained. Based on the spatial point cloud, principal component analysis is used to fit the blade plane equation to generate the first yaw angle. Based on the attitude information, the cabin attitude angles are mapped to the global coordinate system using a coordinate system transformation matrix to generate a second yaw angle. Based on the confidence levels of each data source, the wind turbine yaw angle is generated by fusing the spatial tilt angle, the first yaw angle, and the second yaw angle through an adaptive weighted fusion model.

14. A wind turbine collaborative inspection device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the wind turbine collaborative inspection method as described in any one of claims 1-12.

15. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the wind turbine collaborative inspection method as described in any one of claims 1-12.

16. A computer program product, comprising a computer program, characterized in that, The computer program is executed by the processor to perform each step of the wind turbine collaborative inspection method as described in any one of claims 1-12.

Images