A wind turbine blade service robot control method and system

Abstract

The application discloses a kind of wind power blade operation robot control method and system, the robot includes walking mechanism, and is located at the walking mechanism operation side's maintenance mechanism, obtains the three-dimensional point cloud model of target blade, generates global walking path based on the three-dimensional point cloud model covering target operation area;Acquire blade surface image on global walking path, utilize first neural network model to carry out footing point detection to the blade surface image, verify the footing point detected based on the three-dimensional scene information of the blade of first neural network model, and carry out foot pose calculation to the footing point of valid verification;Utilize second neural network model to carry out defect target detection to blade surface image, and obtain the target posture of multi-degree-of-freedom mechanical arm based on detection result.It solves the technical problems of insufficient reliability of the existing technology of double-foot alternating adsorption type wind power blade robot footing point detection, low walking and maintenance collaborative control precision.

Description

Technical Field

[0001] This invention relates to the field of inspection robot technology, and in particular to a control method and system for a wind turbine blade operation robot. Background Technology

[0002] Wind turbines are exposed to harsh outdoor environments year-round, and their blades are susceptible to damage from factors such as wind and sand erosion, salt spray corrosion, lightning strikes, and ultraviolet aging, resulting in various types of damage such as coating peeling, surface cracks, and leading-edge corrosion.

[0003] Currently, the mainstream operation and maintenance method for wind turbine blades still relies on manual high-altitude operations, where workers use suspended platforms or elevated platforms to reach the blade surface to perform inspections and maintenance tasks. This method has drawbacks such as extremely high operational risks and long downtime for the turbine. To reduce the risks of manual operations, existing technologies have proposed several automated alternatives: for example, drone inspection technology, which can only achieve long-distance visual inspection and cannot perform contact-based maintenance operations such as grinding and painting; general-purpose wall-climbing robots, which cannot adapt to the complex curved surface transitions of wind turbine blades and have poor obstacle-crossing capabilities; and track-based or cableway robots, which require the installation of special tracks on the blade surface or tower in advance, limiting their operating range.

[0004] The following technical problems exist: First, the reliability of foothold detection is insufficient, which cannot meet the requirements of bipedal walking. Foothold detection is based on a single vision sensor, and there are defects in the handling of adsorption anomalies, which may lead to the risk of the robot falling from a height. Second, the maintenance operation control is insufficient, which may lead to problems such as local missed inspections, insufficient maintenance depth, or over-maintenance, resulting in the need for manual rework. Finally, the global path planning optimization dimension is too simplistic, making it difficult to balance work efficiency and coverage. This may result in missed work in the blade edge area or the robot frequently turning and slowing down, which may prolong the overall operation time. Summary of the Invention

[0005] This application provides a control method and system for a wind turbine blade operation robot, which solves the technical problems of insufficient reliability in foothold detection and low accuracy in collaborative control of walking and maintenance of existing bipedal alternating adsorption wind turbine blade robots. It enables the robot to walk safely and stably on the complex curved surface of wind turbine blades and perform automated maintenance.

[0006] In a first aspect, the present invention provides a control method for a wind turbine blade operation robot, wherein the robot includes a walking mechanism and a maintenance mechanism disposed on the working side of the walking mechanism; The walking mechanism includes a left leg and a right leg. Each of the left leg and the right leg includes at least one joint and a suction cup that is attached to the surface of the wind turbine blade. The suction cup is equipped with a negative pressure mechanism. The maintenance mechanism includes a multi-degree-of-freedom robotic arm and a processing arm mounted on the multi-degree-of-freedom robotic arm. The method includes: S1. Obtain a three-dimensional point cloud model of the target blade, and generate a global walking path covering the target working area based on the three-dimensional point cloud model; S2. Obtain the blade surface image on the global walking path, use the first neural network model to detect the landing point on the blade surface image, verify the landing point detected by the first neural network model based on the three-dimensional scene information of the blade, and calculate the foot pose of the verified landing point. S3. Use the second neural network model to detect defects in the surface image of the blade, and obtain the target posture of the multi-degree-of-freedom robotic arm based on the detection results.

[0007] Furthermore, obtaining the three-dimensional point cloud model of the target blade includes: S101. Obtain the original three-dimensional point cloud data of the target blade; S102. Perform denoising and surface reconstruction preprocessing operations on the original three-dimensional point cloud data to generate a three-dimensional point cloud model; S103. Generate a global walking path covering the target work area based on the three-dimensional point cloud model.

[0008] Furthermore, generating a global walking path covering the target work area based on the three-dimensional point cloud model includes: S131. Based on the three-dimensional point cloud model, extract the feature lines of the leading edge, trailing edge, tip, and root of the blade to construct a grid map of the target working area; S132. Using the premise of meeting the preset requirements for operation coverage, minimizing the total path length and the number of turns as the multi-objective optimization function, an initial global walking path is generated; S133. Based on the obstacle distribution information on the blade surface, the initial global travel path is corrected for obstacle avoidance to obtain the final global travel path.

[0009] Further, the step of detecting the landing point of the blade surface image using the first neural network model includes: S201. Input the blade surface image into the pre-trained first neural network model; S202. Extract planar region features and texture features from the image using the first neural network model; S203. Output the pixel coordinates and confidence scores of multiple candidate landing points, and filter out the candidate landing points with confidence scores greater than the preset confidence threshold as landing points to be verified.

[0010] Furthermore, verifying the landing point detected by the first neural network model based on the three-dimensional scene information of the blade includes: S204. Obtain the 3D point cloud data around the landing point to be verified, and construct a local 3D scene model; S205. Calculate the local flatness and local tilt angle for each landing point to be verified; S206. Calculate the comprehensive score of the landing point based on the local flatness and local tilt angle. The formula for calculating the comprehensive score of the landing point is as follows: ,in, The overall score is based on the landing point. This is the flatness weighting coefficient. For the local flatness of the landing point to be verified, To preset the maximum allowable flatness, This is the tilt angle weighting coefficient. The local tilt angle of the landing point to be verified. The preset maximum allowable tilt angle; S207. Select the landing points to be verified that have a comprehensive score greater than the preset score threshold, and use them as valid landing points.

[0011] Furthermore, the step of calculating the foot pose of the verified valid landing point includes: S208. Convert the pixel coordinates of the verified valid landing point into three-dimensional spatial coordinates in the robot coordinate system; S209. Calculate the target pose of the corresponding foot based on the three-dimensional spatial coordinates and the local tilt angle at the corresponding position; S210. Establish the kinematic model of the walking mechanism, calculate the target angles of each joint of the left or right leg through inverse kinematics, and generate joint control commands.

[0012] Furthermore, the defect target detection using the second neural network model on the blade surface image includes: S301. Input the blade surface image into the pre-trained second neural network model; S302. Identify defect targets in the image using the second neural network model, and output the bounding box, defect type, and defect confidence level of the defect targets; S303. Select defect targets with a defect confidence level greater than the preset defect confidence level threshold as defect targets to be processed.

[0013] Furthermore, obtaining the target pose of the multi-degree-of-freedom robotic arm based on the detection results includes: S304. Obtain the depth data corresponding to the defect target to be processed, and calculate the three-dimensional spatial coordinates of the center of the defect target; S305. Based on the three-dimensional spatial coordinates of the defect target and the defect type, determine the working reference point and working range of the processing arm; S306. Determine the target pose of the end effector of the multi-degree-of-freedom robotic arm based on the working reference point and working range of the processing arm; S307. Establish the kinematic model of the multi-degree-of-freedom manipulator, calculate the target angles of each joint of the multi-degree-of-freedom manipulator through inverse kinematics, and generate control commands for the manipulator.

[0014] Furthermore, the method also includes: S401. Obtain the suction force data of the suction cups on the left and right legs; S402. When the suction force of the single leg currently in the support phase is detected to be lower than the safety threshold, a braking command is generated to maintain the current posture of both feet and prohibit the swing phase movement; S403. Update the feasible region map based on the current adsorption position offset and remove the neighborhood range of points with abnormal adsorption force; S404. Replan the landing point coordinates and foot pose for the next cycle within the updated feasible domain map.

[0015] Furthermore, the method also includes: S501. Acquire real-time blade surface image data of the maintenance area, extract defect feature parameters of the area to be evaluated, and construct a comprehensive maintenance quality score based on the defect feature parameters; S502. If the overall inspection quality score is lower than the preset inspection accuracy threshold, the inspection is deemed unqualified and the type of unqualified defect and the coordinates of the unqualified area are determined. S503. Update the global defect map and mark the defective areas as areas to be repaired; S504. Generate corresponding differentiated repair instructions based on the type of non-conforming defect, and output them to the multi-degree-of-freedom robotic arm and the processing arm to control them to perform repair operations on the area to be repaired.

[0016] Secondly, a control system for a wind turbine blade operation robot includes: The path planning module is configured to acquire a three-dimensional point cloud model of the target blade and generate a global walking path covering the target working area based on the three-dimensional point cloud model. The walking and positioning module is configured to acquire images of the blade surface along the global walking path, use a first neural network model to detect footing points on the blade surface images, verify the footing points detected by the first neural network model based on the three-dimensional scene information of the blade, and calculate the foot pose for the verified footing points. The inspection and positioning module is configured to use a second neural network model to detect defects in the surface image of the blade, and obtain the target posture of the multi-degree-of-freedom robotic arm based on the detection results.

[0017] Thirdly, the present invention provides a computer-readable storage medium storing a plurality of instructions adapted to be loaded by a processor of a terminal device and executed by the method thereof.

[0018] Fourthly, the present invention provides a terminal device, including a processor and a computer-readable storage medium, wherein the processor is configured to implement various instructions; and the computer-readable storage medium is configured to store multiple instructions adapted to be loaded by the processor and executed using the method described thereon.

[0019] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By adopting a dual-layer landing point screening mechanism that combines coarse detection using a first neural network with fine verification using 3D scene information, the technical problem of misjudgment caused by single visual detection being easily interfered with by surface contamination defects in existing technologies is effectively solved, thereby significantly improving the reliability of landing point detection.

[0020] 2. By adopting an adsorption anomaly handling logic that distinguishes between the support phase and the swing phase, and using the technical means of dynamically updating the feasible domain map to eliminate the neighborhood of abnormal points, the technical problem of falsely triggering emergency braking and continuous adsorption failure caused by imperfect adsorption anomaly handling logic in the existing technology is effectively solved, thereby reducing the risk of robot falling when operating at height.

[0021] 3. By employing a second neural network to achieve accurate defect classification and location, and generating differentiated maintenance instructions based on different defect types, the technical problem of inconsistent defect detection and robotic arm control in existing technologies, which leads to a one-size-fits-all approach to maintenance parameters, is effectively solved, thereby avoiding insufficient maintenance depth or over-maintenance.

[0022] 4. By employing the technique of extracting defect feature parameters to construct a comprehensive maintenance quality score and combining it with dynamic updates of the global defect map to achieve automatic repair, the technical problems of local missed inspections and manual rework caused by the lack of online quality assessment in existing technologies are effectively solved, thereby constructing a complete closed-loop control system for maintenance quality.

[0023] 5. By employing a multi-objective optimization function to generate the initial global path with the goal of minimizing the total path length and number of turns to meet the preset requirements for operational coverage, and combining obstacle avoidance correction with the distribution of obstacles on the blade surface, the technical problem of insufficient operational coverage and low operational efficiency caused by the single dimension of global path planning optimization in the existing technology is effectively solved, thus balancing operational efficiency and operational coverage. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the robot body in Embodiment 1 of this application; Figure 2 This is a schematic diagram of the robot body from another angle in Embodiment 1 of the application; Figure 3 for Figure 2 A magnified view of part A in the image; Figure 4 This is a flowchart of the method in Embodiment 1 of this application; Figure 5 This is a module diagram of Embodiment 2 of this application.

[0025] 1. Robot body; 2. Left leg; 3. Right leg; 301. Hip joint; 302. Thigh bar; 303. Knee joint; 304. Lower leg bar; 305. Ankle joint; 306. Suction cup mount; 307. Rubber suction cup; 4. Five-bar parallel arm; 401. Fixed base; 402. First base joint motor; 403. First active branch; 404. First driven hinge point; 405. First forward extension branch; 406. Second base joint motor; 407. Second active branch; 408. Second driven hinge point; 409. Second forward extension branch; 5. UBNS type ball spline screw assembly; 501. Screw body; 502. Ball screw nut; 503. Spline nut; 504. First drive motor; 505. First belt drive unit; 506. Second drive motor; 507. Second belt drive unit. Detailed Implementation

[0026] Bipedal alternating adsorption-type wind turbine blade maintenance robots have become the mainstream hardware equipment for addressing the risks of manual high-altitude maintenance of wind turbine blades. However, the current control methods adapted to this type of robot still have significant shortcomings. The core technical problems are: foothold detection relies solely on a single vision sensor, making it susceptible to environmental interference; the handling of adsorption anomalies does not differentiate between gait states, easily leading to false triggers; and the lack of a quality closed loop in maintenance operations can result in substandard maintenance, failing to fully leverage the hardware advantages of bipedal robots.

[0027] To address the aforementioned technical issues, this application provides a control method for a wind turbine blade operation robot. This method employs a hierarchical control architecture. First, it generates a multi-objective optimized global walking path based on a three-dimensional point cloud model of the blade. Second, it selects reliable landing points through coarse detection using neural networks combined with fine verification using three-dimensional scene information, and establishes an adsorption anomaly emergency handling mechanism that distinguishes gait states. Finally, it constructs a complete closed loop for maintenance operation quality by combining precise defect detection with online quality assessment.

[0028] The technical solution of this application embodiment can effectively improve the stability and safety of the robot walking on the complex curved surface of the wind turbine blade, reduce the risk of falling from a height from the root, and significantly improve the first-pass yield of maintenance operations, avoid manual rework, balance work efficiency and work coverage, and realize fully automatic and high-quality operation and maintenance of wind turbine blades.

[0029] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0030] Example 1 A control method for a wind turbine blade operation robot, the robot including a walking mechanism and a maintenance mechanism located on the working side of the walking mechanism; Reference Figure 1 The walking mechanism includes a left leg 2 and a right leg 3. Each of the left leg 2 and the right leg 3 includes at least one joint and a suction cup that is adsorbed onto the surface of the wind turbine blade. The suction cup is equipped with a negative pressure mechanism. The maintenance mechanism includes a multi-degree-of-freedom robotic arm and a processing arm disposed on the multi-degree-of-freedom robotic arm. Reference Figure 2 This embodiment presents a specific robot structure. The robot adopts a modular and lightweight design, and all structural components are made of aerospace-grade aluminum alloy and carbon fiber composite materials. While ensuring the overall structural strength and impact resistance, this effectively reduces the overall weight of the robot, decreases the load on the adsorption mechanism, and improves the robot's continuous operation endurance and high-altitude operation safety. The robot body 1 integrates an embedded controller, a power module, and a communication module. A depth camera and a LiDAR are fixedly mounted on the front end of the robot body 1, with the detection directions of the depth camera and LiDAR both facing the robot's working direction.

[0031] The alternating walking mechanism includes a left leg 2 and a right leg 3, which are symmetrically mounted on the left and right sides of the robot body 1. The left leg 2 and the right leg 3 have identical structures. Each leg is equipped with a hip joint 301, a thigh bar 302, a knee joint 303, a calf bar 304, an ankle joint 305, and a negative pressure suction foot end. Servo motors can be used at each joint.

[0032] The fixed end of the hip joint 301 is fixedly connected to the side end face of the robot body 1, and the output end of the hip joint 301 is fixedly connected to the upper end of the thigh rod 302. The hip joint 301 is a rotary joint, providing rotational freedom about the vertical plane, which is used to adjust the horizontal swing angle of the leg to realize the robot's turning and obstacle-crossing actions.

[0033] The lower end of the thigh rod 302 is fixedly connected to the fixed end of the knee joint 303, and the output end of the knee joint 303 is fixedly connected to the upper end of the lower leg rod 304. The knee joint 303 is a swing joint, providing swing freedom around the horizontal plane, used to adjust the vertical height of the foot end to cross protruding obstacles on the blade surface.

[0034] The lower end of the lower leg rod 304 is fixedly connected to the fixed end of the ankle joint 305, and the output end of the ankle joint 305 is fixedly connected to the mounting seat of the negative pressure suction foot end. The ankle joint 305 is a swing joint, providing swing freedom around the horizontal plane, which is used to adjust the pitch angle of the foot end, ensuring that the suction cup can fit tightly with the curved surface of the blade with different curvatures to form an effective vacuum seal.

[0035] The negative pressure adsorption foot end includes a suction cup mounting base 306, a rubber suction cup 307, a high-speed switching valve, and a high-flow negative pressure generator. The upper end of the suction cup mounting base 306 is fixedly connected to the output end of the ankle joint 305, and the lower end of the suction cup mounting base 306 is sealed and fixedly connected to the edge of the rubber suction cup 307. The high-speed switching valve and the high-flow negative pressure generator are both integrated and installed inside the suction cup mounting base 306. The air inlet of the high-speed switching valve is connected to the internal cavity of the rubber suction cup 307, and the air outlet of the high-speed switching valve is connected to the air inlet of the high-flow negative pressure generator and the outside atmosphere. The response time of the high-speed switching valve is no more than ten milliseconds, and the pumping rate of the high-flow negative pressure generator meets the requirements for rapid establishment and release of the vacuum chamber, enabling millisecond-level adsorption and release actions at the foot end.

[0036] The maintenance mechanism is fixedly installed in the upper region between the left leg 2 and the right leg 3. The maintenance mechanism includes a five-bar parallel arm 4 and a ball spline screw assembly.

[0037] The maintenance mechanism is fixedly installed on the working side end face of the robot body 1, located in the upper area between the left leg 2 and the right leg 3. The maintenance mechanism includes a five-bar parallel arm 4 and a UBNS type ball spline screw assembly 5.

[0038] Reference Figure 2The five-bar parallel arm 4 includes a fixed base 401, a first base joint motor 402, a first active branch 403, a first driven hinge point 404, a first forward extension branch 405, a second base joint motor 406, a second active branch 407, a second driven hinge point 408, and a second forward extension branch 409. The back of the fixed base 401 is fixedly connected to the top surface of the robot body 1. The first base joint motor 402 and the second base joint motor 406 are both fixedly mounted on the front of the fixed base 401 and are symmetrically spaced along the horizontal direction. The output shaft of the first base joint motor 402 is fixedly connected to the proximal end of the first active branch 403, and the distal end of the first active branch 403 is fixedly connected to the housing of the first driven hinge point 404. The first driven hinge point 404 is a passive hinge structure without drive. Its rotating end is hinged to the proximal end of the first forward extension chain 405, and the distal end of the first forward extension chain 405 is fixedly connected to the left side of the ball spline screw assembly. The output shaft of the second base joint motor 406 is fixedly connected to the proximal end of the second active chain 407, and the distal end of the second active chain 407 is fixedly connected to the housing of the second driven hinge point 408. The second driven hinge point 408 is a passive hinge structure without drive. Its rotating end is hinged to the proximal end of the second forward extension chain 409, and the distal end of the second forward extension chain 409 is fixedly connected to the right side of the ball spline screw assembly. The first forward extension chain 405 and the second forward extension chain 409 support the ball spline screw assembly in parallel. The coordinated rotation of the first base joint motor 402 and the second base joint motor 406 drives the first active branch 403 and the second active branch 407 to swing, which in turn drives the first forward branch 405 and the second forward branch 409 to move through the first driven hinge point 404 and the second driven hinge point 408, thus realizing arbitrary planar motion of the ball spline screw assembly in the XY plane. This dual-branch parallel structure eliminates the problem of insufficient fixed-end support caused by the increased cantilever length in traditional three-axis serial robotic arms, significantly improving the overall rigidity and vibration resistance of the mechanism. It also has the advantages of fast response speed and high positioning accuracy, making it particularly suitable for performing high-precision contact operations in the limited space of wind turbine blade operations.

[0039] Reference Figure 3The UBNS type ball spline screw assembly 5 includes a screw body 501, a ball screw nut 502, a spline nut 503, a first drive motor 504, a second drive motor 506, a first belt drive unit 505, a second belt drive unit 507, and a multi-functional quick-change connector. The screw body 501 is an integrated shaft with ball screw raceways and spline raceways, vertically inserted at the center of the ball screw nut 502 and the spline nut 503. The spline nut 503 and the ball screw nut 502 are sequentially sleeved on the screw body 501 from top to bottom. The ball screw nut 502 meshes with the screw raceway of the screw body 501, and the spline nut 503 meshes with the spline raceway of the screw body 501. The first drive motor 504 is fixedly installed inside the housing of the first driven hinge point 404. The drive pulley of the first belt drive unit 505 is fixedly sleeved on the output shaft of the first drive motor 504, and the driven pulley of the first belt drive unit 505 is fixedly sleeved on the outer periphery of the ball screw nut 502. The belt of the first belt drive unit 505 runs along the internal wiring of the first drive branch 403 and the first forward extension branch 405, connecting the drive pulley and the driven pulley. The first drive motor 504 drives the ball screw nut 502 to rotate around the axis of the screw body 501 through the first belt drive unit 505. The second drive motor 506 is fixedly installed inside the housing of the second driven hinge point 408. The drive pulley of the second belt drive unit 507 is fixedly sleeved on the output shaft of the second drive motor 506, and the driven pulley of the second belt drive unit 507 is fixedly sleeved on the outer periphery of the spline nut 503. The belt of the second belt drive unit 507 runs along the internal wiring of the second drive branch 407 and the second forward extension branch 409, connecting the drive pulley and the driven pulley. The second drive motor 506 drives the spline nut 503 to rotate around the axis of the lead screw body 501 through the second belt drive unit 507. A multi-functional quick-change connector is fixedly installed at the lower end of the lead screw body 501, which can quickly change tools such as grinding heads, spray guns, and inspection probes according to different maintenance conditions, meeting various operational needs such as blade surface cleaning, defect grinding, and coating repair.

[0040] The UBNS type ball spline screw assembly 5 can control the start / stop and speed of two drive motors to achieve three working modes: linear motion, rotary motion, and helical motion on a single axis. Linear motion mode: The first drive motor 504 drives the ball screw nut 502 to rotate, the second drive motor 506 stops, and the spline nut 503 remains stationary. At this time, the screw body 501 only moves up and down linearly along its own axis and does not rotate.

[0041] Rotary motion mode: The first drive motor 504 and the second drive motor 506 rotate synchronously at the same speed and in the same direction, and the ball screw nut 502 and the spline nut 503 rotate synchronously. At this time, the screw body 501 only rotates around its own axis and does not undergo axial displacement.

[0042] Helical motion mode: The second drive motor 506 drives the spline nut 503 to rotate, the first drive motor 504 stops, and the ball screw nut 502 remains stationary. At this time, the screw body 501 rotates around its own axis while making up-down linear motion along its own axis, forming a helical motion trajectory.

[0043] Reference Figure 4 The method includes: S1. Obtain a three-dimensional point cloud model of the target blade, and generate a global walking path covering the target working area based on the three-dimensional point cloud model; Furthermore, obtaining the three-dimensional point cloud model of the target blade includes: S101. Obtain the original three-dimensional point cloud data of the target blade; An embedded controller controls a 3D LiDAR mounted on the robot to perform segmented scanning along the axis of the target blade, acquiring raw 3D point cloud data. The 3D LiDAR's scanning resolution is set to at least five points per centimeter, and the scanning range covers the entire surface area of ​​the blade from the root to the tip. During the scanning process, attitude data from the inertial measurement unit is simultaneously acquired to correct the pose of the raw point cloud data, eliminating the impact of the robot's own posture changes on scanning accuracy. The corrected raw 3D point cloud data is stored in the robot's local storage module for subsequent preprocessing operations.

[0044] S102. Perform denoising and surface reconstruction preprocessing operations on the original three-dimensional point cloud data to generate a three-dimensional point cloud model; First, statistical filtering denoising is performed on the original 3D point cloud data. The average distance between each point and its twenty nearest neighbors is calculated. A distance threshold is determined based on a Gaussian distribution, and points with an average distance greater than the threshold are identified as outliers and removed. After statistical filtering denoising, radius filtering denoising is performed. A spherical neighborhood with a radius of five centimeters is set around each point. The number of points within the neighborhood is counted, and points with fewer points than a preset threshold are identified as isolated points and removed.

[0045] After denoising, the Poisson surface reconstruction algorithm is used to reconstruct the surface of the processed point cloud data. The point cloud data is converted into an octree structure, the normal vector of each point is calculated, and a continuous implicit surface is obtained by solving the Poisson equation. Then, isosurface extraction is performed on the implicit surface to generate a high-precision 3D point cloud model of the target blade. This 3D point cloud model completely restores the geometric shape and surface features of the blade.

[0046] S103. Generate a global walking path covering the target work area based on the three-dimensional point cloud model.

[0047] The generation of a global walking path covering the target work area based on the 3D point cloud model includes: S131. Based on the three-dimensional point cloud model, extract the feature lines of the leading edge, trailing edge, tip, and root of the blade to construct a grid map of the target working area; An edge detection algorithm combined with curvature calculation method is used to extract the feature lines of the leaf from a 3D point cloud model. The Gaussian curvature and mean curvature of each point in the 3D point cloud model are calculated, and the set of points with abrupt curvature changes are selected as candidate feature points. The candidate feature points are clustered and fitted to obtain the leading edge feature line, trailing edge feature line, leaf tip feature line, and leaf root feature line of the leaf.

[0048] Based on the extracted feature lines, a grid map of the target operation area is constructed. The blade surface is uniformly divided along the axial and circumferential directions to generate square grids with a side length of 50 centimeters. Each grid stores its corresponding 3D coordinates, average curvature, obstacle markers, and operation status markers. Among them, obstacle markers are used to identify fixed obstacles on the blade surface, and operation status markers are used to indicate whether the operation of that grid has been completed.

[0049] S132. Using the premise of meeting the preset requirements for operation coverage, minimizing the total path length and the number of turns as the multi-objective optimization function, an initial global walking path is generated; Construct a multi-objective optimization function, the expression of which is: ,in, The objective function value, For operational coverage, This is the total path length. To preset the maximum allowed path length, The number of turns on the path. To preset the maximum number of turns, β and γ are the weighting coefficients for operation coverage, path length, and number of turns, respectively, and satisfy the following conditions: A genetic algorithm was used to solve the multi-objective optimization function. The initial population size was fifty, and the number of iterations was one hundred. Through selection, crossover, and mutation operations, individual paths were continuously optimized until the path with the minimum objective function value was obtained as the initial global path. This initial path minimized both the total path length and the number of turns while ensuring 100% task coverage.

[0050] S133. Based on the obstacle distribution information on the blade surface, the initial global travel path is corrected for obstacle avoidance to obtain the final global travel path.

[0051] Based on obstacle markers in the grid map, all obstacle regions on the initial global walking path are identified. The A-Star algorithm is used to locally correct the initial path for obstacle avoidance, replanning local paths to bypass obstacles. The constraints for local path planning are that the minimum distance between the path and obstacles is no less than 30 centimeters, and the length increment of the local path is minimized. After obstacle avoidance correction, the corrected path is smoothed to eliminate sharp inflections and ensure the robot's walking stability. The final global walking path is stored in the robot's local storage module as global navigation commands for autonomous robot movement. The A-Star algorithm is the most commonly used classic heuristic search algorithm in this field for path planning, quickly and accurately finding the shortest path from the starting point to the ending point in an environment with known obstacle distribution.

[0052] S2. Obtain the blade surface image on the global walking path, use the first neural network model to detect the landing point on the blade surface image, verify the landing point detected by the first neural network model based on the three-dimensional scene information of the blade, and calculate the foot pose of the verified landing point. The step of detecting the landing point of the blade surface image using the first neural network model includes: S201. Input the blade surface image into the pre-trained first neural network model; An embedded controller controls a depth camera mounted on the robot to acquire images of the blade surface along its global walking path at a frame rate of 15 frames per second. The acquired images, after grayscale conversion and normalization preprocessing, are input into a pre-trained first neural network model. This first neural network model is an improved single-stage object detection neural network model, pre-trained using a wind turbine blade surface image dataset. The training dataset contains blade surface images under different lighting conditions, pollution levels, and damage states. During training, data augmentation techniques such as random rotation, horizontal flipping, and brightness adjustment are used to improve the model's robustness to complex environments.

[0053] It should be understood that using neural network models for image target detection is an existing and conventional technique in the field, and is also common knowledge to those skilled in the art. The main improvement of this application is not to use neural networks to detect the landing point itself, but to combine the coarse detection results of the neural network with three-dimensional scene information, and to perform secondary verification of the landing point through quantitative calculation of local flatness and tilt angle, thereby significantly improving the reliability of landing point detection.

[0054] S202. Extract planar region features and texture features from the image using the first neural network model; The first neural network model extracts feature information from the input image layer by layer using a multi-layer convolutional neural network structure. Shallow convolutional layers extract texture features, including scratches, stains, and coating peeling on the blade surface; deep convolutional layers extract planar region features, identifying continuous, smooth areas on the blade surface. The model then fuses the shallow texture features with the deep planar region features through a feature fusion network, generating a multi-scale fused feature map.

[0055] S203. Output the pixel coordinates and confidence scores of multiple candidate landing points, and filter out the candidate landing points with confidence scores greater than the preset confidence threshold as landing points to be verified.

[0056] The first neural network model predicts based on multi-scale fused feature maps, outputting bounding boxes, center pixel coordinates, and confidence scores for multiple candidate landing points. The confidence score characterizes the probability that the candidate region is a valid landing point. The embedded controller filters all candidate landing points, eliminating those with confidence scores below a preset confidence threshold, and retaining those with confidence scores greater than or equal to the preset threshold as landing points to be verified. This filtering step effectively filters out most obviously unqualified candidate regions, reducing the computational load of subsequent 3D verification and improving system real-time performance.

[0057] The verification of the landing point detected by the first neural network model based on the three-dimensional scene information of the blade includes: S204. Obtain the 3D point cloud data around the landing point to be verified, and construct a local 3D scene model; For each candidate landing point to be verified, the embedded controller controls the 3D LiDAR to acquire 3D point cloud data within a circular area with a radius of 20 centimeters centered on the pixel coordinates of that candidate landing point. After denoising preprocessing, the acquired point cloud data is used to construct a local 3D scene model of this area using a least-squares plane fitting algorithm. The expression for the local 3D scene model is: ,in, , , To fit the normal vector components of the plane, It is a plane constant term, and satisfies .

[0058] S205. Calculate the local flatness and local tilt angle for each landing point to be verified; Based on a local 3D scene model, the local flatness corresponding to the landing point to be verified is calculated. Local flatness is the root mean square value of the distances from all points within the region to the fitted plane, and its expression is: ,in, For local flatness, This represents the number of point clouds within the region. For the first The three-dimensional coordinates of each point are calculated. The local tilt angle corresponding to the landing point to be verified is calculated. The local tilt angle is the angle between the fitted plane and the horizontal plane, and its expression is: ,in, For local tilt angle, This is to fit the vertical component of the plane normal vector.

[0059] S206. Calculate the comprehensive score of the landing point based on the local flatness and local tilt angle. The formula for calculating the comprehensive score of the landing point is as follows: ,in, The overall score is based on the landing point. This is the flatness weighting coefficient. For the local flatness of the landing point to be verified, To preset the maximum allowable flatness, This is the tilt angle weighting coefficient. The local tilt angle of the landing point to be verified. The maximum allowable tilt angle is preset, and the comprehensive score ranges from 0 to 1. The higher the score, the better the adsorption conditions of the landing point.

[0060] S207. Select the landing points to be verified whose comprehensive score is greater than the preset score threshold, and use them as valid landing points. The embedded controller sorts the comprehensive scores of all landing points to be verified and selects the landing points whose comprehensive scores are greater than the preset score threshold. If the comprehensive scores of all landing points to be verified are less than the preset score threshold, the search range is expanded and the landing point detection and verification process is repeated until a valid landing point that meets the requirements is found.

[0061] The calculation of foot pose for verified valid landing points includes: S208. Convert the pixel coordinates of the verified valid landing point into three-dimensional spatial coordinates in the robot coordinate system; Based on the intrinsic and extrinsic parameter matrices of the depth camera, and combined with the depth information acquired by the LiDAR, the pixel coordinates of the verified landing points are converted into 3D coordinates in the camera coordinate system. Then, using the hand-eye calibration matrix, the 3D coordinates in the camera coordinate system are converted into 3D spatial coordinates in the robot coordinate system.

[0062] S209. Calculate the target pose of the corresponding foot based on the three-dimensional spatial coordinates and the local tilt angle at the corresponding position; The target pose of the foot consists of the target position and the target orientation. The target position is the three-dimensional spatial coordinate in the transformed robot coordinate system. The target orientation is calculated based on the local tilt angle of the corresponding position, ensuring that the plane of the foot suction cup is parallel to the fitting plane of the blade surface, thus ensuring that the suction cup can fit tightly against the blade surface and form an effective vacuum seal.

[0063] S210. Establish the kinematic model of the walking mechanism, calculate the target angles of each joint of the left or right leg through inverse kinematics, and generate joint control commands.

[0064] The forward kinematics models of the left and right legs are established using the DH parameter method to obtain the mapping relationship between joint angles and foot pose. Based on the target pose of the foot, the inverse kinematic equations are solved analytically to obtain the target angles of each joint in the left or right leg. The embedded controller generates corresponding PWM control commands based on the target angles of each joint and sends them to the servo motors of each joint, driving the servo motors to rotate to the target angles and thus moving the leg to the target pose.

[0065] S3. Use the second neural network model to detect defects in the surface image of the blade, and obtain the target posture of the multi-degree-of-freedom robotic arm based on the detection results.

[0066] Furthermore, the defect target detection using the second neural network model on the blade surface image includes: S301. Input the blade surface image into the pre-trained second neural network model; An embedded controller controls a depth camera mounted on the robot to acquire high-resolution images of the blade surfaces in the current maintenance area. The acquired images undergo dehazing, contrast enhancement, and normalization preprocessing before being input into a pre-trained second neural network model. This second neural network model is an improved single-stage object detection neural network model, pre-trained using a dedicated wind turbine blade defect dataset. The training dataset contains blade defect images under different lighting conditions, seasons, and wind farms, covering four typical defect types: coating peeling, surface cracks, leading-edge corrosion, and lightning damage. During training, random cropping, color dithering, and Gaussian blur data augmentation techniques are used to improve the model's robustness to complex operating environments.

[0067] It should be understood that using neural network models for defect target detection is an existing and conventional technique in the field, and is also common knowledge to those skilled in the art. The main improvement of this application is not to use neural networks to detect defects themselves, but to generate differentiated maintenance instructions based on different defect types.

[0068] S302. Identify defect targets in the image using the second neural network model, and output the bounding box, defect type, and defect confidence level of the defect targets; The second neural network model extracts multi-scale features from the input image through the backbone network, performs feature fusion and enhancement through the neck network, and then outputs the prediction results through the detection head network. The prediction results include the bounding box coordinates, defect type label, and defect confidence score for each defect target. The bounding box coordinates are the pixel coordinates of the top-left and bottom-right corners of the defect target in the image, representing the location and size of the defect; the defect type label corresponds to four types of defects: coating peeling, surface cracks, leading-edge corrosion, and lightning damage; and the defect confidence score represents the accuracy of the prediction result.

[0069] S303. Select defect targets with a defect confidence level greater than the preset defect confidence level threshold as defect targets to be processed.

[0070] The embedded controller first performs non-maximum suppression (NMS) on all prediction results to remove duplicate detection boxes for the same defect target. The cross-union ratio (CUI) threshold for NMS is set to 0.5, retaining the detection box with the highest confidence. After NMS, the remaining detection results are filtered by confidence, discarding those with a defect confidence score lower than a preset defect confidence score threshold, and retaining the defect targets with a defect confidence score greater than or equal to the preset defect confidence score threshold as the defect targets to be processed. If no defect target to be processed is detected in the current inspection area, the embedded controller controls the robot to continue moving along the global walking path to the next inspection point.

[0071] The process of obtaining the target pose of the multi-degree-of-freedom robotic arm based on the detection results includes: S304. Obtain the depth data corresponding to the defect target to be processed, and calculate the three-dimensional spatial coordinates of the center of the defect target; For each defect target to be processed, the embedded controller extracts the depth data of the corresponding region from the depth camera based on its bounding box coordinates, and simultaneously calls the 3D LiDAR to acquire high-precision point cloud data of that region. The depth data and point cloud data are fused to obtain the 3D information of the defect target region. The center pixel coordinates of the defect target's bounding box are calculated, and based on the depth camera's intrinsic parameter matrix, these center pixel coordinates are converted into 3D coordinates in the camera coordinate system. The conversion formula is: in, The pixel coordinates of the center of the defect target. The principal point coordinates of the depth camera. , For depth cameras in shaft and Focal length along the axial direction, This represents the depth value corresponding to the center of the defect target. The coordinates are in the camera coordinate system (3D). By using a pre-calibrated hand-eye calibration matrix, the 3D coordinates in the camera coordinate system are converted into 3D spatial coordinates in the robot coordinate system, thus obtaining the global position of the defect target center.

[0072] S305. Based on the three-dimensional spatial coordinates of the defect target and the defect type, determine the working reference point and working range of the processing arm; The reference point for the operation is the projection of the defect target center onto the blade surface, and its three-dimensional coordinates are consistent with those of the defect target center. The corresponding operating range is determined according to the different defect types: for coating peeling defects, the operating range is a rectangular area extending 10 cm beyond the defect boundary frame; for surface crack defects, the operating range is a strip-shaped area extending 5 cm on each side of the crack centerline; for leading-edge corrosion defects, the operating range is an arc-shaped area extending 15 cm beyond the corrosion area; and for lightning damage defects, the operating range is a circular area with a radius of 20 cm around the damage center.

[0073] S306. Determine the target pose of the end effector of the multi-degree-of-freedom robotic arm based on the working reference point and working range of the processing arm; The target pose of the end effector of a multi-degree-of-freedom robotic arm consists of the target position and the target orientation. The target position is a three-dimensional coordinate system located five centimeters directly above the work reference point, with space reserved for the tool's feed. The target orientation is determined based on the normal vector of the blade surface at the location of the defect target, ensuring that the axis of the UBNS-type ball spline screw coincides with the normal vector of the blade surface, thus guaranteeing that the end effector is perpendicular to the blade surface. Using a local three-dimensional scene model of this location, the target orientation is fine-tuned to adapt to changes in the curvature of the blade surface.

[0074] S307. Establish the kinematic model of the multi-degree-of-freedom manipulator, calculate the target angles of each joint of the multi-degree-of-freedom manipulator through inverse kinematics, and generate control commands for the manipulator.

[0075] A forward kinematic model of the five-bar linkage was established using the DH parameter method, obtaining the mapping relationship between the angles of the first and second base joint motors and the position of the ball spline screw assembly in the XY plane. Based on the target position of the ball spline screw assembly, the inverse kinematic equation of the five-bar linkage was solved using a numerical iteration method, yielding the target angles of the first and second base joint motors. Simultaneously, the feed rate and rotational speed of the ball spline screw were determined according to the job type. The embedded controller generates corresponding PWM control commands based on the target angles and motion parameters of each joint, sending them to the first and second base joint motors, the first drive motor, and the second drive motor, driving the five-bar linkage and the ball spline screw assembly to move to the target pose and execute the corresponding maintenance operation.

[0076] The method further includes: S401. Obtain the suction force data of the suction cups on the left and right legs; Both the left and right leg suction cups are equipped with high-precision vacuum pressure sensors. The detection end of the vacuum pressure sensor is directly connected to the vacuum chamber of the suction cup to collect the analog voltage signals output by the two vacuum pressure sensors.

[0077] S402. When the suction force of the single leg currently in the support phase is detected to be lower than the safety threshold, a braking command is generated to maintain the current posture of both feet and prohibit the swing phase movement; The embedded controller tracks the robot's gait state in real time through an internal finite state machine. The robot adopts an inchworm-like alternating walking gait, and the gait state machine includes four states: left support phase, right support phase, left swing phase, and right swing phase. At any given time, only one leg is in the support phase, while the other leg is in the swing phase.

[0078] When the real-time suction force of the suction cup currently in the support phase is detected to be lower than the preset safe suction force threshold for three consecutive sampling cycles, the embedded controller immediately generates an emergency braking command. The braking command is sent to all joint servo motors of the walking mechanism, switching all servo motors to torque-locked mode, locking the current angle of all joints, and maintaining the current posture of the two feet. At the same time, the motion command queue of the swing phase is cleared, prohibiting the execution of subsequent motion commands of the swing phase to prevent the robot's center of gravity from shifting.

[0079] This processing logic only addresses adsorption anomalies of the support phase chuck, completely eliminating false triggering caused by the normal detachment of the swing phase chuck from the blade, thus ensuring the accuracy and timeliness of anomaly handling.

[0080] S403. Update the feasible region map based on the current adsorption position offset and remove the neighborhood range of points with abnormal adsorption force; The feasible region map is a dynamically updated local map based on the global grid map, used to mark areas where the robot can safely traverse. Each grid in the feasible region map contains a feasible state marker, which is divided into three types: feasible, infeasible, and pending verification. When an adsorption anomaly occurs, the embedded controller first obtains the three-dimensional spatial coordinates of the current support point in the robot coordinate system. By combining the coordinate transformation relationships of the global map, the coordinates in the robot's coordinate system are converted into grid coordinates in the global grid map. The coordinate transformation formula is: in, The origin coordinates of the global raster map. This represents the side length of the global grid. This is a floor function. It uses the grid corresponding to the adsorption anomaly points. A circular neighborhood with a radius of 30 centimeters is defined centered on the node. The coordinates of all grid cells within this circular neighborhood are calculated, and the feasible status of these cells is marked as infeasible, updating the feasible region map. This step fundamentally solves the problem of continuous adsorption failure caused by the robot repeatedly falling into the same defective area. Since defects and contamination on the blade surface are usually locally distributed, there is also a high risk of adsorption failure within the neighborhood of abnormal points; therefore, these areas need to be deleted as well.

[0081] S404. Replan the landing point coordinates and foot pose for the next cycle within the updated feasible domain map.

[0082] Within the updated feasible domain map, the embedded controller performs a local replanning process for landing points. The replanning range is limited to a circular area with a radius of one meter around the anomalous points, eliminating the need for global replanning and ensuring real-time processing.

[0083] The replanning process employs the principle of minimizing the cost function to select the optimal landing point. The formula for calculating the landing point cost function is as follows: ,in, The cost function value at the landing point. The overall score is based on the landing point. This is the straight-line distance between the current landing point and the originally planned landing point. This is the maximum permissible distance within the replanning area. and These are the overall score weight and the distance weight, respectively, and satisfy the following conditions: .

[0084] The replanning process repeatedly executes the landing point detection and verification steps: control the depth camera to acquire images of the blade surface in the area, use the first neural network model to detect candidate landing points; control the 3D lidar to acquire 3D point cloud data around the candidate landing points, calculate the local flatness and local tilt angle, and obtain a comprehensive score for each candidate landing point.

[0085] The cost value of each candidate landing point is calculated based on the cost function, and the candidate landing point with the minimum cost value is selected as the optimal landing point. Foot pose calculation is then performed on the optimal landing point to obtain the landing point coordinates and the target angles of the corresponding leg joints for the next control cycle.

[0086] The embedded controller generates corresponding joint control commands to drive the swinging leg to move to the new footing point. After the adsorption force at the new footing point is higher than the preset safe adsorption force threshold for five consecutive sampling cycles, it switches to the support phase, releases the original abnormal support leg, and the robot resumes normal walking.

[0087] The method further includes: S501. Acquire real-time blade surface image data of the maintenance area, extract defect feature parameters of the area to be evaluated, and construct a comprehensive maintenance quality score based on the defect feature parameters; An embedded controller controls a high-resolution depth camera mounted on the robot to re-acquire images of the blade surface in the repaired area under the same shooting angle and lighting conditions. After noise reduction, contrast enhancement, and image registration preprocessing, the acquired images are pixel-level aligned with the original images before repair to obtain a differential image of the area to be evaluated.

[0088] Defect feature parameters of the region to be evaluated are extracted from the difference image. The feature parameters corresponding to different defect types are as follows: For coating peeling defects, extract parameters such as the percentage of remaining coating area, coating thickness uniformity, and edge smoothness; For surface crack defects, extract parameters such as remaining crack length, maximum crack width, and crack depth; For leading-edge corrosion defects, extract the percentage of corrosion residue area and surface roughness parameters; For lightning strike damage defects, the percentage of residual damage area and surface smoothness parameters are extracted.

[0089] Based on the extracted defect feature parameters, a comprehensive maintenance quality score is constructed. The formula for calculating the comprehensive maintenance quality score is as follows: ,in, The overall score for maintenance quality ranges from 0 to 1, with a higher score indicating better maintenance quality. This represents the number of feature parameters corresponding to this defect type; For the first The weight coefficients of each feature parameter are given, and the sum of the weight coefficients of all feature parameters is 1. For the first Measured values ​​of each characteristic parameter; For the first The maximum allowed value for each feature parameter.

[0090] The characteristic parameter weighting coefficients and maximum allowable values ​​for different defect types are pre-stored in the robot's local storage module and can be adjusted according to the operation and maintenance standards of different wind farms.

[0091] S502. If the overall inspection quality score is lower than the preset inspection accuracy threshold, the inspection is deemed unqualified and the type of unqualified defect and the coordinates of the unqualified area are determined. The embedded controller compares the calculated overall maintenance quality score with a preset maintenance accuracy threshold. When the overall maintenance quality score is greater than or equal to the preset maintenance accuracy threshold, the maintenance of that area is deemed qualified, the global defect map is updated, and the area is marked as having completed maintenance.

[0092] When the overall inspection quality score is lower than the preset inspection accuracy threshold, the inspection of that area is deemed unqualified. At this time, the embedded controller reviews the original defect detection results for that area to determine the specific type of the unqualified defect. Simultaneously, based on the bounding box coordinates of the defect residue area in the differential image, combined with the intrinsic parameter matrix of the depth camera and the hand-eye calibration matrix, the three-dimensional spatial coordinates of the center of the unqualified area in the robot coordinate system are calculated.

[0093] If there are multiple types of non-conforming defects in the same area, calculate the comprehensive maintenance quality score for each type of defect separately, take the lowest score as the final comprehensive maintenance quality score for the area, and record the type of all non-conforming defects and the corresponding area coordinates.

[0094] S503. Update the global defect map and mark the defective areas as areas to be repaired; The global defect map is a thematic map built on a global grid map. Each grid cell stores the corresponding defect type, maintenance status, number of repairs, and maintenance quality score. The maintenance status is divided into four types: not under maintenance, under maintenance, maintenance passed, and awaiting repair.

[0095] When a certain area is determined to be unqualified during maintenance, the embedded controller calculates its corresponding grid coordinates in the global defect map based on the three-dimensional spatial coordinates of the unqualified area. The maintenance status of the corresponding grid is updated to "to be repaired," and the type of unqualified defect, the coordinates of the unqualified area, and the quality score of this maintenance are recorded.

[0096] This step enables unified global management of maintenance quality information, providing accurate location and defect information for subsequent automatic repair operations, and avoiding omissions and duplications in repair areas.

[0097] S504. Generate corresponding differentiated repair instructions based on the type of non-conforming defect, and output them to the multi-degree-of-freedom robotic arm and the processing arm to control them to perform repair operations on the area to be repaired.

[0098] The embedded controller retrieves the corresponding standard repair process parameters from the local storage module based on the type of defect. The standard repair process parameters for different defect types are as follows: For coating peeling defects, the repair process parameters include sanding depth, sanding speed, spraying thickness and spraying times; For surface crack defects, the repair process parameters include grinding depth, grinding range, amount of filler material and curing time; For leading-edge corrosion defects, the repair process parameters include grinding depth, grinding speed, anti-corrosion coating thickness, and topcoat thickness. For lightning strike damage defects, the repair process parameters include grinding depth, grinding range, resin filling amount, and curing time.

[0099] Based on the retrieved repair process parameters, differentiated repair instructions are generated. These instructions include the target motion trajectory of the five-bar linkage, the feed rate of the UBNS type ball spline screw, its rotational speed, and the operation time. The embedded controller outputs the repair instructions to the multi-degree-of-freedom robotic arm and the processing arm, controlling them to perform repair operations on the area to be repaired according to the preset process parameters.

[0100] After the repair work is completed, the above-mentioned repair quality assessment process is repeated to conduct a second quality assessment of the repaired area. If the second assessment result is still unsatisfactory, the repair work continues. When the number of consecutive repairs reaches the preset maximum number of repairs, an anomaly reporting command is generated and output to the ground control terminal, and the area is marked as an area that cannot be repaired autonomously.

[0101] Example 2 Reference Figure 5 A control system for a wind turbine blade operation robot includes: The path planning module is configured to acquire a three-dimensional point cloud model of the target blade and generate a global walking path covering the target working area based on the three-dimensional point cloud model. The walking and positioning module is configured to acquire images of the blade surface along the global walking path, use a first neural network model to detect footing points on the blade surface images, verify the footing points detected by the first neural network model based on the three-dimensional scene information of the blade, and calculate the foot pose for the verified footing points. The inspection and positioning module is configured to use a second neural network model to detect defects in the surface image of the blade, and obtain the target posture of the multi-degree-of-freedom robotic arm based on the detection results.

[0102] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0104] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0105] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0106] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0107] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the scope of the invention. The spirit and scope of the invention. Thus, if these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A control method for a wind turbine blade operation robot, characterized in that, The robot includes a walking mechanism and a maintenance mechanism located on the working side of the walking mechanism; The walking mechanism includes a left leg and a right leg. Each of the left leg and the right leg includes at least one joint and a suction cup that is attached to the surface of the wind turbine blade. The suction cup is equipped with a negative pressure mechanism. The maintenance mechanism includes a multi-degree-of-freedom robotic arm and a processing arm mounted on the multi-degree-of-freedom robotic arm. The method includes: Obtain a three-dimensional point cloud model of the target blade, and generate a global travel path covering the target working area based on the three-dimensional point cloud model; The blade surface image on the global walking path is acquired, and the first neural network model is used to detect the landing point on the blade surface image. Based on the three-dimensional scene information of the blade, the landing point detected by the first neural network model is verified, and the foot pose is calculated for the verified landing point. The second neural network model is used to detect defects on the surface image of the blade, and the target posture of the multi-degree-of-freedom robotic arm is obtained based on the detection results.

2. The wind turbine blade operation robot control method according to claim 1, characterized in that, The process of obtaining the three-dimensional point cloud model of the target blade includes: Obtain the original 3D point cloud data of the target blade; The original 3D point cloud data is subjected to denoising and surface reconstruction preprocessing operations to generate a 3D point cloud model; A global walking path covering the target work area is generated based on the three-dimensional point cloud model.

3. The wind turbine blade operation robot control method according to claim 1, characterized in that, The step of detecting the landing point of the blade surface image using the first neural network model includes: The image of the blade surface is input into the pre-trained first neural network model; The first neural network model is used to extract planar region features and texture features from the image. Output the pixel coordinates and confidence scores of multiple candidate landing points, and filter out the candidate landing points with confidence scores greater than the preset confidence threshold as landing points to be verified.

4. The wind turbine blade operation robot control method according to claim 3, characterized in that, The verification of the landing point detected by the first neural network model based on the three-dimensional scene information of the blade includes: Acquire 3D point cloud data around the landing point to be verified, and construct a local 3D scene model; Calculate the local flatness and local tilt angle for each landing point to be verified; The comprehensive score of the landing point is calculated based on the local flatness and local tilt angle. Select the landing points that have a comprehensive score greater than the preset score threshold as the landing points to be verified.

5. The wind turbine blade operation robot control method according to claim 4, characterized in that, The calculation of foot pose for verified valid landing points includes: Convert the pixel coordinates of the verified landing point into three-dimensional spatial coordinates in the robot coordinate system; Calculate the target pose of the corresponding foot based on the three-dimensional spatial coordinates and the local tilt angle at the corresponding position; A kinematic model of the walking mechanism is established, and the target angles of each joint of the left or right leg are calculated by inverse kinematics to generate joint control commands.

6. The wind turbine blade operation robot control method according to claim 1, characterized in that, The defect target detection of the blade surface image using the second neural network model includes: The image of the blade surface is input into the pre-trained second neural network model; The second neural network model identifies defective targets in the image and outputs the bounding box, defect type, and defect confidence level of the defective targets. Defect targets with a defect confidence level greater than a preset defect confidence level threshold are selected as defect targets to be processed.

7. The wind turbine blade operation robot control method according to claim 6, characterized in that, The process of obtaining the target pose of the multi-degree-of-freedom robotic arm based on the detection results includes: Obtain the depth data corresponding to the defect target to be processed, and calculate the three-dimensional spatial coordinates of the center of the defect target; Based on the three-dimensional spatial coordinates of the defect target and the defect type, determine the working reference point and working range of the processing arm; Based on the working reference point and working range of the processing arm, the target pose of the end effector of the multi-degree-of-freedom robotic arm is determined; A kinematic model of the multi-degree-of-freedom robotic arm is established, and the target angles of each joint of the multi-degree-of-freedom robotic arm are calculated by inverse kinematics to generate control commands for the robotic arm.

8. The wind turbine blade operation robot control method according to claim 1, characterized in that, The method further includes: Obtain the suction force data of the suction cups on the left and right legs; When the suction force of the single leg currently in the support phase is detected to be lower than the safety threshold, a braking command is generated to maintain the current posture of both feet and prohibit the swing phase movement. The feasible region map is updated based on the current adsorption location offset, and the neighborhood range of points with abnormal adsorption force is removed. Replan the landing point coordinates and foot posture for the next cycle within the updated feasible domain map.

9. The wind turbine blade operation robot control method according to claim 1, characterized in that, The method further includes: Acquire real-time blade surface image data of the maintenance area, extract defect feature parameters of the area to be evaluated, and construct a comprehensive maintenance quality score based on the defect feature parameters; If the overall inspection quality score is lower than the preset inspection accuracy threshold, the inspection is deemed unqualified and the type of unqualified defect and the coordinates of the unqualified area are determined. Update the global defect map and mark the defective areas as areas to be repaired; Based on the type of non-conforming defect, a corresponding differentiated repair instruction is generated and output to the multi-degree-of-freedom robotic arm and the processing arm to control them to perform repair operations on the area to be repaired.

10. A control system for a wind turbine blade operation robot, characterized in that, The method according to any one of claims 1-9 comprises: The path planning module is configured to acquire a three-dimensional point cloud model of the target blade and generate a global walking path covering the target working area based on the three-dimensional point cloud model. The walking and positioning module is configured to acquire images of the blade surface along the global walking path, use a first neural network model to detect footing points on the blade surface images, verify the footing points detected by the first neural network model based on the three-dimensional scene information of the blade, and calculate the foot pose for the verified footing points. The inspection and positioning module is configured to use a second neural network model to detect defects in the surface image of the blade, and obtain the target posture of the multi-degree-of-freedom robotic arm based on the detection results.

Images