An operating robot control system and method for wind power bolt fastening operation

Abstract

The application provides an operating robot control system and method for wind power bolt fastening operation, relates to the technical field of robot control, and the system comprises a triggering unit that reads bolt passport data; a far-field capturing unit that controls 2D vision to perform image acquisition and fuzzy positioning, performs reacquisition according to fuzzy positioning, and controls an operating robot to track feeding; a near-field calibration unit that performs compliant pose correction of contact force data when tracking feeding error satisfies a preset threshold; an inspection unit that performs fastening testing, and performs fastening inspection by using test feedback and bolt passport data; and an adaptive control unit that establishes an adaptive calibration strategy and performs bolt fastening operation if the fastening inspection result is a pass result. The application can solve the technical problem that, due to insufficient positioning accuracy and inaccurate pose adjustment of an operating robot, fastening operation cannot be accurately completed, thereby affecting fastening quality in the prior art.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and in particular to a control system and method for an operating robot used in wind power bolt tightening operations. Background Technology

[0002] In the operation and maintenance of large wind power equipment, bolt tightening is a crucial and labor-intensive task, leading to the increasing use of operating robots. However, in practical applications, significant manufacturing and installation tolerances in large structural components, as well as complex factors such as varying ambient lighting, bolt corrosion, or oil contamination, make it difficult for the robot's vision system to achieve precise initial bolt positioning. Existing robot control methods largely rely on preset programs or single sensor information, lacking the ability to actively perceive and compensate for the relative posture of the sleeve and bolt head during the tightening process. This results in the robot's end effector failing to accurately align the bolt head, and even collisions and stripping during the contact phase. This not only severely impacts the efficiency of the tightening operation but may also lead to inaccurate applied torque values, failing to meet the designed preload requirements, thus posing a potential safety hazard to the entire structure.

[0003] In summary, existing technologies suffer from technical problems such as insufficient positioning accuracy and inaccurate posture adjustment of the operating robot, which prevent the robot from accurately reaching the bolt position and completing the tightening operation, further affecting the tightening quality and work efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a control system and method for a robot used in wind power bolt tightening operations, in order to solve the technical problems in the prior art where insufficient positioning accuracy and inaccurate posture adjustment of the robot prevent the robot from accurately reaching the bolt position and completing the tightening operation, which further affects the tightening quality and work efficiency.

[0005] In view of the above problems, this application provides a control system and method for an operating robot used in wind power bolt tightening operations.

[0006] In a first aspect, this application provides a control system for an operating robot used in wind power bolt tightening operations. The control system includes: a triggering unit, used to trigger the operating robot to read bolt passport data after detecting that the operating robot has reached the target workstation position; a far-field acquisition unit, used to control 2D vision to perform image acquisition based on the bolt passport data, perform fuzzy positioning using the image acquisition results, and then perform point cloud and 2D vision re-acquisition based on the fuzzy positioning, using the re-acquisition results to control the operating robot's tracking feed; a near-field calibration unit, used to read the contact force data between the electric sleeve and the bolt head when the tracking feed error of the far-field acquisition unit meets a preset threshold, perform confidence-driven compliant posture correction based on the contact force data, and perform compliant posture correction feed; an inspection unit, used to perform a tightening test under a first torque, establish test feedback, perform a tightening inspection using the test feedback and the bolt passport data, and output a tightening inspection result; and an adaptive control unit, used to establish an adaptive calibration strategy based on the test feedback and the bolt passport data if the tightening inspection result is a pass result, and then perform the bolt tightening operation.

[0007] Optionally, the bolt passport data is a bolt digital passport, which is read by scanning the bolt identification code using an operating robot. The bolt passport data includes bolt specifications, bolt grade, target preload, and recent operation records.

[0008] Optionally, the contour extraction subunit is used to extract contours from the image acquisition results and establish contour extraction results; the probability detection subunit is used to perform probability detection of closed contours on the contour extraction results and establish a pre-selected target set using the probability detection results; the fuzzy positioning subunit is used to perform target credibility verification of the pre-selected target set according to the bolt specifications in the bolt passport data, complete fuzzy positioning using the target credibility verification results, and obtain relative fuzzy coordinates; and the re-acquisition subunit is used to perform point cloud and 2D vision re-acquisition based on the relative fuzzy coordinates.

[0009] Optionally, a data preprocessing subunit is used to perform data preprocessing on the image data and point cloud data in the reacquisition results; an initial value establishment subunit is used to extract calibration three-dimensional feature points based on the bolt passport data, use the calibration three-dimensional feature points and 2D corner points in the image data to perform RANSAC-PnP to obtain the initial visual pose value, and output the reprojection error and inlier rate. If the calibration three-dimensional feature points do not exist, geometric fitting and backprojection are performed based on the hole contour in the image data to establish the initial visual pose value; a plane fitting subunit is used to fit a plane on the flange surface using the point cloud data, detect the hole boundary, and use the hole boundary to fit the plane. The results are as follows: Initial point cloud pose values ​​are constructed, and the point cloud fitting residual and inlier rate are calculated. An iterative processing subunit performs weighted fine registration iterative processing based on the initial visual pose values, the initial point cloud pose values, the reprojection error and inlier rate, the point cloud fitting residual, and the inlier rate, establishing the iterative processing results. Both the initial visual pose values ​​and the initial point cloud pose values ​​are represented in the end-effector coordinate system of the operating robot, and the transformation from the acquired coordinate system to the end-effector coordinate system is completed through a hand-eye calibration matrix. A weight reconstruction subunit updates the visual quality and point cloud quality using the iterative processing results, reconstructs the pose weighting weights, and controls the tracking feed of the operating robot based on the reconstructed pose weighted fusion result.

[0010] Optionally, the confidence weight calculation channel is used to calculate the confidence weights of the initial visual pose value and the initial point cloud pose value using the reprojection error and the inlier rate, the point cloud fitting residual, and the inlier rate, respectively. The initial registration value is then established by weighted interpolation of the initial visual pose value and the initial point cloud pose value using these confidence weights. The point-to-surface registration channel is used to perform point-to-surface registration at multiple resolutions using the initial registration value, and a visual prior regularization term is introduced into the registration objective function based on visual confidence. The convergence condition determination channel is used to determine the convergence condition for the point-to-surface registration using the RMS residual threshold, the pose update threshold, or the maximum number of iterations. The strategy activation channel is used to activate a backoff strategy and perform data re-acquisition of the re-acquisition result if the point-to-surface registration result is divergent.

[0011] Optionally, the adaptive filtering subunit is used to perform adaptive filtering on the contact force data under a sliding window to remove transient noise and establish a smooth force set; the mapping subunit is used to establish a force-displacement mapping based on the smooth force set using the end-effector compliance model; and the compliant pose correction feed subunit is used to configure the flexible pose feed parameters based on the force confidence and force-displacement mapping of the smooth force set to complete the compliant pose correction feed.

[0012] Optionally, the test feedback subunit is used when the first torque is a low test torque, and performs initial force extraction and continuous force trend analysis on the test feedback; the first verification subunit is used to perform initial force trust verification using the initial force extraction result and bolt passport data, and establish a first verification result; the second verification subunit is used to generate a second verification result using the continuous force trend analysis; and the inspection result output subunit is used to output a fastening inspection result based on the first verification result and the second verification result.

[0013] Optionally, the result failure subunit is used to generate a test instruction if the fastening inspection result is a failure result; the bolt test subunit is used to perform bolt testing according to the test instruction and to perform anomaly identification using the bolt test results, the anomaly identification including slippage identification, misaligned teeth identification, burr anomaly identification, and adhesion identification.

[0014] Optionally, a current state data establishment subunit is used to establish the current state data of the bolt based on the test feedback; a target preload acquisition subunit is used to acquire the target preload based on the bolt passport data, and use the current state data and the target preload as input data; an adaptive calibration subunit is used to configure posterior parameters based on the most recent work record in the bolt passport data, use the posterior parameters to set an adaptive calibration model, input the input data into the adaptive calibration model, and output an adaptive calibration strategy.

[0015] Optionally, a feedback torque monitoring subunit is used to perform feedback torque monitoring during the bolt tightening operation and establish a sudden change identification anomaly; an operation anomaly reporting subunit is used to report operation anomalies based on the sudden change identification anomaly and send a manual intervention request.

[0016] Optionally, the early warning unit is used to perform early warning level matching, establish an early warning signal, and issue an early warning if the fastening inspection result output by the inspection unit is a failure result.

[0017] Optionally, the planning unit is used to acquire a set of bolt tightening tasks, perform bolt distribution analysis based on the set of bolt tightening tasks, perform bolt operation path optimization, establish a target optimization path, and control the robot to move towards the target workstation position based on the target optimization path.

[0018] Secondly, this application also provides a control method for an operating robot used in wind power bolt tightening operations. The method includes: when the operating robot reaches the target workstation position, triggering the operating robot to read bolt passport data; controlling 2D vision to perform image acquisition based on the bolt passport data; performing fuzzy positioning using the image acquisition results; performing point cloud and 2D vision re-acquisition based on the fuzzy positioning; controlling the operating robot's tracking feed using the re-acquisition results; when the tracking feed error of the far-field acquisition unit meets a preset threshold, reading the contact force data between the electric sleeve and the bolt head; performing confidence-driven compliant posture correction based on the contact force data; performing compliant posture correction feed; performing a tightening test under a first torque; establishing test feedback; performing a tightening inspection using the test feedback and the bolt passport data; and outputting the tightening inspection result; if the tightening inspection result is a pass result, establishing an adaptive calibration strategy based on the test feedback and the bolt passport data; and performing the bolt tightening operation.

[0019] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0020] By reading bolt passport data, 2D vision is guided to perform initial image acquisition and fuzzy positioning. Based on the fuzzy positioning results, more precise point cloud and 2D vision re-acquisition is triggered, achieving accurate coordinate mapping and tracking feed from the far field to the near field. Contact force data is introduced for confidence-driven compliant pose correction, fine-tuning the robot's posture in real time. Before formal tightening, a tightening test under the first torque is performed and test feedback is established. Tightening inspection is conducted based on the test feedback and bolt passport data to determine whether the tightening process under the current working conditions is normal. Only after the inspection is passed is the bolt tightening operation performed according to the adaptive calibration strategy, ensuring that the tightening quality of each bolt meets the requirements, improving the efficiency and accuracy of the tightening operation, and ensuring the tightening quality.

[0021] The above description is merely an overview of the technical solution of this application. To better understand the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the control system for an operating robot used in wind power bolt tightening operations, as described in this application.

[0024] Figure 2 This is a flowchart illustrating a control method for an operating robot used in wind turbine bolt tightening operations, as described in this application.

[0025] Explanation of reference numerals in the attached figures: Trigger unit 11, Far-field acquisition unit 12, Near-field calibration unit 13, Verification unit 14, Adaptive control unit 15. Detailed Implementation

[0026] This application provides a control system and method for a manipulator robot used in wind turbine bolt tightening operations. It solves the technical problem in existing technologies where insufficient positioning accuracy and inaccurate posture adjustment of the manipulator robot prevent it from accurately reaching the bolt position and completing the tightening operation, further affecting tightening quality and work efficiency. By reading bolt passport data, a 2D vision system is used for initial fuzzy positioning. Precise tracking feed is performed based on point cloud and 2D vision data. Confidence-driven compliant posture correction is performed using contact force data. Test feedback is established through tightening tests, and tightening inspection is performed based on the bolt passport data. Once the inspection is passed, the bolt tightening operation is performed according to an adaptive calibration strategy, improving the efficiency and accuracy of the tightening operation and ensuring tightening quality.

[0027] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. It should be understood that this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. It should also be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all of them.

[0028] Example 1, please refer to the appendix. Figure 1This application provides a control system for a robot used in wind turbine bolt tightening operations. The control system is used to implement a robot control method for wind turbine bolt tightening operations. The control system includes:

[0029] Trigger unit 11 is used to trigger the robot to read bolt passport data when the detection robot reaches the target workstation position.

[0030] Furthermore, the trigger unit 11 in the operating robot control system for wind power bolt tightening operations is also used to: the bolt passport data is a bolt digital passport, which is read by the operating robot scanning the bolt identification code. The bolt passport data includes bolt specifications, bolt grade, target preload, and recent operation records.

[0031] Specifically, the robot uses sensors (such as LiDAR and vision sensors) to locate itself, ensuring it accurately reaches the target workstation where the bolt is located. Once the robot is detected to have reached the target workstation location according to the planned path, it is considered to have reached the designated specific location, usually the location of the bolt or the work area.

[0032] The operating robot scans bolt identification codes (such as QR codes or barcodes) to read bolt passport data, or the bolt's digital passport, which includes the bolt's specifications, grade, target preload torque, and recent operation records. Each bolt's identification code is unique, ensuring the operating robot can read the precise information corresponding to a specific bolt. Bolt passport data is a digital record, similar to an ID card for each bolt, containing all the bolt's important information, including specifications, grade, target preload value, and recent operation records. Bolt specifications include basic parameters such as bolt size and type; bolt grade indicates the bolt's quality and performance level; target preload is the preload force the bolt needs to achieve during tightening to ensure structural stability; recent operation records include data from the bolt's last tightening operation, such as tightening torque and tightening time, used to track the bolt's usage status. For example, suppose the data reads that the bolt specification is M20×150 (meaning a bolt with a diameter of 20 mm and a length of 150 mm), the bolt grade is 10.9 (meaning the bolt's strength grade is 10.9, suitable for high-load applications), the target preload torque is 300 N·m (meaning the target preload torque is 300 N·m), and the most recent job record shows the last job date was May 15th, the job type was tightening, and the tightening torque was 290 N·m. By reading the bolt passport data, the operating robot can perform tightening operations according to the specific requirements of each bolt, ensuring the personalization and accuracy of the tightening operation.

[0033] The far-field acquisition unit 12 is used to control 2D vision to perform image acquisition based on bolt passport data, perform fuzzy positioning using the image acquisition results, perform point cloud and 2D vision re-acquisition based on the fuzzy positioning, and control the operation robot to track and feed using the re-acquisition results.

[0034] Furthermore, the far-field acquisition unit 12 in the control system of the operating robot for wind power bolt tightening operation is also used for: a contour extraction subunit, used to extract contours from the image acquisition results and establish contour extraction results; a probability detection subunit, used to perform probability detection of closed contours from the contour extraction results and establish a pre-selected target set using the probability detection results; a fuzzy positioning subunit, used to perform target credibility verification of the pre-selected target set according to the bolt specifications in the bolt passport data, use the target credibility verification results to complete fuzzy positioning, and obtain relative fuzzy coordinates; and a re-acquisition subunit, used to perform point cloud and 2D vision re-acquisition based on the relative fuzzy coordinates.

[0035] Specifically, based on the read bolt passport data, 2D vision is initiated for image acquisition, capturing two-dimensional image data of the target bolt to obtain image acquisition results, including information such as the bolt's shape, size, and position. Contour extraction is then performed on the image acquisition results, extracting the outer edges or contours of the target bolt to identify its shape. Image processing algorithms (such as Canny edge detection, Sobel operator, etc.) are used to process the image acquisition results, extracting the bolt's edge contours and identifying its outer edges or contours. For example, in the image acquisition results, the bolt's shape appears as a circle (approximately 20mm in diameter); the circular edge of the bolt is extracted, identifying its approximate shape. Once contour extraction is complete, a contour extraction result is generated, which is typically a data structure representing the object's edge. For example, the bolt's edge might be represented as a series of coordinate points (each point representing a portion of the contour), or presented as a polygon or closed curve. The contour extraction result can be a set of points, where each point represents a portion of the bolt's outer contour.

[0036] The extracted contours undergo probability detection to confirm whether they are complete closed contours. A closed contour is defined as an object whose edges form a closed loop without any openings or gaps. For objects like bolts, a closed contour typically appears as a complete boundary, which may be hexagonal, circular, or other geometric shapes. Closure detection is performed by analyzing the geometry of the contour. If the contour forms a closed loop, it is considered a closed contour; if the contour has gaps or breaks, it is considered an incomplete contour.

[0037] Probabilistic detection algorithms are used to evaluate the closure of each profile, typically based on calculating the geometric relationships between the profile's parts to assess whether they smoothly connect to form a closed loop. The algorithm checks for openings or errors in the profile that prevent complete closure. By calculating the probability values ​​of these degrees of closure, it determines whether each profile conforms to the true shape of the bolt head. If the profile is circular or hexagonal, the closure level is very high, with a probability value close to 1; conversely, if the profile has gaps or is incomplete, the probability value is lower. A high closure level (close to 1) indicates a high degree of confidence in the profile.

[0038] Based on the probability detection results, high-probability closed contour regions are selected as the pre-selected target set, considered as possible locations of the bolt. For example, if five contours are extracted, after probability detection, only three contours have a probability value greater than a set threshold (e.g., 0.9). These three contours constitute the pre-selected target set, while the others are excluded. The pre-selected target set refers to the set of contour regions that meet certain closure requirements after probability detection, representing potential target bolt locations. For example, if after image acquisition and contour extraction, multiple candidate targets are obtained: contour 1 is a hexagon with a diameter of 22mm and a closure probability of 0.98; contour 2 is a circle with a side length of 22mm and a closure probability of 0.95; contour 3 is a hexagon with a side length of 18mm and a closure probability of 0.85; and contour 4 is a circle with a diameter of 20mm and a closure probability of 0.97, contour 1, contour 2, and contour 4 are selected as the pre-selected target set based on the set closure probability threshold (0.9).

[0039] The target credibility verification verifies whether the targets in the pre-selected target set are indeed bolts based on the bolt specifications in the bolt passport data. For example, if the bolt specification is M20×150 (diameter 20mm, length 150mm), the size and proportion of the target contour are compared to confirm whether the target conforms to the actual bolt specifications. Target credibility verification is the process of verifying candidate targets in the pre-selected target set based on the bolt specifications (such as size and shape) in the bolt passport data, used to determine whether these contour areas truly correspond to the shape of a bolt. For example, using bolt passport data (specification M20×150, target diameter 20mm) to verify the pre-selected target set: contours 1 and 4 conform to the specifications and pass the verification; contour 2 is circular with a diameter of 22mm, which does not conform to the bolt shank specifications and is excluded. After verification, contours 1 and 4 are considered valid bolt targets.

[0040] After completing target credibility verification, fuzzy localization technology is used to estimate the approximate position of the bolt. Fuzzy localization is typically based on image data, sensor data, and preliminary target verification information to provide approximate target position coordinates, i.e., relative fuzzy coordinates. For example, after verification, the robot knows that the center of the bolt is approximately located at image coordinates (x=100±5mm, y=150±5mm). Based on the relative fuzzy coordinates, re-acquisition results are obtained through point cloud and 2D vision. That is, the robot moves to a position close to the bolt based on the relative fuzzy coordinates, then activates the point cloud acquisition device and 2D vision system for re-acquisition, obtaining re-acquisition data to determine the target's position and orientation. Through 2D vision image acquisition, contour extraction, probability detection, target credibility verification, and re-acquisition of point cloud and 2D vision, the robot can achieve precise bolt localization. Target credibility verification combined with bolt passport data helps eliminate false identifications and improves localization accuracy.

[0041] Furthermore, the far-field acquisition unit 12 in the control system of the operating robot for wind power bolt tightening operations is also used for: a data preprocessing subunit, used to perform data preprocessing on the image data and point cloud data in the reacquisition results; an initial value establishment subunit, used to extract calibration three-dimensional feature points based on the bolt passport data, use the calibration three-dimensional feature points and 2D corner points in the image data to perform RANSAC-PnP to obtain the initial visual pose value, and output the reprojection error and inlier rate; if the calibration three-dimensional feature points do not exist, then perform geometric fitting and backprojection based on the hole contour in the image data to establish the initial visual pose value; and a plane fitting subunit, used to fit the point cloud data onto the flange surface. The system employs a plane fitting method and detects the aperture boundary. It uses the aperture boundary fitting results to construct initial point cloud pose values ​​and calculates the point cloud fitting residual and inlier rate. An iterative processing subunit performs weighted fine registration iterative processing based on the initial visual pose value, the initial point cloud pose value, reprojection error and inlier rate, and the point cloud fitting residual and inlier rate. The iterative processing result is established, with both the initial visual pose value and the initial point cloud pose value represented in the end-effector coordinate system of the operating robot. The transformation from the acquired coordinate system to the end-effector coordinate system is completed through a hand-eye calibration matrix. A weight reconstruction subunit updates the visual quality and point cloud quality using the iterative processing result, reconstructs the pose weighting weights, and controls the tracking feed of the operating robot based on the reconstructed pose weighted fusion result.

[0042] Furthermore, the far-field acquisition unit 12 in the control system of the operating robot for wind power bolt tightening operation is also used for: a confidence weight calculation channel, used to calculate the confidence weights of the initial visual pose value and the initial point cloud pose value using the reprojection error and the inlier rate, the point cloud fitting residual and the inlier rate respectively, and to use the confidence weights to perform weighted interpolation of the initial visual pose value and the initial point cloud pose value to establish the initial registration value; a point-to-surface registration channel, used to perform point-to-surface registration under multiple resolutions using the initial registration value, and to introduce a visual prior regularization term in the registration objective function of the registration process based on visual confidence; a convergence condition determination channel, used for the point-to-surface registration to use the RMS residual threshold, the pose update threshold or the maximum number of iterations as the convergence condition; and a strategy activation channel, used to activate the backoff strategy and perform data re-acquisition of the re-acquisition result if the point-to-surface registration result is a divergent result.

[0043] Specifically, for the image and point cloud data in the reacquisition results, data preprocessing is performed, including denoising, filtering, and coordinate normalization, with the aim of improving data quality. Image data can be denoised to remove background noise, and point cloud data can be filtered to remove outliers, thus improving subsequent registration and positioning accuracy. Calibration 3D feature points are extracted from the bolt passport data—that is, the known 3D feature points of the bolts extracted from the bolt passport data—and used to match them with 2D corner points in the image data to calculate the visual pose. 2D corner points are feature points extracted from the image, typically points with significant shape changes (such as corners or edge intersections of objects).

[0044] Using extracted calibrated 3D feature points and 2D corner points from image data, the RANSAC-PnP algorithm is used to calculate the initial visual pose, outputting the reprojection error and inlier rate. RANSAC-PnP combines the RANSAC and PnP algorithms. RANSAC (Random Sample Consensus) is a method commonly used to estimate parameters from data containing outliers, while PnP is a method for calculating camera pose (position and orientation). PnP solves pose estimation but is susceptible to outliers, while RANSAC is more robust to noise, thus complementing each other. Reacquiring point clouds and 2D vision generates 3D-2D point correspondences, which may contain erroneous matches. RANSAC-PnP filters out these errors, improving localization accuracy. Given the 3D point coordinates and their coordinates on the new image plane, solving for the new camera extrinsic parameters is a PnP problem. Three points are sampled each time to obtain the camera pose. The remaining points are projected onto the image plane using the calculated camera pose, and the reprojection error is calculated. Inliers are identified by analyzing the error, and their count is determined. Through iterative sampling and optimization, the maximum number of inliers is reached. These inliers are then used to calculate the camera pose. The reprojection error represents the deviation between the estimated pose and the actual pose, while the inlier ratio represents the proportion of points that match the estimated pose. A high inlier ratio and a low reprojection error generally indicate more accurate pose estimation.

[0045] If calibrated 3D feature points are unavailable, meaning that pose estimation cannot be performed using predefined calibrated 3D feature points in some cases, it may be because the calibration points are not correctly matched or cannot be obtained. In this case, estimation needs to be performed using other available image features (such as hole contours). Hole contours refer to the boundary lines of bolt holes in an image, usually extracted from the image using image processing techniques (such as edge detection or contour extraction). After extracting the hole contours from the image, geometric fitting methods are used to fit these contours. Assuming the target hole is circular, a circular fitting algorithm (such as least squares) is used to obtain the geometric parameters of the hole contour (such as radius, center coordinates, etc.). If the hole contour is not a perfect circle, elliptical fitting or other shape geometric fitting methods are used. After fitting the geometry of the hole, the 2D hole contour in the image is mapped to 3D space using backprojection techniques. The backprojection process involves mapping the circular or elliptical contour of the hole extracted from the image to its position in 3D space through a camera model. Using known camera intrinsic parameters (such as focal length, optical center, etc.), the fitted hole contour is backprojected to calculate the approximate position of the bolt hole, thereby determining the initial value of the visual pose. The three-dimensional coordinates obtained by backprojection provide the spatial position of the target hole, which can then be used to calculate the initial pose of the target object (such as a bolt), i.e., based on the hole's contour and the backprojection results. If the initial visual pose value obtained through geometric fitting and backprojection has a large error, it is further corrected by combining other data sources (such as point cloud data, sensor information, etc.).

[0046] Using point cloud data, a plane is fitted onto the flange surface. The flange surface is typically a planar region in mechanical components, often used to connect different parts; therefore, in point cloud data, it appears as a set of points approximating a plane. In point cloud data processing, the flange surface is an important geometric feature; fitting the flange surface helps identify the relative positions of bolts. Fitting a plane involves using mathematical methods (such as least squares) to fit points in the point cloud data to obtain an optimal plane equation, commonly used to identify planar regions (such as flange surfaces) from 3D point clouds. The spatial position and orientation of the flange surface can be determined using the fitted plane equation.

[0047] The boundary of a bolt hole is its outer contour, typically circular or elliptical. In point cloud data, boundary detection algorithms (such as circular fitting and edge detection) are used to identify and extract the boundary of the bolt hole. By detecting the boundary, the geometry of the bolt hole is determined, enabling precise location of the bolt hole and its geometry. After identifying the boundary, the initial point cloud pose is constructed using the boundary fitting results. That is, a geometric fitting algorithm (such as circular fitting) is used to fit the boundary, obtaining parameters such as the center and radius, to construct the initial point cloud pose. This involves calculating the position and orientation of the bolt hole by fitting a circle or other geometric model. During the fitting process, the point cloud fitting residual is calculated, representing the difference between the fitted model and the point cloud data. A smaller residual indicates a better fit. The inlier ratio is calculated, representing the proportion of points that match the fitted model. A higher inlier ratio indicates a better fit and higher consistency between the model and the point cloud data.

[0048] The confidence weights of the initial visual position values ​​are calculated based on the reprojection error and the inlier rate. A smaller reprojection error and a higher inlier rate indicate higher reliability of the visual pose estimation, thus assigning a higher confidence weight. The confidence weights of the initial point cloud pose values ​​are also calculated based on the point cloud fitting residual and the inlier rate. If the point cloud fitting error is small and the inlier rate is high, the reliability of the initial point cloud pose value is high. For example, assuming that in wind turbine bolt tightening operations, the reprojection error of the calculated initial visual pose value is 0.5 pixels and the inlier rate is 90%, and the fitting residual of the initial point cloud pose value is 1 mm and the inlier rate is 95%, the confidence weights of the initial visual pose value and the initial point cloud pose value are calculated to be 0.9 and 0.95, respectively.

[0049] Based on the calculated confidence weights, weighted interpolation is performed on the initial visual pose value and the initial point cloud pose value to obtain the final initial registration value. The initial registration value is a weighted fusion result of the initial visual pose value and the initial point cloud pose value, providing a comprehensive preliminary pose estimate as the starting point for point cloud and visual registration. Weighted interpolation, based on the confidence weights of different data sources (such as visual and point cloud data), weights and fuses the initial visual pose value and the initial point cloud pose value to obtain a more accurate initial pose value.

[0050] After obtaining the initial registration values, precise registration is performed using a point-to-surface registration method. During registration, each point in the point cloud data is matched with a surface of the target model, minimizing the point-to-surface distance. Point-to-surface registration is a common point cloud registration method where points in the point cloud data are registered with surfaces of the target model. The pose is adjusted by minimizing the point-to-surface distance, thus achieving precise registration. Convergence conditions for the registration process typically include the RMS residual threshold, the pose update threshold, and the maximum number of iterations. The registration process is considered convergent when the registration result meets these conditions. The RMS residual threshold is a standard used to evaluate the residuals (i.e., the distance from a point cloud point to the fitted model) during point cloud registration. When the residual is below a certain threshold, the registration process is considered convergent. During registration, if the change in pose update is less than the pose update threshold, it indicates that registration is complete. The maximum number of iterations is the maximum allowed number of iterations during the registration process; if registration has not converged after reaching this number, it is stopped.

[0051] In point-to-surface registration, pose information from the visual data is used as a visual prior. This prior regularization term is introduced into the registration objective function to improve registration accuracy, especially when the point cloud data is of poor quality or sparse. The visual prior regularization term incorporates prior knowledge of the visual data (such as initial visual pose values) into the point cloud registration objective function. By adding visual priors during the registration process, registration accuracy can be improved, particularly when point cloud data is scarce or incomplete. If convergence fails during registration, i.e., divergence occurs, a backtracking strategy is activated, and the reacquisition results are re-acquired. By re-acquiring image or point cloud data, more effective data is obtained, thereby increasing the probability of successful registration. After point-to-surface registration converges, the visual quality and point cloud quality are updated.

[0052] In each iteration, the visual pose and point cloud pose are finely registered according to weighted values. Through continuous optimization and adjustment, a more accurate registration result is obtained, achieved by minimizing errors and progressively updating the pose. Initial values ​​for both the visual pose and the point cloud pose are represented in the robot's end-effector coordinate system, which is the coordinate system of the robot arm's end effector (such as a gripper or actuator). The hand-eye calibration matrix is ​​used to transform the sensor coordinate system (e.g., camera coordinate system or sensor coordinate system) to the robot's end-effector coordinate system. Hand-eye calibration ensures that the robot's end-effector's operation and perception data are synchronized in the same coordinate system. The hand-eye calibration matrix transforms data collected from the vision system and point cloud system into the robot's end-effector coordinate system, ensuring that all pose estimations and adjustments are performed in the robot's actual operating coordinate system.

[0053] Based on the iterative processing results, the visual quality and point cloud quality are updated. As the iteration progresses, the visual image becomes clearer, and the point cloud data becomes denser and more accurate. Based on the results of the iterative optimization, the weights of the visual and point cloud data are recalculated. High-quality visual and point cloud data will receive higher weights to account for a larger proportion in the final pose estimation. The visual pose and point cloud pose are then weighted and fused according to the updated pose weights to obtain the reconstructed pose weighted fusion result. This result controls the robot to perform tracking feed, ensuring that the robot's end effector accurately aligns with the target position for operations (such as tightening bolts). Tracking feed ensures that the robot accurately tracks the target position during task execution, ensuring the accuracy and efficiency of task completion.

[0054] The near-field calibration unit 13 is used to read the contact force data between the electric sleeve and the bolt head when the tracking feed error of the far-field acquisition unit 12 meets the preset threshold, perform confidence-driven compliant posture correction based on the contact force data, and execute compliant posture correction feed.

[0055] Furthermore, the near-field calibration unit 13 in the control system of the operating robot for wind power bolt tightening operation is also used for: an adaptive filtering subunit, used for adaptive filtering of the contact force data under a sliding window to remove transient noise and establish a smooth force set; a mapping subunit, used for establishing a force-displacement mapping based on the smooth force set using an end-effector compliance model; and a compliant pose correction feed subunit, used for configuring flexible pose feed parameters based on the force confidence and force-displacement mapping of the smooth force set to complete the compliant pose correction feed.

[0056] Specifically, the far-field acquisition unit 12 controls the robot to perform tracking feed, detecting and confirming whether the tracking feed error of the robot's end effector has met a preset threshold. Tracking feed error refers to the difference between the actual position and the target position. Once the error meets the preset threshold, contact force data between the electric sleeve and the bolt head is read from a force sensor or other sensors, reflecting the force applied to the bolt head by the robot's end effector. Contact force data refers to the measured force applied when the robot's end effector contacts the target object (such as the bolt head).

[0057] Adaptive filtering with a sliding window is applied to the read contact force data to remove transient noise and smooth the data. The adaptive filter dynamically adjusts its filtering parameters within the sliding window to adapt to signal changes, ensuring that the smoothed force data better reflects the actual contact force. Sliding window adaptive filtering is a commonly used signal processing technique to remove noise from signals, especially transient noise. By updating the filter coefficients within the sliding window, it can dynamically adapt to signal changes, thereby improving signal stability. After applying adaptive filtering, a smoothed force set is obtained. This smoothed force set is the result of applying adaptive filtering to the contact force data, removing transient noise and retaining the true force signal, thus forming a more stable and continuous force data set.

[0058] Using an end-effector compliance model, a force-displacement mapping is established based on a smoothed force set, describing how applied forces affect the displacement or position of the robot's end effector. The end-effector compliance model describes the mechanical relationship between the robot's end effector and the target object, particularly the interaction between force and displacement. In compliant control, the end effector adaptively adjusts to changes in external forces to maintain proper contact. Based on the force-displacement mapping, appropriate adjustment amounts can be calculated to ensure the end effector can make precise displacement adjustments according to externally applied forces. Force confidence is a measure of the reliability of contact force data; high confidence indicates that the contact force data is stable and accurate, while low confidence indicates that the data may have high uncertainty. Based on the force confidence (i.e., the reliability of the force data) of the smoothed force set and the force-displacement mapping, flexible pose feed parameters are configured for the robot's end effector, determining how the robot adjusts its posture or position according to changes in contact force, ensuring flexible and precise task completion. Flexible pose feed parameters are used to control the motion and attitude adjustment of a robot's end effector under compliant control. Through these parameters, the robot can adaptively adjust the position and attitude of the end effector based on applied force and displacement information. By executing compliant pose correction feed according to the configured parameters, the robot's end effector automatically adjusts itself based on real-time contact force and displacement data to maintain appropriate contact force and position, thereby completing precise tasks such as bolt tightening.

[0059] By establishing adaptive filtering and smoothing force sets, unnecessary noise is removed, making the contact force data more stable and thus improving the accuracy of bolt tightening. Utilizing a compliant model and force-displacement mapping, the robot can automatically adjust the position and orientation of the actuators according to changes in contact force, ensuring accurate task completion under different contact conditions.

[0060] Inspection unit 14 is used to perform a fastening test under a first torque, establish test feedback, perform fastening inspection using the test feedback and the bolt passport data, and output the fastening inspection result.

[0061] Furthermore, the inspection unit 14 in the control system of the operating robot for wind power bolt tightening operations is also used for: a test feedback subunit, used when the first torque is a low test torque, to extract the initial force and perform continuous force trend analysis on the test feedback; a first verification subunit, used to perform initial force trust verification using the initial force extraction result and bolt passport data, and establish a first verification result; a second verification subunit, used to generate a second verification result using the continuous force trend analysis; and an inspection result output subunit, used to output the tightening inspection result based on the first verification result and the second verification result.

[0062] Furthermore, the inspection unit 14 in the operating robot control system for wind power bolt fastening operations is also used for: a result failure subunit, used to generate a test instruction if the fastening inspection result is a failure result; and a bolt testing subunit, used to perform bolt testing according to the test instruction and perform anomaly identification using the bolt test results, wherein the anomaly identification includes slippage identification, misaligned teeth identification, burr anomaly identification, and adhesion identification.

[0063] Specifically, the manipulator automatically adjusts the position and orientation of the end effector based on changes in contact force and displacement to ensure appropriate contact force for task completion (such as bolt tightening). After compliant posture correction feed, the manipulator applies an initial torque, i.e., a low test torque, to perform a preliminary tightening test. This test checks the initial tightness of the bolt, typically applying a small force when the bolt head just begins to contact the surface. The initial torque, a low test torque, is usually used to initially detect and verify the bolt preload. By applying a low torque, the initial tightness of the bolt is checked, and feedback is provided for further operations. During the tightening test, relevant data, such as applied torque, contact force, and displacement, are recorded to form test feedback.

[0064] Extract the initial force data when the first torque is applied from the test feedback, i.e., the magnitude of the force at the start of tightening. Extract the continuous force from the test feedback and perform trend analysis, i.e., analyze the force change trend during the torque application process, such as whether the force is stable, whether it gradually increases, or whether there are fluctuations, to determine whether the bolt is properly tightened and whether problems such as slippage, misalignment, or adhesion have occurred. For example, does the force gradually increase, and are there fluctuations? If the force continues to increase and remains stable, it indicates that the bolt is properly tightened; if the force decreases or there are unstable fluctuations, there may be problems (such as slippage, misalignment, adhesion, burrs, etc.).

[0065] The initial force data is compared with the bolt passport data to confirm whether the initial force is within the expected range. The bolt passport data contains information such as bolt specifications, grade, and target preload. Verification is performed to confirm whether the initial force value meets expectations, and the results are assessed for reliability. If the initial force matches the passport data and meets expectations, the verification is passed, meaning the first verification result is successful.

[0066] By analyzing the continuous trend of force changes, we can determine if there are any abnormal changes. If the force continuously increases and remains stable, it indicates that the bolt is tightened normally. If the force remains constant, decreases, or fluctuates unevenly, it may indicate a problem during the bolt tightening process (such as slippage, misalignment, adhesion, burrs, etc.). Based on the continuous force trend analysis, a second verification result is generated to evaluate the force change trend during the tightening process and help determine if there are any abnormal force changes.

[0067] Based on the first and second verification results, the final tightening inspection result is output to confirm whether the bolt has been properly tightened. If the bolt meets the standard, the tightening is complete; if a problem is detected (such as abnormal force changes), a failed tightening inspection result is generated, indicating that the bolt has not been properly tightened. When a failed tightening inspection result is detected, a set of test instructions is generated, instructing the operating robot to perform further bolt testing. The test instructions include a retest plan for the bolt, which may involve applying different torques, measuring force changes, etc. The purpose of the test instructions is to further determine the reasons for failure during the tightening process and guide the operating robot to perform detailed anomaly diagnosis. For example, suppose the operating robot needs to tighten an M20×150 bolt. The robot precisely aligns with the bolt head through compliant control and performs a preliminary tightening operation. A first torque (e.g., 50 N·m) is applied to perform a preliminary tightening test on the bolt. Feedback data recorded by the force sensor includes: an initial contact force of 5N (torque 0); during torque application, the contact force gradually increases: 10N (torque 5N·m), 25N (torque 15N·m), 45N (torque 40N·m), and 50N (torque 50N·m). The contact force shows a stable increase without significant fluctuations or decreases, indicating stable contact between the bolt and nut and a normal bolt tightening process. The extracted initial force (5N) is compared with the expected initial force range in the bolt passport data. The bolt passport data specifies: target preload 25-30N (standard range); initial force verification range 5-10N (common range during initial force contact). The initial force of 5N is within the expected range, therefore the verification is considered successful. Continuous force trend analysis shows that the force increases steadily during application without fluctuations or decreases. First verification result: the initial force extraction conforms to the bolt passport data, verification successful; Second verification result: the continuous force trend is stable, with no abnormal fluctuations. The tightening inspection result is successful. The bolts were tightened as expected, and no abnormalities were detected.

[0068] The robot operates according to test instructions to perform bolt tests, conducting detailed inspections of the bolts and the tightening process to identify the causes of failures and confirm whether the bolts meet predetermined tightening standards. Bolt testing includes applying different torques, measuring contact forces, and analyzing force changes. Anomalies are identified based on the bolt test results, including slippage, misaligned threads, burrs, and adhesion. During bolt tightening, insufficient friction at the contact surface can cause slippage, which is identified by detecting drops or fluctuations in force. By monitoring force changes, a sudden drop or instability during force increase indicates slippage. When threads are not properly aligned, force cannot be transmitted correctly, potentially resulting in increased torque but no significant force output. If the contact force does not reach the expected value after applying torque, or if the force increases but does not conform to the normal pattern, it may indicate misaligned threads. Burrs on the bolt or nut surface can cause uneven contact forces, interfering with the normal tightening process. Force fluctuations are detected; if the force is uneven during tightening and cannot increase steadily, burrs may be present. Bolts or nuts may stick together due to poor lubrication or corrosion, preventing the force from increasing smoothly. By analyzing the force change trend, if the force value increases suddenly or the rate of increase is uneven, sticking may be present.

[0069] Based on the anomalies identified in the bolt testing results, the system outputs the final anomaly identification results, including slippage, misaligned threads, burrs, and adhesion. Based on the identification results, instructions are issued to the operator to implement corresponding remedial measures, such as retightening, adjusting bolt position, and cleaning the surface. Through initial force extraction and continuous force trend analysis, it accurately determines whether the bolts have been tightened to the predetermined standard. The anomaly identification function can promptly detect problems during the tightening process, such as slippage and misaligned threads, ensuring the reliability of the tightening quality.

[0070] The adaptive control unit 15 is used to establish an adaptive calibration strategy based on the test feedback and the bolt passport data if the fastening test result is a pass result, and then perform the bolt fastening operation.

[0071] Furthermore, the adaptive control unit 15 in the operating robot control system for wind power bolt tightening operations is also configured to: a current state data establishment subunit, for establishing the current state data of the bolt based on the test feedback; a target preload acquisition subunit, for acquiring the target preload based on the bolt passport data, using the current state data and the target preload as input data; and an adaptive calibration subunit, for configuring posterior parameters based on the most recent operation record in the bolt passport data, setting an adaptive calibration model using the posterior parameters, inputting the input data into the adaptive calibration model, and outputting an adaptive calibration strategy.

[0072] Specifically, when the tightening inspection result is "pass," it means the bolt has been tightened to the appropriate preload standard, meeting the design requirements. The process involves obtaining test feedback during bolt tightening, specifically real-time data from sensors measuring force, torque, and displacement, to determine if the tightening operation meets requirements. This real-time test feedback is used to calculate a description of the bolt's tightening state, including applied torque, contact force, displacement, and current tightening progress. For example, the current torque might be 50 N·m, the current contact force 100 N, the current displacement 0.5 mm, and the current feed rate 1 mm / s.

[0073] Based on the bolt passport data, obtain the target preload value for the bolt. For example, the target preload torque is 80 N·m, and the target preload force is 150 N. The target preload is the predetermined tightening force or preload torque that the bolt should achieve, determined according to the bolt specifications and design requirements, to ensure that the bolt has sufficient tightening strength. Based on the most recent operation records in the bolt passport data, such as historical operation records showing the difference between the actual preload force and the target preload force in the past 10 bolt tightening operations, configure posterior parameters. Posterior parameters are parameters that update the bolt tightening process based on historical operation records and test feedback data. Based on past tightening data, they help adapt to different working conditions and improve tightening accuracy. Use the posterior parameters to set up an adaptive calibration model. An adaptive calibration model is a model that dynamically adjusts based on real-time input data (such as current state data, target preload, posterior parameters, etc.) to optimize bolt tightening operations. In other words, based on recent work records, the posterior parameters are updated to adjust the current tightening strategy. For example, if historical data reveals that the applied torque was slightly lower than the target preload in previous operations, the applied torque is slightly increased using the posterior parameters to improve tightening accuracy. Based on historical work records, the applied torque is adjusted from 80 N·m to 90 N·m to compensate for the previous underestimation.

[0074] The current state data and target preload are input into the adaptive calibration model to obtain the adaptive calibration strategy. The current state data (such as the actual applied force and torque) and the target preload (e.g., 80 N·m) are input, adjusted using posterior parameters, and then input into the adaptive calibration model. The adaptive calibration strategy adjusts parameters such as the applied torque, feed rate, and displacement to ensure more precise and efficient bolt tightening. The adaptive calibration strategy is a dynamically generated operation strategy based on the current state data, target preload value, and posterior parameters. It guides the robot in adjusting the applied torque, feed rate, etc., to ensure the bolt is properly tightened. For example, an adaptive calibration strategy might include an applied torque of 90 N·m, a feed rate of 0.9 mm / s, and a target preload of 150 N.

[0075] Through an adaptive calibration strategy, the tightening strategy is adjusted based on real-time test feedback and historical operation records to ensure that bolts meet the precise tightening standards. The adaptive calibration model dynamically adjusts according to actual operation feedback, adapting to different working environments and bolt conditions, thus improving the adaptability and reliability of the operating robot.

[0076] Furthermore, the adaptive control unit 15 in the operating robot control system for wind power bolt tightening operations is also used for: a feedback torque monitoring subunit, used to perform feedback torque monitoring during the bolt tightening operation and establish a sudden change identification anomaly; and an operation anomaly reporting subunit, used to report operation anomalies based on the sudden change identification anomaly and send a request for manual intervention.

[0077] Specifically, during the bolt tightening operation, the robot uses torque sensors to monitor changes in the torque applied to the bolt in real time, tracking the bolt's tightening status and ensuring that the applied force meets predetermined standards. Each time torque is applied, the sensor records the current torque value and sends it to the adaptive control unit for anomaly detection. Under normal circumstances, the torque increases steadily.

[0078] In the monitored torque data, any sudden abrupt change (such as a sudden increase in torque from 70 N·m to 120 N·m or a significant decrease) is identified as an abrupt change anomaly. Such abrupt changes may be caused by slippage, misalignment, burrs, or adhesion. An abrupt change anomaly refers to a sudden and drastic change in torque data during torque monitoring (such as a rapid increase or decrease in torque), indicating an abnormality in the tightening process.

[0079] Based on detected abrupt changes, an anomaly is immediately identified, and a warning is issued, generating an operational anomaly report. The report details the anomaly type (e.g., torque mutation, slippage), and informs the operator of the problem. For example, a warning message might read: "Anomaly detected: torque mutation. Slippage or misalignment may be present. Operator, please check bolt tightness." A manual intervention request is sent to the operator when a problem that cannot be resolved automatically (e.g., anomaly detection) is detected, requesting human intervention to resolve the issue. Manual intervention may include checking bolts, adjusting the tightening process, or correcting operating parameters. Upon receiving a manual intervention request, the operator will check the bolts and nuts according to system prompts. After resolving the problem, the operator continues the tightening task. For example, the operator checks the bolt and nut mating and finds slight burrs on the bolt head surface. After cleaning and lubrication, the appropriate torque is reapplied.

[0080] By monitoring and responding to torque fluctuations in real time and identifying sudden anomalies, problems can be detected and reported promptly, preventing tightening failures caused by slippage, misalignment, or other abnormalities. Timely identification of anomalies and rapid human intervention prevent safety issues arising from improper tightening, ensuring operational safety.

[0081] Furthermore, the control system for the operating robot used for wind power bolt tightening operations also includes: an early warning unit, which is used to perform early warning level matching, establish an early warning signal, and issue an early warning if the tightening inspection result output by the inspection unit 14 is a failure result.

[0082] Specifically, if the tightening inspection result output by inspection unit 14 is a failure, problems occurred during the surface bolt tightening process, including insufficient torque, unstable contact force, and abnormal tightening process. Based on the failure result, the cause of the abnormality is determined, and a warning level is matched. Different warning levels are assigned to the abnormalities according to preset rules based on the type of failure. The warning level is usually determined by the severity of the problem. For example, minor problems may only trigger a low-level warning, while major problems will trigger a high-level warning. Through warning level matching, the urgency of the abnormality is determined, and different response measures are generated based on different situations.

[0083] Based on the determined warning level, warning signals are generated to help operators promptly recognize problems and take appropriate measures. For example, low-level warnings alert operators through on-screen warning messages or visual indicator lights (such as yellow lights) but do not require immediate action; high-level warnings alert operators through audible alarms, red indicator lights, or emergency stop buttons to prompt immediate inspection or adjustment. Pre-warnings are issued based on the warning signals, notifying operators of the anomalies and explaining their type, location, severity, and possible corrective actions. For example, the screen might display "Tightening operation failed, insufficient torque, please check bolt and nut contact," accompanied by an alarm sound, prompting the operator to take emergency measures and triggering an automatic shutdown for manual inspection. By monitoring and matching anomalies in real time, problems during bolt tightening are identified promptly, ensuring bolt tightening quality.

[0084] Furthermore, the control system for the operating robot used for wind power bolt tightening operations also includes: a planning unit, used to acquire a bolt tightening task set, perform bolt distribution analysis based on the bolt tightening task set, perform bolt operation path optimization, establish a target optimization path, and control the operating robot to move towards the target work position based on the target optimization path.

[0085] Specifically, this involves obtaining the bolt tightening task set, which is the collection of all bolts requiring tightening operations within a specific work environment, including bolts of different locations, sizes, and tightening requirements. Bolt distribution analysis is then performed on this task set to reveal the spatial distribution of bolts, understanding their layout, density, and the relative position of each bolt to other bolts. For example, assuming multiple bolts need tightening in a work area, bolt distribution analysis might indicate that some bolts are concentrated in area A, while others are located in area B, with a certain distance between areas A and B.

[0086] After completing the bolt distribution analysis, an optimal work path is calculated using path optimization algorithms (such as Dijkstra's algorithm, A* algorithm, ant colony optimization, etc.). Optimized paths ensure the robot can complete all bolt tightening operations in the shortest time, while avoiding path interference and repetitive movements, maximizing efficiency. The path optimization process needs to consider the bolt tightening sequence, distance, operation time, energy consumption, and efficiency. Specifically, the robot's operation sequence from one bolt to another is determined based on the bolt distribution. During movement, the distance of each path segment needs to be considered, calculating the physical distance between each bolt to select the shortest route during path optimization. The time required for each tightening task may vary depending on the bolt type, tightening requirements, etc., and the time required to complete each bolt is estimated based on the tightening task for each bolt. Simultaneously, the robot's energy consumption and overall work efficiency are considered. By selecting shorter, more efficient paths, energy consumption can be reduced and production efficiency improved. Path calculation is performed based on the selected path optimization algorithm (such as A* algorithm). Algorithm A* evaluates each possible path and calculates the optimal path using a heuristic method, selecting the path with the shortest total distance to generate the target optimization path. For example, starting from bolt 1 (area A): the distance from bolt 1 to bolt 2 is 5 meters, from bolt 2 to bolt 3 is 10 meters, from bolt 3 to bolt 4 is 7 meters, and from bolt 4 to bolt 5 is 3 meters. The total path distance is calculated as follows: from bolt 1 to bolt 5, passing through bolts 2, 3, and 4, the total path length is 25 meters. Based on the calculation of Algorithm A*, the optimal path is determined to be: Bolt 1 → Bolt 2 → Bolt 3 → Bolt 4 → Bolt 5. This path has the shortest total distance and meets the requirements of production efficiency.

[0087] Based on the generated target optimization path, the robot is controlled to perform bolt tightening operations one by one. For example, the robot first moves to station 1, and then sequentially reaches the other bolt positions according to the target path until all bolts are tightened. By analyzing the bolt positions and calculating the optimal path, the robot can complete the tightening task of all bolts in the shortest time, avoiding unnecessary repetitive movements. By reducing movement paths and operation time, energy consumption and operating costs are significantly reduced, improving the overall operating efficiency of the production line.

[0088] In summary, the robot control system for wind turbine bolt tightening operations provided in this application has the following technical advantages:

[0089] By reading bolt passport data, 2D vision is guided to perform initial image acquisition and fuzzy positioning. Based on the fuzzy positioning results, more precise point cloud and 2D vision re-acquisition is triggered, achieving accurate coordinate mapping and tracking feed from the far field to the near field. Contact force data is introduced for confidence-driven compliant pose correction, fine-tuning the robot's posture in real time. Before formal tightening, a tightening test under the first torque is performed and test feedback is established. Tightening inspection is conducted based on the test feedback and bolt passport data to determine whether the tightening process under the current working conditions is normal. Only after the inspection is passed is the bolt tightening operation performed according to the adaptive calibration strategy, ensuring that the tightening quality of each bolt meets the requirements, improving the efficiency and accuracy of the tightening operation, and ensuring the tightening quality.

[0090] Example 2: Based on the same inventive concept as the control system for a wind turbine bolt tightening operation described in Example 1, this application also provides a control method for a wind turbine bolt tightening operation. Please refer to the appendix. Figure 2 The method for controlling an operating robot used in wind turbine bolt tightening operations includes:

[0091] Once the detection robot reaches the target workstation, it is triggered to read the bolt passport data. Based on the bolt passport data, 2D vision is controlled to perform image acquisition. After performing fuzzy positioning using the image acquisition results, point cloud and 2D vision are re-acquired based on the fuzzy positioning. The re-acquisition results are used to control the tracking feed of the robot. When the tracking feed error of the far-field acquisition unit 12 meets the preset threshold, the contact force data between the electric sleeve and the bolt head is read. Based on the contact force data, confidence-driven compliant posture correction is performed, and compliant posture correction feed is executed. A fastening test under the first torque is performed, test feedback is established, and fastening inspection is performed using the test feedback and the bolt passport data. The fastening inspection result is output. If the fastening inspection result is a pass result, an adaptive calibration strategy is established based on the test feedback and the bolt passport data, and the bolt fastening operation is performed.

[0092] Furthermore, the method for controlling the operating robot for wind power bolt tightening operations includes: the bolt passport data is a bolt digital passport, which is read by the operating robot scanning the bolt identification code; the bolt passport data includes bolt specifications, bolt grade, target preload, and recent operation records.

[0093] Furthermore, the step of performing fuzzy localization using the image acquisition results includes: extracting contours from the image acquisition results and establishing contour extraction results; performing probability detection of closed contours on the contour extraction results and establishing a pre-selected target set using the probability detection results; performing target credibility verification of the pre-selected target set based on the bolt specifications in the bolt passport data, completing fuzzy localization using the target credibility verification results, and obtaining relative fuzzy coordinates; and re-acquiring point clouds and 2D vision based on the relative fuzzy coordinates.

[0094] Furthermore, the method of controlling the robot's tracking feed using the reacquisition results includes: performing data preprocessing on the image data and point cloud data in the reacquisition results; extracting calibrated three-dimensional feature points based on the bolt passport data; using the calibrated three-dimensional feature points and 2D corner points in the image data to perform RANSAC-PnP to obtain the initial visual pose value, and outputting the reprojection error and inlier rate; if the calibrated three-dimensional feature points do not exist, then performing geometric fitting and backprojection based on the hole contour in the image data to establish the initial visual pose value; fitting a plane on the flange surface using the point cloud data and detecting the hole boundary, and using... The initial pose of the point cloud is constructed from the fitting results of the aperture boundary, and the point cloud fitting residual and inlier rate are calculated. Weighted fine registration iterative processing is performed based on the initial visual pose, the initial point cloud pose, the reprojection error and inlier rate, and the point cloud fitting residual and inlier rate to establish the iterative processing result. Both the initial visual pose and the initial point cloud pose are represented in the end-effector coordinate system of the operating robot, and the transformation from the acquired coordinate system to the end-effector coordinate system is completed through a hand-eye calibration matrix. After updating the visual quality and point cloud quality using the iterative processing result, the pose weighting is reconstructed, and the tracking feed of the operating robot is controlled based on the reconstructed pose weighted fusion result.

[0095] Furthermore, the step of performing weighted fine registration iterative processing based on the initial visual pose value, the initial point cloud pose value, the reprojection error and inlier rate, the point cloud fitting residual and inlier rate, and establishing the iterative processing result includes: calculating the confidence weights of the initial visual pose value and the initial point cloud pose value using the reprojection error and inlier rate, the point cloud fitting residual and inlier rate respectively; using the confidence weights to perform weighted interpolation of the initial visual pose value and the initial point cloud pose value to establish the initial registration value; using the initial registration value to perform point-to-surface registration under multiple resolutions, and introducing a visual prior regularization term based on visual confidence in the registration objective function of the registration process; the point-to-surface registration uses the RMS residual threshold, the pose update threshold or the maximum number of iterations as the convergence condition; if the point-to-surface registration result is a divergent result, the backoff strategy is activated, and the data of the reacquisition result is reacquired.

[0096] Furthermore, the confidence-driven compliant pose correction based on the contact force data, and the execution of the compliant pose correction feed, includes: performing adaptive filtering on the contact force data under a sliding window to remove transient noise and establish a smooth force set; establishing a force-displacement mapping based on the smooth force set using an end-effector compliance model; and configuring flexible pose feed parameters based on the force confidence of the smooth force set and the force-displacement mapping to complete the compliant pose correction feed.

[0097] Furthermore, the step of performing a fastening inspection using the test feedback and the bolt passport data, and outputting a fastening inspection result, includes: the first torque being a low test torque; extracting the initial force and performing continuous force trend analysis on the test feedback; using the initial force extraction result and the bolt passport data to perform initial force confidence verification and establish a first verification result; generating a second verification result using the continuous force trend analysis; and outputting the fastening inspection result based on the first verification result and the second verification result.

[0098] Furthermore, the output of the fastening inspection result includes: if the fastening inspection result is a failure result, generating a test instruction; performing bolt testing according to the test instruction, and using the bolt test result to perform anomaly identification, the anomaly identification including slippage identification, misaligned thread identification, burr anomaly identification, and adhesion identification.

[0099] Furthermore, the step of establishing an adaptive calibration strategy based on the test feedback and the bolt passport data includes: establishing the current state data of the bolt based on the test feedback; obtaining the target preload based on the bolt passport data, and using the current state data and the target preload as input data; configuring posterior parameters based on the most recent work record in the bolt passport data, setting an adaptive calibration model using the posterior parameters, inputting the input data into the adaptive calibration model, and outputting the adaptive calibration strategy.

[0100] Furthermore, the bolt tightening operation includes: monitoring the feedback torque during the bolt tightening operation and establishing a sudden change identification anomaly; reporting the operation anomaly based on the sudden change identification anomaly and sending a manual intervention request.

[0101] Furthermore, the control method for the operating robot used for wind power bolt tightening operations also includes: if the tightening inspection result output by the inspection unit 14 is a failure result, then according to the failure anomaly, an early warning level matching is performed, an early warning signal is established, and an early warning is issued.

[0102] Furthermore, the method for controlling an operating robot for wind power bolt tightening operations also includes: acquiring a bolt tightening task set, performing bolt distribution analysis based on the bolt tightening task set, performing bolt operation path optimization, establishing a target optimization path, and controlling the operating robot to move towards the target workstation position based on the target optimization path.

[0103] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Figure 1 The control system and specific example of the operating robot for wind power bolt tightening operation in Embodiment 1 are also applicable to the operating robot control method for wind power bolt tightening operation in this embodiment.

[0104] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0105] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. An operation robot control system for a wind power bolt fastening operation, characterized by, The method comprises the following steps: A triggering unit is used to trigger the operation robot to read the bolt passport data when the operation robot detects that it has arrived at the target station position; A far-field capture unit is used to control the 2D vision to perform image acquisition based on the bolt passport data, and after performing fuzzy positioning using the image acquisition result, to perform reacquisition of point cloud and 2D vision according to the fuzzy positioning, and to control the operation robot to track feeding using the reacquisition result; The far-field capture unit comprises: A data preprocessing subunit is used to perform data preprocessing on the image data and the point cloud data in the reacquisition result; An initial value establishing subunit is used to extract a calibration three-dimensional feature point according to the bolt passport data, to perform RANSAC-PnP to obtain a visual pose initial value using the calibration three-dimensional feature point and a 2D corner point in the image data, and to output a re-projection error and an inlier rate, and if the calibration three-dimensional feature point does not exist, to perform geometric fitting and back projection according to a hole contour in the image data to establish a visual pose initial value; A plane fitting subunit is used to fit a plane on a flange face using the point cloud data, to detect a hole circle boundary, to construct a point cloud pose initial value using the hole circle boundary fitting result, and to calculate a point cloud fitting residual error and an inlier rate; An iterative processing subunit is used to perform weighted fine registration iterative processing according to the visual pose initial value, the point cloud pose initial value, the re-projection error and the inlier rate, the point cloud fitting residual error and the inlier rate, to establish an iterative processing result, and the visual pose initial value and the point cloud pose initial value are both expressed in an end coordinate system of the operation robot and are converted into the end coordinate system through a hand-eye calibration matrix; A weight reconstruction subunit is used to reconstruct a pose weighting weight after updating a visual quality and a point cloud quality using the iterative processing result, and to control the operation robot to track feeding according to a reconstructed pose weighting fusion result; A near-field calibration unit is used to read contact force data of an electric sleeve and a bolt head when a tracking feeding error of the far-field capture unit satisfies a preset threshold, to perform confidence-driven compliant pose correction based on the contact force data, and to perform compliant pose correction feeding; An inspection unit is used to perform a fastening test under a first torque, to establish a test feedback, to perform fastening inspection using the test feedback and the bolt passport data, and to output a fastening inspection result; 2. The operating robot control system for a wind power bolt fastening operation according to claim 1, characterized in that, An adaptive control unit is used to establish an adaptive calibration strategy according to the test feedback and the bolt passport data if the fastening inspection result is a pass result, and to perform bolt fastening operation.

3. The operating robot control system for a wind power bolt fastening operation according to claim 2, characterized in that, The bolt passport data is a bolt digital passport, which is read by scanning a bolt identification code using the operation robot, and the bolt passport data comprises a bolt specification, a bolt grade, a target preload, and a nearest operation record. The far-field capture unit comprises: A contour extraction subunit is used to perform contour extraction on the image acquisition result to establish a contour extraction result; A probability detection subunit is used to perform probability detection on the contour extraction result to establish a preselected target set using a probability detection result; A fuzzy positioning subunit is used to perform target credibility verification of the preselected target set according to a bolt specification in the bolt passport data, to complete fuzzy positioning using a target credibility verification result, and to obtain a relative fuzzy coordinate; The reacquisition subunit is configured to perform point cloud and 2D vision reacquisition based on the relative blur coordinates.

4. The operating robot control system for a wind power bolt fastening operation according to claim 1, characterized in that, The iterative processing subunit comprises: The confidence weight calculation channel is configured to calculate confidence weights of the visual pose initial value and the point cloud pose initial value by using the re-projection error and the inlier rate and the point cloud fitting residual and the inlier rate respectively, perform weighted interpolation of the visual pose initial value and the point cloud pose initial value by using the confidence weights, and establish a registration initial value; The point-to-plane registration channel is configured to perform point-to-plane registration under multi-resolution by using the registration initial value, and introduce a visual priori regular term according to the visual confidence in a registration target function in the registration process; The convergence condition determination channel is configured to take the RMS residual threshold, the pose update threshold or the maximum number of iterations as a convergence condition for the point-to-plane registration. The strategy activation channel is configured to activate a fallback strategy to perform data reacquisition of the reacquisition result if the point-to-plane registration result is a divergent result.

5. The operating robot control system for a wind power bolt fastening operation according to claim 1, characterized in that, The near-field calibration unit comprises: The adaptive filtering subunit is configured to perform adaptive filtering on the contact force data under a sliding window, remove transient noise, and establish a smooth force set. The mapping subunit is configured to establish a force-displacement mapping according to the smooth force set by using the end compliance model. The compliant pose correction feed subunit is configured to configure flexible pose feed parameters according to the force confidence of the smooth force set and the force-displacement mapping to complete compliant pose correction feed.

6. The operating robot control system for a wind power bolt fastening operation according to claim 1, wherein The inspection unit comprises: The test feedback subunit is configured to perform initial force extraction and continuous force trend analysis on the test feedback if the first torque is a low test torque. The first verification subunit is configured to perform initial force trust verification by using the initial force extraction result and the bolt passport data to establish a first verification result. The second verification subunit is configured to generate a second verification result by using the continuous force trend analysis. The inspection result output subunit is configured to output a fastening inspection result according to the first verification result and the second verification result.

7. The operating robot control system for a wind power bolt fastening operation according to claim 1, wherein The inspection unit further comprises: The result failure subunit is configured to generate a test instruction if the fastening inspection result is a failure result. The bolt test subunit is configured to perform bolt testing according to the test instruction, and perform abnormality identification by using the bolt test result, wherein the abnormality identification comprises slip identification, wrong tooth identification, burr abnormality identification and adhesion identification.

8. The operating robot control system for a wind power bolt fastening operation according to claim 1, wherein, The adaptive control unit comprises: The current state data establishment subunit is configured to establish current state data of the bolt according to the test feedback. The target pretightening acquisition subunit is configured to acquire a target pretightening according to the bolt passport data, and take the current state data and the target pretightening as input data. The adaptive calibration subunit is configured to configure a posteriori parameter according to a latest operation record in the bolt passport data, set an adaptive calibration model by using the posteriori parameter, input the input data into the adaptive calibration model, and output an adaptive calibration strategy.

9. The operating robot control system for wind power bolt fastening work according to claim 1, characterized by, The operation robot control system for wind power bolt fastening operation comprises: The early warning unit is configured to perform early warning level matching according to a failure abnormality if the fastening inspection result output by the inspection unit is a failure result, establish an early warning signal, and perform early warning reporting.

10. The operating robot control system for a wind power bolt fastening operation according to claim 1, characterized in that, The operation robot control system for wind power bolt fastening operation comprises: A planning unit is configured to obtain a bolt fastening task set, perform bolt distribution analysis based on the bolt fastening task set, execute bolt operation path optimization, establish a target optimization path, and control the operation robot to move to a target work station position based on the target optimization path.

11. The operating robot control system for a wind power bolt fastening operation according to claim 1, characterized in that, The adaptive control unit further comprises: A feedback torque monitoring subunit is configured to perform feedback torque monitoring during bolt fastening operation, establish a mutation identification exception, and perform operation exception reporting based on the mutation identification exception and send a request for manual intervention. The operation robot control method for wind power bolt fastening operation is executed by the operation robot control system for wind power bolt fastening operation according to any one of claims 1 to 11, and comprises:

12. A control method of an operation robot for a wind power bolt fastening work, characterized by, When the operation robot reaches the target work station position, the operation robot reads bolt passport data; Based on the bolt passport data, the 2D vision performs image acquisition, and after fuzzy positioning is performed using the image acquisition result, point cloud and 2D vision reacquisition is performed based on the fuzzy positioning, and the operation robot tracking feeding is controlled using the reacquisition result, including: Data preprocessing is performed on the image data and point cloud data in the reacquisition result; Calibration three-dimensional feature points are extracted based on the bolt passport data, RANSAC-PnP is used to obtain a visual pose initial value using the calibration three-dimensional feature points and 2D corner points in the image data, and a re-projection error and an inlier rate are output, if the calibration three-dimensional feature points do not exist, geometric fitting and back projection are performed based on the hole outline in the image data to establish a visual pose initial value; A plane is fitted on the flange face using the point cloud data, and a hole circle boundary is detected, a point cloud pose initial value is constructed using the hole circle boundary fitting result, and a point cloud fitting residual error and an inlier rate are calculated; Weighted fine registration iterative processing is performed based on the visual pose initial value, the point cloud pose initial value, the re-projection error and the inlier rate, the point cloud fitting residual error and the inlier rate, an iterative processing result is established, the visual pose initial value and the point cloud pose initial value are expressed in an end coordinate system of the operation robot, and conversion from acquisition to the end coordinate system is completed through a hand-eye calibration matrix; After the visual quality and the point cloud quality are updated using the iterative processing result, a pose weighting weight is reconstructed, and the operation robot tracking feeding is controlled based on the reconstructed pose weighted fusion result; When the tracking feeding error of the far-field capture unit meets a preset threshold, contact force data of the electric sleeve and the bolt head are read, confidence-driven compliant pose correction is performed based on the contact force data, and compliant pose correction feeding is executed; Fastening testing under a first torque is performed, a test feedback is established, fastening verification is performed using the test feedback and the bolt passport data, and a fastening verification result is output; If the fastening verification result is a pass result, an adaptive calibration strategy is established based on the test feedback and the bolt passport data, and bolt fastening operation is executed. ​

Images