Large steel structure corrosion damage integrated repair robot and control method

Abstract

The invention belongs to the technical field of steel structure damage repair. The invention provides a large steel structure corrosion damage integrated repairing robot and a control method. The robot comprises a robot body, and a force control servo motor, an automatic polishing module, a damage identification camera, an air compression pump, a paint storage box, a control terminal, a magnetic adsorption module, a pipeline climbing module, a paint film thickness measuring module and an automatic paint spraying module which are arranged on the robot body. A robot compliance control strategy integrating force feedback, visual guidance and other modes is provided, dynamic parameter adjustment is combined, efficient and compliance control over the rust removal and spraying process of the repairing robot is achieved, the high efficiency and stability of the corrosion damage repairing process are ensured, and the corrosion damage repairing efficiency is improved. And flexible movement and efficient operation of the integrated repairing robot in a complex space structure are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of steel structure damage repair, and in particular to a large-scale steel structure corrosion damage integrated repair robot and a control method. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] Corrosion damage poses a serious threat to the safe service of large steel structure facilities. From the perspective of material properties, corrosion can cause significant changes in the material properties of steel structures. During the corrosion process, a layer of corrosion products will form on the surface of steel, which often have significant differences in the mechanical properties of the steel itself. For example, the hardness of the corrosion products may be much higher than that of the steel itself, which makes the rusted area more likely to become an area of ​​stress concentration when subjected to external forces, thereby increasing the risk of structural failure. In addition, corrosion will also weaken the bearing capacity of steel because corrosion will cause the cross-sectional area of ​​steel to decrease, thereby reducing its overall strength. Secondly, from the perspective of structural safety, corrosion damage poses a serious threat to the stability and durability of large steel structure facilities. When steel structure facilities are designed and constructed, they are based on certain material properties and safety factors. However, corrosion damage will change these parameters, causing the performance of the structure in actual use to deviate from the design expectations. This deviation may cause the structure to fail unexpectedly under extreme weather conditions or during normal use, such as collapse or fracture, resulting in serious casualties and property losses.

[0004] At present, the technical difficulty and cost of real-time monitoring of corrosion damage of steel structure facilities are high, and in the case of difficult access or passage for personnel, traditional repair methods may require long construction time and complex operations, which is a challenge for steel structure facilities such as bridges that need to resume work quickly. Traditional methods use sandblasting, shot blasting and other methods to clean the surface of steel structures to remove rust, dirt, etc., and then use appropriate anti-corrosion coatings to enhance the durability and corrosion resistance of steel structures. If the surface treatment is not sufficient, the corrosion problem under the coating may persist, resulting in poor coating effect. Summary of the invention

[0005] In order to address the shortcomings of the prior art, the present invention provides an integrated repair robot for corrosion damage of large steel structures and a control method, proposes a robot compliant control strategy that integrates multi-modal features such as force feedback and visual guidance, and combines dynamic parameter adjustment and high-precision image recognition to achieve efficient and compliant control of the rust removal and spraying processes of the repair robot, thereby ensuring the efficiency and stability of the corrosion damage repair process and realizing the flexible movement and efficient operation of the integrated repair robot in complex spatial structures.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides an integrated repair robot for corrosion damage of large steel structures.

[0007] A large-scale steel structure corrosion damage integrated repair robot, comprising: a robot body and a force-controlled servo motor arranged on the robot body, an automatic grinding module, a damage recognition camera, an air compression pump, a paint storage box, a control terminal, a magnetic adsorption module, a pipeline climbing module, a paint film thickness measurement module and an automatic painting module; The magnetic adsorption module is used to be connected to the pipeline magnetic attraction, the force control servo motor is connected to the automatic polishing module to control the polishing element of the automatic polishing module to rotate and polish the outer wall of the pipeline, the automatic painting module is connected to the air compression pump and the paint storage box through pipelines, and the damage recognition camera, air compression pump, magnetic adsorption module, pipeline climbing module, paint film thickness measurement module and automatic painting module are all connected to the control terminal; The control terminal is configured to: control the pipeline climbing module to drive the robot body to move along the pipeline according to the work task, control the pipeline climbing module to stop action when corrosion damage is identified according to the image of the damage recognition camera, control the magnetic adsorption module to adsorb the pipeline, control the force control servo motor to drive the automatic polishing module to polish the corrosion damage position, and after polishing, use the automatic painting module to paint, and after painting, use the paint film thickness measurement module to measure the thickness, when the paint film thickness is greater than the set threshold, control the magnetic adsorption module to release the pipeline, and control the pipeline climbing module to continue climbing.

[0008] In a second aspect, the present invention provides a control method for a large steel structure corrosion damage integrated repair robot.

[0009] A control method for a large steel structure corrosion damage integrated repair robot includes the following processes: Receive a repair task, obtain sensor data from multiple sensors carried on the robot body, fuse the sensor data from the multiple sensors based on the extended Kalman filter, decompose the repair task according to the fused data, decompose the repair task into several subtasks, and perform repair operations according to each subtask.

[0010] In this implementation, during the execution of the repair task, the control terminal uses reinforcement learning and adaptive control algorithms to continuously optimize the work path and operation strategy, continuously tests and provides feedback through the reinforcement learning algorithm, and dynamically adjusts the robot's motion parameters and operation mode based on real-time operation data through the adaptive control algorithm.

[0011] As a further limitation of the second aspect of the present invention, in the process of performing the repair task, controlling the walking of the terminal robot body includes: The magnetic adsorption module includes two pairs of front and rear adsorption components, the front adsorption component includes two adsorption feet at opposite positions, and the rear adsorption component includes two adsorption feet at opposite positions; The front suction assembly loosens the clamping force on the steel structure, while the rear suction assembly holds the steel structure; Under the precise control of the servo motor, the front suction component and the slide rail slide relative to the rear suction component, thereby driving the entire robot to move; The front adsorption assembly clamps the steel structure again, and the rear adsorption assembly relaxes and slides along the slide rail, repeating the cycle until it reaches the set position.

[0012] As a further limitation of the second aspect of the present invention, in the process of performing the repair task, the control terminal performs the rusted area identification, including: For the collected image, take any pixel as the center and the radius is k The adjacent range of is regarded as a search area; the entropy is calculated by calculating the proportion of pixels of different categories in the search area, and the regional complexity is calculated; the average entropy of all pixels in the area is used to evaluate the uncertainty of the prediction; According to the regional complexity and the average value of all pixel entropies, a regional query function is obtained. The regional query function is used to accurately select the areas with more categories and difficult to classify in the image, and accurate artificial supervision signals are given to assist model training. Image segmentation is performed according to the trained model to obtain multiple segmented rust areas.

[0013] As a further limitation of the second aspect of the present invention, the search area It is expressed as follows: ; In the formula, Represents any pixel in the image, It represents the neighboring area of ​​the pixel, and the size is Round is considered as a region centered at each pixel.

[0014] As a further limitation of the second aspect of the present invention, the proportion of pixels of different categories in the area for: , where represents the pseudo label of the rusted area output during the self-supervised training process, Indicates the pixel category in the region.

[0015] As a further limitation of the second aspect of the present invention, entropy calculation for regional complexity includes: after counting pixels of various categories in a region, obtaining the regional complexity: ,in, Represents the total number of pixels in the set.

[0016] As a further limitation of the second aspect of the present invention, the average value of the entropy of all pixels in the region is used to evaluate the uncertainty of the prediction, and the calculation method is shown in the following formula: , , where represents the predicted entropy of each pixel, represents the rusted pseudo-labels provided by the network, represents the uncertainty of each pixel; The final region query function is expressed as: , where represents the target domain image, To segment the network, is the area query function, G is The set of M is A collection of .

[0017] As a further limitation of the second aspect of the present invention, the output result of the trained model is represented as a set of multiple corrosion areas. ,in For the rusted area, For the background part, Represents the entire image; Break down rusted areas for and , represents the high confidence region obtained by applying the erosion morphological operator to the rusted region, represents the low confidence boundary region obtained by applying the dilation morphological operator to the remaining pixels; Use segmentation algorithm to divide the original image into multiple color and area segments , project the boundary low confidence area to and The boundary enhancement result of the corroded area is obtained.

[0018] As a further limitation of the second aspect of the present invention, the projection boundary low confidence area is and Above, including: ; in, To obtain the precise outline of the rusted area using the projection method, Represents the logical AND operation, for and and and There is also a union of the intersection parts.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention innovatively proposes an integrated repair method for corrosion damage of steel structures such as supports, rods, and welds, and proposes a steel structure corrosion repair wall-climbing robot based on suspended magnetic adsorption, multifunctional integrated lightweight, suitable for steel structures of different shapes such as tubular and I-shaped, to achieve flexible movement and efficient operation of the integrated repair robot in complex spatial structures.

[0020] 2. The present invention proposes a robot compliant control strategy that integrates multi-modal features such as force feedback and visual guidance, and combines it with dynamic parameter adjustment to achieve efficient and compliant control of the rust removal and spraying process of the repair robot, thereby ensuring the efficiency and stability of the corrosion damage repair process.

[0021] 3. Based on deep learning, the present invention realizes the rapid identification of corrosion damage of steel structures such as supports, rods, and welds, proposes a method for identifying rusted areas of steel structures based on active transfer learning, constructs an accurate identification and prediction model for corrosion damage of steel structures, and realizes the rapid identification of corrosion damage with a small amount of labeled data, guiding the repair equipment to perform accurate repair of corrosion damaged areas, providing an efficient and reliable damage repair solution for the operation and maintenance of large steel structures.

[0022] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0024] Figure 1 Schematic diagram of the large steel structure corrosion damage integrated repair robot provided by the present invention Figure 1 ; Figure 2 Schematic diagram of the large steel structure corrosion damage integrated repair robot provided by the present invention Figure 2 ; Figure 3 Schematic diagram of the large steel structure corrosion damage integrated repair robot provided by the present invention Figure 3 ; Figure 4 Schematic diagram of the large steel structure corrosion damage integrated repair robot provided by the present invention Figure 4 ; Figure 5 A schematic diagram of the extended Kalman filter provided by the present invention; Figure 6 A schematic diagram of the structure of the steel structure corrosion area segmentation model provided by the present invention; Figure 7 A schematic diagram of image area division provided by the present invention, Figure 7 (a) is a schematic diagram of fixed grid area division; Figure 7 (b) is a schematic diagram of super-pixel area division. Figure 7 (c) in the figure is a schematic diagram of text area division.

[0025] Figure 8 A schematic diagram of the refinement of the corrosion area boundary based on the projection method provided by the present invention, Figure 8 (a) is the original image of the rusted area. Figure 8 (b) is a schematic diagram of the segmentation network output result. Figure 8 (c) is a schematic diagram of the result after morphological processing. Figure 8 (d) is a schematic diagram of the method for boundary projection results; Among them, 1. force-controlled servo motor; 2. automatic polishing module; 3. damage identification camera; 4. air compression pump; 5. paint storage box; 6. control terminal; 7. magnetic adsorption module; 8. pipeline climbing module; 9. paint film thickness measurement module; 10. automatic painting module. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0028] In this implementation, a large-scale steel structure corrosion damage integrated repair robot is proposed. Figure 1 and Figure 2 As shown, it includes: a force-controlled servo motor 1, an automatic grinding module 2, a damage recognition camera 3, an air compression pump 4, a paint storage box 5, a control terminal 6, a magnetic adsorption module 7, a pipeline climbing module 8, a paint film thickness measurement module 9 and an automatic painting module 10.

[0029] The large-scale integrated steel structure corrosion damage repair robot provided by this implementation method can move flexibly on the surface of complex steel structures, and the magnetic suspension technology ensures its stable adhesion on vertical and inclined surfaces. The force-controlled servo motor 1 can achieve precise pressure grinding, and after the grinding is completed, the automatic painting module 10 can realize precise spraying operations on the repair area.

[0030] The magnetic adsorption module 7 can be accurately clamped on the steel structure at both ends (can be controlled by a special motor), and can be firmly adsorbed on the surface of the steel structure component by virtue of the strong magnetic force. At this time, the synergistic effect of the magnetic adsorption module 7 and the pipeline climbing module 8 enables the robot to complete complex climbing movements. The pipeline climbing module 8 includes a slide rail and a servo motor.

[0031] The magnetic adsorption module 7 includes two pairs of front and rear adsorption components, the front adsorption component includes two opposite adsorption feet, the rear adsorption component includes two opposite adsorption feet, the front adsorption component is fixedly connected to the slide rail, and the rear adsorption component is slidably connected to the slide rail.

[0032] The specific movement process includes: first, the front adsorption component relaxes the clamping force on the steel structure, while the rear adsorption component holds the steel structure; then, under the precise control of the servo motor, the front adsorption component and the slide rail slide relative to the rear adsorption component, thereby driving the entire robot to move; then, the front adsorption component clamps the steel structure again, and the rear adsorption component relaxes and slides along the slide rail. This process is repeated, ensuring that the robot can perform climbing tasks smoothly and efficiently.

[0033] When the high-precision camera at the front end of the robot accurately identifies the damaged part on the steel structure, the robot will quickly move to that position and hold the steel component through the four adsorption feet of the magnetic adsorption module 7 to ensure stable operation; then, the robot will first automatically use the polishing module 2 to finely polish the corroded and damaged parts to completely remove the damage marks; then, the spraying function is activated, and the air compression pump 4 starts to operate efficiently, pushing the paint to be evenly sprayed out from the nozzle of the automatic painting module 10, and finely paint the polished parts; finally, the robot will also use the advanced paint film thickness measurement module 9 to accurately detect the quality of the paint spraying to ensure that the paint spraying effect meets the established standards.

[0034] In order to achieve the coordination and optimization of multifunctional operations, this implementation method proposes an algorithm framework based on task decomposition and collaborative control. This framework can dynamically decompose operation tasks according to real-time environmental information and task requirements, and reasonably schedule the work of detection, cleaning and spraying modules in the robot system.

[0035] Specifically, when the robot receives a task instruction, it first obtains environmental information through sensors and visual systems, and inputs it into the task decomposition module (embedded in the control terminal 6). The task decomposition module uses advanced task decomposition algorithms to break down the overall task into several subtasks and assign them to the corresponding actuators. In the process of performing the repair task, the robot system uses reinforcement learning and adaptive control algorithms to continuously optimize the work path and operation strategy. The reinforcement learning algorithm gradually improves the robot's performance in different tasks and environments through continuous experiments and feedback, while the adaptive control algorithm dynamically adjusts the robot's motion parameters and operation mode based on real-time operation data to cope with various emergencies and environmental changes.

[0036] In this implementation, the reinforcement learning algorithm mainly plays the role of optimizing the decision-making of the robot system. The reinforcement learning algorithm can help the robot to continuously adjust its behavior strategy through interaction with the environment, thereby improving its performance in a variety of tasks and environments. Specifically, the role of reinforcement learning includes: Self-learning and decision-making: The robot learns through trial and error and feedback, and can automatically discover the best work path and operation strategy without explicit programming; Task adaptability: The robot can adaptively adjust its operation strategy according to different task requirements and environmental changes to achieve dynamic optimization; Long-term reward optimization: Through reinforcement learning, robots are able to maximize long-term rewards, rather than just immediate results, thereby gradually improving their performance in complex tasks.

[0037] For example, reinforcement learning algorithms that can be used include: Q-learning: A value-based offline reinforcement learning method that allows a robot to choose the optimal action strategy in a specific state by learning the state-action value function (Q function); Deep Q Network (DQN): It combines deep learning and Q-learning methods, using neural networks to approximate the Q function, and is suitable for problems with very large state spaces.

[0038] During the rust removal process, if the degree of corrosion in a certain area is detected to be more serious, the adaptive control algorithm will automatically adjust the rust removal force and spray thickness to ensure the repair effect. In addition, in order to improve the overall efficiency and reliability of the robot system, the present invention also introduces multi-sensor fusion technology. By integrating the data of force sensors and visual sensors, the robot can more accurately perceive environmental characteristics and state changes, thereby making more accurate operational decisions. For example, when crossing an obstacle, the robot can perceive the shape and position of the obstacle in real time, adjust the action of the magnetic adsorption module 7, and ensure smooth passage.

[0039] The task decomposition algorithm is one of the core algorithms for robot task execution. Its main goal is to split a large task into a series of small, executable subtasks so that the robot can complete them one by one. The decomposition of these subtasks is based on different factors, including the goal of the task, the required resources, the capabilities of the robot, etc. In the task decomposition algorithm, there are two key concepts: task decomposition and task planning. Task decomposition is the process of breaking down a large task into multiple small subtasks. These subtasks can be a series of specific operations or higher-level tasks, depending on the specific application scenario of the robot. The task decomposition process needs to clarify the goal and execution order of each subtask to ensure the smooth execution of the entire task. The core principle of task decomposition is to break down complex tasks into many simple and easy-to-implement subtasks.

[0040] Task planning is the process of determining the execution method and steps for each subtask. In task planning, the robot needs to consider its current environment and state, as well as the requirements and constraints of the task. Task planning requires breaking down each subtask into a series of execution steps and ensuring that they are executed in the correct order. Usually, task planning requires the use of various planning algorithms to help the robot generate the optimal execution path and strategy.

[0041] The task decomposition algorithm can be divided into three main steps: task decomposition, task planning and task execution.

[0042] Task decomposition: Robots need to break down large tasks into multiple small subtasks, which can be achieved by breaking down the task into more specific and executable actions; Task planning: Once the robot breaks down the task into a series of subtasks, it needs to generate an execution plan for each subtask. Task planning involves determining the execution order, path, and strategy of the subtasks. The robot needs to consider the current environment and state, as well as the requirements and constraints of the task, to generate the optimal execution path and strategy. The goal of task planning is to generate an executable plan for each subtask. Task execution: The robot needs to execute the generated task plan. During the task execution process, the robot will execute each subtask in sequence according to the planned order and strategy. The robot needs to perceive and judge the environment in real time, and adjust the execution strategy. The goal of task execution is to complete each subtask smoothly according to the plan.

[0043] In this implementation, task decomposition is the process of breaking down a large task into multiple subtasks. The purpose is to make complex tasks manageable and executable. Current technical methods generally include the following: Hierarchical decomposition: This is a common approach where a large task is decomposed into layer-by-layer subtasks, each of which can be further split into smaller parts. For example, repair task decomposition is performed based on a hierarchical structure of tasks (such as HTN, Hierarchical Task Network); Behavior Trees: Behavior Trees are a structure that converts complex tasks into executable behaviors. In this way, different parts of the task are organized into a tree structure, and each node represents an action or task that can be performed. Automatic task allocation algorithm: Through multi-objective optimization algorithms, machine learning and other methods, complex tasks are automatically decomposed into specific operational subtasks based on the robot's hardware resources, capabilities, task requirements, etc.; Task planning is to generate the execution order, path and strategy of each subtask after task decomposition. This process often uses the following technologies in practical applications: Graph search algorithms: Graph search algorithms such as A* and Dijkstra are widely used in path planning tasks. Robots can use these algorithms to find the shortest path or the optimal path from the current state to the target state. Planning framework: such as PDDL (Planning Domain Definition Language) and its corresponding planning solver. PDDL is widely used to describe task planning problems and combines automatic reasoning and search techniques to generate the optimal order and path for task execution; Reinforcement learning: Reinforcement learning methods can learn task planning strategies through interaction with the environment. Especially in task planning in uncertain or dynamic environments, the robot adjusts its strategy through continuous trial and error and feedback. Motion planning algorithm: In physical space, robots usually need to plan their own movement paths through motion planning (such as RRT, Rapidly-exploring Random Tree) to ensure that they avoid collisions and comply with dynamic constraints during task execution.

[0044] The task execution part involves how to actually execute the subtasks according to the plan generated by the task planning, which usually includes: Real-time perception and adjustment: Rely on sensors to obtain real-time environmental data and make dynamic adjustments based on the plan. For example, if an obstacle appears during path planning, the robot will replan the optimal path through local path planning. Closed-loop control: When executing a task, the robot may use a closed-loop control strategy to monitor the progress and effect of the task in real time and adjust the strategy to ensure the completion of the goal; Multi-task parallel execution: In some cases, multiple subtasks may be executed simultaneously, especially in multi-robot collaborative tasks, where multiple robots are assigned to perform different subtasks simultaneously.

[0045] When performing surface treatment on complex steel structures such as truss structures and grid structures, the environmental contact characteristics are diverse, and the robot frequently interacts with the environment. In order to ensure safe and smooth interaction with the environment and successfully complete the repair operation tasks, it is necessary to study the compliance control of robot rust removal and painting operations. In response to this problem, the present invention proposes a compliant control strategy for repair robots based on real-time force feedback and visual servoing. By installing force sensors to monitor contact force in real time, combining adaptive control algorithms and visual guidance systems, the operating parameters are dynamically adjusted to achieve high-precision compliance control, and ensure the stability and effect of the rust removal and painting processes.

[0046] This implementation integrates and fuses force feedback with the visual servo system. The force feedback system includes a force sensor and a data processing module, which can obtain the force information exerted on the robot during the repair process in real time. The visual servo system is composed of a camera and an image processing module, which provides the relative position and posture information of the robot and the target object by acquiring and processing the environmental image.

[0047] In order to achieve the effective fusion of the two, a multi-sensor data fusion algorithm based on extended Kalman filter (EKF) is proposed to ensure the synchronization and accuracy of force and visual information (the automatic grinding module 2 is at the front end, and the spraying module is at the back end. During the grinding and spraying process, the robot will be affected by the reaction force. Therefore, force sensors need to be deployed on both the front and back ends of the robot. Both devices need to detect force when working together. Designing a compliant control strategy is the core link of the research. The compliant control strategy needs to consider the information of force feedback and visual servoing at the same time, and a method based on force-position mixing is used to dynamically adjust the motion trajectory and posture of the robot.

[0048] Specifically, the force feedback control part will adjust the stiffness and damping parameters of the robot end according to the real-time force information, so that it maintains appropriate compliance when in contact with the steel structure; the visual servo control part will adjust the position and posture of the robot according to the real-time image information to ensure the accuracy of the repair operation. The two are coordinated and optimized through the model predictive control (MPC) framework to ensure the stability and flexibility of the robot in uncertain environments.

[0049] The model of the discrete-time nonlinear process described by the extended Kalman filter model is: (1); The observation equation is: (2); In the formula, the nonlinear function is a dynamic model function, a nonlinear function is the measurement model function, represents the nonlinear mapping equation between state and measurement, for The interfering noise at all times is like white noise, express The state vector of the moving target at any moment, express The observation vector at time, express The state vector of the moving target at any moment, is a zero-mean, white Gaussian process noise matrix.

[0050] Extended Kalman filter linearization processing: Extended Kalman filter converts nonlinear problems into an approximate linear problem. The linearization method is as follows: , and Perform Taylor series expansion and only keep the low-order terms of the Taylor series expansion.

[0051] Calculate the state transfer matrix of the system state equation: (3); in, It is expressed as the posterior state estimate at time k-1, that is, the updated result, express k The transfer matrix of the state vector of the motion process at any moment, Represents a nonlinear function .

[0052] After linearization, the state estimates and the estimation error covariance are calculated: State estimate : (4); in, for The posterior state estimate at time , that is, the result after updating at time k.

[0053] Estimated error covariance : (5); in, is the covariance matrix, is the state transfer matrix, and the superscript T represents the transpose of the matrix, that is, for The transpose of Indicates the time, Represents the covariance matrix of the process noise.

[0054] Measurement Matrix : (6); in, is the nonlinear mapping equation mentioned above , For function Relative to The partial derivative of The Jacobian matrix at time instant.

[0055] Correspondingly, in the rectangular coordinate system, the new information covariance matrix used for filtering is , weight gain matrix for: (7); (8); The superscript T represents transposition, that is, for The transpose of .

[0056] The nonlinear transformation between coordinates is: (9); Filter state estimate : (10); in, To predict the measurement vector at time k+1, It means that the value at time k is obtained by ignoring the measurement noise The noise-free measurement at time , is the noise value after conversion.

[0057] The covariance of the measurement error is , where the estimated error covariance is: (11).

[0058] Compared with other nonlinear filtering algorithms, the advantage of the extended Kalman filter is that it is more convenient to use. However, in practical applications, the truncation error generated by the linear solution of the nonlinear system has a significant impact on the target state estimation result, and in severe cases, it causes the filter to diverge.

[0059] like Figure 5 As shown in the figure, the specific process of extended Kalman filtering includes: preprocessing sensor monitoring data, initializing state equations, covariance and noise equations, combining The optimal estimate is determined by the sensor monitoring data at time k, and the sensor state value at time k is predicted to obtain the mean square error of the predicted value at time k. The mean square error and state equation of the sensor monitoring data correction value at the moment, calculate the Kalman gain, read the The actual sensor observation value at time k is used to determine the optimal estimate of the sensor at time k.

[0060] From a technical and theoretical point of view, hybrid force-position compliant control is the basis of impedance / admittance control. The corresponding impedance / admittance controls are improved and derived on the basis of hybrid force-position control. Hybrid force-position control is widely used in actual robot engineering.

[0061] In general, if Figure 6 As shown in the figure, hybrid force-position control is to split the position and force of the end effector (i.e. the end effector of the robot where the spray module is located) into two independent decoupled subspaces for processing. In practical applications, these two independent subspaces are generally set as the tangential surface in contact with the object as the position space and the normal surface as the force space (in line with the requirements of polishing and grinding in actual industrial applications). The basic idea of ​​hybrid force-position control is to link the task constraints that require force feedback with the design of the controller. The movement of each joint of the robot will affect the posture / force of the final end effector in all directions. The position of each joint can be written as: (12); in, For the i The angle of the joints, refers to the inverse kinematics function, refers to the jth position of the end effector of the robot arm, refers to the jth position of the end effector of the robot arm, Refers to the jth position of the end effector of the robot arm; similarly, for the force information of each joint, the output of each joint will affect the final output of the end, and the output torque of each joint that needs to be controlled can be written as: (13); in, is the torque required to control the output of the i-th joint, is the external force difference of the end effector in the direction of the jth degree of freedom (expected force - measured force), is the position difference of the end effector in the jth direction (expected position - measured position), and is the corresponding force and position compensation function, is the jth item of the corresponding constraint matrix S; the constraint matrix S here is a binary N-tuple (N is the number of joint degrees of freedom of the robot arm), which describes that the end effector of the robot arm is subject to external force control constraints in space ( ) or subject to position control constraints ( ).

[0062] In the corrosion damage repair spraying scenario of this application, it is assumed that only the force in the z-axis direction (normal force of the polishing plane) needs to be constrained, and only the position trajectory is constrained in other directions. Then the constraint matrix of the terminal Cartesian space is That is: (14).

[0063] The function here is to constrain the joint angle of the robot arm. The input signal required by the robot arm is S and the output torque .

[0064] Corrosion damage is a common and serious form of damage to supports, rods, welds and other parts of complex steel structures. Timely and accurate detection and positioning are essential to ensure the safety of the structure and extend its service life. However, traditional detection methods often rely on manual inspection, which is easily affected by the subjective factors of the inspectors, resulting in inaccurate test results. Manual inspection usually takes a long time, especially when facing large-area or complex-shaped steel structures. The efficiency is even lower. In addition, there are great safety risks when manual inspection is carried out at high altitude or in harsh environments. Therefore, it is particularly urgent and important to develop an efficient and automated detection technology to replace the traditional manual inspection method.

[0065] A method for segmenting the corroded area of ​​the steel structure of a stereo garage based on active transfer learning is proposed. Figure 6As shown in the figure, adversarial training is combined with self-supervised training to build a "pre-training-fine-tuning" corrosion damage and coating degradation accurate prediction and identification model, which can realize the rapid identification of steel structure corrosion damage and accurate location prediction with a small amount of data and annotation. The specific plan includes: in-depth analysis of the corrosion damage characteristics of existing complex steel structures in actual use environments, construction of a high-quality image data set suitable for corrosion damage of complex steel structures, and establishment of a corrosion damage feature library; design and optimization of neural network models to improve detection efficiency and accuracy, while enhancing the robustness and generalization ability of the model; development of a corrosion damage rapid identification system that integrates image acquisition, preprocessing, damage identification and result output, and rigorous testing and verification of the system to evaluate its identification performance and stability in different environments and working conditions; application of the system in actual steel structure projects for corrosion damage monitoring, continuous optimization based on actual application results, and improvement of the practicality and reliability of the system. Through these steps, efficient and accurate identification of corrosion damage of complex steel structures can be achieved, ensuring structural safety and extending service life. The steel structure corrosion area segmentation model based on active transfer learning consists of a model backbone, a domain transfer module, an active learning strategy, and a boundary refinement module. The domain transfer module transfers the corrosion knowledge from the source domain (including source domain images and source domain labels) to the unlabeled target domain (including target domain images) by minimizing the adversarial loss to improve the generalization performance of the model. The source domain and the target domain share weights to obtain pseudo labels for the target domain images. The active learning strategy integrates uncertainty calculation and regional information query to generate marked areas, and conducts supervised training on the target domain images. It automatically queries and labels the difficult-to-classify areas in the rust images, and gives the model effective supervision effects with a small amount of annotations. The boundary refinement module projects the high-precision contour of the rusted area obtained by the traditional segmentation method onto the output result of the segmentation network to refine the boundary information of the rusted area.

[0066] Active learning is an enhancement of the existing model. It gives priority to the samples with the most annotation value for annotation, and greatly improves the accuracy of the model with a small amount of supervision signal. Since the domain transfer module adopts unsupervised training, the generated segmentation results of the corroded area of ​​the steel structure of the stereo garage deviate from the actual scene, and the performance lags far behind the fully supervised algorithm, and cannot be directly applied to actual projects. In order to effectively transfer knowledge, reduce the manual annotation cost of steel structure corrosion images, and overcome the problem of pseudo-label noise in the self-supervised training stage, an active learning method based on region query is introduced, which is combined with self-supervised training to automatically query uncertain and difficult-to-classify image areas for manual annotation to assist network training for model fine-tuning, while greatly reducing the workload of data set annotation, and maximizing the segmentation performance of the corrosion model.

[0067] Specifically, the active learning method is different from the unsupervised domain transfer module. Its goal is to show good steel structure corrosion area segmentation performance in the target domain by using a small amount of annotations in the self-supervised training stage. When traditional active learning methods perform region segmentation, most of them simply divide the image into non-overlapping rectangles, or use superpixel algorithms to divide the target boundaries, such as Figure 7 (a) and Figure 7 As shown in (b) in the figure. The present invention takes any pixel as the center of the image and the radius is k The vicinity of is considered as a search area, which is expressed as follows: (15); In the formula, Represents any pixel in the image, It represents the neighboring area of ​​the pixel, that is, the size of Round is considered as a region centered at each pixel, such as Figure 7 As shown in (c) in .

[0068] In order to query the rusted areas that are difficult for the network to distinguish and have the richest information, the regional complexity and uncertainty strategy in the field of active learning is introduced to select the most valuable supervisory information in the image. The regional complexity is calculated by calculating the proportion of pixels of different categories in the region and calculating the entropy. The proportion of different categories is defined as: (16); In the formula, represents the pseudo label of the rusted area output during the self-supervised training process, Indicates the pixel category in the area. After counting the information of each category in an area, the regional complexity The calculation method is shown in the following formula: (17); In the formula, represents the total number of pixels in the set, and C represents the total number of pixel categories. At the same time, the average value of the entropy of all pixels in the region is used to evaluate the uncertainty of the prediction, and the calculation method is shown as follows: (18); (19); In the formula, represents the predicted entropy of each pixel, represents the rusted pseudo-labels provided by the network, Indicates the pixel category in the area, is the uncertainty calculation result, The calculation is performed using formula (18). Therefore, the final area query function is expressed as: (20); In the formula, represents the target domain image, To segment the network, is a region query function. The image can be represented as a two-dimensional matrix. Indicates Line Column pixels, For the pixel The value of G is The set of M is A collection of .

[0069] In summary, this method adds an active learning strategy based on region query to the segmentation network, integrates regional complexity and prediction uncertainty, accurately selects regions with more categories and difficult to classify in the image through the region query function, and provides accurate artificial supervision signals to assist model training. Active transfer learning solves the problems of complicated and difficult manual labeling and the presence of noise in pseudo labels during the migration process of self-supervised training, significantly improves the segmentation performance of the model, and enables the network to accurately segment the rusted area of ​​the steel structure of the stereo garage.

[0070] Although the above methods can effectively segment the corroded area of ​​the steel structure of the stereo garage, the deep learning-based method outputs a rough segmentation result near the boundary of the corroded area due to a large number of downsampling and upsampling operations performed by the segmentation network during the training process. In order to provide accurate boundary information for the stereo garage steel structure robot prefabrication maintenance system, a projection-based corroded area boundary refinement module is introduced. The traditional color segmentation algorithm can accurately retain the contour of the corroded area, and the output of the segmentation model is projected onto the color segmentation result, thereby achieving the purpose of boundary refinement and improving the ability to express the boundary information of the corroded area of ​​the steel structure. Figure 8 shown.

[0071] like Figure 8 (a) is the original image, and the output of the segmentation network is expressed as A collection of rusted areas, such as Figure 8 As shown in (b) in the figure, it can be expressed as: ,in For the rusted area, For the background part, Represents the entire image. Further decomposition of the rusted area for and , represents the high confidence region obtained by applying the erosion morphological operator to the rusted region, represents the low confidence boundary region obtained by applying the dilation morphological operator to the remaining pixels, such as Figure 8 As shown in (c), the green area is the high confidence area, and the yellow boundary is the low confidence area. The clustering-based SLIC and watershed segmentation algorithms are used to segment the original image into multiple color and region segments. , and project the low confidence area of ​​the boundary onto the results of traditional segmentation algorithms (such as erosion morphological operators and dilation morphological operators). The projection method is as follows: (twenty one); In the formula, To obtain the precise outline of the rusted area using the projection method, Meaning is the content of the entire image P, Is a mathematical operation, indicating a logical AND operation, for and and and There is also a union of the intersection parts.

[0072] Based on the above improved deep learning model, a corrosion damage rapid identification system is developed. The system includes an image acquisition module, a data processing module, an identification module, and a result display module. The system will integrate a deep learning model to realize the automatic, rapid identification and precise positioning of corrosion damage in the acquired images. After the system is developed, it will be functionally tested in a laboratory environment to verify the recognition accuracy and stability of the system. Finally, an application test will be carried out in an actual steel structure project to evaluate the practicality and reliability of the system. According to the actual application effect, the recognition system will be continuously optimized to improve its recognition speed and accuracy. Finally, the system will be promoted and applied to more complex steel structure projects to realize the rapid identification and monitoring of corrosion damage, providing strong support for ensuring the safety of steel structures and extending their service life.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A large steel structure corrosion damage integrated repair robot, characterized in that: include: The robot body and the force-controlled servo motor, automatic grinding module, damage recognition camera, air compression pump, paint storage box, control terminal, magnetic adsorption module, pipeline climbing module, paint film thickness measurement module and automatic painting module arranged on the robot body; The magnetic adsorption module is used to be connected to the pipeline magnetic attraction, the force control servo motor is connected to the automatic polishing module to control the polishing element of the automatic polishing module to rotate and polish the outer wall of the pipeline, the automatic painting module is connected to the air compression pump and the paint storage box through pipelines, and the damage recognition camera, air compression pump, magnetic adsorption module, pipeline climbing module, paint film thickness measurement module and automatic painting module are all connected to the control terminal; The control terminal is configured to: control the pipeline climbing module to drive the robot body to move along the pipeline according to the work task, control the pipeline climbing module to stop action when corrosion damage is identified according to the image of the damage recognition camera, control the magnetic adsorption module to adsorb the pipeline, control the force control servo motor to drive the automatic polishing module to polish the corrosion damage position, and after polishing, use the automatic painting module to paint, and after painting, use the paint film thickness measurement module to measure the thickness, when the paint film thickness is greater than the set threshold, control the magnetic adsorption module to release the pipeline, and control the pipeline climbing module to continue climbing.

2. A control method for a large steel structure corrosion damage integrated repair robot, characterized in that: Using the large steel structure corrosion damage integrated repair robot as described in claim 1, The process includes: Receive repair tasks and obtain sensor data from multiple sensors on the robot body; The sensing data of multiple sensors are fused based on the extended Kalman filter; The repair task is decomposed according to the fused data, the repair task is broken down into several subtasks, and the repair operation is performed according to each subtask.

3. The control method of the large steel structure corrosion damage integrated repair robot according to claim 2, characterized in that: During the repair task, the control terminal controls the movement of the robot, including: The magnetic adsorption module includes two pairs of front and rear adsorption components, the front adsorption component includes two adsorption feet at opposite positions, and the rear adsorption component includes two adsorption feet at opposite positions; The front suction assembly loosens the clamping force on the steel structure, while the rear suction assembly holds the steel structure; Under the precise control of the servo motor, the front suction component and the slide rail slide relative to the rear suction component, thereby driving the entire robot to move; The front adsorption assembly clamps the steel structure again, and the rear adsorption assembly relaxes and slides along the slide rail, repeating the cycle until it reaches the set position.

4. The control method of the large steel structure corrosion damage integrated repair robot according to claim 2, characterized in that: During the repair process, the control terminal identifies the rusted area, including: For the collected image, take any pixel as the center and the radius is k The vicinity of is considered as a search area; By calculating the proportion of pixels of different categories in the search area, the entropy is calculated to calculate the complexity of the area; The average entropy of all pixels in the region is used to assess the uncertainty of the prediction; Obtaining a region query function according to the region complexity and an average value of all pixel entropies; The region query function is used to accurately select the regions in the image with many categories and those that are difficult to classify. Accurate artificial supervision signals are given to assist model training. Image segmentation is performed according to the trained model to obtain multiple segmented rust regions.

5. The control method of the large steel structure corrosion damage integrated repair robot according to claim 4, characterized in that: The search area is represented as follows: ； In the formula, Represents any pixel in the image, It represents the neighboring area of ​​the pixel, and the size is Round is considered as a region centered at each pixel.

6. The control method of the large steel structure corrosion damage integrated repair robot according to claim 4, characterized in that: The proportion of pixels of different categories in this area is: , where represents the pseudo label of the rusted area output during the self-supervised training process, Indicates the pixel category in the region.

7. The control method of the large steel structure corrosion damage integrated repair robot according to claim 4, characterized in that: The entropy calculation of regional complexity includes: After counting the pixels of each category in a region, the regional complexity is obtained: ,in, represents the total number of pixels in the set, and C represents the total number of pixel categories.

8. The control method of the large steel structure corrosion damage integrated repair robot according to claim 7, characterized in that: The average entropy of all pixels in the region is used to evaluate the uncertainty of the prediction, which is calculated as follows: , , where represents the predicted entropy of each pixel, represents the rusted pseudo-labels provided by the network, represents the uncertainty of each pixel; The final region query function is expressed as: , where represents the target domain image, To segment the network, is the area query function, G is The set of M is A collection of .

9. The control method of the large steel structure corrosion damage integrated repair robot according to claim 4, characterized in that: The training segmentation result is represented as a collection of multiple rust regions. ,in For the rusted area, For the background part, Represents the entire image; Break down rusted areas for and , represents the high confidence region obtained by applying the erosion morphological operator to the rusted region, represents the low confidence boundary region obtained by applying the dilation morphological operator to the remaining pixels; Use segmentation algorithm to divide the original image into multiple color and area segments , project the boundary low confidence area to and The boundary enhancement result of the corroded area is obtained.

10. The control method of the large steel structure corrosion damage integrated repair robot according to claim 9, characterized in that: Project the boundary low confidence region to and Above, including: ； in, To obtain the precise outline of the rusted area using the projection method, Represents the logical AND operation, for and and and There is also a union of the intersection parts.

Images