Large-model double-arm robot collaborative planning method for underwater equipment repair

Abstract

The invention relates to the technical field of coordinated motion planning and control of double-arm robots, in particular to a large-model double-arm robot collaborative planning method for underwater equipment repair. According to the method, a double-arm kinematics model based on a relative Jacobian matrix is constructed, data are collected by combining vision, sonar and force sense multi-mode sensors, a tail end pose error and integral information are introduced to construct an anti-noise motion planning scheme, a noise compensation strategy is designed, an optimal solution is solved through numerical difference, and the noise compensation precision is improved. And meanwhile, a large model is combined to recognize voice or image signals to realize intelligent action arrangement. According to the method, the problems of low precision and poor adaptability caused by difficulty in adapting to underwater dynamic interference, insufficient noise immunity and lack of large model fusion in the existing scheme are solved, the precision and noise immunity of double-arm cooperation are improved, the intelligent decision-making capability is enhanced, the method is effectively adapted to high-pressure, high-corrosion and other complex underwater environments, and the high-precision and high-reliability requirements of equipment repair are met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coordinated motion planning and control of dual-arm robots, and in particular to a large model dual-arm robot collaborative planning method for underwater equipment repair. BACKGROUND

[0002] Compared with single-arm robots, dual-arm robots have more flexibility due to the operation capability of dual arms, and have obvious advantages in tasks such as multi-tool cooperation and complex structure alignment. For example, through the synchronous operation of dual arms, fine operations such as the disassembly of parts of underwater equipment and the docking of pipelines can be completed. When facing complex environments such as high pressure, high corrosion and low visibility underwater, it is difficult for humans to directly carry out high-precision repair operations. At this time, the motion planning capability of dual-arm robots becomes the key to ensuring the smooth implementation of repair tasks, and its performance is directly related to the efficiency and quality of underwater equipment repair. At present, there are various dual-arm motion planning schemes applied in the field of underwater robots, but there are still obvious limitations in actual application. Specifically, these schemes are usually difficult to adapt to dynamic environmental disturbances such as water flow disturbance, have insufficient robustness to underwater time-varying noise, and lack task semantic understanding and large model fusion capability, resulting in low pose control precision and poor adaptability to complex repair scenarios.

[0003] Therefore, for the underwater equipment repair scene, it is of great practical significance to design a dual-arm robot planning method that can adapt to dynamic environmental disturbances, has strong anti-noise characteristics, and can flexibly respond to diversified task requirements. This not only can improve the precision and reliability of underwater repair operations and reduce the risk of task failure, but also can enhance the adaptability of robots in complex underwater scenarios and promote the development of underwater equipment maintenance technology to a higher level. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a large model dual-arm robot collaborative planning method for underwater equipment repair to solve the technical problems that the existing underwater dual-arm robot planning method is difficult to adapt to dynamic environmental disturbances such as water flow disturbance, has insufficient robustness to underwater time-varying noise, and lacks task semantic understanding and large model fusion capability, resulting in low pose control precision and poor adaptability to complex repair scenarios.

[0005] To achieve the above purpose, the technical scheme provided by the present application is as follows: a large model dual-arm robot collaborative planning method for underwater equipment repair, the method comprising: constructing a dual-arm robot kinematics model based on a relative Jacobian matrix by calculating the Jacobian matrix of the dual arms respectively and combining the rotation and translation relationship thereof;

[0006] Considering the influence of noise environment on the motion planning accuracy of the dual-arm in the underwater collaborative repair process, a collaborative motion planning scheme with noise resistance characteristics is constructed by introducing the pose error of the dual-arm end effector and the integral information of the error according to the kinematic model. To ensure that the dual-arm robot can reach the expected trajectory during the motion planning process, a noise compensation strategy is designed based on the collaborative motion planning scheme. To reduce the error and improve the motion planning accuracy, the collaborative motion planning scheme with noise compensation is discretized by numerical difference calculation, i.e., a numerical difference solving scheme is constructed to solve the optimal solution. The large model receives the front-end voice or graphical signal, identifies the high-level action signal, arranges the action signal, plans the expected action signal, inputs the expected action signal into the numerical difference solving scheme, and outputs the joint angle solution, i.e., the large model identifies and arranges actions.

[0007] Preferably, the motion planning scheme with noise resistance characteristics step is:

[0008] First, the initial joint angle of the robot and the end effector trajectory are initialized. In each control period, the end effector pose data is collected through visual, sonar, and force sensing multi-modal sensors, and the pose error of the dual-arm robot during the coordinated motion planning process is calculated. Then, according to the current task type and environmental disturbance intensity, the time-varying weight of the error correction term is dynamically generated, and the correction term is added to the end effector desired velocity. Further, the joint angular velocity is calculated using the pseudo-inverse Jacobian matrix to control the robot effector action to adjust the pose. Finally, according to the end effector feedback, the error trend prediction module is used to update the pose error and velocity error in advance to realize closed-loop continuous regulation.

[0009] Preferably, the noise compensation strategy step is:

[0010] First, the nonlinear noise caused by environmental disturbance, modeling error, or sensor noise during operation is modeled through a nonlinear mapping function, and the compensation amount used to offset the noise is calculated and added to the original control signal. Then, the compensated signal is low-pass filtered to smooth the high-frequency components and improve the stability of the system. At the same time, the system also dynamically adjusts and corrects the compensation effect according to the accumulated tracking error during long-term operation, combining the integral gain and initial impact rate, to eliminate residual deviations and balance response speed and steady-state accuracy.

[0011] Preferably, the noise compensation strategy expression is:

[0012] First, according to the underwater nonlinear noise, the dynamic recursive model is constructed as follows

[0013]

[0014] where represents nonlinear noise, represents the noise compensation amount, represents the result of noise compensation, is the filter coefficient, filtering out high-frequency noise, represents the result of low-pass filtering, represents the result of cumulative error correction, represents the initial impact rate, represents the integral gain;

[0015] Furthermore, the noise compensation amount output by the signal layer is Introducing collaborative motion planning scheme to design current noise compensation weights , , the specific expression is

[0016]

[0017] is the joint angle vector of the left arm a and the right arm b, is the joint vector corresponding to the joint angular velocity, represents the pseudo-inverse matrix of the relative Jacobian matrix of the dual-arm robot, is the desired trajectory at the end, is the desired terminal velocity, time-varying weight ,satisfy , Represents the integral information of the posture error of the dual-arm robot end effector, It represents the pose error of the end effector of the dual-arm robot during coordinated motion planning and is defined as , Represents the nonlinear mapping function of dual-arm kinematics.

[0018] Preferably, the numerical difference solution is:

[0019] The collaborative motion planning scheme with noise compensation is discretized to obtain the optimal solution, using the following numerical difference formula:

[0020]

[0021] Wherein, the subscript k represents the number of iterations and k=5,6,7,…, , , represents the sampling interval;

[0022] The coordinated motion planning scheme in a noisy environment is discretized by a numerical difference formula and is designed as follows:

[0023]

[0024] wherein, , , , ; for the formula, 5 values are required to complete the initial iterative calculation, that is, , , , , ; in this case, a value can be given first , and the remaining four values are determined according to the following calculation formula:

[0025]

[0026]

[0027]

[0028]

[0029] On the basis of the above-mentioned 5 values, through numerical difference iterative calculation, the optimal solution of high-precision stable planning joint angle of the underwater dual-arm robot can be obtained, that is, , wherein T represents the cycle of dual-arm collaborative planning.

[0030] Preferably, the large model is a multi-modal large model based on a Transformer architecture.

[0031] Preferably, the specific steps of action planning and identification of the large model are:

[0032] After the multi-modal front end collects voice or underwater image signals, it inputs a multi-modal large model based on a Transformer architecture to perform noise robustness preprocessing; when the signal time sequence and spatial features are analyzed by the single-modal Transformer attention module built in the large model, the voice-image features are fused and deeply inferred through the multi-modal cross attention module, and the high-level action intention containing the underwater equipment repair task type and the operation constraint is identified; the action planning sub-module encodes the action intention into an action signal based on the underwater dual-arm relative Jacobian matrix kinematics constraint; the signal is connected to the numerical difference solution scheme of the dual-arm robot collaborative planning method of the large model for underwater equipment repair, and the optimal joint angle solution suitable for the repair task is output.

[0033] The present application can effectively overcome the shortcomings of the prior art, and through the deep integration of large models and motion control technology, the precision, robustness and intelligent decision-making ability of underwater equipment repair operations in complex environments are comprehensively improved. Specifically, by constructing a dual-arm kinematics model based on the relative Jacobian matrix, combining the rotation and translation relationship of the robot arm, the limitations of traditional single-arm modeling are broken through, providing a unified and accurate kinematics foundation for underwater fine repair actions, greatly improving the consistency and accuracy of pose control; the pose error and integral information of the dual-arm end effector are introduced to construct a noise-resistant motion planning scheme, which dynamically suppresses error accumulation in underwater time-varying noise, enhances the robustness of motion planning in complex environments, and ensures the stability of the repair trajectory; a noise compensation strategy is designed and combined with numerical difference discretization processing, which converts the collaborative planning problem into an efficient solvable numerical optimization problem, not only compensating for environmental noise interference, but also reducing calculation delay to meet the real-time control requirements of underwater; with the help of large model fusion of multi-modal signals such as voice and image, the intelligent upgrading of task semantic understanding and action arrangement is realized, that is, the image or voice instructions can be parsed, the damage form can be automatically identified, and the repair strategy can be adapted, breaking through the rigid limitations of traditional pre-programming mode, and significantly improving the task response ability and decision-making efficiency in complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only illustrate the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0035] Figure 1 The control method flowchart of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical scheme and advantages of the present application clearer, the following will further illustrate the present application in detail with reference to the drawings and specific embodiments. Among them Figure 1 The control method flowchart of the present application is a dual-arm robot collaborative planning method for underwater equipment repair. The specific application steps of the present embodiment are as follows.

[0037] Step 1: Construct a dual-arm robot kinematics model based on the relative Jacobian matrix

[0038] For a dual-arm robot consisting of a left arm a and a right arm b in an underwater environment, a hierarchical modeling strategy of "single-arm decoupling and collaborative constraints" is used to construct the kinematic equations. First, joint coordinate systems are defined for the left arm a and the right arm b, and environmentally compensated DH parameters are collected to adapt to the underwater waterproof joint structure. This takes into account the transmission clearance of the sealed joint and the correction of the fluid damping on the link motion relationship. The forward kinematic model of the single arm from the base to the end effector is derived using the exponential product method, and the respective pose transformation matrices are obtained. Subsequently, the collaborative constraints of the dual-arm end-effectors during underwater repair operations are analyzed—including relative pose synchronization accuracy and force interaction coordination rules. Based on these constraints, the single-arm models are coupled to derive the collaborative kinematic equations for the dual arms:

[0039]

[0040] in, It is a two-arm coupled Jacobian matrix that integrates the single-arm Jacobian and the cooperative constraint, which is different from the simple overall modeling and reflects the two-level correlation; is the joint angle vector of the left arm a and the right arm b, is the joint vector corresponding to the joint angular velocity; The desired terminal velocity vector after compensating for underwater fluid resistance and buoyancy interference is denoted by t, where time is the desired velocity vector. This strategy allows the kinematic model to maintain both single-arm control accuracy and accurately describe the coordination between the two arms, making it more suitable for the control requirements of complex underwater repair scenarios.

[0041] Step 2: Build a noise-resistant underwater dual-arm collaborative motion planning scheme

[0042] First, the robot's initial joint angles and end-effector trajectory are initialized. In each control cycle, the end-effector posture data is collected through vision, sonar, and force multimodal sensors to calculate the posture error of the end-effector during the coordinated motion planning process of the dual-arm robot. ,in, is the desired trajectory at the end, Represents the nonlinear mapping function of the dual-arm kinematics; then, the time-varying weights are dynamically generated according to the homogeneous transformation matrix of the end effector of the manipulator and the environmental interference that changes with time t. ,satisfy , online verification of stability, combined with probabilistic noise compensation Feedforward compensation of environmental disturbances is used to predict the velocity correction of water drag and buoyancy fluctuations, construct error correction terms and add them to the desired terminal velocity; further, the pseudo-inverse Jacobian matrix is ​​used to calculate the velocity of the terminal. Calculate the joint angular velocity, and finally, based on the end feedback, update the posture and velocity errors in advance through the error trend prediction module to achieve closed-loop control. The underwater dual-arm collaborative motion planning scheme in the noisy environment is constructed as follows:

[0043]

[0044] wherein, is a Jacobian pseudo-inverse matrix of the underwater environment adapted to the large model, is a terminal desired velocity, is regulated by the task and disturbance dynamics, represents the integral information of the dual-arm robot end effector pose error, integrates multi-modal perception and task semantic constraints, quantifies the noise probability distribution. Through the large model, the "perception-decision-control" is broken through, and the traditional static control is limited, and the underwater high-precision repair demand is adapted.

[0045] Step 3: Design a noise compensation strategy

[0046] In order to further ensure that the underwater dual-arm robot can accurately track the expected trajectory during motion planning, a noise compensation strategy is introduced on the basis of the existing collaborative motion planning scheme. First, the nonlinear noise caused by environmental disturbance, modeling error or sensor noise during operation is modeled through a nonlinear mapping function, and then the compensation amount used to offset the noise is calculated and added to the original control signal; then the compensated signal is low-pass filtered to smooth the high-frequency components and improve the stability of the system; at the same time, the system also dynamically adjusts and corrects the compensation effect according to the accumulated tracking error in the long-term operation, combined with the integral gain and the initial impact rate, to eliminate residual deviation and balance response speed and steady-state accuracy. The noise compensation strategy is:

[0047] First, according to the underwater nonlinear noise, the dynamic recursive model is constructed as follows

[0048]

[0049] wherein represents the nonlinear noise, represents the noise compensation amount, represents the result of noise compensation, represents the result of low-pass filtering, represents the result of cumulative error correction, represents the initial impact rate, represents the integral gain.

[0050] Further, the noise compensation amount output by the above signal layer is introduced into the collaborative motion planning scheme, and the current noise compensation weight , is designed. The specific expression is

[0051]

[0052] Step 4: Constructing a numerical difference solution scheme to solve the optimal solution

[0053] To further optimize the control accuracy and stability, the cooperative motion planning scheme with noise compensation is discretized to solve the optimal solution, and the following numerical difference formula is used:

[0054]

[0055] where subscript k represents the iteration number and k = 5, 6, 7, …, , , denotes the sampling interval.

[0056] The cooperative motion planning scheme in the noise environment is discretized by the numerical difference formula, and the design is:

[0057]

[0058] where , , , . For the formula, 5 numerical values are needed to complete the initial iteration calculation. That is , , , , In this case, a numerical value is given first, and the remaining four numerical values are determined according to the following calculation formula:

[0059]

[0060]

[0061]

[0062]

[0063] Based on the above 5 numerical values, the optimal solution of the high-precision stable planning joint angle of the underwater dual-arm robot can be obtained by numerical difference iteration calculation, that is where T represents the period of dual-arm cooperative planning.

[0064] Step 5: Input voice or image signals, large model recognition and action arrangement

[0065] After the multi-modal front-end collects the speech or underwater image signals, a multi-modal large model pre-training subnetwork based on the Transformer architecture is input to perform noise robustness preprocessing; when the signal time sequence and spatial features are analyzed by the single-modal Transformer attention module built in the large model, the speech-image features are fused and deeply inferred through the multi-modal cross attention module, and the high-level action intention containing the underwater equipment repair task type and operation constraints is identified; the action intention is arranged into an action signal by the action planning sub-module based on the kinematic constraint of the relative Jacobian matrix of the underwater dual arms; the signal is connected to the numerical difference solution scheme of the large model dual-arm robot collaborative planning method for underwater equipment repair, and the optimal joint angle solution adapted to the repair task is output.

[0066] The optimal joint angle of the underwater dual-arm collaborative planning robot is calculated by the large model dual-arm robot collaborative planning method for underwater equipment repair, and the calculation result is transmitted to the robot controller. The robot controller is built in a driving algorithm adapted to the collaborative characteristics of the underwater dual arms, which is based on the relative Jacobian matrix dual-arm kinematics model constructed in step 1, according to the underwater dual-arm collaborative motion planning scheme with noise resistance in step 2, fuses the noise compensation data in step 3, accepts the numerical difference solution scheme in step 4, combines the action signal output by the large model in step 5, and converts the calculated joint angle solution into joint driving control instructions. Through the real-time closed-loop control strategy, the joint motion trajectory is dynamically corrected, and the dual-arm pose collaborative accuracy and the execution stability of complex repair actions are ensured in the underwater noise interference environment.

[0067] Those skilled in the art will understand that the above discussion of any embodiment is only exemplary and is not intended to limit the scope of the present application and the claims to these examples; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the present application as described above. In order to be brief, they are not provided in detail.

[0068] The present application is intended to cover all such alternatives, modifications and variations as fall within the broad scope of the appended claims. Accordingly, any and all such modifications, variations, omissions, and equivalents are intended to be encompassed by the present application.

Claims

1. A large-scale dual-arm robot collaborative planning method for underwater equipment repair, characterized in that: Includes the following steps Step 1: By calculating the Jacobian matrices of the two arms respectively and combining their rotation and translation relationships, a kinematic model of the dual-arm robot based on the relative Jacobian matrix is ​​constructed; Step 2: Introduce the pose error of the dual-arm end effector and the integral information of the error to construct a collaborative motion planning scheme with noise-resistant characteristics; Step 3: Based on the collaborative motion planning scheme, a noise compensation strategy is designed; Step 4: Discretize the collaborative motion planning scheme with the noise compensation by numerical difference calculation, construct a numerical difference solution, and find the optimal solution; Step 5: Input voice or image signals. The large model receives the front-end voice or image signals, identifies high-level motion signals, arranges the motion signals, plans the expected motion signals, inputs the expected motion signals into the numerical difference solution, and outputs the joint angle solution.

2. A large-scale dual-arm robot collaborative planning method for underwater equipment repair according to claim 1, characterized in that: The specific steps of step 2 include: Step 2.

1. Initialize the robot's initial joint angles and end-effector trajectory. In each control cycle, collect end-effector pose data using multimodal sensors including vision, sonar, and force sensing, and calculate the end-effector pose error during coordinated motion planning. Step 2.2: Dynamically generate the time-varying weight of the error correction term based on the current task type and environmental interference intensity, and add the correction term to the desired velocity of the end effector; Step 2.3: Use the pseudo-inverse Jacobian matrix to calculate the joint angular velocity and control the robot actuator movement to adjust the posture; Step 2.4: Based on the feedback from the end effector, the error trend prediction module is used to update the posture error and velocity error in advance to achieve closed-loop continuous control.

3. The collaborative planning method for a large-scale dual-arm robot for underwater equipment repair according to claim 1 is characterized in that: The specific steps of step 3 include: Step 3.1: Model the nonlinear noise caused by environmental interference, modeling errors, or sensor noise during operation using a nonlinear mapping function, and then calculate the compensation amount used to offset the noise and add it to the original control signal; Step 3.2: Low-pass filter the compensated signal to smooth the high-frequency components and improve the stability of the system. Step 3.3: Based on the accumulated tracking error during long-term operation, the system dynamically adjusts and corrects the compensation effect in combination with the integral gain and the initial impact rate to eliminate the residual deviation while taking into account both the response speed and the steady-state accuracy.

4. The collaborative planning method for a large-scale dual-arm robot for underwater equipment repair according to claim 1 is characterized in that: The noise compensation strategy expression of step 3 is: Step 3.1: Based on underwater nonlinear noise, the dynamic recursive model is constructed as follows: in represents nonlinear noise, represents the noise compensation amount, represents the result of noise compensation, is the filter coefficient, filtering out high-frequency noise, represents the result of low-pass filtering, represents the result of cumulative error correction, represents the initial impact rate, represents the integral gain; Step 3.2: Compensate the noise output of the signal layer Introducing collaborative motion planning scheme to design current noise compensation weights , , the specific expression is is the joint angle vector of the left arm a and the right arm b, is the joint vector corresponding to the joint angular velocity, represents the pseudo-inverse matrix of the relative Jacobian matrix of the dual-arm robot, is the desired trajectory at the end, is the desired terminal velocity, time-varying weight ,satisfy , Represents the integral information of the posture error of the dual-arm robot end effector, It represents the pose error of the end effector of the dual-arm robot during coordinated motion planning and is defined as , Represents the nonlinear mapping function of dual-arm kinematics.

5. The collaborative planning method for a large-scale dual-arm robot for underwater equipment repair according to claim 1 is characterized in that: The numerical difference solution of step 4 is: Step 4.1: Discretize the collaborative motion planning scheme with the noise compensation to find the optimal solution, using the following numerical difference formula: Wherein, the subscript k represents the number of iterations and k=5,6,7,…, , , represents the sampling interval; Step 4.2: Discretize the coordinated motion planning scheme in a noisy environment using a numerical difference formula, and design it as follows: in, , , , ; For the formula described above, 5 values ​​are required to complete the initial iterative calculation, namely , , , , ; In this case, you can first give a value , the remaining four values ​​are determined according to the following calculation formula: Based on the above five values, the optimal solution for high-precision and stable planning of joint angles of the underwater dual-arm robot can be obtained through numerical difference iterative calculation, namely: , where T represents the period of dual-arm collaborative planning.

6. The large-scale dual-arm robot collaborative planning method for underwater equipment repair according to claim 1 is characterized in that: The large model in step 5 is a multimodal large model based on the Transformer architecture.

7. The collaborative planning method for a large-scale dual-arm robot for underwater equipment repair according to claim 1 is characterized in that: The specific steps of step 5 include: Step 5.1: After the multimodal front-end collects the voice or underwater image signal, it inputs it into the large model pre-trained sub-network to perform noise robustness preprocessing; Step 5.2: The built-in attention module of the large model analyzes the temporal and spatial features of the signal. The cross-attention module then fuses the speech and image features and performs deep reasoning to identify the high-level action intention, including the underwater equipment repair task type and operation constraints. Step 5.3: The action planning submodule composes the action intention into an action signal based on the kinematic constraints of the underwater dual-arm relative Jacobian matrix; Step 5.4: Connect the signal to the numerical difference solution of the large-scale dual-arm robot collaborative planning method for underwater equipment repair, and output the optimal joint angle solution suitable for the repair task.

Images