A robot long-time assembly task planning method and system based on reversibility constraint

Abstract

The application discloses a kind of long-time assembly task planning method and system of robot based on reversibility constraint. For the problem of step irreversibility in assembly task, high failure cost, reversibility and failure semantics of assembly subtask are explicitly modeled in task planning layer, a subtask irreversible risk measurement model is constructed, and on this basis, risk-aware assembly task sequence planning is carried out, so as to reduce the probability of irreversible failure. In the execution phase, for each assembly subtask obtained by planning, a visual-language-action model is used to directly control the robot to complete specific operations, achieving effective collaboration between high-level planning and low-level operations. The application can significantly improve the safety and success rate of the robot in complex assembly tasks, and is suitable for long-time, multi-step industrial assembly scenarios.

Description

A method and system for planning long-term robot assembly tasks based on reversibility constraints Technical Field

[0001] This invention belongs to the field of robotics and intelligent manufacturing, and particularly relates to a method and system for planning long-term robot assembly tasks based on reversibility constraints. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, robotic arms are increasingly being used in assembly operations. However, the actual assembly process usually consists of multiple steps, with strict sequential constraints between them; some assembly steps have significant irreversibility, such as fastening, pressing, and welding operations, and failure to execute these steps may lead to damage to parts or even scrapping of the entire assembly; the assembly process is easily affected by factors such as environmental uncertainties, positioning errors, assembly tolerances, and changes in contact states, which makes assembly failures occur frequently.

[0003] Therefore, it is necessary to plan the assembly tasks of robotic arms. However, existing robotic arm task planning methods mostly assume that assembly steps can be repeated, and usually only plan and make decisions based on binary feedback of "success / failure," lacking fine-grained modeling and analysis of assembly failure types, risk levels, and recoverability. In addition, existing rule-based or learning-based task planning methods often struggle to effectively assess potential risks before executing irreversible steps when facing long-duration assembly tasks. Once assembly failure occurs, the system usually lacks reasonable recovery or rollback strategies, leading to a reduced assembly success rate and insufficient system safety and reliability. Summary of the Invention

[0004] Purpose of the invention: This invention provides a robot long-term assembly task planning method and system based on reversibility constraints, aiming to solve the problems of existing technologies lacking fine-grained modeling and analysis of assembly failure types, risk levels and recoverability, lacking reasonable recovery or rollback strategies, resulting in reduced assembly success rate and insufficient system safety and reliability.

[0005] Technical solution: This invention provides a robot long-term assembly task planning method based on reversibility constraints, including:

[0006] Obtain assembly task and assembly object information, model the assembly task based on the assembly object information and the preset assembly process flow, decompose the assembly task into multiple sub-tasks, and convert each sub-task into a structured assembly semantic description.

[0007] Define a recovery cost function and an irreversible risk measurement function, and preset a loss cost threshold to construct a sub-task-level reversibility constraint model; calculate the reversibility of each sub-task based on the sub-task-level reversibility constraint model.

[0008] A dependency graph between subtasks is established based on the assembly object information to determine the preceding dependent tasks of each subtask. Based on the reversibility of the subtasks and the dependency graph between them, a restricted topology sorting algorithm based on risk priority is used to generate the global task execution sequence of the assembly task.

[0009] Subtasks are scheduled sequentially according to the global task execution sequence, and the structured assembly semantic description of the scheduled subtasks is used as text prompts to input the pre-trained visual-language-action model to obtain the robot's underlying motion control instructions corresponding to the scheduled subtasks, thereby driving the robotic arm to execute the robot's underlying motion control instructions corresponding to the scheduled subtasks.

[0010] During the execution of the robot's underlying motion control commands corresponding to the scheduled sub-tasks by the robotic arm, the execution status and assembly results are collected. A preset multi-source state comparator is used to compare the execution status, assembly results, and predefined completion status to determine whether the scheduled sub-task has been successfully executed. If the scheduled sub-task is successfully executed, the next sub-task in the global task execution sequence is executed. If the scheduled sub-task fails to execute, a rollback or shutdown operation is performed based on the reversibility of the scheduled sub-task.

[0011] Furthermore, the assembly task is modeled based on the assembly object information and the preset assembly process flow, and the assembly task is decomposed into multiple sub-tasks, including:

[0012] By analyzing the CAD model of the assembly object, the three-dimensional geometric features of the parts are obtained. Then, combined with the preset assembly process, a multi-level assembly state space model is constructed with the initial assembly state as the source point and the final product state as the destination point. The multi-level assembly state space model describes the evolution path from parts to components and from components to the final product state.

[0013] Based on the multi-level assembly state space model, the contact surfaces and connection attributes between parts are identified by analyzing three-dimensional geometric features. These are used as dividing boundaries to decompose the assembly task into multiple sub-tasks. Each sub-task is defined with a state transition function from the initial state to the target state. Each sub-task has a state tuple, including the initial state, target state, operation type, and process constraints.

[0014] Furthermore, the step of converting each subtask into a structured assembly semantic description includes: using a natural language generation algorithm based on predefined rule templates to extract the initial state, target state, operation type, and process constraints from the state tuple of the subtask, and converting the subtask into text prompts understandable by the vision-language-action model. The template is:

[0015] ;

[0016] in, For operation type, For the target object, For reference, These are process constraints.

[0017] Furthermore, the construction of the subtask-level reversibility constraint model includes:

[0018] For subtasks Possible failure states Define the recovery cost function to compute the subtask From failure status The cost of restoring to the initial state The formula is:

[0019] ;

[0020] in, In the failure state The time cost required to perform the recovery action; In the failure state The required level for human intervention is indicated below; In the failure state The following is a quantitative value of the component damage; They are respectively , , The preset weight parameters;

[0021] Combined with failure status The probability of occurrence is defined, and an irreversible risk metric function is used to calculate the subtask. Average loss cost The formula is:

[0022] ;

[0023] in, For subtasks The set of all possible failure states To execute subtasks Failure status occurs The average failure frequency.

[0024] Furthermore, the calculation of the reversibility of each subtask includes:

[0025] The calculated subtasks Average loss cost Compared with the preset loss cost threshold Compare;

[0026] like Then determine the subtask It is a reversible subtask;

[0027] like Then determine the subtask This is an irreversible subtask.

[0028] Furthermore, establishing the dependency graph between subtasks includes:

[0029] Extracting geometric topological constraints: Based on the CAD model of the assembly object, the occlusion relationship of parts in the assembly direction is identified by using degree of freedom analysis; if the assembly path of part A passes through the envelope space of part B, then part B is defined as the prerequisite dependency of part A. Therefore, the subtask containing part B is the prerequisite dependency task containing the subtask of part A.

[0030] Extract physical connection constraints: Identify the fastening relationships in the assembly process by combining the assembly relationships between assembly objects, and define the support as the prerequisite dependency of the connected part. Therefore, the subtask containing the support is the prerequisite dependency task containing the connected part.

[0031] Construct a dependency graph: Summarize the above constraints to form a priority topology graph among subtasks, i.e., a dependency graph. In the graph, nodes represent subtasks, and directed edges represent the order of tasks.

[0032] Furthermore, the global task execution sequence for generating the assembly task includes:

[0033] In the k-th step of generating the global task execution sequence, the following steps are performed, where 1≤k≤M, and M is the number of subtasks in the assembly task:

[0034] Determine the candidate task set : Filter all subtasks that meet the constraints and form a candidate task set. The constraint is that all the prerequisite dependent tasks of the subtask have been completed in steps 1 to k-1 and the subtask does not appear in the generated global task execution sequence.

[0035] If candidate task set If there are reversible subtasks, then the candidate task set will be... Remove all irreversible subtasks to update the candidate task set. ;

[0036] For candidate task set Each subtask calculates the risk perception priority. The formula is:

[0037] ;

[0038] in, This represents the length of the longest path from the initial state to the current subtask in the dependency graph. Positive weights;

[0039] Define the optimization objective function The formula is:

[0040] ;

[0041] in, The position weight function is given by the formula: N is the number of subtasks in the candidate task set; the goal is to minimize the optimization objective function. To achieve the goal, in the k-th step... The task with the smallest value Add to the global task execution sequence. The formula is:

[0042] .

[0043] Furthermore, determining whether the scheduled subtask was executed successfully includes: using a multi-source state comparator to perform pose verification, force verification, and visual semantic verification.

[0044] The pose verification is achieved by calculating the geometric deviation between the assembly result and the predefined completed state, including the Euclidean distance deviation between the position vectors of the assembly result and the predefined completed state and the rotation angle deviation between the attitude quaternions. If the Euclidean distance deviation is less than or equal to the preset distance deviation and the rotation angle deviation is less than or equal to the preset rotation deviation, the pose verification is deemed to be successful; otherwise, the pose verification is deemed to be unsuccessful.

[0045] The force verification is determined by thresholding the peak contact force or steady-state holding force characteristic value extracted from the execution state. If the peak contact force or steady-state holding force characteristic value is within the preset process torque threshold range, the force verification is deemed to pass; otherwise, the force verification is deemed to fail.

[0046] The visual semantic verification calculates the rotation and translation errors between the assembly result and the CAD model of the assembly object using a visual recognition model, and calculates the width of the connected component. If the rotation error is less than or equal to the preset geometric alignment rotation error, the translation error is less than or equal to the preset geometric alignment translation error, and the width of the connected component is less than or equal to the preset gap width threshold, then the visual semantic verification is deemed to pass; otherwise, the visual semantic verification is deemed to fail.

[0047] If the pose verification, force verification, and visual semantic verification all pass, the scheduled subtask is considered to have been executed successfully; otherwise, the scheduled subtask is considered to have failed.

[0048] Furthermore, the step of performing rollback or halt operations based on the reversibility of the scheduled subtasks includes:

[0049] If the scheduled subtask is a reversible subtask, the pre-stored reverse recovery strategy will be automatically invoked to perform a rollback operation. After the scheduled subtask is restored to its initial state, the scheduled subtask will be executed again.

[0050] If the scheduled subtask is an irreversible subtask, it is considered a catastrophic failure, and subsequent task planning is immediately terminated and a safe shutdown command is output.

[0051] This invention also provides a robot long-term assembly task planning system based on reversibility constraints, comprising:

[0052] The task modeling and state awareness module is used to acquire assembly task and assembly object information, model the assembly task according to the assembly object information and the preset assembly process flow, decompose the assembly task into multiple sub-tasks, and convert each sub-task into a structured assembly semantic description.

[0053] The reversibility perception module is used to define the recovery cost function and the irreversibility risk measurement function, and preset the loss cost threshold to construct a sub-task-level reversibility constraint model; and calculate the reversibility of each sub-task based on the sub-task-level reversibility constraint model.

[0054] The execution sequence generation module is used to establish a dependency graph between subtasks based on the assembly object information to determine the preceding dependent tasks of the subtasks; based on the reversibility of the subtasks and the dependency graph between the subtasks, a restricted topology sorting algorithm based on risk priority is used to generate the global task execution sequence of the assembly task.

[0055] The execution control module is used to schedule subtasks sequentially according to the global task execution sequence, and input the structured assembly semantic description of the scheduled subtasks as text prompts into the pre-trained visual-language-action model to obtain the robot's underlying motion control instructions corresponding to the scheduled subtasks, and drive the robotic arm to execute the robot's underlying motion control instructions corresponding to the scheduled subtasks.

[0056] The execution monitoring and feedback module is used to collect the execution status and assembly results during the process of the robotic arm executing the robot's underlying motion control commands corresponding to the scheduled sub-tasks. It uses a preset multi-source state comparator to compare the execution status, assembly results and predefined completion status to determine whether the scheduled sub-task has been executed successfully. If the scheduled sub-task is executed successfully, the next sub-task in the global task execution sequence is executed. If the scheduled sub-task fails, a rollback or shutdown operation is performed based on the reversibility of the scheduled sub-task.

[0057] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0058] 1. By introducing subtask reversibility modeling and recovery cost quantification functions, high-risk steps can be identified during the planning stage, changing the limitation of traditional planning that only focuses on the successful path;

[0059] 2. By combining risk measurement and position weight optimization into a sequence generation mechanism, irreversible operations are forced to be postponed, thus preserving the backtracking and recovery capabilities after a failure in the middle of the task to the greatest extent and reducing the probability of interruption during long-term assembly.

[0060] 3. By using the VLA large model to fuse visual, linguistic and action information, an end-to-end mapping from high-level semantics to low-level control is achieved, enhancing the system's adaptability to minor disturbances in unstructured environments.

[0061] 4. A dynamic fault handling mechanism based on reversibility attributes was established, which realizes automatic recovery of reversible errors and timely loss prevention of irreversible errors, greatly improving the security and robustness of the system. Attached Figure Description

[0062] Figure 1 is a flowchart of the method of the present invention.

[0063] Figure 2 is a system framework diagram of the present invention. Detailed Implementation

[0064] Example 1

[0065] As shown in Figure 1, the robot long-term assembly task planning method based on reversibility constraints according to the present invention includes:

[0066] S1. Obtain assembly task and assembly object information; model the assembly task based on the assembly object information and the preset assembly process flow; decompose the assembly task into multiple sub-tasks; and convert each sub-task into a structured assembly semantic description, specifically:

[0067] S1.1 Information Acquisition and Preprocessing: Assembly task instructions (such as "Assemble the reducer of model X") are acquired through the user interface, and assembly object information is retrieved from the pre-stored product database. The assembly object information includes: the geometric model of the assembly object (such as STEP / STL format CAD data) and the assembly relationships between the assembly objects (such as assembly constraint matrices). The geometric model of the assembly object is used to identify contact surfaces (such as shaft holes, planes, etc.) and the time cost and Euclidean distance for path planning in S2. The assembly relationships between the assembly objects are used to determine the assembly priority of components when establishing the dependency graph in S3.

[0068] S1.2 Assembly process modeling and state space construction: By analyzing the CAD model of the assembly object, the three-dimensional geometric features of the parts are obtained. Then, combined with the preset assembly process flow, a multi-level assembly state space model is constructed with the initial assembly state as the source point and the final product state as the destination point. The multi-level assembly state space model describes the evolution path from parts to components and from components to the final product state, providing a global state reference for assembly task decomposition.

[0069] S1.3 Task Decomposition Based on State Space Model: Based on the multi-level assembly state space model, the assembly process is traversed, and the contact surfaces and connection attributes between parts are identified using three-dimensional geometric features. These are used as dividing boundaries to divide the assembly task into multiple atomic-level action operators, i.e., subtasks. Each subtask It is defined as a unique state transition function, with the following formula:

[0070] ;

[0071] in, The initial state is defined as the pose and connection state of the relevant components in the workspace before the action is executed. The target state, or completed state, is unique and defined as the expected pose and physical constraint state of the relevant components after the action is successfully executed. Each subtask corresponds to only one specific edge in the multi-level assembly state space model, ensuring... The uniqueness of the set of geometric and physical properties. The state tuple for each subtask. ,in, Operation types include grab, align, insert, and fasten; The process constraints that must be met to perform this subtask (such as tightening torque and position tolerance).

[0072] S1.4, Structured Assembly Semantic Conversion: The natural language generation algorithm based on predefined rule templates extracts the initial state, target state, operation type, and process constraints from the state tuples of subtasks, converting the subtasks into text prompts understandable by the visual-language-action model. The template is as follows:

[0073] ;

[0074] in, For operation type, For the target object, For reference, These are process constraints.

[0075] For example, for a subtask with the operation type "Insert", the operation object "Part_A (axis)", the target reference "Part_B_Hole_C (shell hole)" and the constraint condition "H7 fit", the generated structured assembly semantic description is: "Precisely insert part A (axis) into the hole C of part B (shell)", thereby guiding the VLA model to focus on assembly features and generate fine motion trajectories.

[0076] S2. Define the recovery cost function and the irreversible risk measurement function, and preset the loss cost threshold to construct a sub-task-level reversibility constraint model; calculate the reversibility of each sub-task based on the sub-task-level reversibility constraint model, specifically:

[0077] S2.1 Define the recovery cost function:

[0078] For subtasks Possible failure states The subtask is calculated by defining the recovery cost function. From failure status The cost of restoring to the initial state The formula is:

[0079] ;

[0080] in, Indicates a failure state The time cost required to perform the recovery action is obtained by comparing the current pose of the robotic arm with the initial pose of the subtask. The Euclidean distance between them, combined with the average moving speed preset by the robotic arm, is used to calculate the no-load path planning time, and the estimated process time required for the operation (such as rotating out or pulling out) is added on.

[0081] In the failure state Below, the manual intervention requirement indicator is a set of preset discrete tiered values ​​stored in the task risk database. When the failure status... When the obstacle can be avoided autonomously by the robot or by resetting its pose, When in failure state When a part falls outside the workspace or becomes obstructed and requires manual repositioning. When in failure state When severe entanglement or clamp failure requires manual intervention for repair, ;

[0082] In the failure state The degree of damage to the components is a quantitative value, calculated using the Sigmoid function based on the cumulative over-limit impulse, with the following formula:

[0083] ;

[0084] in, To accumulate the over-limit impulse, i.e., the time integral of the over-limit load during execution, the formula is:

[0085] ;

[0086] in, This provides real-time contact force and torque feedback from the sensor; The allowable elastic limit load threshold for the part; The damage threshold, also known as the offset of the center point of the Sigmoid function, represents the degree of damage when the accumulated impulse reaches this value. Reaching 0.5; This is the damage sensitivity coefficient, i.e., the slope of the sigmoid function, used to adjust how steeply the damage value increases with increasing impulse. A larger value indicates that the component is more sensitive to excessive loads. Through the nonlinear mapping of this formula, the progressive failure process of the component from elastic deformation to plastic failure is simulated to output a value. The degree to which the value approaches 1 quantifies the probability of a part being scrapped;

[0087] They are respectively , , The preset weighting parameters are used to balance time, manpower, and material costs.

[0088] S2.2, Define the irreversible risk measurement function:

[0089] Combined with failure status The probability of occurrence is calculated for each subtask by defining an irreversible risk metric function. Average loss cost The formula is:

[0090] ;

[0091] in, For subtasks The set of all possible failure states To execute subtasks based on historical assembly data. Failure status occurs The average failure frequency.

[0092] S2.3, Determine subtasks The reversibility is determined by calculating the average loss cost. Compared with the preset loss cost threshold Compare;

[0093] like Then determine the subtask It is a reversible subtask, meaning that the cost of failure is controllable;

[0094] like Then determine the subtask It is an irreversible subtask, meaning that failure in this subtask may lead to the complete failure of the mission or a high cost.

[0095] S3. Based on the assembly object information, establish a dependency graph between subtasks to determine the prerequisite dependent tasks of each subtask; based on the reversibility of the subtasks and the dependency graph between them, use a risk-priority-based restricted topology sorting algorithm to generate the global task execution sequence for the assembly tasks, specifically:

[0096] S3.1, Establish a dependency graph between subtasks:

[0097] S3.1.1 Extracting geometric topological constraints: Based on the CAD model of the assembly object, the occlusion relationship of parts in the assembly direction is identified by using degree of freedom analysis; if the assembly path of part A passes through the envelope space of part B, then part B is defined as the prerequisite dependency of part A. Therefore, the subtask containing part B is the prerequisite dependency task containing the subtask of part A.

[0098] S3.1.2 Extracting physical connection constraints: Identify the fastening relationships in the assembly process (such as bolted connections) by combining the assembly relationships between assembly objects. Define the support as the prerequisite dependency of the connected parts. Therefore, the subtask containing the support is the prerequisite dependency task containing the connected parts.

[0099] S3.1.3 Summarize the above constraints to form a priority topology graph between subtasks, i.e., a dependency graph. In the graph, nodes represent subtasks, and directed edges represent the order of tasks.

[0100] S3.2. A risk-priority-based restricted topology sorting algorithm is used to generate the global task execution sequence for the assembly task, specifically as follows:

[0101] In the k-th step of generating the global task execution sequence, the following steps are performed, 1≤k≤M, where M is the number of subtasks in the assembly task:

[0102] S3.2.1 Determine the candidate task set : Filter all subtasks that meet the constraints and form a candidate task set. The constraint condition is that all the prerequisite dependent tasks of the subtask (defined by the in-degree / out-degree of the dependency graph between subtasks) have been completed in steps 1 to k-1, and the subtask... This means that the subtask did not appear in the generated sequence;

[0103] If candidate task set If there are reversible subtasks, then the candidate task set will be... Remove all irreversible subtasks to update the candidate task set. ;

[0104] S3.2.2, for the candidate task set Each subtask calculates the risk perception priority. The formula is:

[0105] ;

[0106] in, This represents the length of the longest path from the initial state to the current task in the dependency graph. Positive weights;

[0107] S3.2.3, Define the optimization objective function The formula is:

[0108] ;

[0109] in, The position weight function is given by the formula: N is the number of subtasks in the candidate task set; the optimization objective function varies with the execution order. The increase and monotonically decrease make high-risk tasks (large) (Value) Match small weights (large) as much as possible The value (i.e., executed later) maximizes the system's backtracking capability; this patent aims to minimize the optimization objective function. For the goal, due to The risk term in the objective function is negatively correlated with the position weight in the objective function, therefore the risk term in step k is... The task with the smallest value Add to the global task execution sequence. The formula is:

[0110] .

[0111] S4. Form a set of unexecuted subtasks into a set of subtasks to be executed: Schedule the subtasks in the current set of subtasks to be executed in sequence according to the global task execution sequence. Use the structured assembly semantic description of the scheduled subtask as a text prompt, and combine it with the real-time RGB-D visual information of the assembly environment and the robot joint angle and end-effector pose information to form a multimodal data input pre-trained vision-language-action (VLA) model. Through multimodal fusion inference of the VLA model, jointly encode and decode the input to generate the robot's low-level motion control instructions corresponding to the scheduled subtasks, and drive the robotic arm to execute the motion control instructions. Drive the robotic arm to perform fine assembly actions in an unstructured environment and adaptively handle small disturbances in the environment to achieve end-to-end mapping from high-level semantics to low-level actions.

[0112] S5. During and after the robotic arm execution of each subtask, the execution status and assembly result are collected in real time. A multi-source state comparator is used to compare the execution status and assembly result with a predefined completion status to determine whether the scheduled subtask was successfully executed. The execution status includes the real-time pose data of the robotic arm's end effector and the time-domain data stream of the force sensor. The assembly result is the final pose of the part after the action is completed. The predefined completion state is the target state of the subtask. The multi-source state comparator is a software logic judgment module that integrates visual, force, and robot body pose data.

[0113] The comparison and verification process includes pose verification, force verification, and visual semantic verification.

[0114] Pose verification: Calculate assembly results Compared with predefined completion status The geometric deviations between them include the Euclidean distance deviations between the calculated position vectors. and the rotation angle deviation between attitude quaternions If satisfied and If so, the geometric pose verification is considered successful. and The preset tolerance threshold is based on the accuracy requirements of the basic process.

[0115] Force verification: Read the force sensor data in the execution state and extract the peak contact force or steady-state holding force characteristic value during the action; if the characteristic value falls within the preset process torque threshold range, the physical connection verification is determined to be successful.

[0116] Visual semantic verification: A visual recognition model is used to perform geometric alignment and residual analysis on the acquired local point cloud and RGB images to confirm that there is no misalignment or gap between parts. This verification is achieved by triggering a visual sensor to acquire images of the assembly area and using the Iterative Closest Point (ICP) algorithm to align the real-time point cloud. With standard CAD models Registration is used to calculate the rotation error of the transformation matrix. With translation error Simultaneously calculate the connected component width of depth gradient jumps and high-contrast shadows within the region of interest (ROI). If both conditions are met , and If so, the visual semantic verification is deemed successful. , and These are the preset pose alignment tolerance threshold and gap width threshold, respectively;

[0117] If all of the above verifications—force verification and visual semantic verification—pass, the output result is "Subtask executed successfully." If any verification fails, the output result is "Subtask executed failed," and the current state is marked as the corresponding failure state. The failure status Failure due to pose deviation, torque overload, or visual semantic defect.

[0118] If the scheduled subtask is executed successfully, the scheduled subtask is removed from the set of subtasks to be executed. It is then determined whether all subtasks in the assembly task have been executed successfully. If so, the assembly completion instruction is output and the process ends. If not, the process returns to step S4 and the next subtask in the set of subtasks to be executed is scheduled in sequence according to the global task execution sequence.

[0119] If the scheduled subtask fails to execute, a closed-loop adjustment strategy is triggered based on the reversibility of the scheduled subtask:

[0120] If the scheduled subtask is a reversible subtask ( If the pre-stored reverse recovery strategy is invoked, a rollback operation (such as withdrawing the gripper or resetting the part) will be performed. After the state of the scheduled subtask is restored to the starting state, the process will return to step S4 to retry executing the scheduled subtask.

[0121] If the scheduled subtask is an irreversible subtask ( If the failure is identified as a catastrophic failure (such as permanent damage or jamming of parts), the subsequent task planning will be terminated immediately, a safety shutdown command will be output and an alarm will be triggered, and manual intervention will be required. The process will end abnormally.

[0122] Example 2

[0123] As shown in Figure 2, based on Embodiment 1, the robot long-term assembly task planning system based on reversibility constraints of the present invention includes: a task modeling and state perception module, a reversibility perception module, an execution sequence generation module, an execution control module, and an execution monitoring and feedback module.

[0124] The task modeling and state perception module is used to acquire assembly task and assembly object information, model the assembly task according to the assembly object information and the preset assembly process flow, decompose the assembly task into multiple sub-tasks, and convert each sub-task into a structured assembly semantic description. Specifically, it includes an information acquisition and preprocessing module, an assembly process modeling and state space construction module, a task decomposition module based on the state space model, and a structured assembly semantic conversion module.

[0125] The information acquisition and preprocessing module is used to acquire assembly task instructions (such as "assemble the reducer of model X") through the user interface and retrieve assembly object information from the pre-stored product database. The assembly object information includes: the geometric model of the assembly object (such as STEP / STL format CAD data) and the assembly relationships between the assembly objects (such as assembly constraint matrices). The geometric model of the assembly object is used to identify contact surfaces (such as shaft holes, planes, etc.) and to calculate the time cost and Euclidean distance of path planning in the reversibility perception module. The assembly relationships between the assembly objects are used to determine the assembly priority of components when establishing a dependency graph in the sequence generation module.

[0126] The assembly process modeling and state space construction module is used to obtain the three-dimensional geometric features of the parts by parsing the CAD model of the assembly object, and then combine it with the preset assembly process flow to construct a multi-level assembly state space model with the initial assembly state as the source point and the final product state as the sink point. The multi-level assembly state space model describes the evolution path from part to component and from component to final product state, providing a global state reference for assembly task decomposition.

[0127] The task decomposition based on the state-space model is used to traverse the assembly process flow based on the multi-level assembly state-space model, and to analyze and identify the contact surfaces and connection attributes between parts using three-dimensional geometric features. These are then used as dividing boundaries to divide the assembly task into multiple atomic-level action operators, i.e., subtasks. Each subtask It is defined as a unique state transition function, with the following formula:

[0128] ;

[0129] in, The initial state is defined as the pose and connection state of the relevant components in the workspace before the action is executed. The target state, or completed state, is unique and defined as the expected pose and physical constraint state of the relevant components after the action is successfully executed. Each subtask corresponds to only one specific edge in the multi-level assembly state space model, ensuring... The uniqueness of the set of geometric and physical properties. The state tuple for each subtask. ,in, Operation types include grab, align, insert, and fasten; The process constraints that must be met to perform this subtask (such as tightening torque and position tolerance).

[0130] The structured assembly semantic conversion module uses a natural language generation algorithm based on predefined rule templates to extract the initial state, target state, operation type, and process constraints from the state tuple of the subtask, converting the subtask into text prompts understandable by the visual-language-action model. The template is as follows:

[0131] ;

[0132] in, For operation type, For the target object, For reference, These are process constraints.

[0133] For example, for a subtask with the operation type "Insert", the operation object "Part_A (axis)", the target reference "Part_B_Hole_C (shell hole)" and the constraint condition "H7 fit", the generated structured assembly semantic description is: "Precisely insert part A (axis) into the hole C of part B (shell)", thereby guiding the VLA model to focus on assembly features and generate fine motion trajectories.

[0134] The reversibility awareness module is used to define the recovery cost function and the irreversibility risk measurement function, and preset the loss cost threshold to construct a sub-task-level reversibility constraint model. Based on the sub-task-level reversibility constraint model, the reversibility of each sub-task is calculated, specifically including defining the recovery cost function module, defining the irreversibility risk measurement function module, and judging the sub-task... The reversibility module;

[0135] The defined recovery cost function module is used for subtasks. Possible failure states The subtask is calculated by defining the recovery cost function. From failure status The cost of restoring to the initial state The formula is:

[0136] ;

[0137] in, Indicates a failure state The time cost required to perform the recovery action is obtained by comparing the current pose of the robotic arm with the initial pose of the subtask. The Euclidean distance between them, combined with the average moving speed preset by the robotic arm, is used to calculate the no-load path planning time, and the estimated process time required for the operation (such as rotating out or pulling out) is added on.

[0138] In the failure state Below, the manual intervention requirement indicator is a set of preset discrete tiered values ​​stored in the task risk database. When the failure status... When the obstacle can be avoided autonomously by the robot or by resetting its pose, When in failure state When a part falls outside the workspace or becomes obstructed and requires manual repositioning. When in failure state When severe entanglement or clamp failure requires manual intervention for repair, ;

[0139] In the failure state The degree of damage to the components is a quantitative value, calculated using the Sigmoid function based on the cumulative over-limit impulse, with the following formula:

[0140] ;

[0141] in, To accumulate the over-limit impulse, i.e., the time integral of the over-limit load during execution, the formula is:

[0142] ;

[0143] in, This provides real-time contact force and torque feedback from the sensor; The allowable elastic limit load threshold for the part; The damage threshold, also known as the offset of the center point of the Sigmoid function, represents the degree of damage when the accumulated impulse reaches this value. Reaching 0.5; This is the damage sensitivity coefficient, i.e., the slope of the sigmoid function, used to adjust how steeply the damage value increases with increasing impulse. A larger value indicates that the component is more sensitive to excessive loads. Through the nonlinear mapping of this formula, the progressive failure process of the component from elastic deformation to plastic failure is simulated to output a value. The degree to which the value approaches 1 quantifies the probability of a part being scrapped;

[0144] They are respectively , , The preset weighting parameters are used to balance time, manpower, and material costs.

[0145] The module defining the irreversible risk measurement function is used to incorporate failure states. The probability of occurrence is calculated for each subtask by defining an irreversible risk metric function. Average loss cost The formula is:

[0146] ;

[0147] in, For subtasks The set of all possible failure states To execute subtasks based on historical assembly data. Failure status occurs The average failure frequency.

[0148] The judgment subtask The reversibility module calculates the average loss cost. Compared with the preset loss cost threshold Compare;

[0149] like Then determine the subtask It is a reversible subtask, meaning that the cost of failure is controllable;

[0150] like Then determine the subtask It is an irreversible subtask, meaning that failure in this subtask may lead to the complete failure of the mission or a high cost.

[0151] The execution sequence generation module is used to establish a dependency graph between subtasks based on the assembly object information to determine the preceding dependent tasks of the subtasks; based on the reversibility of the subtasks and the dependency graph between the subtasks, a restricted topology sorting algorithm based on risk priority is used to generate the global task execution sequence of the assembly task, specifically including a module for establishing a dependency graph between subtasks and a sequence optimizer.

[0152] The module for establishing the dependency graph between subtasks extracts geometric topological constraints: based on the CAD model of the assembly object, it uses degree-of-freedom analysis to identify the occlusion relationships of parts in the assembly direction; if the assembly path of part A passes through the envelope space of part B, then part B is defined as a prerequisite dependency of part A, therefore, the subtask containing part B is a prerequisite dependency task containing the subtask containing part A; it also extracts physical connection constraints: combining the assembly relationships between assembly objects to identify the fastening relationships in the assembly process (such as bolted connections between parts), it defines the support as a prerequisite dependency of the connected part, therefore, the subtask containing the support is a prerequisite dependency task containing the connected part; the above constraints are summarized to form a priority topological graph between subtasks, i.e., a dependency graph, in which nodes represent subtasks and directed edges represent the order of tasks.

[0153] The sequence optimizer is used to generate a global task execution sequence for the assembly task using a risk-priority-based restricted topology sorting algorithm, specifically:

[0154] In the k-th step of generating the global task execution sequence, the following steps are performed, 1≤k≤M, where M is the number of subtasks in the assembly task:

[0155] Determine the candidate task set : Filter all subtasks that meet the constraints and form a candidate task set. The constraint condition is that all the prerequisite dependent tasks of the subtask (defined by the in-degree / out-degree of the dependency graph between subtasks) have been completed in steps 1 to k-1, and the subtask... This means that the subtask did not appear in the generated sequence;

[0156] If candidate task set If there are reversible subtasks, then the candidate task set will be... Remove all irreversible subtasks to update the candidate task set. ;

[0157] For candidate task set Each subtask calculates the risk perception priority. The formula is:

[0158] ;

[0159] in, This represents the length of the longest path from the initial state to the current task in the dependency graph. Positive weights;

[0160] Define the optimization objective function The formula is:

[0161] ;

[0162] in, The position weight function is given by the formula: N is the number of subtasks in the candidate task set; the optimization objective function varies with the execution order. The increase and monotonically decrease make high-risk tasks (large) (Value) Match small weights (large) as much as possible The value (i.e., executed later) maximizes the system's backtracking capability; this patent aims to minimize the optimization objective function. For the goal, due to The risk term in the objective function is negatively correlated with the position weight in the objective function, therefore the risk term in step k is... The task with the smallest value Add to the global task execution sequence. The formula is:

[0163] .

[0164] The execution control module is used to form a set of unexecuted subtasks into a set of subtasks to be executed: according to the global task execution sequence, the subtasks in the current set of subtasks to be executed are scheduled sequentially, and the structured assembly semantic description of the scheduled subtasks is used as a text prompt. This text prompt is combined with the real-time RGB-D visual information of the assembly environment and the robot joint angle and end-effector pose information to form a multimodal data input pre-trained vision-language-action (VLA) model. Through multimodal fusion inference of the VLA model, the input is jointly encoded and decoded to generate the robot's underlying motion control instructions corresponding to the scheduled subtasks. The robot arm is then driven to execute the motion control instructions, which enable the robot arm to perform fine assembly actions in an unstructured environment and adaptively handle minor disturbances in the environment, thereby achieving end-to-end mapping from high-level semantics to low-level actions.

[0165] The execution monitoring and feedback module is used to collect the execution status and assembly result in real time during and after the execution of each subtask by the robotic arm. A multi-source state comparator is used to compare the execution status and assembly result with a predefined completion status to determine whether the scheduled subtask was successfully executed. The execution status includes the real-time pose data of the robotic arm's end effector and the time-domain data stream from the force sensor. The assembly result is the final pose of the part after the action is completed. The predefined completion state is the target state of the subtask. The multi-source state comparator is a software logic judgment module that integrates visual, force, and robot body pose data.

[0166] The comparison and verification process includes pose verification, force verification, and visual semantic verification.

[0167] Pose verification: Calculate assembly results Compared with predefined completion status The geometric deviations between them include the Euclidean distance deviations between the calculated position vectors. and the rotation angle deviation between attitude quaternions If satisfied and If so, the geometric pose verification is considered successful. and The preset tolerance threshold is based on the accuracy requirements of the basic process.

[0168] Force verification: Read the force sensor data in the execution state and extract the peak contact force or steady-state holding force characteristic value during the action; if the characteristic value falls within the preset process torque threshold range, the physical connection verification is determined to be successful.

[0169] Visual semantic verification: A visual recognition model is used to perform geometric alignment and residual analysis on the acquired local point cloud and RGB images to confirm that there is no misalignment or gap between parts. This verification is achieved by triggering a visual sensor to acquire images of the assembly area and using the Iterative Closest Point (ICP) algorithm to align the real-time point cloud. With standard CAD models Registration is used to calculate the rotation error of the transformation matrix. With translation error Simultaneously calculate the connected component width of depth gradient jumps and high-contrast shadows within the region of interest (ROI). If both conditions are met , and If so, the visual semantic verification is deemed successful. , and These are the preset pose alignment tolerance threshold and gap width threshold, respectively;

[0170] If all of the above verifications—force verification and visual semantic verification—pass, the output result is "Subtask executed successfully." If any verification fails, the output result is "Subtask executed failed," and the current state is marked as the corresponding failure state. The failure status Failure due to pose deviation, torque overload, or visual semantic defect.

[0171] If the scheduled subtask is executed successfully, the scheduled subtask is removed from the set of subtasks to be executed. It is then determined whether all subtasks in the assembly task have been executed successfully. If so, the assembly completion instruction is output and the process ends. If not, the process returns to the execution sequence generation module and schedules the next subtask in the set of subtasks to be executed in the order of the global task execution sequence.

[0172] If the scheduled subtask fails to execute, a closed-loop adjustment strategy is triggered based on the reversibility of the scheduled subtask:

[0173] If the scheduled subtask is a reversible subtask ( If the pre-stored reverse recovery strategy is invoked to perform a rollback operation (such as withdrawing the gripper or resetting the part), after restoring to the starting state of the scheduled subtask, the execution sequence generation module is returned to retry executing the scheduled subtask.

[0174] If the scheduled subtask is an irreversible subtask ( If the failure is identified as a catastrophic failure (such as permanent damage or jamming of parts), the subsequent task planning will be terminated immediately, a safety shutdown command will be output and an alarm will be triggered, and manual intervention will be required. The process will end abnormally.

Claims

1. A robot long-term assembly task planning method based on reversibility constraints, characterized in that, include: Obtain assembly task and assembly object information, model the assembly task based on the assembly object information and the preset assembly process flow, decompose the assembly task into multiple sub-tasks, and convert each sub-task into a structured assembly semantic description. The assembly task is modeled based on the assembly object information and a preset assembly process flow, and decomposed into multiple sub-tasks. This includes: obtaining the three-dimensional geometric features of the parts by analyzing the CAD model of the assembly object; constructing a multi-level assembly state space model with the initial assembly state as the source and the final product state as the sink, based on the preset assembly process flow; describing the evolution path from part to component and from component to final product state; using the multi-level assembly state space model to analyze and identify the contact surfaces and connection attributes between parts, using these as dividing boundaries to decompose the assembly task into multiple sub-tasks, each sub-task having a defined state transition function from the initial state to the target state; each sub-task having a state tuple including the initial state, target state, operation type, and process constraints; defining a recovery cost function and an irreversible risk measurement function, and preset a loss cost threshold to construct a sub-task-level reversibility constraint model; and calculating the reversibility of each sub-task based on the sub-task-level reversibility constraint model. A dependency graph between subtasks is established based on the assembly object information to determine the pre-dependent tasks of each subtask. Based on the reversibility of the subtasks and the dependency graph, a restricted topology sorting algorithm based on risk priority is used to generate a global task execution sequence for the assembly tasks. Subtasks are scheduled sequentially according to the global task execution sequence, and the structured assembly semantic description of the scheduled subtasks is used as text prompts to input into a pre-trained vision-language-action model to obtain the robot's underlying motion control instructions corresponding to the scheduled subtasks. The robot arm is then driven to execute the robot's underlying motion control instructions corresponding to the scheduled subtasks. During the execution of the robot's underlying motion control instructions corresponding to the scheduled subtasks, the execution status and assembly results are collected. A preset multi-source state comparator is used to compare the execution status, assembly results, and predefined completion status to determine whether the scheduled subtasks have been executed successfully. If the scheduled subtasks are executed successfully, the next subtask in the global task execution sequence is executed. If the scheduled subtasks fail, a rollback or shutdown operation is performed based on the reversibility of the scheduled subtasks.

2. The robot long-term assembly task planning method based on reversibility constraints according to claim 1, characterized in that, The process of converting each subtask into a structured assembly semantic description includes: using a natural language generation algorithm based on predefined rule templates to extract the initial state, target state, operation type, and process constraints from the state tuple of the subtask, and converting the subtask into text prompts understandable by the vision-language-action model. The template is as follows: ;in, For operation type, For the target object, For reference, These are process constraints.

3. The robot long-term assembly task planning method based on reversibility constraints according to claim 2, characterized in that, The construction of the sub-task-level reversibility constraint model includes: for sub-tasks Possible failure states Define the recovery cost function to compute the subtask From failure status The cost of restoring to the initial state The formula is: ;in, In the failure state The time cost required to perform the recovery action; In the failure state The required level for human intervention is indicated below; In the failure state The following is a quantitative value of the component damage; They are respectively 、 、 The preset weight parameters; combined with the failure status The probability of occurrence is defined, and an irreversible risk metric function is used to calculate the subtask. Average loss cost The formula is: ;in, For subtasks The set of all possible failure states To execute subtasks Failure status occurs The average failure frequency.

4. The robot long-term assembly task planning method based on reversibility constraints according to claim 3, characterized in that, The calculation of the reversibility of each subtask includes: calculating the reversibility of the subtasks. Average loss cost Compared with the preset loss cost threshold Compare; if Then determine the subtask It is a reversible subtask; if Then determine the subtask This is an irreversible subtask.

5. The robot long-term assembly task planning method based on reversibility constraints according to claim 4, characterized in that, The process of establishing a dependency graph between subtasks includes: extracting geometric topological constraints: based on the CAD model of the assembly object, using degree-of-freedom analysis, identifying the occlusion relationships of parts in the assembly direction; if the assembly path of part A passes through the envelope space of part B, then part B is defined as a prerequisite dependency of part A, therefore, subtasks containing part B are prerequisite dependencies of subtasks containing part A; extracting physical connection constraints: combining the assembly relationships between assembly objects to identify the fastening relationships in the assembly process, defining the support component as a prerequisite dependency of the connected component, therefore, subtasks containing the support component are prerequisite dependencies of the connected component; constructing a dependency graph: summarizing the above constraints to form a priority topological graph between subtasks, i.e., a dependency graph, where nodes represent subtasks and directed edges represent the order of tasks.

6. The robot long-term assembly task planning method based on reversibility constraints according to claim 5, characterized in that, The generation of the global task execution sequence for the assembly task includes: in the k-th step of generating the global task execution sequence, performing the following steps, where 1≤k≤M, and M is the number of subtasks in the assembly task: determining the candidate task set. : Filter all subtasks that meet the constraints and form a candidate task set. The constraint is that all prerequisite tasks of the subtask have been completed in steps 1 to k-1 and the subtask does not appear in the generated global task execution sequence; if the candidate task set If there are reversible subtasks, then the candidate task set will be... Remove all irreversible subtasks to update the candidate task set. ; for candidate task set Risk perception priority is calculated for each subtask. The formula is: ;in, This represents the length of the longest path from the initial state to the current subtask in the dependency graph. Use positive weights; define the optimization objective function. The formula is: ;in, The position weight function is given by the formula: N is the number of subtasks in the candidate task set; the goal is to minimize the optimization objective function. To achieve the goal, in the k-th step... The task with the smallest value Add to the global task execution sequence. The formula is: 。 7. The robot long-term assembly task planning method based on reversibility constraints according to claim 6, wherein determining whether the scheduled subtask was successfully executed includes: A multi-source state comparator is used for pose verification, force verification, and visual semantic verification. The pose verification calculates the geometric deviation between the assembly result and the predefined completed state, including the Euclidean distance deviation between the position vectors of the assembly result and the predefined completed state and the rotation angle deviation between the attitude quaternions. If the Euclidean distance deviation is less than or equal to a preset distance deviation and the rotation angle deviation is less than or equal to a preset rotation deviation, the pose verification is considered successful; otherwise, the pose verification is considered unsuccessful. The force verification performs threshold judgment on the peak contact force or steady-state holding force feature value extracted from the execution state. If the peak contact force or steady-state holding force feature value is within a preset process torque threshold range, the force verification is considered successful; otherwise, the force verification is considered unsuccessful. The visual semantic verification uses a visual recognition model to calculate the rotation error and translation error between the assembly result and the CAD model of the assembly object, and calculates the width of the connected component. If the rotation error is less than or equal to the preset geometric alignment rotation error, the translation error is less than or equal to the preset geometric alignment translation error, and the connected component width is less than or equal to the preset gap width threshold, then the visual semantic verification is deemed to have passed; otherwise, the visual semantic verification is deemed to have failed. If the pose verification, force verification, and visual semantic verification are all passed, then the scheduled subtask is deemed to have been executed successfully; otherwise, the scheduled subtask is deemed to have failed.

8. The robot long-term assembly task planning method based on reversibility constraints according to claim 7, wherein the step of performing a rollback or stop operation according to the reversibility of the scheduled sub-task includes: If the scheduled subtask is a reversible subtask, the pre-stored reverse recovery strategy will be automatically invoked to perform a rollback operation. After the scheduled subtask is restored to its initial state, the scheduled subtask will be executed again. If the scheduled subtask is an irreversible subtask, it is considered a catastrophic failure, and subsequent task planning is immediately terminated and a safe shutdown command is output.

9. A system for a robot long-term assembly task planning method based on reversibility constraints as described in any one of claims 1-8, characterized in that, include: The task modeling and state awareness module is used to acquire assembly task and assembly object information, model the assembly task according to the assembly object information and the preset assembly process flow, decompose the assembly task into multiple sub-tasks, and convert each sub-task into a structured assembly semantic description. The reversibility perception module is used to define the recovery cost function and the irreversibility risk measurement function, and preset the loss cost threshold to construct a sub-task-level reversibility constraint model; and calculate the reversibility of each sub-task based on the sub-task-level reversibility constraint model. The execution sequence generation module is used to build a dependency graph between subtasks based on the assembly object information, so as to determine the preceding dependent tasks of the subtasks. Based on the reversibility of subtasks and the dependency graph between subtasks, a restricted topology sorting algorithm based on risk priority is used to generate the global task execution sequence of assembly tasks. The execution control module is used to schedule subtasks sequentially according to the global task execution sequence, and input the structured assembly semantic description of the scheduled subtasks as text prompts into the pre-trained visual-language-action model to obtain the robot's underlying motion control instructions corresponding to the scheduled subtasks, and drive the robotic arm to execute the robot's underlying motion control instructions corresponding to the scheduled subtasks; the execution monitoring and feedback module is used to collect the execution status and assembly results during the process of the robotic arm executing the robot's underlying motion control instructions corresponding to the scheduled subtasks, and use a preset multi-source state comparator to compare the execution status, assembly results and predefined completion status to determine whether the scheduled subtasks have been executed successfully; If the scheduled subtask executes successfully, the next subtask in the global task execution sequence will be executed. If a scheduled subtask fails to execute, a rollback or shutdown operation will be performed based on the reversibility of the scheduled subtask.

Images