Robot, sorting control method and device thereof and program product

Abstract

The invention relates to the field of robots, in particular to a robot, a sorting control method and device thereof and a program product. The method comprises the following steps: receiving a sorting task instruction, wherein the sorting task instruction comprises a task for sorting a plurality of objects to be sorted; according to the sorting task instruction, a multi-task planning process corresponding to the to-be-sorted objects is generated; according to the multi-task plan, operation processes of the multiple to-be-sorted objects are executed in sequence, corresponding composite skills are called according to the to-be-sorted objects, and the composite skills comprise sub-skills, state evaluation models and behavior trees; and the state of the robot is determined according to the state evaluation model, sub-skills needing to be executed are determined according to the behavior tree and the state, and operation elements corresponding to the sub-skills are called to execute sorting operation. Through a Gantt chart-behavior tree-operation unit three-layer decoupling design, only task description and a sub-skill library need to be expanded when a sorting target is newly added, so that the expansibility is favorably improved, the environmental uncertainty is dealt with through state evaluation, and the adaptability is favorably improved.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to robots and their sorting control methods, devices and program products. Background Technology

[0002] Robot sorting operations refer to a series of tasks performed by automated robot systems in industrial or logistics scenarios, including object recognition, grasping, moving, and precise placement.

[0003] Traditional robotic sorting methods primarily rely on symbolic logic reasoning for task planning and predefined motion models for control. However, in unstructured environments (such as flexible sorting scenarios in automobile manufacturing), this method suffers from poor scalability due to its reliance on symbolic logic reasoning for task planning, and insufficient adaptability due to its dependence on motion models for motion control. Summary of the Invention

[0004] In view of this, embodiments of this application provide a robot and its sorting control method, apparatus and program product to solve the problems of poor scalability and insufficient adaptability in sorting in the prior art.

[0005] A first aspect of this application provides a sorting control method for a robot, the method comprising:

[0006] Receive a sorting task instruction, which includes a task to sort multiple objects to be sorted.

[0007] Based on the sorting task instruction, a multi-task planning process corresponding to the plurality of objects to be sorted is generated, and the multi-task planning process includes the operation process of the plurality of objects to be sorted.

[0008] According to the multi-task planning process, the sorting tasks of the multiple objects to be sorted are executed sequentially, and the corresponding composite skills are invoked according to the objects to be sorted. The composite skills include sub-skills, state evaluation models and behavior trees.

[0009] The state of the robot is determined according to the state evaluation model. The sub-skills to be executed are determined according to the behavior tree and the state of the robot. According to the pre-set correspondence between sub-skills and operation primitives, the operation primitives corresponding to the sub-skills are called to execute the sorting task.

[0010] In conjunction with the first aspect, in a first possible implementation of the first aspect, after invoking the corresponding composite skill based on the object to be sorted, the method further includes:

[0011] Obtain the observation sequence, which is a sequence of sensor data collected by the robot during the sorting task;

[0012] Determine the cumulative probability of generating the observation sequence by determining the different state sequences of the robot;

[0013] The accumulated probability is compared with a predetermined probability threshold, and the result of the comparison determines whether the execution process of the composite skill is abnormal.

[0014] In conjunction with the first aspect, in a second possible implementation of the first aspect, after invoking the corresponding composite skill based on the object to be sorted, the method further includes:

[0015] Obtain the probability of being in each state at the current moment;

[0016] The state transition probability matrix is ​​determined based on the probability of transitions between different states. The state transition probability matrix is ​​a matrix determined by the probability of transitions between different robot states.

[0017] The observation probability matrix is ​​determined based on the observation probabilities corresponding to different states, where the observation probability is the probability of observation data generated under a specific state.

[0018] Based on the state transition probability matrix and the observation probability matrix, the optimal state sequence in which the state transition occurs at the current moment is obtained;

[0019] The sub-skill sequence is determined based on the optimal state sequence optimization.

[0020] In conjunction with the first possible implementation of the first aspect, in the third possible implementation of the first aspect, the observation sequence of the robot during the execution of sorting task instructions is obtained, including one or more of the following information:

[0021] The robot's arm position information is estimated using robot kinematics;

[0022] Contact force and / or torque are collected using an end force sensor;

[0023] The robot's gripping state is estimated using the robot's hand tactile information;

[0024] The robot's position information is estimated using the robot positioning device;

[0025] The target position of the object is estimated using a visual sensing device;

[0026] Estimate the pose of the target object using a visual sensing device;

[0027] The pose of the box containing the target object is estimated using a visual sensing device;

[0028] Obtain the location of the unmanned transport vehicle provided by the system;

[0029] Obtain the shelf locations provided by the system.

[0030] In conjunction with the first aspect, in a fourth possible implementation of the first aspect, the sorting operation is performed by invoking the operation primitive corresponding to the sub-skill, including:

[0031] Invoke and initialize the operation primitives corresponding to the sub-skill, and determine the pose of the target object and the current state of the robot;

[0032] Based on the pose of the target object and the current state of the robot, the joint angle sequence of the robot is determined by dynamic motion equations;

[0033] The robot is controlled to perform the sorting operation based on the joint angle sequence.

[0034] In conjunction with the fourth possible implementation of the first aspect, in the fifth possible implementation of the first aspect, the joint angle sequence of the robot is determined by dynamic motion equations based on the pose of the target object and the current state of the robot, including:

[0035] Based on the pose of the target object and the current state of the robot, the motion trajectory of the robot's end effector is determined by dynamic motion equations.

[0036] Based on the motion trajectory of the robot's end effector, the joint angle sequence of the robot is determined by inverse kinematics.

[0037] In conjunction with the fifth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, after determining the motion trajectory of the robot's end effector through dynamic motion equations based on the pose of the target object and the current state of the robot, the method further includes:

[0038] Detect the contact force between the target object and / or the robot;

[0039] When the contact force is greater than a predetermined value, a compensation displacement is determined by an admittance model, and the motion trajectory of the robot end effector is adjusted according to the compensation displacement.

[0040] A second aspect of this application provides a sorting control device for a robot, the device comprising:

[0041] The sorting task instruction receiving unit is used to receive sorting task instructions, which include a task of sorting multiple objects to be sorted.

[0042] A multi-task planning unit is used to generate a multi-task planning process corresponding to the plurality of objects to be sorted according to the sorting task instructions. The multi-task planning process includes the operation process of the plurality of objects to be sorted.

[0043] The composite skill invocation unit is used to sequentially execute the sorting task of the multiple objects to be sorted according to the multi-task planning process, and invoke the corresponding composite skill according to the object to be sorted. The composite skill includes sub-skills, state evaluation models and behavior trees.

[0044] An operation primitive execution unit is used to determine the state of the robot according to the state evaluation model, determine the sub-skills to be executed according to the behavior tree and the state of the robot, and call the operation primitive corresponding to the sub-skill to execute the sorting task according to the pre-set correspondence between the sub-skills and operation primitives.

[0045] A third aspect of this application provides a robot including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot performs the method as described in any of the first aspects.

[0046] A fourth aspect of this application provides a computer program product that, when run on a computer, causes the computer to execute the methods described in the first aspect or its various implementations.

[0047] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0048] A sixth aspect of this application provides a chip for implementing the methods in the various implementations of the first aspect described above. Specifically, the chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform the methods as described in the first aspect or its various implementations.

[0049] The beneficial effects of this application embodiment compared with the prior art are as follows: After receiving a sorting task instruction that includes sorting multiple objects to be sorted, this application embodiment generates a multi-task planning process for multiple objects to be sorted, executes the operation process of multiple objects to be sorted sequentially according to the multi-task planning process, calls the corresponding composite skill according to the object to be sorted, determines the robot's state according to the state evaluation model, determines the sub-skills to be executed according to the robot's state and behavior tree, and calls the operation primitives corresponding to the sub-skills to perform sorting operations. Since this method uses a three-layer decoupling design of operation process-behavior tree-operation primitives, when adding a new sorting target, only the task description and sub-skill library need to be expanded, which is beneficial to improving scalability. Furthermore, by using state evaluation to deal with environmental uncertainties, it is beneficial to improve adaptability. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram illustrating an implementation scenario of a robot sorting control method provided in an embodiment of this application;

[0052] Figure 2 This is a schematic diagram illustrating the implementation process of a robot sorting control method provided in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of an initial state distribution table provided in an embodiment of this application;

[0054] Figure 4 This is a schematic diagram of a state transition matrix provided in an embodiment of this application;

[0055] Figure 5 This is a schematic diagram of an observation probability matrix provided in an embodiment of this application;

[0056] Figure 6 This is a schematic diagram of a robot sorting control device provided in an embodiment of this application;

[0057] Figure 7 This is a schematic diagram of a robot provided in an embodiment of this application. Detailed Implementation

[0058] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0059] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0060] Robot sorting operations utilize automated robot systems to perform a series of tasks in industrial or logistics scenarios, including object recognition, grasping, moving, and precise placement.

[0061] Traditional robotic sorting methods primarily rely on symbolic logic reasoning for task planning and predefined motion models for control. However, in unstructured environments such as flexible sorting in automobile manufacturing, this approach suffers from several drawbacks. First, its reliance on symbolic logic reasoning in task planning leads to poor scalability. Second, its dependence on motion models in motion control results in a lack of adaptability.

[0062] This application provides an implementation scenario for a robot sorting control method. This implementation scenario includes a supply platform, a robot, and a sorting platform. The supply platform can be a flow rack or a conveyor belt, for example... Figure 1 The supply platform shown is a flow rack 1, and the sorting platform is an unmanned transport vehicle. The flow rack serves as a dynamic feeding platform for objects to be sorted, storing and transporting them. The robot receives sorting task instructions from the system, determines the Gantt chart of multiple objects to be sorted based on these instructions, scans the flow rack, detects the target object's pose and slide angle, determines the corresponding Gantt chart of each object, executes the sorting task instructions sequentially according to the Gantt chart's operation flow, invokes the corresponding composite skill based on the object to be sorted, determines the robot's state through the state evaluation model in the composite skill, determines the sub-skills to be executed according to the trigger conditions in the behavior tree, invokes the corresponding operation primitives of the sub-skills to perform the sorting operation, and moves the objects to be sorted to a specific position on the sorting platform.

[0063] Figure 2 A schematic diagram illustrating the implementation flow of a robot sorting control method provided in this application embodiment is described in detail below:

[0064] In S201, a sorting task instruction is received, which includes a task to sort multiple objects to be sorted.

[0065] In this embodiment, the robot sorting control method can be applied to various sorting scenarios, such as sorting electronic components or sorting packaged boxes.

[0066] In this embodiment, the robot sorting control method can be applied to an intelligent warehousing environment to control robots to perform sorting tasks. The method first receives a sorting task instruction from a warehouse management system. This sorting task includes sorting multiple objects to be sorted, i.e., the task name is the name of the object to be sorted. For example, retrieving a specific item from a shelf and placing it into a designated sorting box. Each object to be sorted includes the target object's identification information, location information, and target placement location information.

[0067] When controlling a robot to perform the sorting task of precision electronic components, it first receives a sorting task instruction from the production line management system. This instruction contains multiple electronic components to be sorted, such as identifying and removing specific models of chips, resistors, or capacitors from the conveyor belt and placing them into the corresponding sorting boxes. Each object to be sorted contains the component's model information, location information, and target placement location information.

[0068] In S202, a multi-task planning process corresponding to the plurality of objects to be sorted is generated according to the sorting task instructions.

[0069] The multi-task planning process in this embodiment includes the operation flow of multiple objects to be sorted. This multi-task planning process can be a Gantt chart or other task flowchart. The Gantt chart is a task state diagram that includes operation flow information such as the execution time and execution order of the sorting tasks for each object to be sorted.

[0070] Upon receiving a sorting task, the system generates a Gantt chart based on the multiple objects to be sorted. When generating the Gantt chart, the system considers factors such as the priority of the objects, their location distribution, robot path optimization, and resource utilization efficiency. The Gantt chart breaks down the entire sorting task into a series of ordered steps, marking the dependencies between tasks—meaning that subsequent tasks can only be executed after the previous one is completed—ensuring the robot completes the sorting task in the optimal order and manner.

[0071] For example, a sorting task involves moving multiple objects from multiple flow racks to an automated guided vehicle (AGV). The multi-task planning process includes: moving object α from flow rack A to slot 1 of the AGV; moving object β from flow rack B to slot 2 of the AGV; and moving object γ from flow rack C to slot 3 of the AGV. Based on the sorting tasks included in the sorting instructions, the start and end times of each sorting task are determined. A time period is then defined based on the start and end times of each task and displayed in a Gantt chart. During the execution of the sorting task instructions, different colors can be used to distinguish between executed and pending sorting tasks.

[0072] In electronic component sorting scenarios, the generation of multi-task planning processes needs to consider the fragility of components, accuracy requirements, and sorting efficiency. The sorting sequence and time can be determined based on the characteristics of the components and sorting requirements to improve the efficiency and safety of the sorting process.

[0073] In S203, the sorting tasks for the multiple objects to be sorted are executed sequentially according to the multi-task planning process, and the corresponding composite skills are invoked according to the objects to be sorted.

[0074] Among them, composite skills include sub-skills, state assessment models, and behavior trees.

[0075] After generating the multi-task planning process, sorting tasks can be executed sequentially according to the operation flow of the multi-task planning process. For the attribute information of each object to be sorted, the corresponding composite skill can be invoked. A composite skill is a high-level abstraction of operational capability, containing three core components: sub-skills, a state evaluation model, and a behavior tree. Sub-skills are a collection of basic operational capabilities, such as movement, grasping, and placement; the state evaluation model is used to evaluate the state of the robot and the environment in real time; and the behavior tree defines the sequence of sub-skills to be executed in different states and their logical relationships.

[0076] In electronic component sorting scenarios, composite skills include grasping and placing skills. Grasping and placing composite skills include sub-skills such as selection, grasping, moving, and placement. In the application scenario of transporting objects to an automated guided vehicle (AGV), the functions of each sub-skill are described below:

[0077] Selection: Select an object from the boxes on the fluent shelf;

[0078] Grab: Grab an object from a certain location;

[0079] Movement: Moving objects from flow racks to automated guided vehicles;

[0080] Select: Choose a grid on the automated guided vehicle;

[0081] Placement: Places an object at a specified location.

[0082] The attribute information of objects to be sorted can include one or more of the following: object pose, target position, size, mass, material, and placement accuracy. The combined capabilities of objects with different attribute information can be differentiated through dynamic parameter input. For example, for regular metal parts, they can be directly grasped, moved, and placed, with dynamic motion equation parameters and impedance parameters determined according to material information. For fragile objects, such as glass, pre-contact force detection can be added, and corresponding compliance control parameters can be determined, including compliance control impedance parameters and dynamic motion equation parameters.

[0083] In S204, the state of the robot is determined according to the state evaluation model, the sub-skill to be executed is determined according to the behavior tree and the state of the robot, and the operation primitive corresponding to the sub-skill is called to execute the sorting task according to the pre-set correspondence between the sub-skill and the operation primitive.

[0084] During sorting operations, the system first obtains the robot's current state through a state assessment model. This model processes data from the robot's various sensors to evaluate the robot's current state, the target object's state, and the environmental state. These states include the robot's position and orientation, the target object's position and orientation, and the relative relationship between the robot and the target object.

[0085] When acquiring the state, it can be determined based on visual, force, torque, and tactile information provided by the robot's sensors. Among them, visual information can be acquired through RGB image sensors, and tactile information can be estimated from the fingertip and palm dot matrix to determine the object's pose relative to the hand.

[0086] After acquiring the state, the system determines the sub-skills to be executed based on the logical structure of the behavior tree. The behavior tree is a hierarchical decision structure containing selection nodes, sequence nodes, and condition nodes. The system traverses the behavior tree starting from the root node, selecting the appropriate branch path based on the current state and predetermined conditions, ultimately determining the sub-skills to be executed.

[0087] For example, the following structure can be designed for a sorting task:

[0088] Root node: Used to manage the entire sorting process.

[0089] Select Node: Determines the current operation to be performed (such as grabbing a node, moving a node, or placing a node).

[0090] For grabbing nodes, a state check is performed before grabbing the node, including checking whether the object is correctly aligned and whether it can be grabbed. If it can be grabbed, the grabbing action is performed.

[0091] For moving nodes, before executing the moving node, state monitoring is used to ensure that the grabbing is successful and the object has been attached. Then the moving node is executed to move the object according to the target position.

[0092] For placing nodes, status monitoring is used to ensure that the node has reached the correct position before the placement action is performed.

[0093] That is, after each action, a condition node can be added to check the execution result and decide whether to retry or continue to the next step based on the result.

[0094] After a sub-skill is determined, the system calls the corresponding operation primitive to execute the sorting operation. Operation primitives are low-level functional modules in the robot control system, directly controlling the robot's actuators to complete specific actions. For example, a movement operation primitive controls the robot's movement mechanism, causing the robot to move to a designated position; a grasping operation primitive controls the robot's grasping mechanism, causing the robot to grasp a target object.

[0095] During the sorting process, the system also acquires observation sequences of the robot. These observation sequences are data sequences collected by the robot's various sensors during task execution, including robot arm position information, contact force and torque, gripping state, robot position information, target position of the target object placement grid, target object pose, pose of the box containing the target object, position of the automated guided vehicle, and shelf position.

[0096] The forward algorithm (a dynamic programming algorithm that efficiently calculates the probability of an observed sequence under a given model) can be used to calculate the probability of the observed sequence with different state sequences, obtaining the joint probability of all state sequences generating the predicted sequence. The cumulative probability of the observed sequence is then obtained by summing all joint probabilities. A probability threshold can be preset (e.g., a cumulative probability P(O) < 0.3 is considered abnormal). If the cumulative probability falls below the probability threshold, an alarm can be triggered and the abnormal time period recorded for subsequent analysis.

[0097] In possible implementations, embodiments of this application can obtain the optimal state sequence for state transition at the current moment based on the probabilities of each state obtained at the current moment, combined with a pre-set state transition probability matrix and observation probability matrix. The determined sub-skill sequence can be optimized based on the optimal state sequence, or a new sub-skill sequence can be generated.

[0098] The system determines the state transition probability matrix based on the probabilities of transitions between different robot states, and the observation probability matrix represents the probability of observations occurring under a specific state. The system determines the state transition probability matrix based on the likelihood of transitions between different states. This matrix describes the probability of the system transitioning from one state to another, reflecting the inherent laws governing system state changes. Next, the system determines the observation probability matrix based on the corresponding observation probabilities under different states. This matrix describes the probability of a specific observation occurring under a specific state, reflecting the correspondence between states and observations. Finally, based on the state transition probability matrix and the observation probability matrix, the system infers the most likely optimal state sequence using the Viterbi algorithm.

[0099] The system compares the optimal state sequence with a predetermined standard state sequence to determine whether the execution process of the sub-skills within the composite skill is abnormal. The standard state sequence is an ideal sequence of states predefined during the system design phase, representing the state changes that the composite skill should undergo during normal execution. If there is a difference between the optimal state sequence and the standard state sequence, or if the difference exceeds a predetermined threshold, the system will determine that the execution process of the composite skill is abnormal and take corresponding measures, such as replanning the task, adjusting execution parameters, or issuing an alarm.

[0100] For example, in the embodiments of this application, when monitoring the operation status of an action, the model parameters can be determined to include the state space (S), the observation space (O), the initial state distribution (π), the state transition probability matrix (A), and the observation probability matrix (B).

[0101] The state space represents the potential states in the sorting process and may include: S1, waiting to be grabbed; S2, grabbing object state; S3, moving object state; S4, placing object state; and S5, completed or error state.

[0102] The observation space contains rich perceptual information, and the observation sequence determined by the observation space can include one or more of the following: O1, robot arm position information estimated by robot kinematics; O2, contact force and / or torque collected by end effector force sensors (such as six-dimensional force sensors); O3, grasping state estimated by robot hand tactile information; O4, robot positioning estimation of position information; O5, visual estimation of the placement grid position; O6, visual estimation of the target object pose; O7, visual estimation of the box pose; O8, system-provided unmanned transport vehicle position; O9, system-provided shelf position.

[0103] The initial state distribution occurs before the sorting task begins; the system is in a "waiting to grab" state. The initial state distribution table can be shown as follows: Figure 3 As shown, the probability of waiting state S1 is 1.0, and the probability of other states is 0.

[0104] The state transition matrix A describes the possibilities of transitions between different states. Based on the defined state space, S1 represents the waiting-to-grab state, S2 represents the grab-object state, S3 represents the moving-object state, S4 represents the placing-object state, and S5 represents the completed or incorrect state. Figure 4 As shown, the diagonal values ​​in the state transition matrix (e.g., A22 = 0.6) represent the self-holding probability of state S2, the non-zero off-diagonal elements (e.g., A23 = 0.4) reflect the probability of normal process transition, that is, the probability of transitioning from S2 to S3, and the values ​​in the last column (e.g., A15 = 0.1) represent the probability of abnormal jump.

[0105] To represent the probability of generating a specific observation in each state, the observation probability matrix needs to include the mapping relationship between visual, force, and tactile information. The reflection of the state by the observation data can be shown as follows: Figure 5The observation probability matrix B is shown, where O1 represents the robot arm position information estimated by robot kinematics; O2 represents the contact force and / or torque collected by the end effector (e.g., a six-dimensional force sensor); O3 represents the grasping state estimated by the robot's hand tactile information; O4 represents the robot's position information estimated by localization; O5 represents the position of the placement grid estimated by vision; O6 represents the pose of the target object estimated by vision; O7 represents the pose of the box estimated by vision; O8 represents the position of the unmanned transport vehicle provided by the system; and O9 represents the position of the shelf provided by the system.

[0106] Among them, B11 = 0.8 indicates that the arm is likely to be stationary during the waiting state (O1), B22 = 0.6 indicates that the grasping state mainly relies on force feedback (O2), and B23 = 0.3 indicates that tactile data assists in confirming the grasping state (O3).

[0107] After detecting the observation sequence, the probability of generating the observation sequence can be calculated for each state using the forward algorithm based on the observation probability matrix B. Combining the transition relationships in the state transition matrix A, the Viterbi algorithm can be used to solve for the probability of different state sequences corresponding to the observation sequence, and the state sequence with the highest probability can be selected as the optimal state sequence.

[0108] When invoking the operation primitive corresponding to the sub-skill to perform a sorting operation, the system first calls and initializes the operation primitive corresponding to the sub-skill to determine the target object's pose and the robot's current state. The operation primitive contains the parameters and algorithms required to perform a specific operation. The system acquires the target object's position and orientation information through visual sensors or other sensing methods, and simultaneously acquires the robot's current state, including joint angles and end-effector position.

[0109] Based on the target object's pose and the robot's current state, the system determines the robot's joint angle sequence using dynamic motion equations. First, the system determines the robot's end effector trajectory using dynamic motion equations. These equations consider the robot's dynamic characteristics and generate smooth, efficient trajectories. Second, based on the end effector's trajectory, the system determines the robot's joint angle sequence through inverse kinematics calculations. Inverse kinematics is a mathematical method that converts the end effector's position and orientation into joint angles, ensuring the robot moves along the expected trajectory.

[0110] In this application embodiment, complex motion trajectories can be represented by dynamic motion primitives, compliant control is used for dynamic interaction, and operation gestures are used to fine-tune the action to achieve precise and stable grasping operation.

[0111] The dynamic motion element can include a spring-damped model and a nonlinear perturbation term to describe the trajectory from the starting point to the target point. The dynamic motion element can be represented as:

[0112]

[0113] Where x represents position and v represents velocity. Let represent acceleration, g represent target position, τ represent time constant (τ>0), and f(x) represent nonlinear perturbation term. v and β v This represents the gain parameter, typically α. v =4β v .

[0114] The nonlinear disturbance term can be expressed as:

[0115]

[0116] Where, ω i Represents the trajectory shape weight, ψ i (s) is the Gaussian function, where s represents the phase variable, g represents the target position, and x0 represents the initial position of the trajectory.

[0117] In force-position control-based contact operations, the contact force between the target object and / or the robot can be detected. When the contact force exceeds a predetermined value, a compensation displacement is determined using an admittance model, and the motion trajectory of the robot's end effector is adjusted based on this compensation displacement. The admittance model can generate the target motion using a second-order impedance model, such as a mass-spring-damped model. The second-order impedance model can be expressed as:

[0118]

[0119] Among them, F ex x represents the measured value of external force. t Indicates the target displacement. Indicates the target speed. Let M represent the target acceleration, and D, K represent the virtual mass, damping, and stiffness matrix (N×N diagonal positive definite matrices), respectively. Stiffness determines the force-potential balance relationship, while damping and mass determine the response speed.

[0120] After determining the robot's end effector trajectory, the system also detects the contact force between the target object and / or the robot. Contact force is the interaction force between the robot and the target object or environment, and it is crucial for precise robot control. If the detected contact force exceeds a predetermined value, the system determines a compensation displacement using an admittance model and adjusts the robot's end effector trajectory accordingly. The admittance model is a force control method that calculates corresponding displacement compensation based on the magnitude and direction of the contact force, enabling the robot to adapt to environmental changes and avoid damage or operational failure caused by excessive contact force.

[0121] Finally, the system controls the robot to perform sorting operations based on the calculated joint angle sequence. The system converts the joint angle sequence into motor control signals, driving each joint of the robot to move along a predetermined trajectory to complete the sorting operation. During execution, the system monitors the robot's status and operational performance in real time to ensure the accuracy and safety of the sorting operation.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0123] Figure 6 This is a schematic diagram of a robot sorting control device provided in an embodiment of this application. The device includes:

[0124] The sorting task instruction receiving unit 601 is used to receive sorting task instructions, which include a task of sorting multiple objects to be sorted.

[0125] The multi-task planning unit 602 is used to generate a multi-task planning process corresponding to the plurality of objects to be sorted according to the sorting task instruction. The multi-task planning process includes the operation process of the plurality of objects to be sorted.

[0126] The composite skill invocation unit 603 is used to sequentially execute the sorting task of the multiple objects to be sorted according to the multi-task planning process, and invoke the corresponding composite skill according to the object to be sorted. The composite skill includes sub-skills, state evaluation models and behavior trees.

[0127] The operation primitive execution unit 604 is used to determine the state of the robot according to the state evaluation model, determine the sub-skill to be executed according to the behavior tree and the state of the robot, and call the operation primitive corresponding to the sub-skill to execute the sorting task according to the pre-set correspondence between the sub-skill and the operation primitive.

[0128] Figure 6 The robot's sorting control device shown is... Figure 2 The sorting control method for the robot shown corresponds to this.

[0129] Figure 7 This is a schematic diagram of a robot provided in an embodiment of this application. Figure 7As shown, the robot 7 in this embodiment includes a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as a robot sorting control program. When the processor 70 executes the computer program 72, it implements the steps in the sorting control method embodiments of the various robots described above. Alternatively, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the various device embodiments described above.

[0130] For example, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 72 in the robot 7.

[0131] The robot may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of robot 7 and does not constitute a limitation on robot 7. It may include more or fewer parts than shown, or combine certain parts, or different parts. For example, the robot may also include input / output devices, network access devices, buses, etc.

[0132] The processor 70 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0133] The memory 71 can be an internal storage unit of the robot 7, such as a hard drive or memory. The memory 71 can also be an external storage device of the robot 7, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 71 can include both internal and external storage units of the robot 7. The memory 71 is used to store the computer program and other programs and data required by the robot. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0134] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0135] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0136] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0137] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0138] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0140] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0141] In addition, this application also provides a computer program product that, when run on a computer, causes the computer to execute the methods in the above-described implementations.

[0142] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A sorting control method for a robot, characterized in that, The method includes: Receive a sorting task instruction, which includes a task to sort multiple objects to be sorted. Based on the sorting task instruction, a multi-task planning process corresponding to the plurality of objects to be sorted is generated, and the multi-task planning process includes the operation process of the plurality of objects to be sorted. According to the multi-task planning process, the sorting tasks of the multiple objects to be sorted are executed sequentially, and the corresponding composite skills are invoked according to the objects to be sorted. The composite skills include sub-skills, state evaluation models and behavior trees. The state of the robot is determined according to the state evaluation model. The sub-skills to be executed are determined according to the behavior tree and the state of the robot. According to the pre-set correspondence between sub-skills and operation primitives, the operation primitives corresponding to the sub-skills are called to execute the sorting task.

2. The method according to claim 1, characterized in that, After invoking the corresponding composite skill based on the object to be sorted, the method further includes: Obtain the observation sequence, which is a sequence of sensor data collected by the robot during the sorting task; Determine the cumulative probability of generating the observation sequence by determining the different state sequences of the robot; The accumulated probability is compared with a predetermined probability threshold, and the result of the comparison determines whether the execution process of the composite skill is abnormal.

3. The method according to claim 2, characterized in that, Obtain the observation sequence, including one or more of the following information: The robot's arm position information is estimated using robot kinematics; Contact force and / or torque are collected using an end force sensor; The robot's gripping state is estimated using the robot's hand tactile information; The robot's position information is estimated using the robot positioning device; The target position of the object is estimated using a visual sensing device; Estimate the pose of the target object using a visual sensing device; The pose of the box containing the target object is estimated using a visual sensing device; Obtain the location of the unmanned transport vehicle provided by the system; Obtain the shelf locations provided by the system.

4. The method according to claim 1, characterized in that, After invoking the corresponding composite skill based on the object to be sorted, the method further includes: Obtain the probability of being in each state at the current moment; The state transition probability matrix is ​​determined based on the probability of transitions between different states. The state transition probability matrix is ​​a matrix determined by the probability of transitions between different robot states. The observation probability matrix is ​​determined based on the observation probabilities corresponding to different states, where the observation probability is the probability of observation data generated under a specific state. Based on the state transition probability matrix and the observation probability matrix, the optimal state sequence in which the state transition occurs at the current moment is obtained; The sub-skill sequence is determined based on the optimal state sequence optimization.

5. The method according to claim 1, characterized in that, Invoking the operation primitive corresponding to the sub-skill to perform the sorting operation includes: Invoke and initialize the operation primitives corresponding to the sub-skill, and determine the pose of the target object and the current state of the robot; Based on the pose of the target object and the current state of the robot, the joint angle sequence of the robot is determined by dynamic motion equations; The robot is controlled to perform the sorting operation based on the joint angle sequence.

6. The method according to claim 5, characterized in that, Based on the pose of the target object and the current state of the robot, the joint angle sequence of the robot is determined through dynamic motion equations, including: Based on the pose of the target object and the current state of the robot, the motion trajectory of the robot's end effector is determined by dynamic motion equations. Based on the motion trajectory of the robot's end effector, the joint angle sequence of the robot is determined by inverse kinematics.

7. The method according to claim 6, characterized in that, After determining the motion trajectory of the robot's end effector using dynamic motion equations based on the target object's pose and the robot's current state, the method further includes: Detect the contact force between the target object and / or the robot; When the contact force is greater than a predetermined value, a compensation displacement is determined by an admittance model, and the motion trajectory of the robot end effector is adjusted according to the compensation displacement.

8. A sorting control device for a robot, characterized in that, The device includes: The sorting task instruction receiving unit is used to receive sorting task instructions, which include a task of sorting multiple objects to be sorted. A multi-task planning unit is used to generate a multi-task planning process corresponding to the plurality of objects to be sorted according to the sorting task instructions. The multi-task planning process includes the operation process of the plurality of objects to be sorted. The composite skill invocation unit is used to sequentially execute the sorting task of the multiple objects to be sorted according to the multi-task planning process, and invoke the corresponding composite skill according to the object to be sorted. The composite skill includes sub-skills, state evaluation models and behavior trees. An operation primitive execution unit is used to determine the state of the robot according to the state evaluation model, determine the sub-skills to be executed according to the behavior tree and the state of the robot, and call the operation primitive corresponding to the sub-skill to execute the sorting task according to the pre-set correspondence between the sub-skills and operation primitives.

9. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the robot to perform the method as described in any one of claims 1-7.

10. A computer program product comprising computer program instructions, characterized in that, When the computer program is run, the method as described in any one of claims 1-7 is performed.

Images