Robot control method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a robot control method and device, electronic equipment, a computer readable storage medium and a computer program product. The robot control method and device are applied to various robot control scenes such as robots, cloud technologies, artificial intelligence and intelligent transportation. The robot control method comprises the steps of obtaining a to-be-executed main task in response to a task request for a to-be-controlled robot; when the to-be-controlled robot is controlled to execute the to-be-executed main task, auxiliary tasks of the to-be-controlled robot are obtained, L to-be-fused auxiliary tasks are obtained, and the L to-be-fused auxiliary tasks are used for assisting in executing the to-be-executed main task; fusing the L to-be-fused auxiliary tasks based on the redundant allocation weights to obtain C to-be-executed auxiliary tasks; and calculating target motion information of the to-be-controlled robot by combining the C to-be-executed auxiliary tasks, the null-space projection parameters and the to-be-executed main task, and controlling the to-be-controlled robot to move based on the target motion information. According to the invention, multi-task cooperative execution can be realized, and the resource consumption of the robot is reduced.

Description

Technical Field

[0001] The present application relates to intelligent robot technology in the field of automation, and in particular to a robot control method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] A robot is an intelligent machine capable of semi- or fully-automatic operation, performing tasks through programming and automatic control. When robots are used to perform tasks, they typically include a primary task (e.g., food delivery) and multiple secondary tasks (e.g., joint constraints and obstacle avoidance during delivery). To achieve coordinated execution of the primary and secondary tasks, the robot's degrees of freedom are often increased, which in turn increases the robot's resource consumption. Summary of the Invention

[0003] The embodiments of the present application provide a task processing method, device, electronic device, computer-readable storage medium and computer program product, which can realize the collaborative execution of multiple tasks and reduce the resource consumption of robots.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] An embodiment of the present application provides a robot control method, the method comprising:

[0006] In response to a task request for the robot to be controlled, a main task to be executed is obtained;

[0007] When controlling the robot to be controlled to perform the main task to be performed, acquiring the auxiliary tasks of the robot to be controlled to obtain L auxiliary tasks to be fused, wherein the L auxiliary tasks to be fused are used to assist in performing the main task to be performed, and L is a positive integer greater than 1;

[0008] Fusing the L auxiliary tasks to be fused based on a redundancy allocation weight to obtain C auxiliary tasks to be executed, wherein the redundancy allocation weight represents a probability of allocating the C redundant degrees of freedom to the L auxiliary tasks to be fused, and C is a positive integer;

[0009] Calculating target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, the null space projection parameters, and the main task to be executed, wherein the null space projection parameters are used to project onto the null space of the main task to be executed;

[0010] The movement of the robot to be controlled is controlled based on the target movement information.

[0011] An embodiment of the present application provides a robot control device, the robot control device comprising:

[0012] A request response module, configured to respond to a task request for the robot to be controlled and obtain a main task to be executed;

[0013] a task acquisition module, configured to acquire auxiliary tasks of the robot to be controlled when controlling the robot to be controlled to perform the main task to be performed, to obtain L auxiliary tasks to be fused, wherein the L auxiliary tasks to be fused are used to assist in performing the main task to be performed, and L is a positive integer greater than 1;

[0014] a task fusion module, configured to fuse the L auxiliary tasks to be fused based on a redundancy allocation weight to obtain C auxiliary tasks to be executed, wherein the redundancy allocation weight represents the probability of allocating C redundant degrees of freedom to the L auxiliary tasks to be fused, and C is a positive integer;

[0015] a task projection module, configured to calculate target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, null space projection parameters, and the main task to be executed, wherein the null space projection parameters are used to project onto the null space of the main task to be executed;

[0016] A motion control module is used to control the motion of the robot to be controlled based on the target motion information.

[0017] In an embodiment of the present application, the task acquisition module is further used to acquire the auxiliary tasks of the robot to be controlled to obtain multiple initial auxiliary tasks; and determine the multiple initial auxiliary tasks as L auxiliary tasks to be fused.

[0018] In an embodiment of the present application, the task acquisition module is also used to perform the following processing on each of the multiple initial auxiliary tasks: decompose the initial auxiliary task to obtain at least one auxiliary meta-task; obtain L auxiliary meta-tasks corresponding to the multiple initial auxiliary tasks from the at least one auxiliary meta-task corresponding to each of the initial auxiliary tasks; and determine the L auxiliary meta-tasks as the L auxiliary tasks to be fused.

[0019] In an embodiment of the present application, the execution process of the main task to be executed corresponds to the execution cycles of multiple auxiliary tasks, and the L fused auxiliary tasks, the redundant allocation weights and the target motion information are the information of the current execution cycle; the robot control device also includes a weight acquisition module for acquiring the weight update speed of the current execution cycle; the fusion result between the weight update speed and the execution cycle duration is determined as the information to be updated; based on the information to be updated, the previous redundant allocation weight of the previous execution cycle is updated to obtain the redundant allocation weight, wherein the previous execution cycle is an adjacent execution cycle before the current execution cycle.

[0020] In an embodiment of the present application, the weight acquisition module is further used to determine the combination result of the information to be updated and the previous redundant allocation weight of the previous execution cycle as the first initial allocation weight; determine the minimum of the maximum redundant allocation weight and the first initial allocation weight as the second initial allocation weight; and determine the maximum of the minimum redundant allocation weight and the second initial allocation weight as the redundant allocation weight.

[0021] In an embodiment of the present application, the weight acquisition module is also used to obtain the unfinished degree of each auxiliary task to be merged, and obtain L unfinished degrees corresponding to the L auxiliary tasks to be merged; based on the previous redundant allocation weight, obtain the weight update priority; combine the weight update priority, the L unfinished degrees and the previous redundant allocation weight to determine the weight update speed of the current execution cycle.

[0022] In an embodiment of the present application, the weight acquisition module is also used to perform the following processing on the jth redundant degree of freedom and the i-th auxiliary task to be fused based on the previous redundant allocation weight, wherein 1≤j≤C, 1≤i≤L, and i and j are positive integer variables: based on the weights of allocating the first j-1 redundant degrees of freedom to the i-th auxiliary task to be fused, determine the redundant priority of allocating the jth redundant degree of freedom to the i-th auxiliary task to be fused; based on the weights of allocating the jth redundant degree of freedom to the first i-1 auxiliary tasks to be fused, determine the task priority of allocating the jth redundant degree of freedom to the i-th auxiliary task to be fused; based on the weights of allocating the redundant degrees of freedom other than the j-th redundant degree of freedom to the i-th auxiliary task to be fused, determine the relative priority of allocating the jth redundant degree of freedom to the i-th auxiliary task to be fused; and combining the redundant priority, the task priority and the relative priority to obtain the weight update priority corresponding to the L auxiliary tasks to be fused and the number C of redundant degrees of freedom.

[0023] In an embodiment of the present application, the weight acquisition module is also used to obtain the target combination result and target difference result between the sensitivity range and the auxiliary task to be fused, wherein the sensitivity range belongs to the normalization parameter corresponding to the auxiliary task to be fused; obtain the first fusion result of the response slope and the target combination result, wherein the normalization parameter corresponding to the auxiliary task to be fused also includes the response slope; obtain the second fusion result of the response slope and the target difference result; and quantify the incompleteness of the auxiliary task to be fused in combination with the first fusion result and the second fusion result.

[0024] In an embodiment of the present application, the task projection module is also used to obtain the pseudo-inverse matrix corresponding to the Jacobian matrix of the main task to be executed; obtain the target fusion result between the Jacobian matrix and the pseudo-inverse matrix; and obtain the null space projection parameter that is negatively correlated with the target fusion result.

[0025] In an embodiment of the present application, the task projection module is also used to obtain the first joint motion component for executing the main task to be executed based on the pseudo-inverse matrix of the main task to be executed; combine the C auxiliary tasks to be executed and the null space projection parameters of the main task to be executed to calculate the second joint motion component for executing the C auxiliary tasks to be executed; and combine the first joint motion component and the second joint motion component into the target motion information of the robot to be controlled.

[0026] An embodiment of the present application provides an electronic device for controlling a robot, the electronic device comprising:

[0027] a memory for storing computer-executable instructions or computer programs;

[0028] The processor is used to implement the robot control method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0029] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the robot control method provided in the embodiment of the present application is implemented.

[0030] An embodiment of the present application provides a computer program product, including computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the robot control method provided in the embodiment of the present application is implemented.

[0031] The embodiments of the present application have at least the following beneficial effects: when executing the main task to be executed in response to a task request for the robot, L auxiliary tasks to be fused are fused into C auxiliary tasks to be executed based on the corresponding redundant allocation weights, and then the target motion information for controlling the movement of the robot is calculated by combining the C auxiliary tasks to be executed, the zero space projection parameters of the main task to be executed and the main task to be executed; a robot control method is implemented by fusing L auxiliary tasks to be fused and projecting the fused C auxiliary tasks to be executed to the zero space of the main task to be executed, so that the execution of the L auxiliary tasks to be fused is independent of the execution of the main task to be executed; that is, while ensuring the execution of the main task to be executed, the C redundant degrees of freedom included in the robot can be used to realize the execution of multiple auxiliary tasks, thereby realizing the collaborative execution of multiple tasks and reducing the resource consumption of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Schematic diagram of the architecture of the robot control system provided in an embodiment of the present application;

[0033] Figure 2 This embodiment of the present application provides a Figure 1 A schematic diagram of the structure of the terminal in FIG.

[0034] Figure 3 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 1 ;

[0035] Figure 4 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 2 ;

[0036] Figure 5 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 3 ;

[0037] Figure 6 This is a flow chart of obtaining the weight update speed provided by an embodiment of the present application;

[0038] Figure 7 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 4 ;

[0039] Figure 8 This is an exemplary multi-task collaborative execution diagram provided by an embodiment of the present application;

[0040] Figure 9 This is a schematic diagram of a robot control application provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0042] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] In the following description, the terms "first\second" are used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0044] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0045] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0046] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0047] 1) Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that aims to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. By studying the design principles and implementation methods of various intelligent machines, AI enables them to possess the capabilities of perception, reasoning, and decision-making.

[0048] It should be noted that artificial intelligence technology is an interdisciplinary subject that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models; after fine-tuning, pre-trained models can be widely used in downstream tasks in various directions of artificial intelligence. Artificial intelligence software technology includes several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. In the embodiment of the present application, redundant robots can be realized through artificial intelligence technology and are the product of artificial intelligence.

[0049] 2) Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It is used to study computer simulation or implementation of human learning behavior to acquire new knowledge or skills; reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Machine learning applications are spread across all areas of artificial intelligence. Machine learning / deep learning generally includes technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning. Large models are the latest developments in machine learning / deep learning, integrating the above technologies. In an embodiment of the present application, the interactive processing of redundant robots and task execution scenarios can be implemented in combination with machine learning / deep learning.

[0050] 3) Artificial neural networks are mathematical models that mimic the structure and function of biological neural networks. Exemplary structures of artificial neural networks in the embodiments of this application include graph convolutional networks (GCNs, a type of neural network used to process graph-structured data), deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), neural state machines (NSMs), and phase-functioned neural networks (PFNNs). In the embodiments of this application, the interaction between redundant robots and task execution scenarios can be implemented using artificial neural network models.

[0051] It should be noted that with the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence generated content (AIGC), conversational interaction, smart medical care, smart customer service and virtual scenes. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role. The robot control method provided in the embodiment of the present application describes the application of artificial intelligence technology in the fields of smart home and robots, and is specifically illustrated by the following embodiment.

[0052] 4) Redundant robots, also known as redundant degree-of-freedom robots or redundant robots, are robots with more degrees of freedom than required (i.e., they include additional degrees of freedom, also known as redundant degrees of freedom). These robots can use these redundant degrees of freedom to optimize performance when performing tasks, such as obstacle avoidance. These redundant degrees of freedom are typically added to the robot's end effector to enable rotation, translation, or other types of motion, thereby increasing the robot's flexibility in performing tasks. The robots in the embodiments of this application may be redundant robots.

[0053] 5) Degree of freedom refers to the number of independent motion parameters of the robot; for example, a component that supports single-dimensional rotation has 1 degree of freedom; a component that supports single-dimensional movement has 1 degree of freedom.

[0054] 6) Null space, a type of vector space; for matrix B, the null space of B is also called the kernel, which refers to the set of all solutions y for the linear equation system By=0; in the embodiment of the present application, the null space refers to the null space of the main task to be executed, which represents the space outside the space involved in the execution of the main task by the redundant robot.

[0055] It should be noted that when robots are used to perform tasks, the tasks performed usually include a main task and multiple auxiliary tasks; however, when robots are directly used to achieve the coordinated execution of the main task and multiple auxiliary tasks, due to the limitation of the number of redundant degrees of freedom, there are often situations where execution cannot be performed, which affects the task execution scenarios in which the robots are applicable.

[0056] In addition, in order to achieve the coordinated execution of the main task and multiple auxiliary tasks, it is also possible to increase the robot's degree of freedom, thereby increasing the robot's resource consumption.

[0057] Based on this, the embodiments of the present application provide a robot control method, device, electronic device, computer-readable storage medium and computer program product, which can broaden the task execution scenarios applicable to the robot, realize multi-task collaborative execution, and reduce the resource consumption of the robot. The following describes an exemplary application of the electronic device for controlling the robot provided by the embodiments of the present application (hereinafter referred to as the robot control device). The robot control device provided by the embodiments of the present application can be implemented as various types of terminals such as robots, smart phones, smart watches, laptops, tablet computers, desktop computers, smart home appliances, set-top boxes, smart car devices, portable music players, personal digital assistants, dedicated messaging devices, intelligent voice interaction devices, portable gaming devices and smart speakers. It can also be implemented as a server, or a combination of the two. Below, an exemplary application of the robot control device when it is implemented as a terminal will be described.

[0058] See also Figure 1 , Figure 1 Schematic diagram of the robot control system provided in the embodiment of the present application; Figure 1 As shown, to support a robot control application, in the robot control system 100, the terminal 400 (terminal 400-1 and terminal 400-2 are shown as examples) is connected to the server 200 via the network 300, and the terminal 400 is used to control the robot 600 to be controlled; wherein the network 300 can be a wide area network or a local area network, or a combination of the two, and the server 200 is used to provide computing services to the terminal 400 via the network 300. In addition, the robot control system 100 also includes a database 500 for providing data support to the server 200; and, Figure 1 The example shown in FIG. 5 is a case where the database 500 is independent of the server 200. In addition, the database 500 may also be integrated into the server 200, which is not limited in the embodiment of the present application. Figure 1 , the terminal 400 is independent of the robot 600 to be controlled. In addition, the terminal 400 can also be integrated into the robot 600 to be controlled, which is not limited in this embodiment of the present application.

[0059] Terminal 400 is used to respond to a task request for the robot to be controlled, obtain the main task to be executed, and when controlling the robot to be controlled to execute the main task to be executed, obtain the auxiliary tasks of the robot to be controlled to obtain L auxiliary tasks to be fused; fuse the L auxiliary tasks to be fused based on the current redundant allocation weight to obtain C auxiliary tasks to be executed; combine the C auxiliary tasks to be executed, the null space projection parameters of the main task to be executed and the main task to be executed to calculate the target motion information of the robot to be controlled (when presenting the target motion information, the graphical interface 410-1 and the graphical interface 410-2 are exemplarily shown for the presented target motion information); control the motion of the robot to be controlled based on the target motion information.

[0060] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal and the server may be connected directly or indirectly via wired or wireless communication, which is not limited in the embodiments of the present application.

[0061] See also Figure 2 , Figure 2 This embodiment of the present application provides a Figure 1 The schematic diagram of the terminal structure in Figure 2 As shown, the terminal 400 includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 2 Various buses are labeled as bus system 440 .

[0062] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0063] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0064] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0065] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0066] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0067] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0068] A network communication module 452 for reaching other electronic devices via one or more (wired or wireless) network interfaces 420 , exemplary network interfaces 420 including Bluetooth, Wi-Fi, and Universal Serial Bus (USB);

[0069] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);

[0070] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.

[0071] In some embodiments, the robot control device provided in the embodiments of the present application can be implemented in a software manner. Figure 2 A robot control device 455 stored in memory 450 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: a request response module 4551, a task acquisition module 4552, a task fusion module 4553, a task projection module 4554, a motion control module 4555, and a weight acquisition module 4556. These modules are logical and can be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0072] In some embodiments, the robot control device provided in the embodiments of the present application can be implemented in hardware. As an example, the robot control device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the robot control method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0073] In some embodiments, the terminal or server can implement the robot control method provided in the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be microprogram-level commands, machine instructions or software instructions. The computer program can be a native program or software module in the operating system; it can be a local (Native) application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as a robot APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0074] The following describes the robot control method provided by the present invention in conjunction with exemplary applications and implementations of the robot control device provided by the present invention. Furthermore, the robot control method provided by the present invention is applicable to various robot control scenarios, including robotics, cloud technology, artificial intelligence, and smart transportation.

[0075] See also Figure 3 , Figure 3 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 1 , Figure 3 The execution subject of each step is the robot control device; Figure 3 The steps shown are explained.

[0076] Step 101: In response to a task request for a robot to be controlled, a main task to be executed is obtained.

[0077] In an embodiment of the present application, when a task is performed by the robot to be controlled, for example, serving tea or delivering meals, the robot to be controlled also receives a task request for the robot to be controlled; since the task request is used to request the robot to be controlled to perform a task, at this time, the robot control device responds to the task request and can obtain the task requested to be performed by the task request; here, the task requested to be performed by the task request is called the main task to be performed, such as serving tea, delivering meals, etc.

[0078] It should be noted that the movement of the robot to be controlled is controlled by a robot control device, and the robot control device performs corresponding tasks by controlling the robot to be controlled to perform different movements; wherein, the robot control device can be a device independent of the robot to be controlled, or it can be a device integrated in the robot to be controlled, and the embodiments of the present application do not limit this.

[0079] Step 102: When controlling the robot to be controlled to perform the main task to be performed, the auxiliary tasks of the robot to be controlled are acquired to obtain L auxiliary tasks to be fused.

[0080] In an embodiment of the present application, when the robot to be controlled is executing the main task to be executed, it will also execute tasks used to assist the execution of the main task to be executed, that is, auxiliary tasks, such as obstacle avoidance, joint constraints, singularity, etc.; here, the robot control device acquires auxiliary tasks when controlling the robot to be controlled to execute the main task to be executed, and thus obtains L auxiliary tasks to be fused.

[0081] It should be noted that the L auxiliary tasks to be fused are used to assist in executing the main task to be executed, L is a positive integer greater than 1, and each auxiliary task to be fused is an auxiliary task.

[0082] See also Figure 4 , Figure 4 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 2 , Figure 4 The execution subject of each step is the robot control device; Figure 4As shown, step 102 can be implemented through step 1021 and step 1022A; that is, when the robot control device controls the robot to be controlled to perform the main task to be performed, it obtains the auxiliary task of the robot to be controlled and obtains L auxiliary tasks to be fused, including step 1021 and step 1022A. Each step is explained below.

[0083] Step 1021 : When controlling the robot to be controlled to execute the main task to be executed, the auxiliary tasks of the robot to be controlled are acquired to obtain a plurality of initial auxiliary tasks.

[0084] It should be noted that the multiple initial auxiliary tasks are multiple auxiliary tasks obtained by the robot control device, and are the results obtained by the robot control device from obtaining the auxiliary tasks for controlling the robot.

[0085] Step 1022A: Determine the multiple initial auxiliary tasks as L auxiliary tasks to be merged.

[0086] In an embodiment of the present application, the robot control device can directly determine multiple initial auxiliary tasks as L auxiliary tasks to be fused, and can also obtain L auxiliary tasks to be fused by decomposing multiple initial auxiliary tasks. This embodiment of the present application is not limited to this.

[0087] In an embodiment of the present application, step 1021 also includes steps 1022B1 to 1022B3; that is, the robot control device acquires the auxiliary tasks of the robot to be controlled, and after obtaining multiple initial auxiliary tasks, the robot control method also includes steps 1022B1 to 1022B3, and each step is explained below.

[0088] In the embodiment of the present application, the robot control device performs the following processing (steps 1022B1 to 1022B3) on each of the multiple initial auxiliary tasks.

[0089] Step 1022B1: Decompose the initial auxiliary task to obtain at least one auxiliary meta-task.

[0090] It should be noted that the robot control device decomposes each of the multiple initial auxiliary tasks into at least one auxiliary meta-task. Here, when the initial auxiliary task cannot be decomposed, the initial auxiliary task becomes an auxiliary meta-task. Furthermore, the robot control device may decompose the initial auxiliary task based on the directional dimension, determine the decomposition method based on the type of the initial auxiliary task, and so on, which are not limited in this embodiment of the present application.

[0091] For example, when the initial auxiliary task is an obstacle avoidance task, the initial auxiliary task can be decomposed into three auxiliary meta-tasks: obstacle avoidance in the first dimensional direction (for example, the X direction of the two-dimensional coordinate axis), obstacle avoidance in the second dimensional direction (for example, the Y direction of the two-dimensional coordinate axis), and yaw rotation obstacle avoidance.

[0092] Step 1022B2: Obtain L auxiliary meta-tasks corresponding to the multiple initial auxiliary tasks from the at least one auxiliary meta-task corresponding to each initial auxiliary task.

[0093] In an embodiment of the present application, after the robot control device completes the decomposition of each initial auxiliary task in a plurality of initial auxiliary tasks, it combines at least one auxiliary meta-task corresponding to each of the plurality of initial auxiliary tasks, thereby obtaining L auxiliary meta-tasks corresponding to the plurality of initial auxiliary tasks; that is, the L auxiliary meta-tasks include at least one auxiliary meta-task corresponding to each of the plurality of initial auxiliary tasks.

[0094] Step 1022B3: Determine the L auxiliary meta-tasks as L auxiliary tasks to be fused.

[0095] In the embodiment of the present application, the robot control device determines the obtained L auxiliary meta-tasks as L auxiliary tasks to be fused, that is, the auxiliary meta-tasks are auxiliary tasks to be fused.

[0096] It can be understood that by decomposing the obtained initial auxiliary tasks to obtain basic auxiliary meta-tasks with finer granularity than the initial auxiliary tasks, the feasibility of compact and asynchronous execution of auxiliary tasks can be improved, thereby improving the utilization of redundant degrees of freedom.

[0097] Step 103: Fuse the L auxiliary tasks to be fused based on the redundancy allocation weights to obtain C auxiliary tasks to be executed.

[0098] In an embodiment of the present application, the redundancy allocation weight represents the probability of allocating C redundant degrees of freedom to L auxiliary tasks to be fused, and is used to fuse the L auxiliary tasks to be fused into C auxiliary tasks to be executed that can be executed by the remaining redundant degrees of freedom; thus, after the robot control device obtains the L auxiliary tasks to be fused, the L auxiliary tasks to be fused are fused based on the redundancy allocation weight, and the obtained fusion result is C auxiliary tasks to be executed.

[0099] It should be noted that C is the number of redundant degrees of freedom of the robot to be controlled, and C is a positive integer. When the robot to be controlled includes N degrees of freedom and M degrees of freedom are used to execute the main task to be executed, C is NM. The redundancy allocation weight includes the weight (also known as the likelihood or probability) assigned to each redundant degree of freedom to each auxiliary task to be fused. Through the redundancy allocation weight, L auxiliary tasks to be fused can be fused into C auxiliary tasks to be executed by C redundant degrees of freedom, thereby completing the execution of the auxiliary tasks of the main task to be executed.

[0100] Step 104 : Calculate the target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, the null space projection parameters, and the main task to be executed.

[0101] In an embodiment of the present application, the robot control device uses C to-be-executed auxiliary tasks as secondary tasks of the to-be-executed main task, and projects the secondary tasks onto the null space of the to-be-executed main task based on the null space projection parameters of the to-be-executed main task, to obtain motion information of the robot to be controlled, which performs the to-be-executed tasks with M degrees of freedom and performs the auxiliary tasks with the remaining C redundant degrees of freedom. The obtained motion information of the robot to be controlled is referred to as target motion information. It is easy to understand that target motion information refers to the motion information of the robot to be controlled, such as joint velocities, and this target motion information can be used to execute the auxiliary tasks of the to-be-executed main task within the null space of the to-be-executed main task while executing the to-be-executed main task.

[0102] In an embodiment of the present application, the robot control device combines C auxiliary tasks to be executed, the zero-space projection parameters and the main task to be executed to calculate the target motion information of the robot to be controlled, including: the robot control device first obtains the first joint motion component for executing the main task to be executed based on the pseudo-inverse matrix of the main task to be executed; then combines the C auxiliary tasks to be executed and the zero-space projection parameters of the main task to be executed to calculate the second joint motion component for executing the C auxiliary tasks to be executed; finally, the first joint motion component and the second joint motion component are combined into the target motion information of the robot to be controlled.

[0103] It should be noted that the Jacobian matrix and other linear approximation information of the main task to be executed can be obtained from the main task to be executed, and the pseudo-inverse linear approximation information of the Jacobian matrix of the main task to be executed can be obtained from the Jacobian matrix of the main task to be executed; here, the robot control device can use the product of the pseudo-inverse matrix of the main task to be executed and the main task to be executed as the first joint motion component. The null space projection parameter is used to project the C auxiliary tasks to be executed into the null space of the main task to be executed. Here, the robot control device can first obtain the product of the linear approximation information of the C auxiliary tasks to be executed (such as the augmented Jacobian matrix, etc.) and the first joint motion component, and then obtain the difference between the C auxiliary tasks to be executed and the product, then multiply the linear approximation information of the C auxiliary tasks to be executed with the null space projection parameter, and then multiply the pseudo-inverse matrix of the multiplication result with the null space projection parameter and the above difference to obtain the second joint motion component. The robot control device can combine the first joint motion component and the second joint motion component in an additive manner to obtain the target motion information.

[0104] Step 105: Control the movement of the robot to be controlled based on the target movement information.

[0105] It should be noted that since the target motion information is the motion information of the robot, the robot control device controls the robot to be controlled to move based on the target motion information, so that the robot to be controlled can independently perform the corresponding auxiliary task while performing the main task to be performed.

[0106] It can be understood that when the main task to be executed is executed in response to the task request for the robot to be controlled, the L auxiliary tasks to be fused are fused into C auxiliary tasks to be executed based on the corresponding redundant allocation weights, and then the target motion information for controlling the motion of the robot to be controlled is calculated by combining the C auxiliary tasks to be executed, the null space projection parameters of the main task to be executed and the main task to be executed; a robot control method is implemented by fusing L auxiliary tasks to be fused and projecting the fused C auxiliary tasks to be executed to the null space of the main task to be executed, so that the execution of the L auxiliary tasks to be fused is independent of the execution of the main task to be executed; that is, while ensuring the execution of the main task to be executed, the C redundant degrees of freedom included in the robot to be controlled can be used to realize the execution of multiple auxiliary tasks, thereby realizing the collaborative execution of multiple tasks and reducing the resource consumption of the robot to be controlled.

[0107] See also Figure 5 , Figure 5 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 3 , Figure 5 The execution subject of each step is the robot control device; Figure 5As shown, in an embodiment of the present application, steps 106 to 108 are also included before step 103; that is, before the robot control device fuses L auxiliary tasks to be fused based on the redundant allocation weights and obtains C auxiliary tasks to be executed, the robot control method also includes steps 106 to 108, and each step is explained below.

[0108] Step 106: Obtain the weight update speed of the current execution cycle.

[0109] It should be noted that the process of the robot control device controlling the robot to be controlled to perform the main task to be performed corresponds to multiple execution cycles for performing auxiliary tasks; that is, the robot control device divides the process of the robot to be controlled to perform the main task to be performed into multiple execution cycles; wherein, each execution cycle refers to a sub-process of the robot to be controlled to perform the main task to be performed, and the cycle lengths of multiple execution cycles can be the same, different, or a combination of the two, etc., which is not limited in the embodiments of the present application. Here, the robot control device executes steps 102 to 105 in each execution cycle of controlling the robot to be controlled to perform the main task to be performed; thus, the L fused auxiliary tasks, redundant allocation weights, and target motion information are information of the current execution cycle; the robot control device acquires auxiliary tasks in units of execution cycles, and thus obtains L auxiliary tasks to be fused corresponding to each execution cycle; the L auxiliary tasks to be fused are used to assist in the execution of the main task to be performed within the corresponding execution cycle, L is a positive integer greater than 1, and each auxiliary task to be fused is an auxiliary task within the current execution cycle.

[0110] In an embodiment of the present application, when the current execution cycle is the first execution cycle, the redundancy allocation weight can be specified information, and the specified information can be obtained by obtaining input information, or can be obtained by artificial intelligence, etc., and the embodiment of the present application does not limit this. When the current execution cycle is an execution cycle after the first execution cycle, the update speed of the redundant allocation weight corresponding to the current execution cycle is called the weight update speed. The weight update speed includes the update speed corresponding to each weight in the redundant allocation weight.

[0111] See also Figure 6 , Figure 6 This is a flow chart of obtaining the weight update speed provided by an embodiment of the present application. Figure 6 The execution subject of each step is the robot control device; Figure 6 As shown, in an embodiment of the present application, step 106 can be implemented through steps 1061 to 1063; that is, the robot control device obtains the weight update speed of the current execution cycle, including steps 1061 to 1063, and each step is explained below.

[0112] Step 1061: Obtain the uncompleted degree of each auxiliary task to be merged, and obtain L uncompleted degrees corresponding to the L auxiliary tasks to be merged.

[0113] It should be noted that the unfinished degree indicates the degree to which the auxiliary tasks to be merged are to be completed, and is negatively correlated with the degree to which the auxiliary tasks to be merged have been completed. There is a one-to-one correspondence between the L auxiliary tasks to be merged and the L unfinished degrees.

[0114] In an embodiment of the present application, the robot control device obtains the incompleteness of each auxiliary task to be fused, including: the robot control device first obtains the target combination result and target difference result between the sensitivity range and the auxiliary task to be fused; then obtains a first fusion result of the response slope and the target combination result; then, obtains a second fusion result of the response slope and the target difference result; finally, the incompleteness of the auxiliary task to be fused is quantified by combining the first fusion result and the second fusion result.

[0115] It should be noted that the normalization parameters of the auxiliary task to be fused include the sensitivity range and response slope corresponding to the normalization function, and the normalization parameters of different auxiliary tasks to be fused can be different, or of course, they can be the same. The robot control device can obtain a target combination result by adding the sensitivity range and the auxiliary task to be fused, and obtain a target difference result by subtracting the sensitivity range from the auxiliary task to be fused. The first fusion result can be obtained by multiplying the response slope and the target combination result, and the second fusion result can be obtained by multiplying the response slope and the target difference result. Here, the robot control device quantifies the incompletion of the auxiliary task to be fused by combining the first fusion result and the second fusion result, including: the robot control device uses the first fusion result as an exponent of a natural constant (e) to obtain a first index result; uses the second fusion result as an exponent of the natural constant to obtain a second index result; obtains a first state component that is negatively correlated with the first index result, and obtains a second state component that is negatively correlated with the second index result; and finally, determines the sum of the first state component and the second state component as the incompletion.

[0116] Step 1062: Based on the previous redundant allocation weight, obtain the weight update priority.

[0117] It should be noted that the robot control device determines the weight update priority for the current execution cycle based on the historical allocation of redundant degrees of freedom to the auxiliary tasks to be fused (called allocation preference). The previous redundant allocation weight includes the weight assigned to each redundant degree of freedom to each auxiliary task to be fused in the previous execution cycle, that is, the historical allocation of redundant degrees of freedom to the auxiliary tasks to be fused, and the previous execution cycle is the execution cycle immediately preceding the current execution cycle. Here, the robot control device obtains the weight update priority based on the previous redundant allocation weight, so that the obtained weight update priority can ensure the stability of the allocation relationship between redundant degrees of freedom and auxiliary tasks to be fused.

[0118] In an embodiment of the present application, the robot control device obtains a weight update priority based on the previous redundant allocation weight, including: the robot control device performs the following processing on the j-th redundant degree of freedom and the i-th auxiliary task to be fused based on the previous redundant allocation weight: based on the weight of allocating the first j-1 redundant degrees of freedom to the i-th auxiliary task to be fused, the redundant priority of allocating the j-th redundant degree of freedom to the i-th auxiliary task to be fused is determined, and based on the weight of allocating the j-th redundant degree of freedom to the first i-1 auxiliary tasks to be fused, the task priority of allocating the j-th redundant degree of freedom to the i-th auxiliary task to be fused is determined, and based on the weight of allocating redundant degrees of freedom other than the j-th redundant degree of freedom to the i-th auxiliary task to be fused, the relative priority of allocating the j-th redundant degree of freedom to the i-th auxiliary task to be fused is determined; finally, combining the redundant priority, task priority and relative priority, the weight update priority corresponding to the L auxiliary tasks to be fused and the number C of redundant degrees of freedom.

[0119] It should be noted that 1≤j≤C, 1≤i≤L, and i and j are positive integer variables. Here, the robot control device obtains the weights for assigning the first j-1 redundant degrees of freedom to the i-th auxiliary task to be fused from the previous redundant allocation weights. That is, the first j-1 redundant degrees of freedom are respectively assigned to the j-1 weights of the i-th auxiliary task to be fused, and the obtained result negatively correlated with the j-1 weights (for example, obtaining the j-1 differences between the specified weight (1, etc.) and the j-1 weights, and then obtaining the product of the j-1 differences) is determined as the redundancy priority for assigning the j-th redundant degree of freedom to the i-th auxiliary task to be fused. The robot control device obtains the weight for allocating the j-th redundant degree of freedom to the first i-1 auxiliary tasks to be fused from the previous redundant allocation weight, that is, obtains the i-1 weights for allocating the j-th redundant degree of freedom to the first i-1 auxiliary tasks to be fused, and uses the obtained results that are negatively correlated with the i-1 weights (for example, obtain the i-1 differences between the specified weights and the i-1 weights, and then obtain the product of the i-1 differences) to determine the task priority for allocating the j-th redundant degree of freedom to the i-th auxiliary task to be fused. The robot control device obtains the weights of the redundant degrees of freedom allocated to the i-th auxiliary task to be fused except the j-th redundant degree of freedom from the previous redundant allocation weights, that is, obtains the C-1 weights corresponding to the C-1 redundant degrees of freedom allocated to the i-th auxiliary task to be fused except the j-th redundant degree of freedom, and uses the obtained results that are negatively correlated with the C-1 weights (for example, obtain C-1 differences between the maximum weight and the C-1 weights, and then obtain the product of the C-1 differences) to determine the relative priority of allocating the j-th redundant degree of freedom to the i-th auxiliary task to be fused.

[0120] It should also be noted that the robot control device can achieve combination by obtaining the product of redundant priority, task priority and relative priority, and the obtained combination result is the target priority corresponding to the jth redundant degree of freedom and the i-th auxiliary task to be fused; thus, C*L target priorities can be obtained, and C*L target priorities are the weighted update priorities corresponding to the L auxiliary tasks to be fused and the number C of redundant degrees of freedom.

[0121] It can be understood that by determining the weight update priority based on the historical distribution of redundant degrees of freedom in the auxiliary tasks to be fused, the current redundant allocation weight obtained through the weight update priority can improve the stability of the distribution relationship between redundant degrees of freedom and the auxiliary tasks to be fused, thereby improving the task execution efficiency.

[0122] Step 1063: Determine the weight update speed of the current execution cycle by combining the weight update priority, the L unfinished degrees, and the previous redundant allocation weight.

[0123] In an embodiment of the present application, the robot control device obtains L incomplete diagonal matrices and uses a winner-takes-all strategy to process the weight update priority, the diagonal matrix and the previous redundant allocation weight to obtain the weight update speed of the current execution cycle.

[0124] It can be understood that by obtaining the weight update speed corresponding to each execution cycle and then updating the redundant fusion weights based on the weight update speed, the dynamic fusion of the auxiliary tasks to be fused is achieved, thereby improving the effect of multi-task collaborative execution.

[0125] Step 107: Determine the fusion result between the weight update speed and the execution cycle duration as the information to be updated.

[0126] In an embodiment of the present application, the robot control device may use the product of the weight update speed and the execution cycle duration as the fusion result of the weight update speed and the execution cycle duration; thus, the information to be updated is the product of the weight update speed and the execution cycle duration. The execution cycle duration refers to the duration of the current execution cycle.

[0127] Step 108: Update the previous redundancy allocation weight of the previous execution cycle based on the information to be updated to obtain the redundancy allocation weight.

[0128] It should be noted that the robot control device may directly use the superposition result of the previous redundancy allocation weight and the information to be updated as the redundancy allocation weight, or may determine the redundancy allocation weight based on the judgment result after judging the superposition result based on a specified range.

[0129] See also Figure 7 , Figure 7 This is a schematic diagram of the process of the robot control method provided in the embodiment of the present application. Figure 4 , Figure 7 The execution subject of each step is the robot control device; Figure 7 As shown, in an embodiment of the present application, when the robot control device obtains the superposition result of the previous redundant allocation weight and the information to be updated, and judges the superposition result based on the specified range to obtain the redundant allocation weight, step 108 can be implemented through steps 1081 to 1083; that is, the robot control device updates the previous redundant allocation weight of the previous execution cycle based on the information to be updated to obtain the redundant allocation weight, including steps 1081 to 1083, and each step is explained below.

[0130] Step 1081: Determine the first initial allocation weight as a result of combining the information to be updated with the previous redundancy allocation weight of the previous execution cycle.

[0131] It should be noted that the first initial allocation weight refers to the combination result of the information to be updated and the previous redundant allocation weight, for example, the sum of the two.

[0132] Step 1082: Determine the minimum of the maximum redundancy allocation weight and the first initial allocation weight as the second initial allocation weight.

[0133] It should be noted that the redundancy allocation weight includes a specified range, which is bounded by a minimum redundancy allocation weight and a maximum redundancy allocation weight. Here, the robot control device compares the first initial allocation weight with the maximum redundancy allocation weight. If the maximum redundancy allocation weight is less than the first initial allocation weight, the maximum redundancy allocation weight is the minimum of the maximum redundancy allocation weight and the first initial allocation weight, and thus, in this case, the maximum redundancy allocation weight is the second initial allocation weight. If the maximum redundancy allocation weight is greater than the first initial allocation weight, the first initial allocation weight is the minimum of the maximum redundancy allocation weight and the first initial allocation weight, and thus, in this case, the first initial allocation weight is the second initial allocation weight. It is easy to understand that when the maximum redundancy allocation weight and the first initial allocation weight are equal, the second initial allocation weight is either the maximum redundancy allocation weight or the first initial allocation weight.

[0134] Step 1083: Determine the maximum of the minimum redundancy allocation weight and the second initial allocation weight as the redundancy allocation weight.

[0135] It should be noted that the robot control device compares the second initial allocation weight with the minimum redundant allocation weight. If the minimum redundant allocation weight is greater than the second initial allocation weight, the minimum redundant allocation weight is the maximum of the minimum redundant allocation weight and the second initial allocation weight, so at this time, the minimum redundant allocation weight is the redundant allocation weight; if the minimum redundant allocation weight is less than the second initial allocation weight, the second initial allocation weight is the maximum of the minimum redundant allocation weight and the second initial allocation weight, so at this time, the second initial allocation weight is the redundant allocation weight; it is easy to know that when the minimum redundant allocation weight and the second initial allocation weight are equal, the redundant allocation weight is any one of the minimum redundant allocation weight and the second initial allocation weight.

[0136] It can be understood that the current redundant allocation weight obtained is determined by the minimum redundant allocation weight and the maximum redundant allocation weight, so that the obtained redundant allocation weight is within the specified range, which improves the rationality of the redundant freedom degree allocation and thus improves the task execution effect.

[0137] In an embodiment of the present application, before the robot control device calculates the target motion information of the robot to be controlled by combining C auxiliary tasks to be executed, the null space projection parameters and the main task to be executed, the robot control method also includes: the robot control device first obtains the pseudo-inverse matrix corresponding to the Jacobian matrix of the main task to be executed; then obtains the target fusion result between the Jacobian matrix and the pseudo-inverse matrix; finally, obtains the null space projection parameters that are negatively correlated with the target fusion result.

[0138] It should be noted that the target fusion result can be the product of the Jacobian matrix and the pseudo-inverse matrix; here, the null space projection parameter is negatively correlated with the target fusion result, for example, the null space projection parameter is the difference between the unit matrix and the target fusion result.

[0139] Below, we will describe an exemplary application of the embodiments of the present application in a practical application scenario. This exemplary application describes the process of collaboratively executing the main task of tea delivery and subtasks such as obstacle avoidance and joint constraints (referred to as auxiliary tasks) in a tea delivery scenario using redundant robots. This is a multi-task collaborative execution method based on subtask decomposition and fusion. It is easy to understand that the robot to be controlled in the embodiments of the present application can be any robot. The redundant robot is used as an example for illustration.

[0140] It should be noted that the multi-task collaborative execution method based on subtask decomposition and fusion described in this exemplary application includes four processes: subtask decomposition, meta-task fusion, dynamic update of fusion matrix and null space projection of secondary tasks.

[0141] See also Figure 8 , Figure 8 This is an exemplary multi-task collaborative execution diagram provided by an embodiment of the present application; Figure 8 As shown, the main task 8-1 (called the main task to be executed) is to deliver tea; the secondary task 8-2 (recorded as C auxiliary tasks to be executed) is obtained by performing the meta-task set 8-3 (denoted as Or recorded as The meta-task set 8-3 is obtained by performing the meta-task fusion processing 8-41 on multiple sub-tasks 8-5 (exemplarily showing pedestrian obstacle avoidance (Humans Obstacles), joint constraints (Joint Limits), ..., singularity (Singularity), called multiple initial auxiliary tasks); here, the secondary task 8-2 is projected into the null space of the main task 8-1 to calculate the joint speed 8-6 (called target motion information) of the redundant robot at the current moment, that is, the null space projection processing of the secondary task is performed. Among them, in the meta-task fusion processing 8-41 process, the winner-takes-all strategy is adopted to process the state matrix (called L incompleteness), soft preference matrix (called weight update priority) and the previous moment (or initial) fusion matrix (called previous redundant allocation weight) of the meta-task set 8-3 to obtain the update rate (called weight update speed) at the current moment, and then update the previous moment (or initial) fusion matrix according to the update rate to obtain the fusion matrix at the current moment, and then fuse the meta-task set 8-3 based on the fusion matrix at the current moment, and thus obtain the secondary task 8-2.

[0142] The subtask decomposition process is described below.

[0143] Subtask decomposition refers to decomposing each multidimensional subtask (called initial auxiliary task) into multiple one-dimensional meta-tasks (called auxiliary meta-tasks). In this way, for multiple subtasks, a meta-task set can be obtained. As shown in formula (1).

[0144]

[0145] in, is the i-th (i=1,2,…,L) meta-task, L is the total number of meta-tasks, f i represents the result of the ith meta-task based on the state of the ith meta-task and other related parameters, ξ i Represents the status and other related parameters of the i-th meta-task.

[0146] in addition, The kinematic representation of is shown in formula (2).

[0147]

[0148] in, express The Jacobian matrix of ; q represents the joint position corresponding to the redundant robot with N degrees of freedom; Represents the joint velocity of the redundant robot, which is unknown data.

[0149] Based on formula (1) and formula (2), multiple meta-task sets It can be expressed by formula (3), which is as follows.

[0150]

[0151] Among them, J su$ (q) is the augmented Jacobian matrix of the L meta-tasks.

[0152] The following describes the meta-task fusion process.

[0153] Meta-task fusion refers to the process of combining meta-tasks Integration as the main task A secondary task As shown in formula (4).

[0154]

[0155] Among them, H represents the allocation of NM redundancies (redundancy is the abbreviation of redundant degrees of freedom) of the redundant robot to L meta-tasks γ is the upper limit of the fusion matrix elements (called the maximum weight); NM represents the number of redundant degrees of freedom; A ("*+)×L (t) is the fusion matrix of the meta-task set at time t (called the redundant allocation matrix), as shown in formula (5).

[0156]

[0157] Among them, α ji (t) represents the weight of assigning the jth (i=1,2,…,NM) redundancy to the i-th meta-task at time t, and,

[0158] The dynamic update process of the fusion matrix is described below.

[0159] Dynamic updating of the fusion matrix refers to dynamically updating the redundant weights assigned to meta-tasks in meta-task fusion based on meta-task characteristics and personalized settings (e.g., preference settings). The fusion matrix is updated as shown in Equation (6).

[0160]

[0161] in, is the update rate of the fusion matrix at time t-1 (called weight update speed), E is the all-one matrix, and Δt is the update interval (called execution cycle duration). Here, when the winner-takes-all strategy W is used to define the update rate of the fusion matrix, As shown in formula (7).

[0162]

[0163] Among them, P(t-1) is the soft preference matrix (called weight update priority), and S is the state matrix of the meta-task.

[0164] The soft preference matrix is shown in formula (8).

[0165]

[0166] Among them, p ji (t-1)∈(0,1), represents α determined based on the state and preference of the meta-task ji The update priority of (t-1) can be obtained by formula (9), which is as follows.

[0167]

[0168] in, (called redundancy priority) means that if the weight of assigning the first j-1 redundancies to the i-th meta-task is greater, the priority of assigning the j-th redundancy to the i-th meta-task will be lower; and if the weight of assigning the first j-1 redundancies to the i-th meta-task is smaller, the priority of assigning the j-th redundancy to the i-th meta-task will be higher. (called task priority) means that if the weight assigned to the jth redundancy of the first i-1 meta-tasks is greater, the priority assigned to the jth redundancy of the i-th meta-task is lower; and if the weight assigned to the jth redundancy of the first i-1 meta-tasks is smaller, the priority assigned to the jth redundancy of the i-th meta-task is higher. uXj Eγ-α ui (t-1)I (called relative priority) means that if the weight assigned to the i-th meta-task except the j-th redundancy is greater, the priority assigned to the i-th meta-task to the j-th redundancy is lower; and if the weight assigned to the i-th meta-task except the j-th redundancy is smaller, the priority assigned to the i-th meta-task to the j-th redundancy is higher.

[0169] The state matrix is shown in formula (10).

[0170]

[0171] Where diag represents the diagonal matrix function; It represents the active state of the ith meta-task, that is, the degree of incompletion, as shown in formula (11).

[0172]

[0173] Among them, k i is the response slope of the normalized function corresponding to the i-th meta-task, di is the sensitivity range of the normalized function corresponding to the i-th meta-task.

[0174] The null space projection process of the secondary task is described below.

[0175] The null space projection of the secondary task refers to projecting the secondary task into the null space of the main task for execution, so as to ensure that the execution of the secondary task is independent of the execution of the main task and reduce the impact on the main task.

[0176] The kinematic representation of is shown in formula (12).

[0177]

[0178] Where J2(q) represents the Jacobian matrix of the secondary task, as shown in Equation (13).

[0179] J2(q)=(1 / y)A (N-M)×L (t)J sub (q) (13);

[0180] Based on equations (3), (4) and (13), the inverse kinematics solution combining the main task and the secondary task can be obtained: (called target motion information), as shown in formula (14).

[0181]

[0182] Thus, at time t, we can calculate Control the movement of redundant robots to complete the coordinated execution of the main task and multiple subtasks; is the pseudo-inverse matrix of the Jacobian matrix J1 of the main task; N1 is the operator projected to the null space of the main task (called the null space projection parameter), as shown in formula (15).

[0183]

[0184] Where I is the identity matrix.

[0185] It should be noted that the second joint motion component described above refers to the following content in formula (14):

[0186] The first joint motion component is

[0187] For example, see Figure 9 , Figure 9 Schematic diagram of robot control application provided by the embodiment of the present application; Figure 9As shown: When the redundant robot 9-1 performs the tea delivery task, it performs auxiliary tasks such as joint constraints and obstacle avoidance to improve the execution effect of the tea delivery task.

[0188] It can be understood that the subtask merging strategy dynamically adjusts auxiliary tasks to timely execute all meta-tasks. This merging strategy combines task status and soft priorities to improve redundancy allocation efficiency, ensuring that redundant robots effectively execute the main task and multiple subtasks. In addition, a control method using null space mapping dynamically merges multiple subtasks into a secondary task within the null space of the main task. This ensures that, under the constraint of insufficient redundancy, the redundant robots can not only perfectly execute the main task, but also effectively execute multiple subtasks within the redundant space. This eliminates the need for hardware reconstruction of the redundant robots and enables multi-task collaboration among the redundant robots.

[0189] The following continues to describe the exemplary structure of the robot control device 455 provided in the embodiment of the present application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the robot control device 455 of the memory 450 may include:

[0190] The request response module 4551 is used to respond to the task request for the robot to be controlled and obtain the main task to be executed;

[0191] A task acquisition module 4552 is configured to acquire auxiliary tasks of the robot to be controlled when controlling the robot to be controlled to perform the main task to be performed, thereby obtaining L auxiliary tasks to be integrated, wherein the L auxiliary tasks to be integrated are used to assist in performing the main task to be performed, and L is a positive integer greater than 1;

[0192] The task fusion module 4553 is configured to fuse the L auxiliary tasks to be fused based on a redundancy allocation weight to obtain C auxiliary tasks to be executed, wherein the redundancy allocation weight represents the probability of allocating C redundant degrees of freedom to the L auxiliary tasks to be fused, and C is a positive integer;

[0193] a task projection module 4554 for calculating target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, null space projection parameters, and the main task to be executed, wherein the null space projection parameters are used for projection onto the null space of the main task to be executed;

[0194] The motion control module 4555 is used to control the motion of the robot to be controlled based on the target motion information.

[0195] In the embodiment of the present application, the task acquisition module 4552 is further used to acquire the auxiliary tasks of the robot to be controlled to obtain multiple initial auxiliary tasks; and determine the multiple initial auxiliary tasks as L auxiliary tasks to be fused.

[0196] In an embodiment of the present application, the task acquisition module 4552 is also used to perform the following processing on each of the multiple initial auxiliary tasks: decompose the initial auxiliary task to obtain at least one auxiliary meta-task; obtain L auxiliary meta-tasks corresponding to the multiple initial auxiliary tasks from the at least one auxiliary meta-task corresponding to each of the initial auxiliary tasks; and determine the L auxiliary meta-tasks as the L auxiliary tasks to be fused.

[0197] In an embodiment of the present application, the execution process of the main task to be executed corresponds to the execution cycles of multiple auxiliary tasks, and the L fused auxiliary tasks, the redundant allocation weights and the target motion information are the information of the current execution cycle; the robot control device 455 also includes a weight acquisition module 4556, which is used to obtain the weight update speed of the current execution cycle; the fusion result between the weight update speed and the execution cycle duration is determined as the information to be updated; based on the information to be updated, the previous redundant allocation weight of the previous execution cycle is updated to obtain the redundant allocation weight, wherein the previous execution cycle is an adjacent execution cycle before the current execution cycle.

[0198] In an embodiment of the present application, the weight acquisition module 4556 is also used to determine the combination result of the information to be updated and the previous redundant allocation weight of the previous execution cycle as the first initial allocation weight; determine the minimum of the maximum redundant allocation weight and the first initial allocation weight as the second initial allocation weight; and determine the maximum of the minimum redundant allocation weight and the second initial allocation weight as the redundant allocation weight.

[0199] In an embodiment of the present application, the weight acquisition module 4556 is also used to obtain the unfinished degree of each auxiliary task to be merged, and obtain L unfinished degrees corresponding to the L auxiliary tasks to be merged; based on the previous redundant allocation weight, obtain the weight update priority; combine the weight update priority, the L unfinished degrees and the previous redundant allocation weight to determine the weight update speed of the current execution cycle.

[0200] In an embodiment of the present application, the weight acquisition module 4556 is also used to perform the following processing on the jth redundant degree of freedom and the i-th auxiliary task to be fused based on the previous redundant allocation weight, wherein 1≤j≤C, 1≤i≤L, and i and j are positive integer variables: based on the weights of allocating the first j-1 redundant degrees of freedom to the i-th auxiliary task to be fused, determine the redundant priority of allocating the jth redundant degree of freedom to the i-th auxiliary task to be fused; based on the weights of allocating the jth redundant degree of freedom to the first i-1 auxiliary tasks to be fused, determine the task priority of allocating the jth redundant degree of freedom to the i-th auxiliary task to be fused; based on the weights of allocating the redundant degrees of freedom other than the j-th redundant degree of freedom to the i-th auxiliary task to be fused, determine the relative priority of allocating the jth redundant degree of freedom to the i-th auxiliary task to be fused; and combining the redundant priority, the task priority and the relative priority to obtain the weight update priority corresponding to the L auxiliary tasks to be fused and the number C of redundant degrees of freedom.

[0201] In an embodiment of the present application, the weight acquisition module 4556 is also used to obtain the target combination result and target difference result between the sensitivity range and the auxiliary task to be fused, wherein the sensitivity range belongs to the normalization parameter corresponding to the auxiliary task to be fused; obtain the first fusion result of the response slope and the target combination result, wherein the normalization parameter corresponding to the auxiliary task to be fused also includes the response slope; obtain the second fusion result of the response slope and the target difference result; and quantify the incompleteness of the auxiliary task to be fused in combination with the first fusion result and the second fusion result.

[0202] In an embodiment of the present application, the task projection module 4554 is also used to obtain the pseudo-inverse matrix corresponding to the Jacobian matrix of the main task to be executed; obtain the target fusion result between the Jacobian matrix and the pseudo-inverse matrix; and obtain the null space projection parameter that is negatively correlated with the target fusion result.

[0203] In an embodiment of the present application, the task projection module 4554 is also used to obtain the first joint motion component for executing the main task to be executed based on the pseudo-inverse matrix of the main task to be executed; combine the C auxiliary tasks to be executed and the null space projection parameters of the main task to be executed to calculate the second joint motion component for executing the C auxiliary tasks to be executed; and combine the first joint motion component and the second joint motion component into the target motion information of the robot to be controlled.

[0204] The present invention provides a computer program product comprising computer-executable instructions or a computer program stored in a computer-readable storage medium. A processor of a robot control device reads the computer-executable instructions or the computer program from the computer-readable storage medium and executes the computer-executable instructions or the computer program, causing the robot control device to perform the robot control method described above in the present invention.

[0205] The embodiment of the present application provides a computer-readable storage medium in which computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the robot control method provided by the embodiment of the present application, for example, Figure 3 The robot control method is shown.

[0206] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0207] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0208] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0209] As an example, computer-executable instructions may be deployed to be executed on one electronic device (in which case, this one electronic device is a robot control device), or on multiple electronic devices located at one location (in which case, the multiple electronic devices located at one location are robot control devices), or on multiple electronic devices distributed at multiple locations and interconnected by a communication network (in which case, the multiple electronic devices distributed at multiple locations and interconnected by a communication network are robot control devices).

[0210] It is understood that in the embodiments of this application, when the embodiments of this application are applied to specific products or technologies, the relevant data, such as the information of the service object to be served, etc., must be obtained, and the user's permission or consent must be obtained. The collection, use, and processing of the relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, the collection and processing of the relevant data in this application should be strictly in accordance with the requirements of the relevant national laws and regulations when applied in the examples. The informed consent or separate consent of the personal information subject must be obtained, and subsequent data use and processing activities must be carried out within the scope of the authorization of the laws and regulations and the personal information subject.

[0211] In summary, when executing the main task to be executed in response to the task request for the robot, the embodiment of the present application fuses L auxiliary tasks to be fused into C auxiliary tasks to be executed based on the corresponding redundant allocation weights in each execution cycle, and then combines the C auxiliary tasks to be executed, the null space projection parameters of the main task to be executed and the main task to be executed to calculate the target motion information for controlling the robot movement; a robot control method is implemented by fusing L auxiliary tasks to be fused and projecting the fused C auxiliary tasks to be executed to the null space of the main task to be executed, so that the execution of the L auxiliary tasks to be fused is independent of the execution of the main task to be executed; that is, it is possible to use the C redundant degrees of freedom included in the robot to realize the execution of multiple auxiliary tasks while ensuring the execution of the main task to be executed, thereby realizing the collaborative execution of multiple tasks when the redundant degrees of freedom are insufficient, and reducing the resource consumption of the robot. In addition, the embodiment of the present application can improve the allocation efficiency of the redundant degrees of freedom by decomposing the initial auxiliary tasks and allocating the redundant degrees of freedom based on the decomposed auxiliary meta-tasks, thereby improving the task execution effect.

[0212] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A robot control method, characterized in that: The method comprises: In response to a task request for the robot to be controlled, a main task to be executed is obtained; When controlling the robot to be controlled to perform the main task to be performed, acquiring the auxiliary tasks of the robot to be controlled to obtain L auxiliary tasks to be fused, wherein the L auxiliary tasks to be fused are used to assist in performing the main task to be performed, and L is a positive integer greater than 1; Fusing the L auxiliary tasks to be fused based on a redundancy allocation weight to obtain C auxiliary tasks to be executed, wherein the redundancy allocation weight represents a probability of allocating the C redundant degrees of freedom to the L auxiliary tasks to be fused, and C is a positive integer; Calculating target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, the null space projection parameters, and the main task to be executed, wherein the null space projection parameters are used to project onto the null space of the main task to be executed; The movement of the robot to be controlled is controlled based on the target movement information.

2. The method according to claim 1, characterized in that The auxiliary tasks of the robot to be controlled are acquired to obtain L auxiliary tasks to be fused, including: Acquiring auxiliary tasks of the robot to be controlled to obtain a plurality of initial auxiliary tasks; The plurality of initial auxiliary tasks are determined as L auxiliary tasks to be fused.

3. The method according to claim 2, characterized in that After acquiring the auxiliary tasks of the robot to be controlled to obtain a plurality of initial auxiliary tasks, the method further includes: The following process is performed on each of the plurality of initial auxiliary tasks: Decomposing the initial auxiliary task to obtain at least one auxiliary meta-task; Obtaining L auxiliary meta-tasks corresponding to the plurality of initial auxiliary tasks from the at least one auxiliary meta-task corresponding to each of the initial auxiliary tasks; The L auxiliary meta-tasks are determined as the L auxiliary tasks to be fused.

4. The method according to any one of claims 1 to 3, characterized in that The execution process of the to-be-executed main task corresponds to the execution cycle of multiple execution auxiliary tasks, and the L fused auxiliary tasks, the redundant allocation weights and the target motion information are information of the current execution cycle; Before fusing the L auxiliary tasks to be fused based on the redundancy allocation weights to obtain C auxiliary tasks to be executed, the method further includes: Obtain the weight update speed of the current execution cycle; Determine the fusion result between the weight update speed and the execution cycle duration as the information to be updated; The previous redundancy allocation weight of the previous execution cycle is updated based on the information to be updated to obtain the redundancy allocation weight, wherein the previous execution cycle is an execution cycle adjacent to the current execution cycle.

5. The method according to claim 4, characterized in that The updating of the previous redundancy allocation weight of the previous execution cycle based on the information to be updated to obtain the redundancy allocation weight includes: Determine a first initial allocation weight by combining the information to be updated and the previous redundancy allocation weight of the previous execution cycle; Determine the minimum of the maximum redundancy allocation weight and the first initial allocation weight as the second initial allocation weight; The maximum of the minimum redundancy allocation weight and the second initial allocation weight is determined as the redundancy allocation weight.

6. The method according to claim 4, characterized in that The obtaining of the weight update speed of the current execution cycle includes: Obtaining the uncompleted degree of each auxiliary task to be merged, and obtaining L uncompleted degrees corresponding to the L auxiliary tasks to be merged; Based on the previous redundant allocation weight, obtaining a weight update priority; The weight update speed of the current execution cycle is determined in combination with the weight update priority, the L incompleteness degrees and the previous redundancy allocation weight.

7. The method according to claim 6, characterized in that The acquiring the weight update priority based on the previous redundancy allocation weight includes: Based on the previous redundancy allocation weight, the following processing is performed on the j-th redundant degree of freedom and the i-th auxiliary task to be fused, where 1≤j≤C, 1≤i≤L, and i and j are positive integer variables: Determining a redundancy priority for allocating the jth redundant degree of freedom to the ith auxiliary task to be fused based on the weights of allocating the first j-1 redundant degrees of freedom to the i-th auxiliary task to be fused; Determining, based on the weights of allocating the j-th redundant degree of freedom to the first i-1 auxiliary tasks to be merged, the task priority for allocating the j-th redundant degree of freedom to the i-th auxiliary task to be merged; Determining a relative priority for allocating the jth redundant degree of freedom to the ith auxiliary task to be fused based on the weights of the redundant degrees of freedom other than the jth redundant degree of freedom allocated to the ith auxiliary task to be fused; The weight update priority corresponding to the L auxiliary tasks to be merged and the number C of the redundant degrees of freedom is obtained by combining the redundancy priority, the task priority and the relative priority.

8. The method according to claim 6, characterized in that The obtaining of the incompleteness of each auxiliary task to be merged includes: Obtaining a target combination result and a target difference result between a sensitivity range and the auxiliary task to be fused, wherein the sensitivity range belongs to a normalized parameter corresponding to the auxiliary task to be fused; Obtaining a first fusion result of a response slope and the target combination result, wherein the normalized parameter corresponding to the auxiliary task to be fused also includes the response slope; obtaining a second fusion result of the response slope and the target difference result; The incompleteness of the auxiliary task to be fused is quantified by combining the first fusion result and the second fusion result.

9. The method according to any one of claims 1 to 3, characterized in that Before calculating the target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, the null space projection parameters, and the main task to be executed, the method further includes: Obtaining the pseudo-inverse matrix corresponding to the Jacobian matrix of the main task to be executed; Obtaining a target fusion result between the Jacobian matrix and the pseudo-inverse matrix; Acquire the null space projection parameter that is negatively correlated with the target fusion result.

10. The method according to any one of claims 1 to 3, characterized in that The calculating target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, the null space projection parameter, and the main task to be executed includes: Based on the pseudo-inverse matrix of the main task to be performed, obtaining a first joint motion component for performing the main task to be performed; Calculate the second joint motion components for executing the C auxiliary tasks to be performed by combining the null space projection parameters of the C auxiliary tasks to be performed and the main task to be performed; The first joint motion component and the second joint motion component are combined into the target motion information of the robot to be controlled.

11. A robot control device, characterized in that: The robot control device comprises: A request response module, configured to respond to a task request for the robot to be controlled and obtain a main task to be executed; a task acquisition module, configured to acquire auxiliary tasks of the robot to be controlled when controlling the robot to be controlled to perform the main task to be performed, to obtain L auxiliary tasks to be fused, wherein the L auxiliary tasks to be fused are used to assist in performing the main task to be performed, and L is a positive integer greater than 1; a task fusion module, configured to fuse the L auxiliary tasks to be fused based on a redundancy allocation weight to obtain C auxiliary tasks to be executed, wherein the redundancy allocation weight represents the probability of allocating C redundant degrees of freedom to the L auxiliary tasks to be fused, and C is a positive integer; a task projection module, configured to calculate target motion information of the robot to be controlled by combining the C auxiliary tasks to be executed, null space projection parameters, and the main task to be executed, wherein the null space projection parameters are used to project onto the null space of the main task to be executed; A motion control module is used to control the motion of the robot to be controlled based on the target motion information.

12. An electronic device for controlling a robot, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions or computer programs; The processor is configured to implement the robot control method according to any one of claims 1 to 10 when executing the computer executable instructions or computer program stored in the memory.

13. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the robot control method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising computer executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the robot control method according to any one of claims 1 to 10 is implemented.

Images