A robot skills learning and sharing method and system

Abstract

The application provides a robot skill learning and sharing method and system, and relates to the field of artificial intelligence.The method comprises the following steps: uploading first-person perspective data collected when a robot performs an entire initial task to a core control processor, uploading third-person perspective data collected by a spatial intelligent machine when the robot performs the entire initial task to the core control processor, obtaining complete task data of the initial task and storing the complete task data in the core control processor, and using the complete task data of the initial task to perform robot skill learning and sharing, so that repeated operations are reduced and the efficiency of the robot in completing a task is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to a robot skill learning and sharing method and system. BACKGROUND

[0002] The embodied intelligent device is an intelligent system device based on physical body for perception and action. In current robot technology, the embodied intelligent device generally faces the problems of repeated development of environment modeling and difficulty in reuse of robot task skills. Robot manufacturers often independently model the spatial environment, resulting in repeated work, and the experience accumulated by the robot after performing a task cannot be quickly shared with other robots. Specifically, the traditional method mainly relies on first-person perspective data collected by the robot's own sensors (such as IMU and camera), which makes it difficult to obtain global environmental dynamics (such as changes in the state of collaborative objects and displacement of spatial obstacles) outside the robot's own body, resulting in repeated work. SUMMARY

[0003] To solve the above technical problems, the technical scheme adopted by the present application is as follows:

[0004] According to the first aspect of the present application, a robot skill learning and sharing method is provided, which comprises the following steps:

[0005] uploading first-person perspective data collected by the robot when performing the entire initial task to the core control processor, the dimensions of the first-person perspective data including at least: motion trajectory of the robot, motor operating parameters of the robot, and multimedia information collected by the robot when performing the initial task;

[0006] uploading third-person perspective data collected by the spatial intelligent machine when the robot performs the entire initial task to the core control processor, the dimensions of the third-person perspective data including at least: environmental data when the robot performs the initial task, multimedia data when the robot performs the initial task, object attribute data interacting with the robot, and historical behavior data of objects in the space where the robot performs the initial task; the spatial intelligent machine is a device or system with a preset function in the space where the robot performs the initial task, and the spatial intelligent machine does not include the robot; the preset function at least includes: multi-modal perception function, environment adaptive decision function, and space modeling function;

[0007] obtaining complete task data of the initial task and storing the complete task data in the core control processor, using the complete task data of the initial task for robot skill learning and sharing, and the complete task data is the data obtained by aligning the first-person perspective data and the third-person perspective data in time by the core control processor.

[0008] According to a second aspect of the present invention, a robot skill learning and sharing system is provided, which includes: a processor, a memory, and a computer program stored in the memory and runnable on the processor, and the processor implements the aforementioned robot skill learning and sharing method when executing the computer program.

[0009] The present invention has at least the following beneficial effects: In summary, the first-person perspective data collected when the robot performs the entire initial task is uploaded to the core control processor, the third-person perspective data collected by the spatial intelligent machine when the robot performs the entire initial task is uploaded to the core control processor, the complete task data of the initial task is obtained and stored in the core control processor, and the complete task data of the initial task is used to learn and share robot skills. The present invention collects the first-person perspective data and third-person perspective data of the robot performing the initial task to achieve complete data recording of the robot performing the initial task, so that other tasks can learn and share skills based on the complete data record, reduce repetitive operations, and improve the efficiency of the robot in completing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A flowchart of a robot skill learning and sharing method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0013] It is to be understood that the terms "first", "second", and the like, used in the description and the claims of the application, as well as above-described accompanying drawings, are used to distinguish similar objects and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments of the application described herein can be carried out in other than the order shown or described herein. Furthermore, the terms "comprise" and "have", and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or server that includes a list of steps or units is not necessarily limited to those steps or units that are clearly listed, but can include other steps or units that are not clearly listed or inherent to such processes, methods, products, or devices.

[0014] The embodiment of the application provides a robot skill learning and sharing method, which comprises the following steps of Figure 1 As shown in the figure, the method comprises the following steps:

[0015] S100, first-person perspective data collected when a robot performs an initial task is uploaded to a core control processor, and the dimensions of the first-person perspective data at least include: a motion track of the robot, motor operation parameters of the robot, and multimedia information collected when the robot performs the initial task.

[0016] Specifically, the multimedia information collected when the robot performs the initial task includes video information, audio information and picture information of the robot performing the initial task.

[0017] S200, third-person perspective data collected by a space intelligent machine when the robot performs the initial task is uploaded to the core control processor, and the dimensions of the third-person perspective data at least include: environment data when the robot performs the initial task, multimedia data when the robot performs the initial task, object attribute data interacting with the robot, and historical behavior data of objects in a space where the robot performs the initial task; the space intelligent machine is a device or system with a preset function in the space where the robot performs the initial task, and the space intelligent machine does not include the robot; the preset function at least includes: a multi-modal perception function, an environment adaptive decision function, and a space modeling function.

[0018] Specifically, the third-person perspective data is data collected by the space intelligent machine when the robot performs the initial task, and the first-person perspective data is data collected by the robot itself when the robot performs the initial task.

[0019] Specifically, the attribute data of objects interacting with the robot include: attribute information of people and objects interacting with the robot, such as the identity information of the people interacting with the robot, the brand and model of the objects interacting with the robot, etc.; the historical behavior data of objects in the space where the robot performs the initial task includes: historical behavior data of people, objects, and places in the space where the robot performs the initial task, such as the business responsibility data of people in the space where the robot performs the initial task, equipment usage records, and business characteristic data of the place.

[0020] S300, obtain the complete task data of the initial task and store the complete task data in the core control processor, use the complete task data of the initial task for robot skill learning and sharing, the complete task data is the data after the core control processor aligns the first-person perspective data and the third-person perspective data in time.

[0021] Specifically, after receiving the first-person perspective data and the third-person perspective data, the core control processor aligns the first-person perspective data and the third-person perspective data in time to obtain complete task data. Furthermore, when the acquisition frequencies of the first-person perspective data and the third-person perspective data are not equal, the third-person perspective data is converted to relatively align the third-person perspective data and the first-person perspective data in the time dimension.

[0022] In summary, the first-person perspective data collected when the robot performs the entire initial task is uploaded to the core control processor, and the third-person perspective data collected by the spatial intelligent machine when the robot performs the entire initial task is uploaded to the core control processor, the complete task data of the initial task is obtained and stored in the core control processor, and the complete task data of the initial task is used to learn and share robot skills. The present invention collects the first-person perspective data and third-person perspective data of the robot performing the initial task to achieve complete data recording of the robot performing the initial task, so that other tasks can learn and share skills based on the complete data record, reduce repetitive operations, and improve the efficiency of the robot in completing tasks.

[0023] Specifically, a spatial environment model for the robot when performing the initial task is constructed based on third-person perspective data. Specifically, a spatial environment model for the robot when performing the initial task is constructed based on environmental data when the robot performs the initial task, multimedia data when the robot performs the initial task, attribute data of objects interacting with the robot, and historical behavior data of objects in the space where the robot performs the initial task.

[0024] Specifically, when the robot performs the initial task, it also includes:

[0025] S310, collect the dimension value corresponding to the dimension of the first-person perspective data at the current time node and the dimension value corresponding to the dimension of the third-person perspective data at the current time node.

[0026] S320, input the initial task, the dimension value corresponding to the dimension of the first-person perspective data at the current time node, and the dimension value corresponding to the dimension of the third-person perspective data at the current time node into the target processor, and obtain the execution data of the initial task at the preset time node list, the preset time node list being one or more time nodes after the current time node.

[0027] In an embodiment of the present application, the target processor is a core control processor.

[0028] Specifically, in the process of the robot executing the initial task, the space where the robot executes the initial task may change, for example, a moving obstacle appears, at this time, the environment changes, and the first-person perspective data and the third-person perspective data also change, therefore, the execution data of the initial task is reinitialized based on the changed environment.

[0029] S330, the robot executes based on the execution data.

[0030] In summary, when the robot executes the initial task, the dimension value corresponding to the dimension of the first-person perspective data at the current time node and the dimension value corresponding to the dimension of the third-person perspective data at the current time node are collected, the initial task, the dimension value corresponding to the dimension of the first-person perspective data at the current time node, and the dimension value corresponding to the dimension of the third-person perspective data at the current time node are input into the target processor, the robot executes based on the execution data, the execution data at the subsequent time node is predicted through the data at the current time node, and the change of the environment is timely responded.

[0031] Further, after S300, it further includes: when a new robot executes a target task, the new robot is a robot that has not executed the target task, and the following steps are executed:

[0032] Obtain an intermediate task, the intermediate task being an initial task with a similarity degree to the target task satisfying a preset similarity degree condition.

[0033] Specifically, before the new robot executes the target task, it further includes: obtaining skill data of the new robot, the skill data at least including: robot hardware attribute data; further, the skill data further includes: robot vision data. Specifically, the robot hardware attribute data includes: length, width, and height of the robot, and number of mechanical arms; the robot vision data includes: color of the robot, etc.

[0034] The core control processor obtains subtask node data of the intermediate task based on the complete task data corresponding to the intermediate task. Specifically, the subtask node data of the intermediate task is data of the subtask of the robot executing the intermediate task.

[0035] Based on the subtask node data of the intermediate task, the subtask node data of the new robot to perform the target task is determined to realize robot skill learning and sharing.

[0036] In summary, the intermediate task is obtained, and the core control processor obtains the subtask node data of the intermediate task based on the complete task data corresponding to the intermediate task, and determines the subtask node data of the new robot to perform the target task based on the subtask node data of the intermediate task, so as to realize robot skill learning and sharing, and guide the target task by using an initial task similar to the target task to realize skill learning and sharing between robots.

[0037] Furthermore, the similarity between the target task and the initial task is determined by the following steps:

[0038] The skill data used by the robot when executing the initial task is obtained as the initial skill data.

[0039] If the new robot's skill data fully includes the initial skill data, the task corresponding to the initial skill data is considered an intermediate task, and the initial skill data is used as the intermediate skill data. Otherwise, the similarity between the target task and the initial task corresponding to the initial skill data is recorded as zero. It should be understood that if the new robot's skill data does not fully include the initial skill data, the new robot will not be able to learn the skills for the initial task. Alternatively, even if the new robot learns the skills for the initial task, it may not have the hardware support to execute the initial task.

[0040] The degree of similarity between the skill data of the new robot and the intermediate skill data is obtained as a first degree of similarity. It can be understood that if the skill data of the new robot and the intermediate skill data are exactly the same, then the first degree of similarity is the highest. If the skill data of the new robot includes data other than the intermediate skill data, the degree of similarity between the skill data of the new robot and the intermediate skill data is obtained for each attribute of the robot hardware attribute, thereby obtaining the degree of similarity between the skill data of the new robot and the intermediate skill data as the first degree of similarity.

[0041] The similarity between the target task and the intermediate task is obtained as the second similarity.

[0042] Specifically, obtaining the similarity between the target task and the intermediate task also includes:

[0043] The target task is extracted as a target feature vector containing several preset features, and the intermediate task is extracted as an intermediate feature vector containing several preset features. Specifically, the preset features include at least the task type. In one embodiment of the present invention, the preset features also include a task description statement. The task description statement includes at least core information such as the target, object, conditions, standards, and time nodes.

[0044] The similarity between the target feature vector and the intermediate feature vector is obtained as the similarity between the target task and the intermediate task.

[0045] Based on the first similarity and the second similarity, the similarity between the target task and the initial task corresponding to the intermediate task is determined. Specifically, the first similarity and the second similarity are weighted and summed to serve as the similarity between the target task and the initial task corresponding to the intermediate task. Optionally, the weight of the second similarity is not less than the weight of the first similarity. It is understood that when the skill data can meet the requirements, the similarity between tasks is given priority, and therefore, the weight of the second similarity is set to be not less than the weight of the first similarity.

[0046] In summary, the skill data used by the robot when performing the initial task is obtained as the initial skill data. If the skill data of the new robot all contain the initial skill data, the task corresponding to the initial skill data is used as the intermediate task, and the initial skill data is used as the intermediate skill data; otherwise, the similarity between the target task and the initial task corresponding to the initial skill data is recorded as zero, and the similarity between the skill data and the intermediate skill data of the new robot is obtained as the first similarity. The similarity between the target task and the intermediate task is obtained as the second similarity. Based on the first similarity and the second similarity, the similarity between the target task and the initial task corresponding to the intermediate task is determined. Through the similarity between the skill data and the task, the similarity is obtained more accurately.

[0047] Specifically, determining the subtask node data of the target task to be performed by the new robot based on the subtask node data of the intermediate task also includes:

[0048] Gets the dimension value of the third-person perspective data when the new robot performs the target task.

[0049] The subtask node data for the new robot to perform the target task is determined based on the dimension value of the third-person perspective data when the new robot performs the target task and the subtask node data of the intermediate task.

[0050] The embodiment of the present application also provides a robot skill learning and sharing system, which comprises a processor, a memory and a computer program stored on the memory and capable of running on the processor, and the processor implements the method provided by the above embodiment when executing the computer program.

[0051] Although some specific embodiments of the present application have been described in detail by way of example with reference to the drawings, it is to be understood that the examples are for illustration only and are not intended to limit the scope of the present application. Those skilled in the art should understand that various modifications can be made to the embodiments without departing from the scope and spirit of the present application.

Claims

1. A robot skill learning and sharing method, characterized in that: The method comprises the following steps: Uploading first-person perspective data collected when the robot performs the entire initial task to the core control processor, the dimensions of the first-person perspective data at least including: the robot's motion trajectory, the robot's motor operating parameters, and multimedia information collected when the robot performs the initial task; Uploading third-person perspective data collected by the spatial intelligent machine during the robot's entire initial task to the core control processor, the dimensions of the third-person perspective data including at least: environmental data during the robot's initial task, multimedia data during the robot's initial task, attribute data of objects interacting with the robot, and historical behavior data of objects within the space where the robot performed the initial task; the spatial intelligent machine is a device or system with preset functions within the space where the robot performs the initial task, and the spatial intelligent machine does not include the robot; the preset functions include at least: multimodal perception function, environmental adaptive decision-making function, and spatial modeling function; Acquire the complete task data of the initial task and store it in the core control processor. Use the complete task data of the initial task for robot skill learning and sharing. The complete task data is the data obtained by the core control processor by aligning the first-person perspective data and the third-person perspective data in time. Using the complete task data of the initial task to perform robot skill learning and sharing also includes: when a new robot performs the target task, the new robot is a robot that has not performed the target task, performing the following steps: Acquire an intermediate task, where the intermediate task is an initial task whose similarity to the target task satisfies a preset similarity condition; The core control processor obtains the subtask node data of the intermediate task based on the complete task data corresponding to the intermediate task; Determine the subtask node data of the new robot to perform the target task based on the subtask node data of the intermediate task, so as to realize robot skill learning and sharing; The similarity between the target task and the initial task is determined by the following steps: Acquire the skill data used by the robot when performing the initial task as initial skill data; If all the skill data of the new robot include the initial skill data, the task corresponding to the initial skill data is regarded as the intermediate task, and the initial skill data is regarded as the intermediate skill data; otherwise, the similarity between the target task and the initial task corresponding to the initial skill data is recorded as zero; Obtaining a degree of similarity between the skill data of the new robot and the intermediate skill data as a first degree of similarity; Obtain the similarity between the target task and the intermediate task as the second similarity; Based on the first similarity level and the second similarity level, a similarity level between the target task and the initial task corresponding to the intermediate task is determined.

2. The robot skill learning and sharing method according to claim 1, characterized in that: A spatial environment model is constructed based on the third-person perspective data when the robot performs the initial task.

3. The robot skill learning and sharing method according to claim 2, characterized in that: When the robot performs the initial task, it also includes: Collect the dimension value corresponding to the dimension of the first-person perspective data at the current time node and the dimension value corresponding to the dimension of the third-person perspective data at the current time node; Inputting the initial task, the dimension value corresponding to the dimension of the first-person perspective data at the current time node, and the dimension value corresponding to the dimension of the third-person perspective data at the current time node into the target processor, and obtaining the execution data of the initial task at a preset time node list, wherein the preset time node list is one or more time nodes after the current time node; The robot executes based on the execution data.

4. The robot skill learning and sharing method according to claim 1, characterized in that: Before the new robot performs the target task, the method further includes: obtaining skill data of the new robot, wherein the skill data at least includes: robot hardware attribute data.

5. The robot skill learning and sharing method according to claim 1, characterized in that: Determining the subtask node data of the target task to be executed by the new robot based on the subtask node data of the intermediate task also includes: Get the dimension value of the third-person perspective data when the new robot performs the target task; The subtask node data for the new robot to perform the target task is determined based on the dimension value of the third-person perspective data when the new robot performs the target task and the subtask node data of the intermediate task.

6. The robot skill learning and sharing method according to claim 1, characterized in that: Obtaining the similarity between the target task and the intermediate task also includes: Extracting the target task into a target feature vector containing several preset features, and extracting the intermediate task into an intermediate feature vector containing several preset features; The similarity between the target feature vector and the intermediate feature vector is obtained as the similarity between the target task and the intermediate task.

7. The robot skill learning and sharing method according to claim 6, characterized in that: The preset features include at least: task type.

8. A robot skill learning and sharing system, comprising: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot skill learning and sharing method according to any one of claims 1 to 7 is implemented.

Images