Multimodal control algorithm fusion method for autonomous robot collaboration

Abstract

The present invention discloses a multi-modal control algorithm fusion method suitable for autonomous collaboration of robots. An algorithm layer is set to form a huge algorithm library to provide algorithm support for various aspects of robot work; a task layer is set to translate the work tasks given by humans into a task set executable by the robot through big data and AI classification methods, and the target task is imported into a task parsing method that matches a similar task, and then connected, and through the built-in reasoning, parsing and other algorithms, it is split into N specific robot-executable task points or steps separately; a matching layer is set to deploy a special matching algorithm, a matching platform, and a matching interface to connect and complete the matching of tasks and algorithms; a decision layer is set to provide decisions for matching results, and the control learning algorithms responsible for control common sense, decision-making, and reasoning are integrated with model algorithms to achieve efficient and high-precision work, thereby meeting the high-precision and flexible work of the robot.

Description

Technical Field

[0001] The present invention relates to the field of algorithms, and in particular to a multi-modal control algorithm fusion method suitable for autonomous collaboration of robots. Background Art

[0002] The role of learning algorithms in robots is mainly in decision-making, task decomposition, and common sense understanding. They can self-adjust in different environments to complete different work tasks, that is, improve the ability to work flexibly. However, learning algorithms do not have and are not suitable for robot planning and control tasks that require real-time and accuracy. Therefore, although the advantages of learning algorithms are good wide-area performance and their working status can be determined through training, it is difficult to achieve high precision and the training cost is high. Although pattern recognition algorithms have high algorithm accuracy, they need to be recognized in a fixed mode, so flexible recognition is difficult. Summary of the invention

[0003] In order to overcome the deficiencies of the prior art solutions, the present invention provides a multi-modal control algorithm fusion method suitable for autonomous collaboration of robots, which can effectively solve the problems raised by the background technology.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] A multi-modal control algorithm fusion method suitable for robot autonomous collaboration, the method comprising the following steps:

[0006] Set up the algorithm layer, adopt an open architecture, and access the algorithm technology of other organizations or institutions through SDK, API, authorization, purchase, and cooperation; or access the algorithm modules developed independently to form a huge algorithm library to provide algorithm support for all aspects of robot work;

[0007] Set up the task layer, which is divided into task database, task analysis system, and task key points. Use big data and AI classification methods to translate the work tasks given by humans into a set of tasks that can be executed by robots. Import the target tasks into matching similar task analysis methods, and then connect them. Use the built-in reasoning, analysis and other algorithms to split them into N specific task points or steps that can be executed by robots separately.

[0008] Set up the matching layer, deploy specialized matching algorithms, matching platforms, and matching interfaces to connect and complete the matching of tasks and algorithms;

[0009] Set the decision layer to provide decisions for matching results; the details are as follows:

[0010] Step (1) developing a dedicated algorithm suitable for the fusion of multi-modal control algorithms of robots;

[0011] Step (2) When applying the algorithm, a special task layer is set up to parse and extract the key points of the task. At the decision layer, various factors are used to determine the final task adaptation algorithm.

[0012] Step (3) first solves the problem of flexible work through a learning algorithm, then extracts the task points that do not meet the requirements of high-precision calculation, adapts the corresponding algorithm in the algorithm matching layer, and obtains a multi-modal fusion algorithm of the learning algorithm and the adaptive matching algorithm;

[0013] Step (4) splits the decision of how the robot should perform the task into two parts. The learning algorithm is used for big data training to output usable high-level motion instructions. The dedicated part represents what the robot can do in the current environment. At the same time, the robot's commonly used multimodal data is solidified into executable steps. The two are then combined through the value function to jointly decide which instruction to select for actual execution.

[0014] In one of the preferred solutions, the setting of the task layer includes the following steps:

[0015] Step S1: First, a task database is established, in which different entries are established according to the type of work that the robot needs to complete;

[0016] Step S2: Import the target work into the task database according to the classification;

[0017] Step S3: Start the task analysis system, which has a built-in learning algorithm specifically for task analysis and is responsible for analyzing or translating the tasks given by humans into key points or steps to be performed by N robots. The specific method is as follows:

[0018] Step S31: The deployed task parsing algorithm first performs the training task by taking the human work task and the parsed execution points or steps as samples, and then goes through multiple training, iteration and improvement algorithm processes:

[0019] Step S32: Execution, parsing or translating the task into key points or steps to be specifically executed by N robots;

[0020] Step S33: If the analysis result does not meet the requirements, return to step 32, otherwise execute 35;

[0021] Step S34: If the requirements are not met after executing step 33 M times, manual intervention is requested to generate a result that meets the requirements, and then step 35 is executed, and the result is returned to step 31 to supplement or iterate the calculation loopholes;

[0022] Step S35: Output the parsing result and end the parsing;

[0023] Step S4: Check the N key points or steps of the obtained task execution. If the robot cannot execute the task alone, return to step S3 for further analysis or splitting; otherwise, go to step S5;

[0024] Step S5: Complete the parsing task and output N task points or steps as the final result. The N task points or steps execution algorithm completes the matching through the matching layer.

[0025] In one of the preferred solutions, the setting of the matching layer comprises the following steps:

[0026] Step S11: N matchings need to be performed to provide algorithm support for the execution of the task;

[0027] Step S12: first send a single task point / step to the matching platform;

[0028] Step S13: Start the matching interface to connect the algorithm layer and the independently executable task points or steps;

[0029] Step S14: Recommend appropriate algorithms for independently executable task points or steps through the built-in big data system of the matching platform:

[0030] Step S15: Start the matching algorithm and calculate the matching result. If the requirements are met, execute step S16; otherwise, execute step S13;

[0031] Step S16: After completing the matching task, a decision is provided to the matching result through the decision layer.

[0032] In one of the preferred solutions, the setting of the decision layer includes the following steps:

[0033] When the decision layer provides a decision on the matching result: if the matching result of the matching layer meets the task requirements, the decision is "agree" and the result is output at the same time; if the matching result of the matching layer does not meet the task requirements, the decision is "disagree" and returns to the matching layer for re-matching.

[0034] In one of the preferred embodiments, the step (1) further comprises the following steps:

[0035] The learning algorithms include support vector machines, Bayesian linear analysis algorithms, deep learning, and transfer learning algorithms.

[0036] In one of the preferred embodiments, the step (2) further comprises the following steps:

[0037] According to the relevance of the key points of the task, a special matching big data analysis algorithm is designed to find the best algorithm and suboptimal algorithm that meet the requirements.

[0038] In one of the preferred embodiments, the step (2) further comprises the following steps:

[0039] If necessary, algorithm engineers are required to manually add multiple algorithms, perform multi-algorithm weight analysis, and discuss whether there is a better match when multiple algorithms are integrated.

[0040] In one of the preferred embodiments, the step (2) further comprises the following steps:

[0041] The factors include compatibility, cost and efficiency.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The multimodal control algorithm fusion method suitable for robot autonomous collaboration provided by the present invention integrates the control learning algorithm responsible for control common sense, decision-making, and reasoning with the model algorithm to achieve efficient and high-precision work, meeting the high-precision and flexible work requirements of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The figure is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] This embodiment provides a multimodal control algorithm fusion method suitable for robot autonomous collaboration, which combines pattern recognition and learning algorithms. The problem of flexible work is first solved by using learning algorithms, and then the task points that do not meet the requirements of high-precision calculations are extracted. The corresponding algorithms are adapted in the algorithm matching layer to obtain a multimodal fusion algorithm of the learning algorithm and the adaptive matching algorithm.

[0047] In this embodiment, first of all, in terms of algorithms, we focus on developing algorithms such as support vector machines and Bayesian linear analysis (BLDA) that are suitable for flexible work of robots; as well as algorithms such as deep learning and transfer learning, and place these algorithms in a dedicated algorithm layer. When the algorithm is applied, a special task layer is set to parse and extract the key points of the task. According to the relevance of the key points of the task, a special matching big data analysis algorithm is designed to find the best algorithm that meets the requirements, the suboptimal algorithm, and so on; if necessary, algorithm engineers are required to manually add multiple algorithms, perform multi-algorithm weight analysis, and discuss whether there is a better match when multiple algorithms are integrated. At the decision-making level, the final task adaptation algorithm is determined based on factors such as matching, cost, and efficiency.

[0048] The algorithm design logic of this robot in this embodiment is as follows: the decision of how the robot should perform the task is split into two parts, and the learning algorithm is used for big data training to output available high-level motion instructions; the dedicated represents what the robot can do in the current environment, and at the same time, the robot's commonly used multimodal data (mainly from the robot's sensors, such as images, robot status, scene environment information, etc.) is solidified into executable steps, such as outputting the robot's machine position instructions, motion instructions, and grasping and releasing instructions. Then the two are combined by means of a value function to jointly decide which instruction to select for actual execution. In this way, the robot can better generalize its original capabilities to new scenarios.

[0049] like Figure 1 As shown in the figure, the robot system is given the input "How do you screw a screw into a screw hole?" For this task, the learning class intervenes in advance, breaking down the task into the most appropriate working sequence (steps) for the robot, and combining the machine vision data to establish the task objectives of the stage steps:

[0050] 1. Pick up the screw (task target: the X, Y, X spatial coordinate position of the screw);

[0051] 2. Move the screw (task target: screw center point, X, Y, X spatial coordinate position of object outline);

[0052] 3. Find the screw hole (task goal: the X, Y, X spatial coordinate position of the screw);

[0053] 4. Tighten the screws (task objective: the tightening data of the screws, such as 26 turns means it is tightened).

[0054] Then, the specific steps will expect the intervention of special algorithms. For example, in the second step of moving the screw, the path planning special algorithm is started first to convert the robot's observation space (usually three-dimensional space and the object to be operated) into a 3D value map, and then a mature path search algorithm (such as the Probabilistic RoadMap) can be used to search and generate an available robot motion path on the 3D value map. With an available path, the trajectory planning and control of the robot movement will be done next.

[0055] In the above embodiments, by setting the algorithm layer, an open architecture is adopted, and the algorithm technology of other organizations or institutions is accessed through SDK, API, authorization, purchase, cooperation and other methods; or the algorithm module independently developed is accessed to form a huge algorithm library to provide algorithm support for all aspects of the robot's work; the task layer is set, which is divided into a task database, a task analysis system, and task points. The work tasks given by humans are translated into a set of tasks executable by robots through big data and AI classification methods. By importing the target task into a task analysis method that matches similar tasks, and then connecting them, it is split into N specific robot-executable task points or steps through the built-in reasoning, analysis and other algorithms; the matching layer is set, and a special matching algorithm, matching platform, and matching interface are deployed to connect and complete the matching of tasks and algorithms; the decision layer is set to provide decisions for the matching results; the details are as follows:

[0056] Step (1) developing a special algorithm suitable for the fusion of robot multi-modal control algorithms; the learning algorithms include support vector machines, Bayesian linear analysis algorithms, deep learning, and transfer learning algorithms.

[0057] Step (2) When applying the algorithm, set up a special task layer to parse and extract the key points of the task. At the decision layer, make decisions based on various factors to determine the final task adaptation algorithm. According to the relevance of the key points of the task, design a special matching big data analysis algorithm to find the best algorithm and suboptimal algorithm that meet the requirements. If necessary, algorithm engineers are required to manually add multiple algorithms, perform multi-algorithm weight analysis, and discuss whether there is a better match when multiple algorithms are integrated. The factors include matching, cost, and efficiency.

[0058] Step (3) first solves the problem of flexible work through a learning algorithm, then extracts the task points that do not meet the requirements of high-precision calculation, adapts the corresponding algorithm in the algorithm matching layer, and obtains a multi-modal fusion algorithm of the learning algorithm and the adaptive matching algorithm;

[0059] Step (4) splits the decision of how the robot should perform the task into two parts. The learning algorithm is used for big data training to output usable high-level motion instructions. The dedicated part represents what the robot can do in the current environment. At the same time, the robot's commonly used multimodal data is solidified into executable steps. The two are then combined through the value function to jointly decide which instruction to select for actual execution.

[0060] The setting of the task layer includes the following steps:

[0061] Step S1: First, a task database is established, in which different entries are established according to the type of work that the robot needs to complete;

[0062] Step S2: Import the target work into the task database according to the classification;

[0063] Step S3: Start the task analysis system, which has a built-in learning algorithm specifically for task analysis and is responsible for analyzing or translating the tasks given by humans into key points or steps to be performed by N robots. The specific method is as follows:

[0064] Step S31: The deployed task parsing algorithm first performs the training task by taking the human work task and the parsed execution points or steps as samples, and then goes through multiple training, iteration and improvement algorithm processes:

[0065] Step S32: Execution, parsing or translating the task into key points or steps to be specifically executed by N robots;

[0066] Step S33: If the analysis result does not meet the requirements, return to step 32, otherwise execute 35;

[0067] Step S34: If the requirements are not met after executing step 33 M times, manual intervention is requested to generate a result that meets the requirements, and then step 35 is executed, and the result is returned to step 31 to supplement or iterate the calculation loopholes;

[0068] Step S35: Output the parsing result and end the parsing;

[0069] Step S4: Check the N key points or steps of the obtained task execution. If the robot cannot execute the task alone, return to step S3 for further analysis or splitting; otherwise, go to step S5;

[0070] Step S5: Complete the parsing task and output N task points or steps as the final result. The N task points or steps execution algorithm completes the matching through the matching layer.

[0071] The setting of the matching layer comprises the following steps:

[0072] Step S11: N matchings need to be performed to provide algorithm support for the execution of the task;

[0073] Step S12: first send a single task point / step to the matching platform;

[0074] Step S13: Start the matching interface to connect the algorithm layer and the independently executable task points or steps;

[0075] Step S14: Recommend appropriate algorithms for independently executable task points or steps through the built-in big data system of the matching platform:

[0076] Step S15: Start the matching algorithm and calculate the matching result. If the requirements are met, execute step S16; otherwise, execute step S13;

[0077] Step S16: After completing the matching task, a decision is provided to the matching result through the decision layer.

[0078] The setting of the decision layer includes the following steps:

[0079] When the decision layer provides a decision on the matching result: if the matching result of the matching layer meets the task requirements, the decision is "agree" and the result is output at the same time; if the matching result of the matching layer does not meet the task requirements, the decision is "disagree" and returns to the matching layer for re-matching.

[0080] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A multi-modal control algorithm fusion method suitable for autonomous collaboration of robots, characterized by: The method comprises the following steps: Set up the algorithm layer, adopt an open architecture, and access the algorithm technology of other organizations or institutions through SDK, API, authorization, purchase, and cooperation; or access the algorithm modules developed independently to form an algorithm library to provide algorithms for robot work; Set up the task layer, which is divided into task database, task analysis system, and task key points. Use big data and AI classification methods to translate the work tasks given by humans into a set of tasks that can be executed by robots. Import the target tasks into matching similar task analysis methods, and then connect them. Through the built-in reasoning and analysis algorithms, split them into N specific task points or steps that can be executed by robots separately. Set up the matching layer, deploy the matching algorithm, matching platform, and matching interface to connect and complete the matching of tasks and algorithms; Setting a decision layer to provide a decision for the matching result; the setting of the decision layer includes the following steps: when the decision layer provides a decision for the matching result: if the matching result of the matching layer meets the task requirements, the decision is "agree" and the result is output at the same time; if the matching result of the matching layer does not meet the task requirements, the decision is "disagree" and returns to the matching layer for re-matching; The specific steps of the above method are as follows: Independently develop learning algorithms suitable for the fusion of robot multi-modal control algorithms; the learning algorithms include support vector machines, Bayesian linear analysis algorithms, deep learning, and transfer learning algorithms; When applying the algorithm, set up the task layer to parse and extract the key points of the task. At the decision layer, make decisions based on various factors to determine the final task adaptation algorithm. Design a big data analysis algorithm based on the relevance of the key points of the task to find an algorithm that meets the requirements. Algorithm engineers manually add multiple algorithms, perform multi-algorithm weight analysis, and discuss whether there is a better match when multiple algorithms are integrated. The factors include match, cost, and efficiency. First, the problem of flexible work is solved through learning algorithms, and then the task points that do not meet the requirements of high-precision calculation are extracted. The corresponding algorithms are adapted in the algorithm matching layer to obtain a multi-modal fusion algorithm of learning algorithms and adaptive matching algorithms. The decision of how the robot should perform the task is divided into two parts: the learning algorithm is used for big data training to output usable high-level motion instructions; the dedicated representation of what the robot can do in the current environment, while solidifying the robot's common multimodal data into executable steps, and then combining the two through the value function to jointly decide which instruction to select for actual execution; The setting of the task layer includes the following steps: Step S1: First, a task database is established, in which different entries are established according to the type of work that the robot needs to complete; Step S2: Import the target work into the task database according to the classification; Step S3: Start the task analysis system, which has a built-in learning algorithm for task analysis and is responsible for analyzing the work tasks given by humans into task points or steps to be performed by N robots, specifically: Step S31: The deployed task parsing algorithm first performs a training task, and the method is to first use the work task given by humans and the parsed execution task points or steps as samples, and iterate and improve the algorithm process through multiple trainings; Step S32: Execution, parsing the task into task points or steps specifically executed by N robots; Step S33: If the analysis result does not meet the requirements, return to step S32, otherwise execute S35; Step S34: If the requirements are not met after executing step S33 M times, manual intervention is required to generate a result that meets the requirements, and then step S35 is executed, and the result is returned to step S31 to supplement or iterate the calculation loopholes; Step S35: output the analysis result and end the analysis; Step S4: Check the N key points or steps of the obtained task execution. If the robot cannot execute the task alone, return to step S3 for further analysis or splitting; otherwise, go to step S5; Step S5: Complete the parsing task and output N task points or steps as the final result. The N task points or steps execute the algorithm to complete the matching by setting the matching layer. The setting of the matching layer includes the following steps: Step S11: perform N matches to provide an algorithm for task execution; Step S12: first send a single task point or step to the matching platform; Step S13: Start the matching interface to connect the algorithm layer and the independently executable task points or steps; Step S14: recommending appropriate algorithms for independently executable task points or steps through the big data system built into the matching platform; Step S15: Start the matching algorithm and calculate the matching result. If the requirements are met, execute step S16; otherwise, execute step S13; Step S16: After completing the matching task, a decision is provided for the matching result by setting a decision layer.

Images