A robot control method and device combining a scenario and a policy model

Abstract

This application discloses a robot control method and device that combines scenario and strategy models, relating to the field of robot control, including: obtaining first scenario data of a robot to be controlled; wherein the first scenario data includes three-dimensional environment point cloud data and robot posture data; obtaining a first external instruction, and parsing the first external instruction in combination with the three-dimensional environment point cloud data to obtain target task features; inputting the target task features into a pre-trained strategy model to generate an initial action sequence, and adjusting the initial action sequence according to the robot posture data to obtain a target action sequence; wherein the pre-trained strategy model is trained based on the historical task data of the robot to be controlled; and controlling the robot to be controlled according to the target action sequence. This application can improve the flexibility and accuracy of robot control.

Description

Technical Field

[0001] The present application relates to the field of robot control, and in particular to a robot control method and device combining scenario and strategy models. Background Art

[0002] In existing robot control methods, the preliminary task analysis work is generally done by relevant staff who manually analyze and decompose it to obtain the corresponding specific parameters, and preset the task process and control strategy based on these specific parameters to achieve control of the robot. However, when the robot's task environment, task objectives or task content changes, this traditional method based on task pre-analysis and pre-setting must be manually analyzed and the corresponding control strategy must be set again. It lacks flexibility and cannot adapt to complex task scenarios, and will cause unnecessary consumption of manpower and material resources. At the same time, the robot lacks perception of external instructions during the execution of the task, and cannot adjust the task actions according to external instructions, and lacks corresponding flexibility. Therefore, how to improve the flexibility and accuracy of robot control is still one of the problems that need to be solved urgently in the existing technology. Summary of the Invention

[0003] The present application provides a robot control method and device that combines scenario and strategy models to solve the technical problem that existing robot control methods lack flexibility and accuracy.

[0004] According to a first aspect of the embodiments of the present application, a robot control method combining a scenario and a strategy model is provided, comprising:

[0005] Acquire first scene data of the robot to be controlled; wherein the first scene data includes three-dimensional environment point cloud data and robot posture data;

[0006] Obtaining a first external instruction, and parsing the first external instruction in combination with the three-dimensional environment point cloud data to obtain target task features; wherein the first external instruction is obtained by receiving natural language input from a user;

[0007] Inputting the target task features into a pre-trained strategy model to generate an initial action sequence, and adjusting the initial action sequence according to the robot posture data to obtain a target action sequence; wherein the pre-trained strategy model is trained based on historical task data of the robot to be controlled;

[0008] The robot to be controlled is controlled according to the target action sequence.

[0009] The present application first obtains the first scene data of the robot to be controlled, and combines the first external instruction obtained by parsing the three-dimensional environmental point cloud data in the first scene data to obtain the target task characteristics, and then inputs the target task characteristics into the pre-trained strategy model to generate an initial action sequence, and then adjusts the initial action sequence in combination with the robot posture data in the first scene data to obtain the target action sequence, and finally controls according to the target action sequence. Compared with the existing technology, the present application obtains the first scene data and the first external instruction in the task execution process based on the perception of the external environment, and improves the flexibility of subsequent control of the robot through the comprehensiveness of the data type; at the same time, unlike the existing technology that manually sets the task process and control strategy, the present application generates the initial action sequence based on the pre-trained strategy model, and adjusts it in combination with the robot posture data to obtain the target action sequence, thereby improving the adaptability to different task scenarios, thereby improving the flexibility of subsequent control of the robot, and improving the degree of matching with different task scenarios, thereby improving the accuracy of subsequent control of the robot.

[0010] In certain embodiments of the present application, obtaining first scene data of the robot to be controlled specifically includes:

[0011] Based on radar technology, the initial point cloud data of the robot to be controlled is obtained, and based on the binocular camera, the environmental image data of the robot to be controlled is obtained;

[0012] According to the environmental image data, combined with the posture setting parameters of the robot to be controlled, an initial rotation matrix is obtained;

[0013] According to the initial rotation matrix and based on the feature point extraction method, a three-dimensional model of the initial point cloud data is constructed and aligned to obtain three-dimensional environment point cloud data;

[0014] The robot posture data is mapped based on the three-dimensional environment point cloud data and combined with the initial rotation matrix.

[0015] This application obtains initial point cloud data and environmental image data based on radar technology and binocular cameras, and then obtains an initial rotation matrix in combination with posture setting parameters, and then obtains three-dimensional environmental point cloud data based on a feature point extraction method. Finally, the robot posture data is obtained by combining the three-dimensional environmental point cloud data and the initial rotation matrix. Through the perception and processing of external environmental data, the first scene data that is more compatible with the current task scene can be obtained, so that the subsequent initial action sequence and target action sequence are more compatible with the current character scene, thereby improving the accuracy of controlling the robot.

[0016] In certain embodiments of the present application, the first external instruction is parsed in combination with the three-dimensional environment point cloud data to obtain target task features, specifically including:

[0017] Parsing the first external instruction based on a preset large language model to obtain a first task instruction, and performing feature extraction on the first task instruction based on a self-attention encoder to obtain a first task feature;

[0018] According to the first task feature, searching for a plurality of target task points in the three-dimensional environment point cloud data;

[0019] Based on the multiple target task points and combined with the first task features, a target task feature is obtained.

[0020] This application first parses the first external instruction based on a preset large language model to obtain a first task instruction, and then performs feature extraction based on a self-attention encoder to obtain a first task feature, which can convert the first external instruction in a natural language format into a first task feature that is easy for the computer to understand and process, which is beneficial for subsequent processing; then, based on the first task feature, multiple target task points in the three-dimensional environment point cloud data are searched, and the target task feature is obtained by combining the first task feature. The features of multiple target task points and the first task feature can be combined to obtain a more accurate target task feature, thereby making the initial action sequence generated according to the target task feature more accurate and improving the accuracy of controlling the robot.

[0021] In certain embodiments of the present application, adjusting the initial motion sequence according to the robot posture data to obtain a target motion sequence specifically includes:

[0022] Mapping the initial action sequence to the three-dimensional environment point cloud data according to the robot posture data to obtain a mapped action point cloud and a task action path;

[0023] determining a plurality of unreachable action points in the task action path according to the three-dimensional environment point cloud data and the mapped action point cloud;

[0024] Finding the nearest reachable points corresponding to the multiple unreachable action points respectively, and correcting the task action path based on the nearest reachable points to obtain a target action path;

[0025] According to the target action path, the initial action sequence is adjusted to obtain a target action sequence.

[0026] This application first maps the initial action sequence to the three-dimensional environment point cloud data based on the robot posture data to obtain the mapped action point cloud and the task action path, and then determines multiple unreachable action points of the task action path, and respectively searches for their nearest reachable points, and then corrects the target action path. It can eliminate and correct unreasonable action points in the initial action sequence generated by the strategy model, so as to obtain a more reasonable target action path, and then reversely adjust the initial action sequence according to the target action path to obtain a more accurate target action sequence, thereby improving the accuracy of controlling the robot.

[0027] In certain embodiments of the present application, controlling the robot to be controlled according to the target action sequence specifically includes:

[0028] Determine a task pose sequence according to the target action sequence, and calculate a target joint angle sequence corresponding to the task pose sequence based on an inverse kinematics algorithm;

[0029] Determining a drive signal sequence based on the target joint angle sequence and a preset motion control instruction of the robot to be controlled;

[0030] The robot to be controlled is controlled according to the drive signal sequence.

[0031] This application first determines the task posture sequence according to the target action sequence, and calculates the target joint angle sequence based on the inverse kinematics algorithm, and then combines the preset action control instructions to match the obtained drive signal sequence with the current target action sequence, avoiding blind control and thus improving the accuracy of controlling the robot.

[0032] According to a second aspect of the embodiments of the present application, there is provided a robot control device combining a scenario and a strategy model, comprising a scenario data acquisition module, a task feature analysis module, an action sequence generation module, and a robot control module;

[0033] The scene data acquisition module is used to acquire first scene data of the robot to be controlled; wherein the first scene data includes three-dimensional environment point cloud data and robot posture data;

[0034] The task feature parsing module is configured to obtain a first external instruction and, in combination with the three-dimensional environment point cloud data, parse the first external instruction to obtain target task features; wherein the first external instruction is obtained by receiving a natural language input from a user;

[0035] The action sequence generation module is configured to input the target task features into a pre-trained strategy model to generate an initial action sequence, and adjust the initial action sequence according to the robot posture data to obtain a target action sequence; wherein the pre-trained strategy model is trained based on historical task data of the robot to be controlled;

[0036] The robot control module is used to control the robot to be controlled according to the target action sequence.

[0037] In certain embodiments of the present application, the scene data acquisition module includes an initial data acquisition unit, a rotation matrix acquisition unit, a three-dimensional point cloud construction unit, and a robot pose mapping unit;

[0038] The initial data acquisition unit is used to acquire initial point cloud data of the robot to be controlled based on radar technology, and to acquire environmental image data of the robot to be controlled based on a binocular camera;

[0039] The rotation matrix acquisition unit is used to obtain an initial rotation matrix based on the environmental image data and the posture setting parameters of the robot to be controlled;

[0040] The three-dimensional point cloud construction unit is used to construct a three-dimensional model of the initial point cloud data and align it based on the initial rotation matrix and a feature point extraction method to obtain three-dimensional environment point cloud data;

[0041] The robot posture mapping unit is used to map the robot posture data according to the three-dimensional environment point cloud data in combination with the initial rotation matrix.

[0042] In certain embodiments of the present application, the task feature parsing module includes an instruction parsing and extracting unit, a target task point searching unit, and a task feature acquiring unit;

[0043] The instruction parsing and extraction unit is configured to parse the first external instruction based on a preset large language model to obtain a first task instruction, and perform feature extraction on the first task instruction based on a self-attention encoder to obtain a first task feature;

[0044] The target task point search unit is configured to search for a plurality of target task points in the three-dimensional environment point cloud data according to the first task feature;

[0045] The task feature acquisition unit is configured to obtain a target task feature based on the multiple target task points and in combination with the first task feature.

[0046] In certain embodiments of the present application, the action sequence generation module includes an action sequence mapping unit, an unreachable point determination unit, an action path correction unit, and an action sequence adjustment unit;

[0047] The action sequence mapping unit is used to map the initial action sequence to the three-dimensional environment point cloud data according to the robot posture data to obtain a mapped action point cloud and a task action path;

[0048] The unreachable point determination unit is configured to determine a plurality of unreachable action points in the task action path based on the three-dimensional environment point cloud data and the mapped action point cloud;

[0049] The action path correction unit is used to respectively find the nearest reachable points corresponding to the multiple unreachable action points, and based on the nearest reachable points, correct the task action path to obtain a target action path;

[0050] The action sequence adjustment unit is used to adjust the initial action sequence according to the target action path to obtain a target action sequence.

[0051] In certain embodiments of the present application, the robot control module includes a joint angle calculation unit, a drive signal determination unit, and a robot control unit;

[0052] The joint angle calculation unit is used to determine a task posture sequence according to the target action sequence, and calculate a target joint angle sequence corresponding to the task posture sequence based on an inverse kinematics algorithm;

[0053] The drive signal determination unit is configured to determine a drive signal sequence based on the target joint angle sequence and a preset motion control instruction of the robot to be controlled;

[0054] The robot control unit is used to control the robot to be controlled according to the drive signal sequence.

[0055] The present application first obtains the first scene data of the robot to be controlled, and combines the first external instruction obtained by parsing the three-dimensional environmental point cloud data in the first scene data to obtain the target task characteristics, and then inputs the target task characteristics into the pre-trained strategy model to generate an initial action sequence, and then adjusts the initial action sequence in combination with the robot posture data in the first scene data to obtain the target action sequence, and finally controls according to the target action sequence. Compared with the existing technology, the present application obtains the first scene data and the first external instruction in the task execution process based on the perception of the external environment, and improves the flexibility of subsequent control of the robot through the comprehensiveness of the data type; at the same time, unlike the existing technology that manually sets the task process and control strategy, the present application generates the initial action sequence based on the pre-trained strategy model, and adjusts it in combination with the robot posture data to obtain the target action sequence, thereby improving the adaptability to different task scenarios, thereby improving the flexibility of subsequent control of the robot, and improving the degree of matching with different task scenarios, thereby improving the accuracy of subsequent control of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 : A flowchart of a robot control method combining scenario and strategy models shown in certain embodiments of the present application;

[0057] Figure 2 : A module structure diagram of a robot control device that combines scenario and strategy models shown in certain embodiments of the present application. DETAILED DESCRIPTION

[0058] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of the present application and should not be understood as limiting the embodiments of the present application. Based on the embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0059] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly indicate the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of this application, unless otherwise clearly specified, "multiple" and "several" mean two or more.

[0060] Existing robot control methods typically rely on manual task analysis and pre-setting of task processes and control strategies. However, this traditional approach, based on task pre-analysis and pre-setting, requires manual re-analysis and re-setting of control strategies. This lacks flexibility, cannot adapt to complex task scenarios, and results in unnecessary waste of manpower and resources. Furthermore, the robot lacks awareness of external commands during task execution, making it unable to adjust task actions based on external commands, resulting in a lack of flexibility. Therefore, improving the flexibility and accuracy of robot control remains a pressing issue in existing technologies.

[0061] Based on the above technical background, please refer to Figure 1 The embodiment of the present application provides a robot control method combining a scenario and a strategy model, including steps S101 to S104, each of which is as follows:

[0062] Step S101: Acquire first scene data of a robot to be controlled; wherein the first scene data includes three-dimensional environment point cloud data and robot posture data.

[0063] In certain embodiments of the present application, obtaining first scene data of the robot to be controlled specifically includes:

[0064] Based on radar technology, the initial point cloud data of the robot to be controlled is obtained, and based on the binocular camera, the environmental image data of the robot to be controlled is obtained;

[0065] According to the environmental image data, combined with the posture setting parameters of the robot to be controlled, an initial rotation matrix is obtained;

[0066] According to the initial rotation matrix and based on the feature point extraction method, a three-dimensional model of the initial point cloud data is constructed and aligned to obtain three-dimensional environment point cloud data;

[0067] The robot posture data is mapped based on the three-dimensional environment point cloud data and combined with the initial rotation matrix.

[0068] In certain embodiments of the present application, the initial rotation matrix is obtained based on the environmental image data in combination with the posture setting parameters of the robot to be controlled, specifically: the first posture is obtained based on the environmental image data in combination with the posture setting parameters; the initial rotation matrix is calculated based on the coordinate system where the first posture is located in combination with a preset coordinate system.

[0069] This application obtains initial point cloud data and environmental image data based on radar technology and binocular cameras, and then obtains an initial rotation matrix in combination with posture setting parameters, and then obtains three-dimensional environmental point cloud data based on a feature point extraction method. Finally, the robot posture data is obtained by combining the three-dimensional environmental point cloud data and the initial rotation matrix. Through the perception and processing of external environmental data, the first scene data that is more compatible with the current task scene can be obtained, so that the subsequent initial action sequence and target action sequence are more compatible with the current character scene, thereby improving the accuracy of controlling the robot.

[0070] Step S102: Obtain a first external instruction, and parse the first external instruction in combination with the three-dimensional environment point cloud data to obtain target task features; wherein the first external instruction is obtained by receiving natural language input from the user.

[0071] In certain embodiments of the present application, the first external instruction is parsed in combination with the three-dimensional environment point cloud data to obtain target task features, specifically including:

[0072] Parsing the first external instruction based on a preset large language model to obtain a first task instruction, and performing feature extraction on the first task instruction based on a self-attention encoder to obtain a first task feature;

[0073] According to the first task feature, searching for a plurality of target task points in the three-dimensional environment point cloud data;

[0074] Based on the multiple target task points and combined with the first task features, a target task feature is obtained.

[0075] In certain embodiments of the present application, preferred solutions for the large language model include but are not limited to GPT, DeepSeek, Wenxin Yiyan, iFlytek Spark, Tongyi Qianwen, or an open source model fine-tuned by Lora.

[0076] This application first parses the first external instruction based on a preset large language model to obtain a first task instruction, and then performs feature extraction based on a self-attention encoder to obtain a first task feature, which can convert the first external instruction in a natural language format into a first task feature that is easy for the computer to understand and process, which is beneficial for subsequent processing; then, based on the first task feature, multiple target task points in the three-dimensional environment point cloud data are searched, and the target task feature is obtained by combining the first task feature. The features of multiple target task points and the first task feature can be combined to obtain a more accurate target task feature, thereby making the initial action sequence generated according to the target task feature more accurate and improving the accuracy of controlling the robot.

[0077] Step S103: Input the target task features into a pre-trained strategy model to generate an initial action sequence, and adjust the initial action sequence according to the robot posture data to obtain a target action sequence; wherein the pre-trained strategy model is trained based on the historical task data of the robot to be controlled.

[0078] In certain embodiments of the present application, adjusting the initial motion sequence according to the robot posture data to obtain a target motion sequence specifically includes:

[0079] Mapping the initial action sequence to the three-dimensional environment point cloud data according to the robot posture data to obtain a mapped action point cloud and a task action path;

[0080] determining a plurality of unreachable action points in the task action path according to the three-dimensional environment point cloud data and the mapped action point cloud;

[0081] Finding the nearest reachable points corresponding to the multiple unreachable action points respectively, and correcting the task action path based on the nearest reachable points to obtain a target action path;

[0082] According to the target action path, the initial action sequence is adjusted to obtain a target action sequence.

[0083] In certain embodiments of the present application, searching for the nearest reachable point corresponding to an unreachable action point and correcting the task action path based on the nearest reachable point are based on a preset pathfinding algorithm, which includes but is not limited to a depth-first search algorithm, a breadth-first search algorithm, a Dijkstra algorithm, an A* algorithm, and a simulated annealing algorithm.

[0084] This application first maps the initial action sequence to the three-dimensional environment point cloud data based on the robot posture data to obtain the mapped action point cloud and the task action path, and then determines multiple unreachable action points of the task action path, and respectively searches for their nearest reachable points, and then corrects the target action path. It can eliminate and correct unreasonable action points in the initial action sequence generated by the strategy model, so as to obtain a more reasonable target action path, and then reversely adjust the initial action sequence according to the target action path to obtain a more accurate target action sequence, thereby improving the accuracy of controlling the robot.

[0085] Step S104: controlling the robot to be controlled according to the target action sequence.

[0086] In certain embodiments of the present application, controlling the robot to be controlled according to the target action sequence specifically includes:

[0087] Determine a task pose sequence according to the target action sequence, and calculate a target joint angle sequence corresponding to the task pose sequence based on an inverse kinematics algorithm;

[0088] Determining a drive signal sequence based on the target joint angle sequence and a preset motion control instruction of the robot to be controlled;

[0089] The robot to be controlled is controlled according to the drive signal sequence.

[0090] This application first determines the task posture sequence according to the target action sequence, and calculates the target joint angle sequence based on the inverse kinematics algorithm, and then combines the preset action control instructions to match the obtained drive signal sequence with the current target action sequence, avoiding blind control and thus improving the accuracy of controlling the robot.

[0091] Compared to the prior art, the present application first obtains the first scene data of the robot to be controlled, and combines the first external instructions obtained by parsing the three-dimensional environmental point cloud data in the first scene data to obtain the target task characteristics, and then inputs the target task characteristics into the pre-trained strategy model to generate an initial action sequence, and then adjusts the initial action sequence in combination with the robot posture data in the first scene data to obtain the target action sequence, and finally controls according to the target action sequence. Compared to the prior art, the present application obtains the first scene data and the first external instructions in the task execution process based on the perception of the external environment, and improves the flexibility of subsequent robot control through the comprehensiveness of data types; at the same time, unlike the prior art that manually sets task processes and control strategies, the present application generates the initial action sequence based on the pre-trained strategy model, and adjusts it in combination with the robot posture data to obtain the target action sequence, improving the adaptability to different task scenarios, thereby improving the flexibility of subsequent robot control, and improving the degree of matching with different task scenarios, thereby improving the accuracy of subsequent robot control.

[0092] Corresponding to the above method, see Figure 2 , the embodiment of the present application provides a robot control device that combines a scenario and a strategy model, including a scenario data acquisition module 210, a task feature analysis module 220, an action sequence generation module 230 and a robot control module 240;

[0093] The scene data acquisition module 210 is used to acquire first scene data of the robot to be controlled; wherein the first scene data includes three-dimensional environment point cloud data and robot posture data;

[0094] The task feature parsing module 220 is configured to obtain a first external instruction and, in combination with the three-dimensional environment point cloud data, parse the first external instruction to obtain target task features; wherein the first external instruction is obtained by receiving a natural language input from a user;

[0095] The action sequence generation module 230 is configured to input the target task features into a pre-trained strategy model to generate an initial action sequence, and adjust the initial action sequence according to the robot posture data to obtain a target action sequence; wherein the pre-trained strategy model is trained based on historical task data of the robot to be controlled;

[0096] The robot control module 240 is used to control the robot to be controlled according to the target action sequence.

[0097] In certain embodiments of the present application, the scene data acquisition module 210 includes an initial data acquisition unit, a rotation matrix acquisition unit, a three-dimensional point cloud construction unit, and a robot pose mapping unit;

[0098] The initial data acquisition unit is used to acquire initial point cloud data of the robot to be controlled based on radar technology, and to acquire environmental image data of the robot to be controlled based on a binocular camera;

[0099] The rotation matrix acquisition unit is used to obtain an initial rotation matrix based on the environmental image data and the posture setting parameters of the robot to be controlled;

[0100] The three-dimensional point cloud construction unit is used to construct a three-dimensional model of the initial point cloud data and align it based on the initial rotation matrix and a feature point extraction method to obtain three-dimensional environment point cloud data;

[0101] The robot posture mapping unit is used to map the robot posture data according to the three-dimensional environment point cloud data in combination with the initial rotation matrix.

[0102] In certain embodiments of the present application, the task feature parsing module 220 includes an instruction parsing and extracting unit, a target task point searching unit, and a task feature acquiring unit;

[0103] The instruction parsing and extraction unit is configured to parse the first external instruction based on a preset large language model to obtain a first task instruction, and perform feature extraction on the first task instruction based on a self-attention encoder to obtain a first task feature;

[0104] The target task point search unit is configured to search for a plurality of target task points in the three-dimensional environment point cloud data according to the first task feature;

[0105] The task feature acquisition unit is configured to obtain a target task feature based on the multiple target task points and in combination with the first task feature.

[0106] In certain embodiments of the present application, the action sequence generation module 230 includes an action sequence mapping unit, an unreachable point determination unit, an action path correction unit, and an action sequence adjustment unit;

[0107] The action sequence mapping unit is used to map the initial action sequence to the three-dimensional environment point cloud data according to the robot posture data to obtain a mapped action point cloud and a task action path;

[0108] The unreachable point determination unit is configured to determine a plurality of unreachable action points in the task action path based on the three-dimensional environment point cloud data and the mapped action point cloud;

[0109] The action path correction unit is used to respectively find the nearest reachable points corresponding to the multiple unreachable action points, and based on the nearest reachable points, correct the task action path to obtain a target action path;

[0110] The action sequence adjustment unit is used to adjust the initial action sequence according to the target action path to obtain a target action sequence.

[0111] In certain embodiments of the present application, the robot control module 240 includes a joint angle calculation unit, a drive signal determination unit, and a robot control unit;

[0112] The joint angle calculation unit is used to determine a task posture sequence according to the target action sequence, and calculate a target joint angle sequence corresponding to the task posture sequence based on an inverse kinematics algorithm;

[0113] The drive signal determination unit is configured to determine a drive signal sequence based on the target joint angle sequence and a preset motion control instruction of the robot to be controlled;

[0114] The robot control unit is used to control the robot to be controlled according to the drive signal sequence.

[0115] The present application first obtains the first scene data of the robot to be controlled, and combines the first external instruction obtained by parsing the three-dimensional environmental point cloud data in the first scene data to obtain the target task characteristics, and then inputs the target task characteristics into the pre-trained strategy model to generate an initial action sequence, and then adjusts the initial action sequence in combination with the robot posture data in the first scene data to obtain the target action sequence, and finally controls according to the target action sequence. Compared with the existing technology, the present application obtains the first scene data and the first external instruction in the task execution process based on the perception of the external environment, and improves the flexibility of subsequent control of the robot through the comprehensiveness of the data type; at the same time, unlike the existing technology that manually sets the task process and control strategy, the present application generates the initial action sequence based on the pre-trained strategy model, and adjusts it in combination with the robot posture data to obtain the target action sequence, thereby improving the adaptability to different task scenarios, thereby improving the flexibility of subsequent control of the robot, and improving the degree of matching with different task scenarios, thereby improving the accuracy of subsequent control of the robot.

[0116] It should be understood that the device provided in the embodiment of the present application corresponds to the aforementioned method. The robot control device provided in the embodiment of the present application that combines the scenario and the strategy model can implement the robot control method that combines the scenario and the strategy model provided in any embodiment of the present application.

[0117] Adaptively, the embodiments of the present application further provide a computer device and a computer-readable storage medium.

[0118] The computer device comprises: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor;

[0119] Among them, when the processor executes the computer program, it implements a robot control method combining scenario and strategy models of the present application.

[0120] The computer-readable storage medium stores a plurality of instructions, which are suitable for loading by a processor to execute a robot control method combining a scenario and a strategy model of the present application.

[0121] The above description is a partial embodiment of the present application, which further describes the purpose, technical solutions, and beneficial effects of the present application in detail. It should be understood that the above description of the partial embodiment of the present application is not to be construed as limiting the present application. In particular, it is pointed out that for those skilled in the art, any changes, modifications, equivalent substitutions, and variations made within the spirit and principles of the present application should be included within the scope of protection of the present application.

Claims

1. A robot control method combining scenario and strategy models, characterized in that: include: Acquire first scene data of the robot to be controlled; wherein the first scene data includes three-dimensional environment point cloud data and robot posture data; Obtaining a first external instruction, and parsing the first external instruction in combination with the three-dimensional environment point cloud data to obtain target task features; wherein the first external instruction is obtained by receiving natural language input from a user; Inputting the target task features into a pre-trained strategy model to generate an initial action sequence, and adjusting the initial action sequence according to the robot posture data to obtain a target action sequence; wherein the pre-trained strategy model is trained based on historical task data of the robot to be controlled; controlling the robot to be controlled according to the target action sequence; The adjusting of the initial action sequence according to the robot posture data to obtain the target action sequence specifically includes: mapping the initial action sequence to the three-dimensional environment point cloud data according to the robot posture data to obtain a mapping action point cloud and a task action path; determining multiple unreachable action points in the task action path according to the three-dimensional environment point cloud data and the mapping action point cloud; searching for the nearest reachable points corresponding to the multiple unreachable action points respectively, and correcting the task action path based on the nearest reachable points to obtain a target action path; adjusting the initial action sequence according to the target action path to obtain a target action sequence.

2. A robot control method combining scenario and strategy models according to claim 1, characterized in that: The obtaining of first scene data of the robot to be controlled specifically includes: Based on radar technology, the initial point cloud data of the robot to be controlled is obtained, and based on the binocular camera, the environmental image data of the robot to be controlled is obtained; According to the environmental image data, combined with the posture setting parameters of the robot to be controlled, an initial rotation matrix is obtained; According to the initial rotation matrix and based on the feature point extraction method, a three-dimensional model of the initial point cloud data is constructed and aligned to obtain three-dimensional environment point cloud data; The robot posture data is mapped based on the three-dimensional environment point cloud data and combined with the initial rotation matrix.

3. The robot control method combining scenario and strategy model according to claim 1, characterized in that: The step of analyzing the first external instruction in combination with the three-dimensional environment point cloud data to obtain target task features specifically includes: Parsing the first external instruction based on a preset large language model to obtain a first task instruction, and performing feature extraction on the first task instruction based on a self-attention encoder to obtain a first task feature; According to the first task feature, searching for a plurality of target task points in the three-dimensional environment point cloud data; Based on the multiple target task points and combined with the first task features, a target task feature is obtained.

4. The robot control method combining scenario and strategy model according to claim 1, characterized in that: The controlling the robot to be controlled according to the target action sequence specifically includes: Determine a task pose sequence according to the target action sequence, and calculate a target joint angle sequence corresponding to the task pose sequence based on an inverse kinematics algorithm; Determining a drive signal sequence based on the target joint angle sequence and a preset motion control instruction of the robot to be controlled; The robot to be controlled is controlled according to the drive signal sequence.

5. A robot control device combining scenario and strategy models, characterized in that: It includes scene data acquisition module, task feature analysis module, action sequence generation module and robot control module; The scene data acquisition module is used to acquire first scene data of the robot to be controlled; wherein the first scene data includes three-dimensional environment point cloud data and robot posture data; The task feature parsing module is configured to obtain a first external instruction and, in combination with the three-dimensional environment point cloud data, parse the first external instruction to obtain target task features; wherein the first external instruction is obtained by receiving a natural language input from a user; The action sequence generation module is configured to input the target task features into a pre-trained strategy model to generate an initial action sequence, and adjust the initial action sequence according to the robot posture data to obtain a target action sequence; wherein the pre-trained strategy model is trained based on historical task data of the robot to be controlled; The robot control module is used to control the robot to be controlled according to the target action sequence; The action sequence generation module includes an action sequence mapping unit, an inreachable point determination unit, an action path correction unit and an action sequence adjustment unit; the action sequence mapping unit is used to map the initial action sequence to the three-dimensional environment point cloud data according to the robot posture data to obtain a mapping action point cloud and a task action path; the inreachable point determination unit is used to determine multiple inreachable action points in the task action path according to the three-dimensional environment point cloud data and the mapping action point cloud; the action path correction unit is used to respectively search for the nearest reachable points corresponding to the multiple inreachable action points, and based on the nearest reachable points, correct the task action path to obtain a target action path; the action sequence adjustment unit is used to adjust the initial action sequence according to the target action path to obtain a target action sequence.

6. A robot control device combining scenario and strategy models according to claim 5, characterized in that: The scene data acquisition module includes an initial data acquisition unit, a rotation matrix acquisition unit, a three-dimensional point cloud construction unit and a robot posture mapping unit; The initial data acquisition unit is used to acquire initial point cloud data of the robot to be controlled based on radar technology, and to acquire environmental image data of the robot to be controlled based on a binocular camera; The rotation matrix acquisition unit is used to obtain an initial rotation matrix based on the environmental image data and the posture setting parameters of the robot to be controlled; The three-dimensional point cloud construction unit is used to construct a three-dimensional model of the initial point cloud data and align it based on the initial rotation matrix and a feature point extraction method to obtain three-dimensional environment point cloud data; The robot posture mapping unit is used to map the robot posture data according to the three-dimensional environment point cloud data in combination with the initial rotation matrix.

7. The robot control device combining scenario and strategy model according to claim 5, characterized in that: The task feature parsing module includes an instruction parsing and extracting unit, a target task point searching unit and a task feature acquiring unit; The instruction parsing and extraction unit is configured to parse the first external instruction based on a preset large language model to obtain a first task instruction, and perform feature extraction on the first task instruction based on a self-attention encoder to obtain a first task feature; The target task point search unit is configured to search for a plurality of target task points in the three-dimensional environment point cloud data according to the first task feature; The task feature acquisition unit is configured to obtain a target task feature based on the multiple target task points and in combination with the first task feature.

8. The robot control device combining scenario and strategy model according to claim 5, characterized in that: The robot control module includes a joint angle calculation unit, a drive signal determination unit and a robot control unit; The joint angle calculation unit is used to determine a task posture sequence according to the target action sequence, and calculate a target joint angle sequence corresponding to the task posture sequence based on an inverse kinematics algorithm; The drive signal determination unit is configured to determine a drive signal sequence based on the target joint angle sequence and a preset motion control instruction of the robot to be controlled; The robot control unit is used to control the robot to be controlled according to the drive signal sequence.

Images