Robot automatic control method, device and equipment and storage medium

Abstract

The invention discloses a robot automatic control method and device, equipment and a storage medium. The method is characterized by comprising the following steps: receiving multi-modal interaction information input by a user, analyzing the multi-modal interaction information through a pre-trained robot control large model, and determining a subtask code list corresponding to the multi-modal interaction information; industrial operation information of a target robot is obtained, target control logic corresponding to the target robot is generated according to the industrial operation information and the subtask code list, and the target robot is automatically controlled through the target control logic. Through intelligent analysis of the large model on robot task demands in different scenes, the method has high flexibility and applicability, can effectively reduce the user operation threshold, and improves the control efficiency of the robot.

Description

Technical Field

[0001] The present invention relates to the field of industrial robot control technology, and in particular to a robot automatic control method, device, equipment and storage medium. Background Art

[0002] With the development of industrial automation, industrial robots are increasingly used in manufacturing, assembly, logistics and other fields. Industrial robots are controlled to perform tasks such as manufacturing, assembly and logistics. However, traditional industrial robot control methods mostly rely on pre-programmed task processes and fixed operation modes. This rigid control method often takes a lot of time to reprogram and debug when facing complex or frequently changing tasks, resulting in low production efficiency. In the existing technology, attempts are made to automate complex tasks through machine learning, computer vision, natural language processing and other technologies. However, existing artificial intelligence-based control methods are usually still limited to specific task scenarios and require users to have high professional knowledge. The ease of use and efficiency of artificial intelligence control methods are limited. Summary of the invention

[0003] The present invention provides a robot automatic control method, device, equipment and storage medium to solve the technical problem that the control method of industrial robots in the prior art cannot adapt to complex industrial scenarios.

[0004] According to one aspect of the present invention, there is provided a robot automatic control method, comprising:

[0005] Receive multimodal interaction information input by a user, parse the multimodal interaction information through a pre-trained robot control large model, and determine a subtask code list corresponding to the multimodal interaction information; wherein the subtask code list includes at least one subtask code and a subtask execution logic;

[0006] The industrial operation information of the target robot is obtained, the target control logic corresponding to the target robot is generated according to the industrial operation information and the subtask code list, and the target robot is automatically controlled by the target control logic.

[0007] According to another aspect of the present invention, there is provided a robot automatic control device, comprising:

[0008] The large model module is used to receive multimodal interaction information input by the user, parse the multimodal interaction information through the pre-trained robot control large model, and determine the subtask code list corresponding to the multimodal interaction information;

[0009] The robot control module is used to obtain industrial operation information of the target robot, generate target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically control the target robot through the target control logic.

[0010] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0011] at least one processor; and

[0012] A memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the robot automatic control method described in any embodiment of the present invention.

[0014] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the robot automatic control method described in any embodiment of the present invention when executed.

[0015] The technical solution of the embodiment of the present invention receives multimodal interaction information input by the user, parses the multimodal interaction information through a pre-trained robot control large model, determines the subtask code list corresponding to the multimodal interaction information, and can parse the multimodal interaction information through the robot control large model to automatically generate a subtask code list, effectively improving the flexibility and adaptability of robot control and lowering the user's operation threshold; obtains the industrial operation information of the target robot, generates the target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically controls the target robot through the target control logic. The industrial operation information can ensure the accuracy and stability of the operation when executing the task on the basis of intelligence and flexibility, and meet the high requirements for program execution in industrial scenarios. The technical problem that the control method of industrial robots in the prior art cannot adapt to complex industrial scenarios is solved, and at the same time, it has high flexibility and applicability, can effectively lower the user's operation threshold, and improve the beneficial effect of robot control efficiency.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flowchart of a robot automatic control method is provided for an embodiment of the present invention;

[0019] Figure 2 A flowchart of another robot automatic control method provided by an embodiment of the present invention;

[0020] Figure 3 A flowchart of another robot automatic control method provided by an embodiment of the present invention;

[0021] Figure 4 A flowchart of another robot automatic control method provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of the structure of a robot automatic control device provided by an embodiment of the present invention;

[0023] Figure 6 A schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Figure 1 A flowchart of a robot automatic control method is provided for an embodiment of the present invention. This embodiment can be applied to the situation where the control logic of the robot is generated in an industrial scenario and the robot is automatically controlled by the control logic. The method can be executed by a robot automatic control device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0027] S110, receiving multimodal interaction information input by a user, parsing the multimodal interaction information through a pre-trained robot control large model, and determining a subtask code list corresponding to the multimodal interaction information.

[0028] Among them, the multimodal interaction information may include robot task requirements and robot task execution conditions. The robot task requirements may be natural language input by the user, voice sent by the user, and behavior information sent by the user; the robot task execution conditions may be at least one of images and / or videos uploaded by the robot user, voice sent by the user, and behavior information sent by the user. Optionally, the natural language input by the user may be natural language text sent by the user. For example, "grab the apple on the desktop"; the images and videos uploaded by the user may be images and videos selected by the user to upload through the image interface; the voice sent by the user may be voice information uploaded through the audio interface or the voice input interface; the behavior information sent by the user may be behavior information uploaded by the user performing a specific action through a human-computer interaction device. Exemplarily, the human-computer interaction device may be a virtual reality device or a brain-computer device.

[0029] Optionally, an embodiment of the present invention provides a visual interactive interface for inputting multimodal interactive information, and a text input box, a voice input box, an audio upload interface, a video input interface and an interactive device connection interface are set in the visual interactive interface. The user can input natural language text in the text input box of the visual interactive interface; the user can also connect a voice input device through the voice input box and input voice information through the voice input device; the user can also upload voice files and input voice information through the audio upload interface; the user can also upload images or video files through the video input interface and input image and video information, and can also connect a camera device or a shooting device through the video input interface, shoot images through the shooting device, and record video information through the camera device; the user can also connect to a human-computer interaction device through the interactive device connection interface and obtain behavior information through the human-computer interaction device.

[0030] Among them, the robot control large model can be a neural network large model that is pre-trained to understand the multimodal interaction information input by the user and to perform task planning for the industrial robot; it should be noted that the robot control large model can recognize the multimodal interaction information input by the user, understand the user's needs and the robot's industrial scenario, and then plan tasks for the industrial robot based on user needs in the current industrial scenario.

[0031] Optionally, an industrial robot is a kind of robot that can be automatically controlled, reprogrammed, multifunctional, multi-degree-of-freedom, and multi-purpose for industrial applications. An industrial robot can carry materials, workpieces, or handle tools. By programming the industrial robot, the industrial robot can be controlled to perform various industrial operations.

[0032] Optionally, the robot control large model can be an LLM model (Large Language Model). The robot control large model needs to have language understanding function, industrial scene recognition function, code generation function and robot task decomposition function; when training the robot control large model, on the basis of designing a natural language training set, design a language training set corresponding to the professional vocabulary, semantic relationship and logical structure in the field of industrial robots; design a scene recognition training set corresponding to the image, video and voice corresponding to the industrial scene; design a robot programming code training set corresponding to robot programming; design a robot action training set corresponding to robot action prediction; train the preset multimodal model through the above training sets, and optimize it for the industrial robot application tasks to obtain the robot control large model.

[0033] Optionally, the robot control large model establishes a connection with the visual interaction interface. The robot control large model can interact with the user in the visual interaction interface, and is directly connected to the text input box, voice input box, audio upload interface, video input interface and interactive device connection interface of the visual interaction interface to receive multimodal interaction information input by the user and display the answer results in the visual interaction interface.

[0034] Among them, the subtask code list includes at least one subtask code and subtask execution logic; it should be noted that working robots in industrial scenarios usually decompose complex user needs into multiple subtasks to meet the execution operations of industrial robots. A subtask can be an action of a robot, for example, controlling the end effector of the robot (such as a mechanical gripper, suction cup, etc.) to grasp an object; the subtask code can be a program code that controls the robot to execute an action; the subtask execution logic can be the specific steps for the industrial robot to execute each robot subtask, and the industrial robot can execute each robot subtask in accordance with the robot subtask execution logic sequence to complete the task goal of the industrial robot. For example, the subtask execution logic can be a grasping subtask-moving subtask-placing subtask, the grasping subtask is: the end effector of the robot grasps the object, the moving subtask is: the robot moves to the target placement position, and the placing subtask is: the end effector of the robot places the object; the robot executes the grasping subtask, the moving subtask and the placing subtask in sequence through the subtask execution logic to achieve the task goal of the robot to move an object to the target placement position.

[0035] Specifically, users can input multimodal interaction information through a visual interactive interface. After receiving the multimodal interaction information, the multimodal interaction information is parsed through a pre-trained robot control model, and the robot control model outputs a list of sub-task codes corresponding to the multimodal interaction information.

[0036] S120. Obtain the industrial operation information of the target robot, generate the target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically control the target robot through the target control logic.

[0037] Among them, the industrial operation information may be the status operation information of the industrial robot in the industrial scenario; it should be noted that for the industrial robot in different industrial scenarios, it is necessary to perform corresponding initialization settings for the industrial robot based on the current industrial scenario, define the initial state of the industrial robot, and set the workspace of the industrial robot to set the task execution environment for the current industrial scenario.

[0038] Exemplarily, industrial operation information may include: sensor status information of the industrial robot, workspace setting information, initial state of the industrial robot, end tool and task cycle and termination conditions of the industrial robot; sensor status information of the industrial robot may be status information of sensors equipped with the industrial robot, and the sensors may include position sensors, force sensors, visual sensors, etc., through which the position, posture, force conditions and surrounding environment of the robot can be monitored in real time, providing a basis for the control and decision-making of the industrial robot; workspace setting information may be the space range that can be operated in the current work scene, and the workspace setting information includes parameters such as the maximum motion range, accessible space, and work area of ​​the industrial robot, and the workspace setting information determines the robot's operation and operation. The initial state of an industrial robot refers to the state of the robot when it is started or reset, including parameters such as the position, speed, acceleration of each joint, and the posture and position of the end effector. The end tool is a tool installed on the end effector of the industrial robot and is used to complete specific work tasks. The end tools include fixtures, spray guns, welding guns, and tools. The task cycle and termination conditions can be in industrial scenarios, requiring the industrial robot to execute in a cycle to complete the cycle operation, and setting the termination conditions in the cycle. Each time the industrial robot performs a cycle operation, it checks whether the termination conditions of the operation are met. If the termination conditions of the operation are met, the operation is terminated. If the termination conditions of the operation are not met, the cycle operation continues.

[0039] Among them, the target control logic can be the control logic for controlling the industrial robot to perform operations. Exemplarily, the target control logic can be used to sequentially control the industrial robot to perform various operation actions, and can control the industrial robot to move joints, use end tools and move positions in the current workspace; for an industrial scene of goods sorting, the target control logic can control the industrial robot's sensor to identify the goods on the conveyor belt, and obtain the location information of the goods, and grab the goods by moving the joints and controlling the end tools, and then move the industrial robot to bypass the conveyor belt and other industrial robots to carry the goods to the designated sorting area or container, and move the industrial robot to the starting position, and detect whether there are still goods on the conveyor belt within 1 minute. If there are no goods, the termination condition is met and the industrial robot is shut down. If there are goods, the loop condition is met and the industrial robot continues to sort goods until the termination condition is met.

[0040] Optionally, after obtaining the subtask code list, the industrial operation information of the target robot and the subtask code list are combined to generate a target control logic that conforms to the corresponding industrial operation information of the target robot.

[0041] Specifically, the industrial operation information of the target robot is obtained, the target control logic corresponding to the target robot is generated according to the industrial operation information and the subtask code list, and the target robot is automatically controlled through the target control logic.

[0042] The technical solution of the embodiment of the present invention receives multimodal interaction information input by the user, parses the multimodal interaction information through a pre-trained robot control large model, determines the subtask code list corresponding to the multimodal interaction information, and can parse the multimodal interaction information through the robot control large model to automatically generate a subtask code list, effectively improving the flexibility and adaptability of robot control and lowering the user's operation threshold; obtains the industrial operation information of the target robot, generates the target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically controls the target robot through the target control logic. The industrial operation information can ensure the accuracy and stability of the operation when executing the task on the basis of intelligence and flexibility, and meet the high requirements for program execution in industrial scenarios. The technical problem that the control method of industrial robots in the prior art cannot adapt to complex industrial scenarios is solved, and at the same time, it has high flexibility and applicability, can effectively lower the user's operation threshold, and improve the beneficial effect of robot control efficiency.

[0043] Figure 2 This is a flowchart of another robot automatic control method provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for parsing multi-modal interactive information through a robot control large model. Figure 2 As shown, the method includes:

[0044] S210: Receive multimodal interaction information input by a user.

[0045] S220: parsing the multimodal interaction information through the robot control large model to determine a robot subtask list.

[0046] Among them, the robot subtask list may include at least one robot subtask and a subtask logical order; the subtask logical order may be the specified information of the execution sequence and dependency relationship followed by the industrial robot when executing the robot subtasks; the subtask logical order can be used to control the industrial robot to execute each robot subtask in sequence and follow the dependency relationship between each robot subtask.

[0047] Optionally, the robot subtask list may arrange each robot subtask in sequence according to the logical order of subtasks associated with the robot subtask, with the execution order of the robot subtask recorded first being before the execution order of the robot subtask recorded later, and the arrangement order of the robot subtasks in the robot subtask list serving as the logical order of the subtasks of the robot subtasks.

[0048] Optionally, when recording robot subtasks, the robot task list may also set a task identifier for each robot subtask, and uniquely identify a robot subtask by a task identifier. The robot task list arranges each robot subtask in a logical order according to the execution order and dependency relationship of each robot subtask, and arranges and combines the task identifiers of each robot subtask in a logical order in turn to obtain a subtask logical order composed of the task identifiers; and then in the robot task list, it is only necessary to record the robot subtasks and the subtask logical order, without arranging the robot subtasks; the execution order and dependency relationship of each robot subtask can be determined directly by reading the subtask logical order.

[0049] Among them, the robot subtask includes subtask conditions, subtask goals and subtask execution constraints; it should be noted that, in the present invention, the task conditions of the robot subtask can be the prerequisite requirements or states that must be met before executing the current robot subtask, and the task conditions of the robot subtask are used to ensure whether the robot subtask can start to execute; the subtask goals of the robot subtask can be the results and states that the robot subtask expects to achieve; the subtask goals clarify the execution direction and purpose of the robot subtask; the subtask execution constraints of the robot subtask can be the restrictions and regulations on the industrial robot during the execution of the subtask; the subtask execution constraints of the robot subtask can be used to ensure that the industrial robot is in a safe state when executing the robot subtask. Exemplarily, the subtask conditions of the robot subtask can be understood as the conditions of the input industrial robot, the subtask goals of the robot subtask can be understood as the output of the industrial robot, and the subtask constraints of the robot subtask can be understood as the constraint information of the industrial robot.

[0050] Specifically, the multimodal interaction information is parsed through the robot control model to determine the robot subtask list.

[0051] Optionally, in another optional embodiment of the present invention, the multimodal interaction information is parsed by the robot control macro model to determine the robot subtask list, including:

[0052] Analyze the robot's mission requirements through the robot control model and determine the robot's mission objectives;

[0053] Through the robot control model, the task is decomposed according to the robot task conditions and robot task goals, and the robot subtask list is determined.

[0054] Among them, the robot task goal can be that the robot control large model analyzes user needs to obtain the goals that the industrial robot needs to complete.

[0055] Optionally, in the present invention, the user inputs natural language, sends voice and / or sends behavioral information as the user input robot task requirements, and the robot control large model understands the robot task requirements, extracts key elements of the corresponding tasks of the industrial robot, and determines the robot task goals corresponding to the robot task requirements.

[0056] Optionally, in the present invention, after understanding the robot task requirements, the robot control large model obtains the robot task goals corresponding to the robot task requirements. The robot control large model decomposes the robot task goals into multiple interrelated robot subtasks in combination with the robot task conditions, and determines the logical order of the subtasks according to the sequential execution relationship and dependency relationship of the robot subtasks, and constructs a robot subtask list according to the robot subtasks and the logical order of the subtasks.

[0057] S230: For each of the robot subtasks, perform meta-skill matching on the robot subtask according to a preset meta-skill description text through the robot control macro model to determine the target meta-skill matched by the robot subtask.

[0058] Among them, the preset meta-skill description text can be a text file used to describe all meta-skills corresponding to the pre-built meta-skill library.

[0059] Optionally, in the present invention, a meta-skill library for robot control large models is pre-built, and the meta-skill library includes a series of basic robot operation skills. The meta-skill library can expand and optimize the robot operation skills according to actual needs. Each meta-skill is encapsulated through an API (Application Programming Interface), an SDK (Software Development Kit) or a function. It should be noted that each meta-skill includes a detailed functional description, and a set of configurable parameters are configured for each meta-skill. The specific execution mode of the meta-skill is determined by the configurable parameters, and the parameter input is received through the API, SDK or function, and the execution result of the meta-skill or the status information of the meta-skill is returned. Exemplarily, in the meta-skill library, the meta-skill can be a grasping skill, and the technical description of the grasping skill can be "locating to a specified position" and "grasping a specific object"; the parameterized design of the grasping skill can be the grasping position, grasping force, and grasping angle corresponding to the grasping skill. It should be noted that the meta-skill description text is a fixed standard text structure, and the meta-skill description text can be used as a system prompt (system prompt text); by combining the meta-skill name, the meta-skill corresponding encapsulation and calling method, and the meta-skill description text into text information, all the text information corresponding to the meta-skill is combined into the meta-skill description text. Exemplarily, the meta-skill description text can use the json file format, the meta-skill name is: name, the meta-skill corresponding encapsulation and calling method can be: xxx_skill (arg1, arg2, arg3...), and the skill description of the meta-skill is "description".

[0060] Among them, the target meta-skill includes robot meta-skills and meta-skill parameters; the robot meta-skill can be a meta-skill in the meta-skill library that matches the robot subtask, and the meta-skill parameters can be parameters configured for the robot meta-skill.

[0061] Optionally, the robot control model performs meta-skill matching in the meta-skill description text for each robot sub-task, and obtains a target meta-skill in the meta-skill description text that matches the robot sub-task.

[0062] Optionally, in another optional embodiment of the present invention, the robot control macromodel is used to perform meta-skill matching with the subtask conditions and subtask goals of the robot subtask according to a preset meta-skill description text, so as to determine the robot meta-skill that matches the robot subtask; the robot control macromodel is used to set parameters for the robot meta-skill according to the subtask execution constraints of the robot subtask, so as to determine the meta-skill parameters corresponding to the robot meta-skill.

[0063] Optionally, for each robot subtask, the subtask conditions and subtask goals of the robot subtask are matched with the meta-skill description text through the robot control big model. When matching the robot control big model, the similarity between the robot subtask and each meta-skill is determined by comparing the subtask goals and subtask conditions of the robot subtask with the skill description and skill name of each meta-skill in the meta-skill description text, and the meta-skill with the highest similarity is selected as the robot meta-skill of the robot subtask.

[0064] Optionally, after the robot control macromodel matches the subtask meta-skill for the robot subtask, the meta-skill parameters of the subtask meta-skill are generated according to the subtask constraints of the robot subtask, and the subtask meta-skill and meta-skill parameters are determined as the target meta-skill of the robot subtask.

[0065] Optionally, the robot control model can connect to the meta-skill library, obtain sub-task meta-skills in the meta-skill library, and receive meta-skill parameters through the API, SDK or function of the sub-task meta-skill.

[0066] S240. Determine the subtask code list according to the subtask logical sequence and each of the target meta-skills through the robot control macro model.

[0067] Optionally, after matching each robot subtask, the robot control model obtains the target meta-skill corresponding to each robot subtask, and by connecting to the meta-skill library, the robot control model calls the target meta-skill in the meta-skill library in sequence according to the logical order of the subtasks to obtain a subtask code list.

[0068] Optionally, in another optional embodiment of the present invention, the subtask code list is determined by the robot control macro model according to the subtask logical sequence and each of the target meta-skills, including:

[0069] The robot control macromodel performs meta-skill calling in a preset meta-skill library according to each of the target meta-skills, and determines a sub-task code corresponding to each of the target meta-skills;

[0070] The robot control macromodel is used to combine the subtask codes corresponding to each of the robot subtasks according to the subtask logical sequence to determine the subtask code list.

[0071] Specifically, the meta-skill parameters of each sub-task meta-skill are uploaded to the API, SDK or function of the sub-task meta-skill through the robot control model, and the sub-task meta-skills are called in the meta-skill library in sequence according to the sub-task logical sequence to generate the sub-task code corresponding to each sub-task meta-skill. The sub-task logical sequence is used as the sub-task execution logic, and a sub-task code list is constructed according to each sub-task code and sub-task execution logic.

[0072] S250. Acquire industrial operation information of the target robot, generate a target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically control the target robot through the target control logic.

[0073] The technical solution of the embodiment of the present invention clarifies the user's needs through the robot control big model, understands and analyzes the user's needs, obtains the robot subtask list, matches the robot subtask through the preset meta-skill library, obtains the target meta-skill of each robot subtask, and then determines the subtask code list. Through the intelligent analysis and dynamic skill combination of the big model, the system can quickly adapt to different industrial scenarios and task requirements, reduces the dependence on fixed programming, can effectively lower the user's operation threshold, and meets the high requirements for program execution in industrial scenarios. It solves the technical problem that the control method of industrial robots in the prior art cannot adapt to complex industrial scenarios, and at the same time has high flexibility and applicability, can effectively lower the user's operation threshold, and improves the beneficial effect of robot control efficiency.

[0074] Figure 3 This is a flowchart of another robot automatic control method provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is that the subtask code list is obtained through the robot control model, and the user performs a specific method for personalizing the subtask code list. Figure 3 As shown, the method includes:

[0075] S310: Receive multimodal interaction information input by a user.

[0076] S320: parsing the multimodal interaction information through the robot control large model to determine a robot subtask list.

[0077] S330: For each of the robot subtasks, perform meta-skill matching on the robot subtask according to a preset meta-skill description text through the robot control macro model to determine the target meta-skill matched by the robot subtask.

[0078] S340. Determine the subtask code list according to the subtask logical sequence and each of the target meta-skills through the robot control macro model.

[0079] S350: In response to a task editing operation input by a user through a preset editing interface, the subtask code list is updated according to the task editing operation, and the updated subtask code list is determined.

[0080] Among them, the task editing operation can be an operation for the user to edit the subtask code list; the task editing operation can be used to modify the subtask list. The task editing operation includes the subtask logical sequence editing operation, the robot subtask list editing operation and the meta-skill parameter editing operation.

[0081] Optionally, the robot control large model can understand the user's needs for the industrial robot and output a subtask code list. When the user receives the subtask code list output by the robot control large model, the user can view the subtask code list. If the user believes that the subtask code list does not meet the needs, the user has the authority to adjust the subtask code list and can adjust the subtask code list.

[0082] The editing interface may be used to receive the user's editing operation. Optionally, the robot control model also provides the user with an editing interface, through which the user can connect to the robot control model, and the user inputs the editing operation through the editing interface.

[0083] Optionally, the present invention further provides a user editing interface, which can display the subtask code list output by the robot control large model after the robot control large model outputs the subtask code list. The user editing interface is connected to the robot control large model through the editing interface. The user can input editing operations on the subtask code list through the user editing interface, and the robot control large model receives the editing operations through the editing interface.

[0084] Optionally, the subtask logic sequence editing operation may be an operation of editing and adjusting the subtask execution logic of the subtask code list. It should be noted that the user views the subtask code list in the user editing interface, and the user editing interface displays each subtask code and subtask execution logic. The user can modify the execution order and dependency relationship between the subtask codes; the user can also modify the logical order of the subtask execution logic in the subtask execution logic, thereby completing the subtask logic sequence editing operation. Among them, the modification operation may be to drag the arrangement order of the subtask code.

[0085] Optionally, the robot subtask list editing operation can be an operation of editing and adjusting the subtask code list. The robot subtask list editing operation can be used to add new subtask codes or delete subtask codes in the subtask code list. It should be noted that the user views the subtask code list in the user editing interface. The user editing interface displays each subtask code and subtask execution logic. The user can choose to directly delete any subtask code and delete the execution sequence and dependency relationship of the subtask code in the subtask execution logic. The user can also choose to add a new subtask code and add the execution sequence and dependency relationship of the subtask code in the subtask execution logic.

[0086] Optionally, the meta-skill parameter editing operation may be an operation for editing and adjusting the meta-skill parameters in the sub-task code; it should be noted that the user views the sub-task code list in the user editing interface, which displays each sub-task code and sub-task execution logic. The user can select the code corresponding to the meta-skill parameter in a sub-task code, and edit the meta-skill parameter in the sub-task code by directly modifying the code corresponding to the meta-skill parameter.

[0087] Optionally, when the user performs a task editing operation in the user editing interface, the subtask code list will respond to the task editing operation input through the editing interface, and update the subtask code and subtask execution logic of the subtask code list accordingly to obtain an updated subtask code list, and when the user selects a save operation in the user editing interface, the updated subtask code list is saved to obtain a new subtask code list. The user editing interface provides a save button, and the user can implement the save operation by triggering the save button.

[0088] Optionally, when the subtask code list is updated in the user editing interface, the user editing interface provides different update methods; one update method may be: in the user editing interface, when the user performs a task editing operation on the subtask code list, the user editing interface can automatically read each task editing operation, the subtask code list is synchronously updated and automatically saved through the editing interface, and the updated subtask code list is synchronized in the user editing interface, and each synchronized updated subtask code list is used as a new subtask code list; another update method may be: in the user editing interface, after the user performs a task editing operation on the subtask code list, the user editing interface records the user's task editing operation, and displays a preview result of the subtask code list after the task editing operation is performed, the subtask code list is not modified or updated, and after the user completes all task editing operations in the user editing interface and triggers the save button, the user editing interface sends all task editing operations to the subtask code list through the editing interface, and the subtask code list responds to all task editing operations to obtain a new subtask code list.

[0089] S360. Obtain industrial operation information of the target robot, generate a target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically control the target robot through the target control logic.

[0090] Specifically, after the user updates the subtask code list, the industrial operation information of the target robot is obtained, and the target control logic corresponding to the target robot is generated according to the industrial operation information and the updated subtask code list, and the target robot is automatically controlled through the target control logic.

[0091] The technical solution of the embodiment of the present invention can provide users with the authority to customize the subtask code list after the robot control large model outputs the subtask code list. The user can modify the subtask code according to their own needs, and the intelligence and flexibility of the large model can be used to ensure the accuracy and stability of the industrial robot when performing tasks. The technical problem that the control method of industrial robots in the prior art cannot adapt to complex industrial scenes is solved, and at the same time, it has high flexibility and applicability, can effectively reduce the user's operation threshold, and improve the control efficiency of the robot.

[0092] Figure 4 This is a flowchart of another robot automatic control method provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is a specific method for constructing a target control logic for a target robot. Figure 4 As shown, the method includes:

[0093] S410: Receive multimodal interaction information input by a user.

[0094] S420: parsing the multimodal interaction information through the robot control large model to determine a robot subtask list.

[0095] S430: For each of the robot subtasks, perform meta-skill matching on the robot subtask according to a preset meta-skill description text through the robot control macromodel to determine the target meta-skill matched by the robot subtask.

[0096] S440. Determine the subtask code list according to the subtask logical sequence and each of the target meta-skills through the robot control macro model.

[0097] S450: In response to a task editing operation input by a user through a preset editing interface, the subtask code list is updated according to the task editing operation, and the updated subtask code list is determined.

[0098] S460. Obtain the industrial operation information of the target robot, call each of the subtask codes in the subtask code list in sequence according to the subtask execution logic, and determine the target task sequence; construct the target control logic of the target robot according to the target task sequence and the industrial operation information; and automatically control the target robot through the target control logic.

[0099] Among them, the target task sequence can be obtained by connecting various subtask codes to obtain a task sequence.

[0100] Optionally, each subtask code in the subtask code list corresponds to its own robot subtask. When each subtask code is called in sequence based on the subtask execution logic, for one subtask code: establish a connection relationship between the subtask code and the previous subtask code in the subtask execution logic, pass the output of the previous subtask code as the input of the subtask code, and pass the output of the subtask code as the input of the next subtask code in the subtask execution logic; thereby connecting each subtask code in the subtask code list to construct a target task sequence corresponding to the subtask code list.

[0101] Optionally, after obtaining the target task sequence, the industrial scene corresponding to the target robot is determined according to the industrial operation information of the target robot, and the target task sequence is combined with the industrial operation information in the industrial scene to construct the target control logic corresponding to the target robot. Exemplarily, corresponding to the target robot, the initial state of the target robot, the sensor state information of the target robot, the operating range and capability of the target robot in the industrial scene are set based on the industrial operation information, and the end tool of the target robot is adjusted. Based on the task cycle of the target robot, the subtask codes in the target task sequence are embedded in the loop structure in sequence, and the termination condition is set in the loop structure. When the target robot performs a cyclic operation each time, it checks whether the termination condition of the task is met. If the termination condition of the operation is met, the task is terminated. If the termination condition of the task is not met, the cyclic operation continues until the termination condition of the task is met.

[0102] Specifically, the industrial operation information of the target robot is obtained, and the subtask codes corresponding to each robot subtask in the subtask code list are called in sequence according to the subtask execution logic to obtain the target task sequence, and the target control logic of the target robot is constructed according to the target task sequence and the industrial operation information, and the target robot is automatically controlled through the target control logic.

[0103] The technical solution of the embodiment of the present invention receives multimodal interaction information input by the user, parses the multimodal interaction information through a pre-trained robot control large model, determines the subtask code list corresponding to the multimodal interaction information, and can parse the multimodal interaction information through the robot control large model to automatically generate a subtask code list, effectively improving the flexibility and adaptability of robot control and lowering the user's operation threshold; obtains the industrial operation information of the target robot, generates the target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically controls the target robot through the target control logic. The industrial operation information can ensure the accuracy and stability of the operation when executing the task on the basis of intelligence and flexibility, and meet the high requirements for program execution in industrial scenarios. The technical problem that the control method of industrial robots in the prior art cannot adapt to complex industrial scenarios is solved, and at the same time, it has high flexibility and applicability, can effectively lower the user's operation threshold, and improve the beneficial effect of robot control efficiency.

[0104] Figure 5 The following is a schematic diagram of the structure of a robot automatic control device provided by an embodiment of the present invention. Figure 5 As shown, the device includes: a large model module 510 and a robot control module 520; wherein,

[0105] The large model module 510 is used to receive multimodal interaction information input by a user, parse the multimodal interaction information through a pre-trained robot control large model, and determine a subtask code list corresponding to the multimodal interaction information;

[0106] The robot control module 520 is used to obtain the industrial operation information of the target robot, generate the target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically control the target robot through the target control logic.

[0107] The technical solution of the embodiment of the present invention receives multimodal interaction information input by the user, parses the multimodal interaction information through a pre-trained robot control large model, determines the subtask code list corresponding to the multimodal interaction information, and can parse the multimodal interaction information through the robot control large model to automatically generate a subtask code list, effectively improving the flexibility and adaptability of robot control and lowering the user's operation threshold; obtains the industrial operation information of the target robot, generates the target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically controls the target robot through the target control logic. The industrial operation information can ensure the accuracy and stability of the operation when executing the task on the basis of intelligence and flexibility, and meet the high requirements for program execution in industrial scenarios. The technical problem that the control method of industrial robots in the prior art cannot adapt to complex industrial scenarios is solved, and at the same time, it has high flexibility and applicability, can effectively lower the user's operation threshold, and improve the beneficial effect of robot control efficiency.

[0108] Optionally, the large model module 510 is specifically used for:

[0109] The multimodal interaction information is parsed through the robot control macro model to determine a robot subtask list; wherein the robot subtask list includes robot subtasks and subtask logical sequences; the robot subtasks include subtask conditions, subtask goals, and subtask execution constraints;

[0110] For each of the robot subtasks, meta-skill matching is performed on the robot subtasks according to a preset meta-skill description text through the robot control macro model to determine a target meta-skill matched to the robot subtask; wherein the target meta-skill includes a robot meta-skill and a meta-skill parameter;

[0111] The robot control model determines the subtask code list according to the subtask logical sequence and each target meta-skill.

[0112] Optionally, the large model module 510 is specifically used for:

[0113] The multimodal interactive information includes robot task requirements and robot task execution conditions;

[0114] Analyze the robot task requirements through the robot control macro model to determine the robot task objectives;

[0115] The robot control macromodel is used to perform task decomposition according to the robot task conditions and the robot task objectives, and the robot subtask list is determined.

[0116] Optionally, the large model module 510 is further used for:

[0117] Performing meta-skill matching with the subtask conditions and subtask goals of the robot subtask through the robot control macromodel according to a preset meta-skill description text, and determining the robot meta-skill that matches the robot subtask;

[0118] The robot control macromodel is used to set parameters of the robot meta-skill according to the sub-task execution constraints of the robot sub-task, and the meta-skill parameters corresponding to the robot meta-skill are determined.

[0119] Optionally, the large model module 510 is further used for:

[0120] The robot control macromodel performs meta-skill calling in a preset meta-skill library according to each of the target meta-skills, and determines a sub-task code corresponding to each of the target meta-skills;

[0121] The robot control macromodel is used to combine the subtask codes corresponding to each of the robot subtasks according to the subtask logical sequence to determine the subtask code list.

[0122] Optionally, the device further includes a task editing module;

[0123] A task editing module is used to respond to a task editing operation input by a user through a preset editing interface, update the subtask code list according to the task editing operation, and determine the updated subtask code list; wherein the task editing operation includes a subtask logical sequence editing operation, a robot subtask list editing operation, and a meta-skill parameter editing operation.

[0124] Optionally, the robot control module 520 is specifically used for:

[0125] Call each of the subtask codes in the subtask code list in sequence according to the subtask execution logic to determine a target task sequence;

[0126] The target control logic of the target robot is constructed according to the target task sequence and the industrial operation information.

[0127] The robot automatic control device provided in the embodiment of the present invention can execute the robot automatic control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0128] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their modes are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0129] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0130] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer grid such as the Internet and / or various telecommunication grids.

[0131] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a robot automatic control method.

[0132] In some embodiments, the robot automatic control method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the robot automatic control method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the robot automatic control method in any other appropriate manner (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the modes / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present invention, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. More specific examples of machine readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a grid browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication grid). Examples of communication grids include: a local area network (LAN), a wide area network (WAN), a blockchain grid, and the Internet.

[0138] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication grid. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0139] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0140] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the robot automatic control method provided in any embodiment of the present invention are implemented. The method includes:

[0141] Receive multimodal interaction information input by a user, parse the multimodal interaction information through a pre-trained robot control large model, and determine a subtask code list corresponding to the multimodal interaction information; wherein the subtask code list includes at least one subtask code and a subtask execution logic;

[0142] The industrial operation information of the target robot is obtained, the target control logic corresponding to the target robot is generated according to the industrial operation information and the subtask code list, and the target robot is automatically controlled by the target control logic.

[0143] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0144] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0146] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0147] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a grid composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0149] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A robot automatic control method, characterized in that ,include: Receive multimodal interaction information input by a user, parse the multimodal interaction information through a pre-trained robot control model, and determine a subtask code list corresponding to the multimodal interaction information; wherein the subtask code list includes at least one subtask code and a subtask execution logic; The industrial operation information of the target robot is obtained, the target control logic corresponding to the target robot is generated according to the industrial operation information and the subtask code list, and the target robot is automatically controlled by the target control logic.

2. The method according to claim 1, characterized in that , the multimodal interaction information is parsed by a pre-trained robot control model to determine the robot task function data corresponding to the multimodal interaction information, including: The multimodal interaction information is parsed through the robot control macro model to determine a robot subtask list; wherein the robot subtask list includes robot subtasks and subtask logical sequences; the robot subtasks include subtask conditions, subtask goals, and subtask execution constraints; For each of the robot subtasks, meta-skill matching is performed on the robot subtasks according to a preset meta-skill description text through the robot control macro model to determine a target meta-skill matched to the robot subtask; wherein the target meta-skill includes a robot meta-skill and a meta-skill parameter; The robot control model determines the subtask code list according to the subtask logical sequence and each target meta-skill.

3. The method according to claim 2, characterized in that ,The multimodal interaction information includes robot task requirements and robot task execution conditions; The multimodal interaction information is parsed by the robot control macro model to determine a robot subtask list, including: Analyze the robot task requirements through the robot control macro model to determine the robot task objectives; The robot control macromodel is used to perform task decomposition according to the robot task conditions and the robot task objectives, and the robot subtask list is determined.

4. The method according to claim 2, characterized in that , the robot control macromodel performs meta-skill matching on the robot sub-task according to a preset meta-skill description text, and determines the target meta-skill matched by the robot sub-task, including: Performing meta-skill matching with the subtask conditions and subtask goals of the robot subtask through the robot control macromodel according to a preset meta-skill description text, and determining the robot meta-skill that matches the robot subtask; The robot control macromodel is used to set parameters of the robot meta-skill according to the sub-task execution constraints of the robot sub-task, and the meta-skill parameters corresponding to the robot meta-skill are determined.

5. The method according to claim 4, characterized in that , through the robot control model according to the subtask logical sequence and each of the target meta-skills to determine the subtask code list, including: The robot control macromodel performs meta-skill calling in a preset meta-skill library according to each of the target meta-skills, and determines a sub-task code corresponding to each of the target meta-skills; The robot control macromodel is used to combine the subtask codes corresponding to each of the robot subtasks according to the subtask logical sequence to determine the subtask code list.

6. The method according to claim 1, characterized in that , before generating the target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, it also includes: In response to a task editing operation input by a user through a preset editing interface, the subtask code list is updated according to the task editing operation to determine the updated subtask code list; wherein the task editing operation includes a subtask logical sequence editing operation, a robot subtask list editing operation and a meta-skill parameter editing operation.

7. The method according to claim 1, characterized in that , the target control logic corresponding to the target robot is generated according to the industrial operation information and the subtask code list, including: Call each of the subtask codes in the subtask code list in sequence according to the subtask execution logic to determine a target task sequence; The target control logic of the target robot is constructed according to the target task sequence and the industrial operation information.

8. A robot automatic control device, characterized in that ,include: The large model module is used to receive multimodal interaction information input by the user, parse the multimodal interaction information through the pre-trained robot control large model, and determine the subtask code list corresponding to the multimodal interaction information; The robot control module is used to obtain industrial operation information of the target robot, generate target control logic corresponding to the target robot according to the industrial operation information and the subtask code list, and automatically control the target robot through the target control logic.

9. An electronic device, characterized in that , the electronic device comprises: at least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the robot automatic control method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the robot automatic control method described in any one of claims 1-4 when executed.

Images