Robot control methods and devices

Abstract

This application discloses a robot control method and apparatus, comprising: acquiring a preset training set and a preset large-scale language model, wherein the preset training set includes multiple training texts and corresponding instruction text sets; fine-tuning the preset large-scale language model based on the preset training set to obtain a fine-tuned large-scale language model; upon acquiring a text to be processed, parsing the text based on the fine-tuned large-scale language model to obtain a target instruction text set corresponding to the text to be processed; and controlling the robot based on the instruction text in the target instruction text set. This application can improve the accuracy of robot control.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a robot control method and device. Background Technology

[0002] With the increasing application of intelligent robots in indoor home settings, the demands for their intelligence are rising, including intelligent navigation, intelligent companionship, and intelligent services. This has led to the frequent use of pre-trained natural language processing (NLP) models in these environments. However, because pre-trained NLP models are comprehensive performance models, they exhibit uncertainty or even fail to respond to specific tasks. Currently, robots are primarily driven by machine control commands, which are mainly recognized using conventional models, resulting in low accuracy in robot control.

[0003] In other words, the accuracy of robot control in existing technologies is relatively low. Summary of the Invention

[0004] This application provides a robot control method and apparatus, which can improve the accuracy of robot control.

[0005] Firstly, the robot control method provided in this application includes:

[0006] Obtain a preset training set and a preset large-scale language model, wherein the preset training set includes multiple training texts and a set of instruction texts corresponding to the training texts;

[0007] The preset large-scale language model is fine-tuned based on the preset training set to obtain a fine-tuned large-scale language model.

[0008] Given the text to be processed, the text to be processed is parsed based on the fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed.

[0009] The robot is controlled based on the instruction text in the target instruction text set.

[0010] Secondly, the robot control device provided in this application includes:

[0011] The acquisition module is used to acquire a preset training set and a preset large-scale language model, wherein the preset training set includes multiple training texts and a set of instruction texts corresponding to the training texts;

[0012] The fine-tuning module is used to fine-tune the preset large-scale language model based on the preset training set to obtain a fine-tuned large-scale language model.

[0013] The parsing module is used to parse the text to be processed based on the fine-tuned large-scale language model when the text to be processed is obtained, so as to obtain the target instruction text set corresponding to the text to be processed.

[0014] The control module is used to control the robot based on the instruction text in the target instruction text set.

[0015] Thirdly, the electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and the processor is used to run the computer program in the memory to implement the steps in the robot control method provided in this application.

[0016] Fourthly, the computer-readable storage medium provided in this application stores multiple instructions that are adapted for loading by a processor to implement the steps in the robot control method provided in this application.

[0017] Fifthly, the computer program product provided in this application includes a computer program or instructions that, when executed by a processor, implement the steps in the robot control method provided in this application.

[0018] In this application, compared to related technologies, a preset training set and a preset large-scale language model are obtained. The preset training set includes multiple training texts and corresponding instruction text sets. The preset large-scale language model is fine-tuned based on the preset training set to obtain a fine-tuned large-scale language model. When a text to be processed is obtained, the text is parsed based on the fine-tuned large-scale language model to obtain a target instruction text set corresponding to the text to be processed. The robot is controlled based on the instruction text in the target instruction text set. This application constructs training corpora and corresponding instruction text sets, and then uses the training corpora and corresponding instruction text sets to fine-tune an existing large-scale language model. The resulting fine-tuned large-scale language model can accurately process the text to be processed into an instruction text set that the robot can understand and execute. When a text to be processed is obtained, the fine-tuned large-scale language model is used to process the text to be processed into an instruction text set that the robot can understand and execute, thereby controlling the robot. Because the fine-tuned large-scale language model can accurately process the text to be processed into an instruction text set that the robot can understand and execute, the accuracy of robot control can be improved. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a scenario for the robot control system provided in an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating one embodiment of the robot control method provided in this application.

[0022] Figure 3 This is a schematic diagram of the training text and corresponding instruction text set in one embodiment of the robot control method provided in this application;

[0023] Figure 4 This is a schematic diagram of the Adapter module in one embodiment of the robot control method provided in this application;

[0024] Figure 5 This is a schematic diagram of a fine-tuning method based on prompt word prefix optimization in one embodiment of the robot control method provided in this application.

[0025] Figure 6 This is a schematic diagram of the Prompt-tuning fine-tuning algorithm in one embodiment of the robot control method provided in this application.

[0026] Figure 7 This is a schematic diagram of the P-tuning fine-tuning algorithm in one embodiment of the robot control method provided in this application;

[0027] Figure 8 This is a schematic diagram of adding a bypass module and a fusion module to a preset large-scale language model in one embodiment of the robot control method provided in this application.

[0028] Figure 9 This is a schematic diagram of a pre-constructed prompt template in one embodiment of the robot control method provided in this application;

[0029] Figure 10 This is a flowchart illustrating another embodiment of the robot control method provided in this application.

[0030] Figure 11 This is a schematic diagram of the structure of one embodiment of the robot control device provided in this application;

[0031] Figure 12 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0032] It should be noted that the principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application that are illustrated, and should not be regarded as limiting other specific embodiments not detailed herein.

[0033] In the following description of this application, "some embodiments" are referred to, which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments, and may be combined with each other without conflict.

[0034] In the following description of this application, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0036] To improve the efficiency of updating metrics in a global federated model, embodiments of this application provide a robot control method, a robot control device, an electronic device, a computer-readable storage medium, and a computer program product. The robot control method can be executed by the robot control device or by an electronic device integrating the robot control device.

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] Please refer to Figure 1 This application also provides a control system for a robot, such as Figure 1 As shown, the robot's control system includes an electronic device 100 and a robot 300 connected to the electronic device 100. The electronic device 100 integrates the robot control device provided in this application.

[0039] Among them, electronic device 100 can be any device equipped with a processor and having processing capabilities, such as mobile electronic devices with processors such as smartphones, tablets, PDAs, laptops, and smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, and industrial equipment.

[0040] In addition, the robot's control system may also include a memory 200 for storing raw data, intermediate data, and result data.

[0041] In this embodiment of the application, the storage device can be a cloud storage device. Cloud storage is a new concept that is extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file system functions to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) through application software or application interfaces to work together to provide data storage and business access functions to the outside world.

[0042] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0043] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0044] It should be noted that, Figure 1The schematic diagram of the robot control system shown is merely an example. The robot control system and scenario described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of robot control systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0045] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0046] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating one embodiment of the robot control method provided in this application, as shown below. Figure 2 As shown, the flow of the robot control method provided in this application is as follows:

[0047] 201. Obtain the preset training set and preset large-scale language model.

[0048] The preset training set includes multiple training texts and corresponding instruction text sets. The instruction text set contains multiple instruction texts. The training texts are high-level human language texts. The instruction texts are low-level language execution instructions that can be used to control the robot.

[0049] The pre-defined training set consists of corpus data of robots performing tasks in indoor home scenarios. Since there are virtually no publicly available datasets for corpus data on robots performing tasks in indoor home scenarios, it is necessary to design and construct a custom corpus dataset to obtain the pre-defined training set. This application constructs six indoor home scene environments and 12 major categories of indoor home corpus types, including: leisure, work, room cleaning, room arrangement, food preparation, hygiene, social interaction, diet, activities, sleep, and preparation work, resulting in nearly 2700 corpus data entries.

[0050] See Figure 3 The training text can be "Read book", and the corresponding instruction text set includes multiple instruction texts. For example, the instruction text set might contain multiple instruction texts such as "1.{walk}".<living_room> “2.{walk}” <book>”、"3.{grab} <book>”、"4.{walk} <sofa>”、"5.{lie} <sofa>”、"6.{read} <book>More training texts and corresponding instruction text sets can be found in [reference needed]. Figure 3 For example, the training text could be "Drink water" or "Cookbread", which will not be elaborated further here.

[0051] 202. Fine-tune the preset large-scale language model based on the preset training set to obtain the fine-tuned large-scale language model.

[0052] The default large-scale language model is Chatglm-6b. The GLM model is a framework developed by Tsinghua University. Using Chatglm-6b, which is a modification of the Transformer model, it employs a two-way approach of autoencoder and autoregression, transforming the problem into a cloze test. It has 6.2 billion parameters and uses a 1TB bilingual corpus as its training dataset to optimize Chinese question-and-answer and dialogue. Currently, this model can generate answers that conform to human preferences. Of course, in other embodiments, the default large-scale language model can also be a ChatGPT series or LLama series model, depending on the specific circumstances.

[0053] Model fine-tuning improves accuracy by initializing the weights of your own model using a pre-trained model obtained from large datasets. This requires that the quality of the pre-trained model be guaranteed. Fine-tuning is typically faster and more accurate. Of course, you can also use your own pre-trained model as a pre-trained model and then train it on your own dataset to adapt the model to your specific scenario and task.

[0054] In one specific embodiment, a preset large-scale language model can be directly trained using a preset training set to obtain a fine-tuned large-scale language model.

[0055] However, training a large model for a specific domain using full parameter tuning requires both a large amount of high-quality dataset and significant computing power, sometimes even hundreds or thousands of GPUs.

[0056] To overcome the above problems, in another specific embodiment, fine-tuning a pre-set large-scale language model based on a pre-set training set to obtain a fine-tuned large-scale language model includes: performing parameter-efficient fine-tuning on the pre-set large-scale language model based on the pre-set training set to obtain a fine-tuned large-scale language model. Parameter-efficient fine-tuning (PEFT) aims to solve this problem by training only a small number of parameters, which may be a subset of the existing model parameters or a newly added set of parameters. These methods differ in terms of parameter efficiency, memory efficiency, training speed, the final quality of the model, and additional inference costs.

[0057] In a specific embodiment, performing parameter-efficient fine-tuning on a pre-set large-scale language model based on a pre-set training set to obtain a fine-tuned large-scale language model includes:

[0058] (1) Adding an Adapter module to each network layer of the pre-set large-scale language model to obtain a first language model.

[0059] As Figure 4 shown, Figure 4 in, Adapter Layer is the Adapter module, and Transformer Layer is the network layer in the pre-set large-scale language model. In the embodiments of the present application, for each Transformer layer of the pre-set large-scale language model, an Adapter module is added. The Adapter module includes two Adapter structures, namely, after the projection of the multi-head attention and after the second feed-forward layer. During training, the parameters of the pre-set large-scale language model are fixed, and only the newly added Adapter structures are fine-tuned, thus ensuring the efficiency of training. Whenever a new downstream task appears, a downstream model that is easy to expand is generated by adding an Adapter module, thereby avoiding the problems of full-scale fine-tuning and catastrophic forgetting.

[0060] Each Adapter module is mainly composed of two feed-forward sub-layers. The first feed-forward sub-layer (down-project) takes the output of the Transformer block as input, projects the original input dimension d (high-dimensional features) to m (low-dimensional features), and restricts the number of parameters of the Adapter module by controlling the size of m. Usually, m << d. Then, there is a non-linear layer in the middle. At the output stage, the input dimension is restored through the second feed-forward sub-layer (up-project), and m (low-dimensional features) is remapped back to d (the original high-dimensional features) as the output of the Adapter module.

[0061] (2) Train the first language model based on the preset training set to obtain the fine-tuned large-scale language model. The parameters of the preset large-scale language model in the first language model are not updated during training, while the parameters of the Adapter module in the first language model are updated during training.

[0062] In another specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including: using a fine-tuning algorithm optimized by prompt word prefixes to fine-tune the preset large-scale language model based on a preset training set to obtain a fine-tuned large-scale language model.

[0063] Prefix-Tuning is a fine-tuning method based on prompt word prefix optimization. Its principle is to construct a set of task-related virtual tokens as a prefix before the input token, and then only update the parameters of the prefix part during training, while the parameters of other parts in the large-scale language model are fixed.

[0064] like Figure 5 As shown, the task input is a linearized table (e.g., "name:Starbucks|type:coffeeshop"), and the output is a text description (e.g., "Starbucks serves coffee."). The red portion in the lower left of the diagram represents a series of consecutive task-specific vectors represented by prefixes, which are also involved in the attention calculation, similar to virtual tokens. Fine-tuning updates all Transformer parameters, so a copy of the fine-tuned model weights must be saved for each task. Prefix Tuning, on the other hand, only updates the parameters of the prefix portion, so different tasks only need to save different prefixes, resulting in lower fine-tuning costs.

[0065] In another specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including: using the Prompt-tuning fine-tuning algorithm to fine-tune the preset large-scale language model based on a preset training set to obtain a fine-tuned large-scale language model.

[0066] like Figure 6 As shown, Prompt-tuning doesn't manually set prompts. Instead, it learns the prompts by freezing the model's original weights and training only the prompt parameters, updating the parameters through backpropagation. Prompt Tuning can be seen as a simplified version of Prefix Tuning. It defines its own prompt for each task and concatenates it to the data as input, but only adds prompt tokens to the input layer and doesn't require adjusting the original model to address training difficulties. The diagram illustrates the difference between traditional fine-tuning and Prompt Tuning: Model Tuning: Each downstream task requires customized fine-tuning, maintaining different copies of the pre-trained model, and inference must be performed in separate batches. Prompt Tuning: Only a small task-specific prompt needs to be stored for each task, allowing for mixed-task inference using the original pre-trained model.

[0067] In another specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including: using the P-tunin fine-tuning algorithm to fine-tune the preset large-scale language model based on a preset training set to obtain a fine-tuned large-scale language model.

[0068] like Figure 7 As shown, P-tuning is divided into two versions, V1 and V2. P-tuningV1 converts the prompt into a learnable embedding layer, that is, it fixes the original parameters of the large model, uses a multilayer perceptron and a two-layer LSTM to encode the prompt, and then concatenates it with other vectors before inputting it into the large LLM model. P-tuningV2 adds a layer prompt to each layer of the model, so that different tasks can share the same network parameters and support multi-task learning.

[0069] In another specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including:

[0070] (1) Add a bypass module and a fusion module to the preset large-scale language model to obtain a second language model.

[0071] (2) Train the second language model based on the preset training set to obtain the fine-tuned large-scale language model. The parameters of the preset large-scale language model in the second language model are not updated during training, while the parameters of the bypass module in the second language model are updated during training.

[0072] Specifically, a bypass module and a fusion module are added to a preset large-scale language model; the bypass module is used to process the text to be processed to obtain bypass output data; the preset large-scale language model is used to process the text to be processed to obtain model output data; and the fusion module is used to fuse the bypass output data and model output data to obtain at least one instruction text.

[0073] Specifically, the bypass module includes a dimensionality reduction matrix and an dimensionality increase matrix. The dimensionality reduction matrix is ​​used to process the text to be processed to obtain dimensionality reduction data; the dimensionality increase matrix is ​​used to process the dimensionality increase data to obtain bypass output data.

[0074] like Figure 8 As shown, Figure 8 The left side of the middle section represents the weights of the pre-defined large-scale language model, with both input and output dimensions being d; x represents the processed text to be processed, with dimension d; W0 represents the weights of the pre-defined large-scale language model, which can be decomposed using low-rank decomposition, as shown in formula (1). A and B are decomposition matrices, where A is the dimension reduction matrix and B is the dimension increase matrix. During training, the weights W0 are frozen and cannot be updated with gradients, but the dimension reduction matrix A and the dimension increase matrix B contain trainable parameters, and their forward propagation formula is shown in formula (2).

[0075] W0+ΔW=W0+BA (1)

[0076] h=W0x+ΔWx=W0x+BAx (2)

[0077] In this model, the reduced-dimensional matrix A is initialized using a random Gaussian distribution, while the increased-dimensional matrix B is initialized with zero values. Therefore, in the early stages of model training, ΔW = BA is zero. r is the rank of the matrix, which scales ΔWx by α / r, where α is a constant in r.

[0078] In one specific embodiment, a preset training set is first obtained, which is in JSON format. Then, the preset training set is preprocessed to convert the JSON data into JSONL data format. At the same time, each training text in the preset training set is segmented and the data is saved. Then, the chatGLM-6b large model is fine-tuned by setting training parameters, saving the training model, and obtaining a fine-tuned large-scale language model.

[0079] 203. Given the text to be processed, parse the text based on a fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed.

[0080] In one specific embodiment, after obtaining the text to be processed, the text to be processed is input into a fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed.

[0081] In another specific embodiment, upon obtaining the text to be processed, the text is parsed based on a fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed, including:

[0082] (1) Obtain a pre-constructed prompt template, which is used to instruct the text to be processed to be split into at least one instruction.

[0083] A prompt template, or Prompt, is used in machine learning and natural language processing to guide the model in generating text or training language models. A Prompt can be a question, a topic, a passage of text, or a set of keywords to direct the model to generate relevant text. For example, in text generation tasks, a Prompt can be used to specify the topic or style of the generated text, ensuring it meets the requirements. In language model training, a Prompt can be part of the input sequence, specifying the language rules and contextual information the model needs to learn.

[0084] Among them, the pre-constructed prompt templates are as follows: Figure 9 As shown, where, Figure 9 The text to be processed is "take a bottle of milk from the refrigerator", and LLM is for fine-tuning a large-scale language model.

[0085] (2) Input the prompt template and the text to be processed into the large-scale language model to obtain the target instruction text set corresponding to the text to be processed.

[0086] 204. Robot controlled by instruction text based on target instruction text set.

[0087] In one specific embodiment, each instruction text in the target instruction text set is input into the robot to control the robot to execute each instruction text.

[0088] In another specific embodiment, controlling the robot based on instruction text in a target instruction text set includes: converting instruction text in the target instruction text set into corresponding robot control instructions based on a preset mapping relationship, and inputting the robot control instructions into the robot to control the robot.

[0089] For example, the target instruction text set includes 6 instruction texts, which are: "1.{find} <milk>”、"2.{walk} <milk>”、"3.{find} <fridge>”、"4.{open} <fridge>”、"5.{grab} <milk>”、"6.{open} <fridge>Based on a preset mapping relationship, the six instruction texts in the target instruction text set are converted into six corresponding robot control instructions: Move_hand(); Grasp_object(); Release_object(); Move_object(); Attatch_to_plane(); and Open_by_slide(). These six robot control instructions are then input into the robot to control it.

[0090] Please refer to Figure 10 , Figure 10 This is a flowchart illustrating another embodiment of the robot control method provided in this application, as shown below. Figure 10 As shown, the flow of the robot control method provided in this application is as follows:

[0091] 301. Obtain the preset training set and preset large-scale language model.

[0092] The preset training set includes multiple training texts and corresponding instruction text sets. The instruction text set contains multiple instruction texts. The training texts are high-level human language texts. The instruction texts are low-level language execution instructions that can be used to control the robot.

[0093] 302. Based on the preset training set, perform efficient parameter fine-tuning on the preset large-scale language model to obtain the fine-tuned large-scale language model.

[0094] In one specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including:

[0095] (1) Add an Adapter module to each network layer of the pre-set large-scale language model to obtain the first language model.

[0096] (2) Train the first language model based on the preset training set to obtain the fine-tuned large-scale language model. The parameters of the preset large-scale language model in the first language model are not updated during training, while the parameters of the Adapter module in the first language model are updated during training.

[0097] In another specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including: using a fine-tuning algorithm optimized by prompt word prefixes to fine-tune the preset large-scale language model based on a preset training set to obtain a fine-tuned large-scale language model.

[0098] In another specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including: using the Prompt-tuning fine-tuning algorithm to fine-tune the preset large-scale language model based on a preset training set to obtain a fine-tuned large-scale language model.

[0099] In another specific embodiment, the parameters of a preset large-scale language model are efficiently fine-tuned based on a preset training set to obtain a fine-tuned large-scale language model, including: using the P-tunin fine-tuning algorithm to fine-tune the preset large-scale language model based on a preset training set to obtain a fine-tuned large-scale language model.

[0100] In another specific embodiment, a pre-set large-scale language model is efficiently fine-tuned based on a pre-set training set to obtain a fine-tuned large-scale language model. This includes adding a bypass module and a fusion module to the pre-set large-scale language model to obtain a second language model. The second language model is then trained based on the pre-set training set to obtain the fine-tuned large-scale language model. In this second language model, the parameters of the pre-set large-scale language model are not updated during training, while the parameters of the bypass module in the second language model are updated during training.

[0101] 303. If the text to be processed is obtained, obtain a pre-constructed prompt template, which is used to indicate that the text to be processed is split into at least one instruction text.

[0102] A prompt template, or Prompt, is used in machine learning and natural language processing to guide the model in generating text or training language models. A Prompt can be a question, a topic, a passage of text, or a set of keywords to direct the model to generate relevant text. For example, in text generation tasks, a Prompt can be used to specify the topic or style of the generated text, ensuring it meets the requirements. In language model training, a Prompt can be part of the input sequence, specifying the language rules and contextual information the model needs to learn.

[0103] Among them, the pre-constructed prompt templates are as follows: Figure 9 As shown, where, Figure 9 The text to be processed is "take a bottle of milk from the refrigerator", and LLM is for fine-tuning a large-scale language model.

[0104] 304. Input the prompt template and the text to be processed into the large-scale language model to obtain the target instruction text set corresponding to the text to be processed.

[0105] For example, if the text to be processed is "Get a bottle of milk from the refrigerator", the text to be processed is filled into the prompt template. The text to be processed and the prompt template are then input into a fine-tuning large-scale language model to obtain the target instruction text set. The target instruction text set includes 6 instruction texts, namely: "1.{find} <milk>”、"2.{walk} <milk>”、"3.{find} <fridge>”、"4.{open} <fridge>”、"5.{grab} <milk>”、"6.{open} <fridge>".

[0106] 305. Based on the preset mapping relationship, convert the instruction text in the target instruction text set into the corresponding robot control instructions.

[0107] For example, the target instruction text set includes 6 instruction texts, which are: "1.{find} <milk>”、"2.{walk} <milk>”、"3.{find} <fridge>”、"4.{open} <fridge>”、"5.{grab} <milk>”、"6.{open} <fridge>Based on a preset mapping relationship, the six instruction texts in the target instruction text set are converted into six corresponding robot control instructions: Move_hand(); Grasp_object(); Release_object(); Move_object(); Attatch_to_plane(); and Open_by_slide(). These six robot control instructions are then input into the robot to control it.

[0108] 306. Input robot control commands into the robot to control the robot.

[0109] This application enables integrated indoor intelligence. First, training text for indoor home scenarios is constructed. Second, the parameters of a large language model are efficiently fine-tuned to make the model an "expert" in a specific domain. Finally, prompt templates are constructed to further improve the model's accuracy, thereby efficiently achieving specific sub-tasks. This solution converts high-level language text into low-level language instructions that the robot can understand, which are then executed in conjunction with the robot's control module to enable the robot to possess its own capabilities. This application also provides greater intelligence possibilities for smart home robots, allowing for closer interaction between intelligent robots and humans. It extends the natural language model beyond simply generating chat-like dialogues, enabling systems that directly drive robots through text to achieve embodied intelligence.

[0110] To facilitate better implementation of the robot control method provided in the embodiments of this application, the embodiments of this application also provide a robot control device based on the above-described robot control method. The meanings of the terms used are the same as in the robot control method described above, and for specific implementation details, please refer to the descriptions in the above method embodiments.

[0111] Please refer to Figure 11 , Figure 11 This is a schematic diagram of the structure of the robot control device provided in an embodiment of this application. The robot control device may include an acquisition module 601, a fine-tuning module 602, a parsing module 603, and a control module 604, wherein...

[0112] The acquisition module 601 is used to acquire a preset training set and a preset large-scale language model, wherein the preset training set includes multiple training texts and a set of instruction texts corresponding to the training texts;

[0113] The fine-tuning module 602 is used to fine-tune the preset large-scale language model based on the preset training set to obtain a fine-tuned large-scale language model.

[0114] The parsing module 603 is used to parse the text to be processed based on the fine-tuned large-scale language model when the text to be processed is obtained, so as to obtain the target instruction text set corresponding to the text to be processed.

[0115] The control module 604 is used to control the robot based on the instruction text in the target instruction text set.

[0116] For details on the implementation of each of the above modules, please refer to the previous examples, which will not be repeated here.

[0117] This application also provides an electronic device, including a memory and a processor, wherein the processor executes steps in the robot control method provided in this embodiment by calling a computer program stored in the memory.

[0118] Please refer to Figure 12 , Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0119] The electronic device may include components such as a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, and an input unit 104. Those skilled in the art will understand that the electronic device structure shown in the figures does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0120] The processor 101 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 102, and calls data stored in the memory 102, to perform various functions and process data. Optionally, the processor 101 may include one or more processing cores; alternatively, the processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 101.

[0121] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.

[0122] The electronic device also includes a power supply 103 that supplies power to the various components. Optionally, the power supply 103 can be logically connected to the processor 101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 103 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0123] The electronic device may also include an input unit 104, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0124] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 101 in the electronic device loads one or more executable codes corresponding to computer programs into the memory 102 according to the following instructions, and the processor 101 executes the steps in the robot control method provided in this application, such as:

[0125] Obtain a preset training set and a preset large-scale language model. The preset training set includes multiple training texts and corresponding instruction text sets. Fine-tune the preset large-scale language model based on the preset training set to obtain a fine-tuned large-scale language model. Given the text to be processed, parse the text based on the fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed. Control the robot based on the instruction text in the target instruction text set.

[0126] It should be noted that the electronic device provided in this application embodiment and the robot control method in the above embodiment belong to the same concept. The specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0127] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program stored thereon is executed on the processor of the electronic device provided in the embodiments of this application, the processor of the electronic device performs the steps in the robot control method provided in this application. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0128] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform various optional implementations of the robot control method described above.

[0129] The above provides a detailed description of a robot control method and apparatus provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0130] It should be noted that when the above embodiments of this application are applied to specific products or technologies, and user-related data is involved, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.< / fridge> < / milk> < / fridge> < / fridge> < / milk> < / milk> < / fridge> < / milk> < / fridge> < / fridge> < / milk> < / milk> < / fridge> < / milk> < / fridge> < / fridge> < / milk> < / milk> < / book> < / sofa> < / sofa> < / book> < / book>

Claims

1. A control method of a robot characterized by, include: Obtain a preset training set and a preset large-scale language model, wherein the preset training set includes multiple training texts and a set of instruction texts corresponding to the training texts; Based on the preset training set, the parameters of the preset large-scale language model are efficiently fine-tuned to obtain a fine-tuned large-scale language model. Given the text to be processed, the text to be processed is parsed based on the fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed. The robot is controlled based on the instruction text in the target instruction text set.

2. The control method of the robot according to claim 1, characterized by, The step of efficiently fine-tuning the parameters of the preset large-scale language model based on the preset training set to obtain the fine-tuned large-scale language model includes: An Adapter module is added to each network layer of the preset large-scale language model to obtain a first language model; The first language model is trained based on the preset training set to obtain a fine-tuned large-scale language model. The parameters of the preset large-scale language model in the first language model are not updated during training, while the parameters of the Adapter module in the first language model are updated during training.

3. The control method of the robot according to claim 1, characterized by, The step of efficiently fine-tuning the parameters of the preset large-scale language model based on the preset training set to obtain the fine-tuned large-scale language model includes: The fine-tuning algorithm, which optimizes the prefix of the prompt word, is used to fine-tune the preset large-scale language model based on the preset training set to obtain the fine-tuned large-scale language model.

4. The robot control method according to claim 1, characterized in that, The step of efficiently fine-tuning the parameters of the preset large-scale language model based on the preset training set to obtain the fine-tuned large-scale language model includes: The preset large-scale language model is fine-tuned using the Prompt-tuning algorithm based on the preset training set to obtain a fine-tuned large-scale language model.

5. The robot control method according to claim 1, characterized in that, The process of parsing the text to be processed based on the fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed includes: Obtain a pre-constructed prompt template, which is used to instruct the text to be processed to be split into at least one instruction text; The prompt template and the text to be processed are input into the fine-tuned large-scale language model to obtain the target instruction text set corresponding to the text to be processed.

6. A control device for a robot, characterized in that, include: The acquisition module is used to acquire a preset training set and a preset large-scale language model, wherein the preset training set includes multiple training texts and a set of instruction texts corresponding to the training texts; The fine-tuning module is used to efficiently fine-tune the parameters of the preset large-scale language model based on the preset training set, so as to obtain a fine-tuned large-scale language model. The parsing module is used to parse the text to be processed based on the fine-tuned large-scale language model when the text to be processed is obtained, so as to obtain the target instruction text set corresponding to the text to be processed. The control module is used to control the robot based on the instruction text in the target instruction text set.

7. An electronic device, characterized in that, The device includes a memory and a processor, the memory storing a computer program and the processor running the computer program in the memory to perform the steps in the robot control method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the robot control method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the steps in the robot control method according to any one of claims 1 to 5.

Images