A method, device, equipment and storage medium for training instruction generation model

Abstract

The embodiment of the present invention relates to a method, device, equipment and storage medium for training an instruction generation model. The method comprises the following steps: inputting a data set into an initial model, wherein the data set includes a first instruction; extracting text features from the first instruction using the initial model; determining a hidden layer state sequence corresponding to the text features; encoding the hidden layer state sequence to obtain a semantic coding vector of a preset length; determining an output instruction corresponding to the first instruction according to the semantic coding vector, the text features and the hidden layer state sequence; and determining that the initial model training is completed when the similarity between the output instruction and the second instruction is greater than a set threshold, and obtaining a trained instruction generation model. Thus, the instruction generation model can be used to generate robot instructions from the fuzzy description input by the user, thereby achieving the purpose of fast and simple programming, simplifying the process of robot programming and control, and improving the efficiency of instruction generation.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of robots, and in particular to an instruction generation model training method, device, equipment and storage medium. Background Art

[0002] The robot's control instructions include motion instructions, I / O instructions, network communication instructions, logic judgment instructions, program management instructions, and related process packages. There are many instructions and they are very professional. Some internal parameters of the instructions also require reselection of data types and optional parameters and creation of related variables, so the robot programming process is cumbersome and difficult.

[0003] When programming a robot, the user needs to be very familiar with the robot's program instructions, statement format, and program logic, and the user also needs to master the robot's motion knowledge. Existing robot programming methods require a lot of user operations, which makes programming difficult and inefficient for users with poor professional skills. Missing or missing operations will cause the program to be unusable or achieve incorrect robot motion effects. Summary of the invention

[0004] In view of this, in order to solve the technical problem of the difficulty and low efficiency of the above-mentioned robot programming, the embodiment of the present invention provides an instruction generation model training method, device, equipment and storage medium

[0005] In a first aspect, an embodiment of the present invention provides an instruction generation model training method, comprising:

[0006] Inputting a data set into an initial model, wherein the data set includes a first instruction, and the first instruction is a fuzzy description input by a user;

[0007] extracting text features from the first instruction using the initial model;

[0008] Determining a hidden layer state sequence corresponding to the text feature;

[0009] Encoding the hidden layer state sequence to obtain a semantic coding vector of a preset length;

[0010] Determine an output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence;

[0011] When the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold, it is determined that the initial model training is completed and a trained instruction generation model is obtained.

[0012] In a possible implementation, determining the hidden layer state sequence corresponding to the text feature includes:

[0013] Generate a text feature sequence according to the text features;

[0014] Determine the hidden layer state corresponding to each moment of the text feature sequence;

[0015] Determine the forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence according to the hidden layer state at each moment;

[0016] The forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence are superimposed to obtain the hidden layer state sequence.

[0017] In a possible implementation, determining the output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence includes:

[0018] Determine the semantic coding vector corresponding to each moment of the hidden layer state sequence according to the normalized weight;

[0019] Determine an output instruction sequence corresponding to the semantic coding vector according to the semantic coding vector at each moment and the hidden layer state sequence;

[0020] An output instruction corresponding to the first instruction is determined according to the output instruction sequence.

[0021] In a possible implementation, the method further includes:

[0022] Determining category information of the first instruction according to the text feature;

[0023] determining an execution priority of the first instruction according to the category information;

[0024] The initial model is trained according to the execution priority to obtain a trained new instruction generation model.

[0025] In a possible implementation, the similarity is determined in the following manner:

[0026] Constructing a loss function of the initial model according to a gradient optimization algorithm and a maximum likelihood condition;

[0027] Model training is performed using the loss function as a training condition of the initial model, and the similarity between the output instruction and the second instruction is obtained based on a calculation result of the loss function.

[0028] In a second aspect, an embodiment of the present invention provides an instruction generation method, comprising:

[0029] Inputting a new first instruction into a trained instruction generation model, wherein the instruction generation model is trained by any of the methods described in the first aspect, and the new first instruction is a fuzzy description input by a user;

[0030] Outputting a third instruction using the instruction generation model, wherein the format of the third instruction is the same as the format of the instruction executable by the robot;

[0031] The third instruction is sent to the robot so that the robot executes the three instructions.

[0032] In a possible implementation, the method further includes:

[0033] displaying the third instruction;

[0034] receiving an update operation for the third instruction;

[0035] According to the updating operation, the third instruction is updated to a fourth instruction, and the fourth instruction is sent to the robot.

[0036] In a third aspect, an embodiment of the present invention provides an instruction generation model training device, comprising:

[0037] An input module, used to input a data set into an initial model, wherein the data set includes a first instruction, and the first instruction is a fuzzy description input by a user;

[0038] An extraction module, configured to extract text features from the first instruction using the initial model;

[0039] A determination module, used to determine a hidden layer state sequence corresponding to the text feature;

[0040] An encoding module, used for encoding the hidden layer state sequence to obtain a semantic encoding vector of a preset length;

[0041] The determination module is further used to determine the output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence;

[0042] The determination module is also used to determine that the initial model training is completed and obtain a trained instruction generation model when the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold.

[0043] In a fourth aspect, an embodiment of the present invention provides an instruction generation device, including:

[0044] An input module, used for inputting a new first instruction into a trained instruction generation model, wherein the instruction generation model is trained by any one of the methods described in the first aspect above, and the new first instruction is a fuzzy description input by a user;

[0045] An output module, used to output a third instruction using the instruction generation model, wherein the format of the third instruction is the same as the format of the instruction executable by the robot;

[0046] The sending module is used to send the third instruction to the robot so that the robot executes the three instructions.

[0047] In a fifth aspect, an embodiment of the present invention provides a device comprising: a processor and a memory, wherein the processor is used to execute an instruction generation model training program stored in the memory to implement the instruction generation model training method described in any one of the first aspects above.

[0048] The instruction generation model training scheme provided by the embodiment of the present invention is as follows: inputting a data set into an initial model, wherein the data set includes a first instruction, which is a fuzzy description input by a user; extracting text features from the first instruction using the initial model; determining a hidden layer state sequence corresponding to the text feature; encoding the hidden layer state sequence to obtain a semantic coding vector of a preset length; determining an output instruction corresponding to the first instruction according to the semantic coding vector, the text feature and the hidden layer state sequence; when the similarity between the output instruction and the second instruction is greater than a set threshold, determining that the initial model training is completed, and obtaining a trained instruction generation model, wherein the output instruction and the second instruction are robot instructions. This is to realize the generation of robot instructions from fuzzy descriptions through an instruction generation model, achieve the purpose of fast and simple programming, simplify the process of robot programming and control, and improve the efficiency of instruction generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of an instruction generation model training method provided by an embodiment of the present invention;

[0050] Figure 2 A flowchart of an instruction generation method provided by an embodiment of the present invention;

[0051] Figure 3 An architectural diagram of an instruction generation model training method provided by an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of an instruction generation model training device provided by an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of a structure of an instruction generation device provided by an embodiment of the present invention

[0054] Figure 6 A schematic diagram of the structure of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.

[0057] Figure 1 A flowchart of an instruction generation model training method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method specifically includes:

[0058] S11, inputting a data set into an initial model, wherein the data set includes a first instruction;

[0059] The instruction generation model training method provided in the embodiment of the present application can be applied to a terminal that can perform model training, wherein the terminal can be a mobile phone, a computer, a host computer, etc. The model training is specifically performed according to the first instruction and the corresponding second instruction, so that the trained model output has an output instruction whose similarity with the second instruction is greater than a set threshold.

[0060] In this embodiment, the first instruction is a fuzzy description input by the user for controlling the robot, and a data set formed by the fuzzy description and the corresponding robot instructions is pre-constructed. The data set includes the fuzzy description, the robot instructions corresponding to the fuzzy description, and the correspondence between the fuzzy description and the robot instructions.

[0061] For example, the data set may include: 1. Fuzzy description: let the robot move in a straight line to point P0 using the tool coordinate system tool1, the corresponding robot instruction: MoveL(P0, tool1); 2. Fuzzy description: output IO out3 is forced to 0, the corresponding robot instruction: SetDo(out3, FALSE); 3. Fuzzy description: wait for 1000ms, the corresponding robot instruction: WaitTime(1000); 4. Fuzzy description: create a socket server socket1, the corresponding robot instruction: SocketCreate(socket1).

[0062] Furthermore, the initial model is a seq2seq model, and the formed data set is input into the seq2seq model for deep learning training.

[0063] S12, extracting text features from the first instruction using the initial model;

[0064] In this embodiment, a first instruction is obtained from a data set, the first instruction is recognized, and text features of the first instruction are extracted (for example, text features are extracted through One-hot), and the text features are used for model training.

[0065] In a possible implementation, the text feature can be directly input by the user through an input position of the terminal, or the user can directly select the target text feature in the text feature display interface of the terminal.

[0066] S13, determining a hidden layer state sequence corresponding to the text feature;

[0067] In this embodiment, generating a text feature sequence according to text features includes: extracting the text features according to generating a text feature sequence {x1, x2, ..., x t}, the text feature sequence is used as the original word vector input sequence.

[0068] Further, the hidden layer state corresponding to each moment of the text feature sequence is determined;

[0069] Determine the forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence according to the hidden layer state at each moment;

[0070] The forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence are superimposed to obtain the hidden layer state sequence.

[0071] Specifically, this embodiment selects RNN as Encoder and Decoder. The structure of RNN includes input layer, hidden layer and output layer. Assume that the hidden layer state at the current time t is ht, and the hidden layer state at the previous time t-1 is ht-1. The current hidden layer state ht can be obtained by the hidden layer state ht-1 at the previous time t-1 and the input xt at the current time. Therefore, the hidden layer state corresponding to each time can be obtained by Calculated.

[0072] Among them, h t =tanh(a t ), a t =b+W*h t-1 +U*x t , xt is the current input data of the input layer, ht is the data of the hidden layer. b is the weight parameter, U is the weight from the input layer to the hidden layer, and W is the weight from the hidden layer at the previous moment to the hidden layer at the current moment.

[0073] Furthermore, the hidden layer state generated by the forward RNN is The hidden layer state sequence is The hidden layer state generated by the reverse RNN is The hidden layer state sequence is The forward and reverse hidden layer states are superimposed together to obtain the hidden layer state at time t The hidden layer state sequence is composed of the hidden layer states under each time series. Therefore, according to the original input word vector {x1,x2,...,x t} can get the hidden layer state sequence {h1,h2,...,h t}.

[0074] S14, encoding the hidden layer state sequence to obtain a semantic coding vector of a preset length;

[0075] In this embodiment, the hidden layer state sequence is summarized and encoded to obtain a fixed-length semantic encoding vector C, where C is the relevant feature vector obtained by encoding the user's fuzzy description, C = f(h1, h2, ..., h t )=h t , where f represents the mapping function.

[0076] S15, determining an output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence;

[0077] In this embodiment, determining the semantic coding vector corresponding to each moment of the hidden layer state sequence according to the normalized weight includes:

[0078] First, the semantic vector h output by the encoder t After normalizing the weights, we get the semantic coding vector Ci corresponding to each moment, where i=1,2,3,......,t.

[0079]

[0080] where a ij is the normalized weight

[0081]

[0082] where e ij is the correlation between the i-th output hidden layer state hi-1 and the j-th input hidden layer vector hj, which is calculated as follows:

[0083]

[0084] Where V is the weight from the hidden layer to the output layer, U is the weight from the input layer to the hidden layer, and W is the weight from the hidden layer at the previous moment to the hidden layer at the current moment.

[0085] Furthermore, the output instruction sequence corresponding to the semantic coding vector is determined according to the semantic coding vector and the hidden layer state sequence at each moment, including:

[0086] Determine the generated output sequence y1,y2,...,yt-1,

[0087] O t =c+V*h T

[0088] y t =softmax(o t )

[0089] Among them, ot is the data of the output layer, V is the weight from the hidden layer to the output layer, and c is the weight parameter.

[0090] At the current time t, the current output yt is predicted based on the semantic coding vector Ci obtained in the encoding stage and the generated output instruction sequence y1, y2, ..., yt-1.

[0091] y t =f(y t-1 ,h t-1 ,C i )

[0092] y t-1 is the output of the decoder at time t-1, h t-1 is the state of the hidden layer of neurons in the decoder at time t-1.

[0093] The output instruction corresponding to the first instruction is determined according to the generated output instruction sequence y1, y2, ..., yt-1 and the current output yt.

[0094] S16. When the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold, it is determined that the initial model training is completed and a trained instruction generation model is obtained.

[0095] In this embodiment, the second instruction is a robot instruction corresponding to the fuzzy description in the above data set. The loss function of the initial model is constructed according to the gradient optimization algorithm and the maximum likelihood condition; the loss function is used as the training condition of the initial model to perform model training, so that the output result continuously converges to the real result.

[0096] Specifically, the gap between the word vector sequence formed by the model output and the word vector sequence obtained by the robot instruction is mathematically described, and the gradient optimization algorithm and the maximum likelihood conditional probability are used as the loss function to train and optimize the model:

[0097]

[0098] xn is the input text feature sequence, and yn is the output instruction sequence.

[0099] In order to make the output result close to the actual instruction, the gap between them should be as small as possible. Therefore, this function should take the minimum value as the optimal value. The training process is stopped when the loss function is the set threshold. At this time, the similarity between the output instruction and the second instruction is greater than the set threshold, and the output result is basically the same as the robot instruction. The error between the output result and the robot instruction meets the application requirements. The learning process can be ended to generate a trained instruction generation model.

[0100] In one possible implementation, after extracting text features, the category information of the first instruction is determined according to the control type or parameter type corresponding to the text features; the execution priority of the first instruction is determined according to the category information (for example, the category information includes: network communication instructions, I / O instructions, motion instructions, and the corresponding execution priorities are: first priority, second priority, and third priority respectively); the first instructions are classified according to the category information, and the initial model is trained according to the execution priority and the classified first instructions to obtain a trained new instruction generation model. When multiple new first instructions are input into the trained new instruction generation model; the trained new instruction generation model generates output instructions according to the execution priority of each new first instruction input by the user.

[0101] The instruction generation model training scheme provided by the embodiment of the present invention is as follows: a data set is input into an initial model, wherein the data set includes a first instruction; a text feature is extracted from the first instruction using the initial model; a hidden layer state sequence corresponding to the text feature is determined; the hidden layer state sequence is encoded to obtain a semantic coding vector of a preset length; an output instruction corresponding to the first instruction is determined according to the semantic coding vector, the text feature and the hidden layer state sequence; when the similarity between the output instruction and the second instruction is greater than a set threshold, the initial model training is determined to be completed, and a trained instruction generation model is obtained. The first instruction description is converted into a robot instruction format using a deep learning method based on a seq2seq model, and the original complex steps such as instruction selection, instruction addition, parameter data type modification, and optional parameter selection are converted into an automatically generated form, and the corresponding parameters therein are added at the same time, so as to achieve the purpose of fast and simple programming, simplify the process of robot programming and control, improve the efficiency of robot programming by users, and make robot programming not require great professionalism, which is simpler and can be operated by non-professionals.

[0102] Figure 2 A flowchart of a method for generating instructions according to an embodiment of the present invention is shown in FIG. Figure 2As shown, the method specifically includes:

[0103] S21, inputting a new first instruction into a trained instruction generation model; and outputting a third instruction using the instruction generation model;

[0104] In this embodiment, the new first instruction is a fuzzy description input by the user for controlling robot programming. The trained instruction generation model generates a third instruction based on the fuzzy description. The format of the third instruction is the same as the format of the instructions executable by the robot, so the third instruction can be recognized by the robot.

[0105] Specifically, the instruction generation model is run on the robot teach pendant platform, and a user input interface is provided on the teach pendant program editor interface. After the user inputs a fuzzy description of the operation he or she wants the robot to perform, the program editor automatically generates a third instruction corresponding to the fuzzy description through the instruction generation model.

[0106] S22. Send the third instruction to the robot so that the robot executes the three instructions.

[0107] In this embodiment, the generated third instruction may be directly sent to the robot so that the robot executes the third instruction.

[0108] In one possible implementation, the third instruction is displayed on the program editor interface, the teach pendant automatically generates corresponding parameters according to the generated instruction, and provides a user input interface on the teach pendant program editor interface to receive the user's adjustment operation on the third instruction and the parameter. The third instruction is updated based on the adjustment operation, a fourth instruction corresponding to the third instruction is obtained according to the update operation, and the fourth instruction is sent to the robot so that the robot executes the fourth instruction.

[0109] The instruction generation model training scheme provided by the embodiment of the present invention inputs a new first instruction into the trained instruction generation model; uses the instruction generation model to output a third instruction; and sends the third instruction to the robot so that the robot executes the three instructions. This enables the robot to automatically generate robot instructions through the fuzzy description input by the user, thereby achieving the purpose of fast and simple programming.

[0110] Figure 3 An architectural diagram of an instruction generation model training method provided by an embodiment of the present invention, such as Figure 3 As shown, the architecture specifically includes: a robot teaching pendant and a computer for model training.

[0111] In this embodiment, a corresponding data set between the fuzzy description input by the user and the robot instruction is established and input into a computer, a seq2seq model is trained on the computer, and the trained model is run on a teach pendant so that the trained model can convert the fuzzy description randomly input by the user into robot instructions.

[0112] Figure 4 A schematic diagram of the structure of an instruction generation model training device provided by an embodiment of the present invention, such as Figure 4 As shown, the device specifically includes:

[0113] An input module 41 is used to input a data set into an initial model, wherein the data set includes a first instruction, and the first instruction is a fuzzy description input by a user;

[0114] An extraction module 42, configured to extract text features from the first instruction using the initial model;

[0115] A determination module 43, used to determine a hidden layer state sequence corresponding to the text feature;

[0116] An encoding module 44, used to encode the hidden layer state sequence to obtain a semantic encoding vector of a preset length;

[0117] The determination module 43 is further used to determine the output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence;

[0118] The determination module 43 is further used to determine that the initial model training is completed and obtain a trained instruction generation model when the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold.

[0119] In a possible implementation, the determination module 43 is specifically configured to generate a text feature sequence according to the text feature;

[0120] Determine the hidden layer state corresponding to each moment of the text feature sequence;

[0121] Determine the forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence according to the hidden layer state at each moment;

[0122] The forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence are superimposed to obtain the hidden layer state sequence.

[0123] In a possible implementation manner, the determination module 43 is further configured to determine the semantic encoding vector corresponding to each moment of the hidden layer state sequence according to the normalized weight;

[0124] Determine an output instruction sequence corresponding to the semantic coding vector according to the semantic coding vector at each moment and the hidden layer state sequence;

[0125] An output instruction corresponding to the first instruction is determined according to the output instruction sequence.

[0126] In a possible implementation manner, the determination module 43 is further configured to determine category information of the first instruction according to the text feature;

[0127] determining an execution priority of the first instruction according to the category information;

[0128] The initial model is trained according to the execution priority to obtain a trained new instruction generation model.

[0129] In a possible implementation, the determination module 43 is further configured to construct a loss function of the initial model according to a gradient optimization algorithm and a maximum likelihood condition;

[0130] Model training is performed using the loss function as a training condition of the initial model, and the similarity between the output instruction and the second instruction is obtained based on a calculation result of the loss function.

[0131] The instruction generation model training device provided in this embodiment can be as follows Figure 4 The device shown in FIG. 1 can perform the following steps: Figure 1 All the steps in order to achieve Figure 1 The technical effect of the instruction generation model training method shown is shown in the figure. For details, please refer to Figure 2 For the sake of brevity, the relevant description is not repeated here.

[0132] Figure 5 A schematic diagram of the structure of an instruction generation device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device specifically includes:

[0133] The input module 51 is also used to input a new first instruction into the trained instruction generation model, wherein the new first instruction is a fuzzy description input by the user;

[0134] The output module 52 is further used to output a third instruction using the instruction generation model, wherein the format of the third instruction is the same as the format of the instruction executable by the robot;

[0135] The sending module 53 is used to send the third instruction to the robot so that the robot executes the three instructions.

[0136] In a possible implementation, the device further includes: a teaching module 54, configured to display the third instruction;

[0137] receiving an update operation for the third instruction;

[0138] According to the updating operation, the third instruction is updated to a fourth instruction, and the fourth instruction is sent to the robot.

[0139] The instruction generating device provided in this embodiment may be as follows Figure 5 The device shown in FIG. 1 can perform the following steps: Figure 2 All the steps in order to achieve Figure 2 For details, please refer to the technical effect of the instruction generation method shown in Figure 2 For the sake of brevity, the relevant description is not repeated here.

[0140] Figure 6 A schematic diagram of a device provided in an embodiment of the present invention is provided. Figure 6 The device 600 shown includes: at least one processor 601, a memory 602, at least one network interface 604 and other user interfaces 603. The various components in the device 600 are coupled together via a bus system 605. It is understood that the bus system 605 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 605 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 605 is not shown in FIG. Figure 6 Various buses are labeled as bus system 605 .

[0141] The user interface 603 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0142] It can be understood that the memory 602 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 602 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0143] In some implementations, the memory 602 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 6021 and an application program 6022 .

[0144] The operating system 6021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 6022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 6022.

[0145] In the embodiment of the present invention, by calling the program or instruction stored in the memory 602, specifically, the program or instruction stored in the application 6022, the processor 601 is used to execute the method steps provided by each method embodiment, for example, including:

[0146] Inputting a data set into an initial model, wherein the data set includes a first instruction, and the first instruction is a fuzzy description input by a user;

[0147] extracting text features from the first instruction using the initial model;

[0148] Determining a hidden layer state sequence corresponding to the text feature;

[0149] Encoding the hidden layer state sequence to obtain a semantic coding vector of a preset length;

[0150] Determine an output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence;

[0151] When the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold, it is determined that the initial model training is completed and a trained instruction generation model is obtained.

[0152] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0153] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0154] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0155] The device provided in this embodiment can be Figure 6 The device shown in , can be executed as Figure 1-2 The instructions in the code generate all the steps of the model training method, thereby achieving Figure 1-2 The technical effect of the instruction generation model training method shown is shown in the figure. For details, please refer to Figure 1-2 For the sake of brevity, the relevant description is not repeated here.

[0156] The embodiment of the present invention also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0157] When one or more programs in the storage medium can be executed by one or more processors to implement the above-mentioned instruction generation model training method executed on the device side.

[0158] The processor is used to execute the instruction generation model training program stored in the memory to implement the following steps of the instruction generation model training method executed on the device side:

[0159] Inputting a data set into an initial model, wherein the data set includes a first instruction, and the first instruction is a fuzzy description input by a user;

[0160] extracting text features from the first instruction using the initial model;

[0161] Determining a hidden layer state sequence corresponding to the text feature;

[0162] Encoding the hidden layer state sequence to obtain a semantic coding vector of a preset length;

[0163] Determine an output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence;

[0164] When the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold, it is determined that the initial model training is completed and a trained instruction generation model is obtained.

[0165] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0166] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0167] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for training an instruction generation model, characterized in that: include: Inputting a data set into an initial model, wherein the data set includes a first instruction, and the first instruction is a fuzzy description input by a user; extracting text features from the first instruction using the initial model; Determining a hidden layer state sequence corresponding to the text feature; Encoding the hidden layer state sequence to obtain a semantic coding vector of a preset length; Determine an output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence; When the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold, it is determined that the initial model training is completed, and a trained instruction generation model is obtained; The step of determining a hidden layer state sequence corresponding to the text feature comprises: Generate a text feature sequence according to the text features; Determine the hidden layer state corresponding to each moment of the text feature sequence; Determine the forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence according to the hidden layer state at each moment; The forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence are superimposed to obtain the hidden layer state sequence.

2. The method according to claim 1, characterized in that The step of determining the output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence includes: Determine the semantic coding vector corresponding to each moment of the hidden layer state sequence according to the normalized weight; Determine an output instruction sequence corresponding to the semantic coding vector according to the semantic coding vector at each moment and the hidden layer state sequence; An output instruction corresponding to the first instruction is determined according to the output instruction sequence.

3. The method according to claim 1, characterized in that The method further comprises: Determining category information of the first instruction according to the text feature; determining an execution priority of the first instruction according to the category information; The initial model is trained according to the execution priority to obtain a trained new instruction generation model.

4. The method according to claim 1, characterized in that: The similarity is determined by: Constructing a loss function of the initial model according to a gradient optimization algorithm and a maximum likelihood condition; Model training is performed using the loss function as a training condition of the initial model, and the similarity between the output instruction and the second instruction is obtained based on a calculation result of the loss function.

5. A method for generating an instruction, characterized in that: include: Inputting a new first instruction into a trained instruction generation model, wherein the instruction generation model is trained by the method according to any one of claims 1 to 4, and the new first instruction is a fuzzy description input by a user; Outputting a third instruction using the instruction generation model, wherein the format of the third instruction is the same as the format of the instruction executable by the robot; The third instruction is sent to the robot so that the robot executes the three instructions.

6. The method according to claim 5, characterized in that The method further comprises: displaying the third instruction; receiving an update operation for the third instruction; According to the updating operation, the third instruction is updated to a fourth instruction, and the fourth instruction is sent to the robot.

7. A command generation model training device, characterized in that: include: An input module, used to input a data set into an initial model, wherein the data set includes a first instruction, and the first instruction is a fuzzy description input by a user; An extraction module, configured to extract text features from the first instruction using the initial model; A determination module, used to determine a hidden layer state sequence corresponding to the text feature; An encoding module, used for encoding the hidden layer state sequence to obtain a semantic encoding vector of a preset length; The determination module is further used to determine the output instruction corresponding to the first instruction according to the semantic encoding vector, the text feature and the hidden layer state sequence; The determination module is further configured to determine that the initial model training is completed and obtain a trained instruction generation model when the similarity between the output instruction and the second instruction corresponding to the first instruction is greater than a set threshold; The determination module is specifically used to generate a text feature sequence according to the text feature; Determine the hidden layer state corresponding to each moment of the text feature sequence; Determine the forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence according to the hidden layer state at each moment; The forward RNN hidden layer state sequence and the reverse RNN hidden layer state sequence are superimposed to obtain the hidden layer state sequence.

8. An instruction generating device, characterized in that: include: An input module, used for inputting a new first instruction into a trained instruction generation model, wherein the instruction generation model is trained by any method of claims 1-4, and the new first instruction is a fuzzy description input by a user; An output module, used to output a third instruction using the instruction generation model, wherein the format of the third instruction is the same as the format of the instruction executable by the robot; The sending module is used to send the third instruction to the robot so that the robot executes the three instructions.

9. A device, characterized in that: include: A processor and a memory, wherein the processor is used to execute an instruction generation model training program stored in the memory to implement the instruction generation model training method according to any one of claims 1 to 4.

Images