A Language Model-Based Control Command Generation Method and Companion Robot

Abstract

This application relates to a language model-based control instruction generation method and a companion robot. The method includes: acquiring the text corresponding to a natural language instruction; parsing the text using a trained lightweight language model deployed on the companion robot to output an intent string; wherein the intent string is an intermediate semantic result before generating control instructions for the companion robot, and includes one or more output semantic values, which characterize the emotional state and / or behavioral state to be presented by the companion robot; after obtaining the intent string, for any output semantic value, determining the control field type and control field content corresponding to the output semantic value based on the position of the output semantic value in the intent string and the output semantic value itself, combined with a target mapping relationship; and generating control instructions for the companion robot based on the control field type and control field content. This method can improve the efficiency and accuracy of instruction generation.

Description

Technical Field

[0001] This application relates to the field of companion robot control technology, and in particular to a language model-based control instruction generation method and a companion robot. Background Technology

[0002] With the development of artificial intelligence and robot control technology, companion robots are gradually acquiring the ability to interact with natural language and respond to autonomous actions. Companion robots usually need to be controlled to complete interactive actions such as displaying facial expressions, providing voice feedback, moving, and adjusting posture based on the user's voice input.

[0003] Currently, to improve natural language understanding capabilities, some companion robots employ cloud-based large language models to semantically parse user-input natural language commands. These models then output structured control commands for controlling the robot. However, because structured commands include format symbols such as curly braces, quotation marks, colons, field names, and parameter names, the cloud-based large language model needs to output numerous redundant characters when generating these commands, resulting in lengthy command generation times. Furthermore, companion robots rely on the network to upload user commands to the cloud and receive control commands returned from the cloud, making them susceptible to network conditions. This makes it difficult to meet the low-latency, continuous interactive control requirements of companion robots, impacting their response efficiency and control accuracy to natural language commands, a problem that urgently needs to be addressed. Summary of the Invention

[0004] Therefore, it is necessary to provide a language model-based control instruction generation method and companion robot that can improve the efficiency and accuracy of instruction generation, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a method for generating control instructions based on a language model, including:

[0006] Retrieve the text corresponding to natural language commands;

[0007] The text is parsed using a lightweight language model deployed on the companion robot after training, and an intent string is output. The intent string is an intermediate semantic result before generating control commands for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot.

[0008] After obtaining the intent string, for any output semantic value, based on the position of the output semantic value in the intent string and the output semantic value itself, and in conjunction with the target mapping relationship, determine the control field type and control field content corresponding to the output semantic value;

[0009] Based on the control field type and control field content, control instructions are generated for the companion robot.

[0010] In one embodiment, the target mapping relationship includes a position mapping relationship and a content mapping relationship; accordingly, based on the position of the output semantic value in the intent string and the output semantic value itself, combined with the target mapping relationship, the control field type and control field content corresponding to the output semantic value are determined, including:

[0011] Based on the position of the output semantic value in the intent string, and in conjunction with the position mapping relationship, determine the control field type corresponding to the output semantic value;

[0012] Based on the output semantic value and its corresponding control field type, and in conjunction with the content mapping relationship, determine the content of the control field corresponding to the output semantic value.

[0013] In one embodiment, the content of the control field corresponding to the output semantic value is determined based on the output semantic value and its corresponding control field type, combined with the content mapping relationship, including:

[0014] Based on the control field type, determine the set of legal values ​​corresponding to the control field type;

[0015] If the output semantic value belongs to the set of legal values, the output semantic value is used as the target semantic value;

[0016] If the output semantic value does not belong to the set of legal values, the default semantic value corresponding to the control field type will be used as the target semantic value.

[0017] Based on the target semantic value and its corresponding control field type, and in conjunction with the content mapping relationship, determine the content of the control field corresponding to the output semantic value.

[0018] In one embodiment, control instructions for the companion robot are generated based on the control field type and control field content, including:

[0019] The control subsystems of the companion robot are determined based on the control field type;

[0020] Determine the control parameters of the subsystem to be controlled based on the contents of the control fields;

[0021] Based on the subsystem to be controlled and the control parameters, control instructions are generated for the companion robot.

[0022] In one embodiment, before determining the control field type and control field content corresponding to the output semantic value by combining the target mapping relationship, the method further includes:

[0023] Determine the mapping selection information; the mapping selection information is the number of output semantic values ​​in the intent string, or the interaction scenario to which the natural language instruction belongs;

[0024] Based on the mapping selection information, select the target mapping relationship from multiple candidate mapping relationships;

[0025] Specifically, when the mapping selection information is the number of output semantic values, different candidate mapping relationships correspond to different numbers of output semantic values; when the mapping selection information is the interaction scenario, different candidate mapping relationships correspond to different interaction scenarios.

[0026] In one embodiment, the lightweight language model is trained in the following manner:

[0027] Determine the model generation protocol corresponding to different interaction scenarios; wherein, the model generation protocol is used to specify the format of the intent string output by the lightweight language model in the corresponding interaction scenario; the format includes at least one of the following: the number of output semantic values, the control field type corresponding to the output semantic values, the order of the output semantic values, and the separation method;

[0028] Based on the model generation protocol corresponding to each interaction scenario, training samples corresponding to each interaction scenario are constructed; wherein, the training samples include sample text and sample intent strings corresponding to the sample text; the sample text is the text corresponding to the sample natural language instruction; the sample intent string includes one or more sample semantic values;

[0029] Based on the training samples, the lightweight language model is fine-tuned under supervision to obtain the trained lightweight language model.

[0030] In one embodiment, the training samples further include sample scene prompts; based on the model generation protocol corresponding to each interaction scene, training samples corresponding to each interaction scene are constructed, including:

[0031] Based on the model generation protocol corresponding to each interaction scenario, determine the sample scenario prompt words corresponding to the corresponding interaction scenario;

[0032] Based on the sample scenario prompts, sample text, and sample intent strings, training samples corresponding to the respective interaction scenarios are constructed.

[0033] In one embodiment, the intent string includes at least two output semantic values ​​arranged in a preset order;

[0034] The output semantic value located in the first position is used to characterize the emotional state to be presented by the companion robot; the output semantic value located in the second position is used to characterize the behavioral state to be presented by the companion robot.

[0035] In one embodiment, the intent string also includes an output semantic value at a third position, which is used to characterize the intensity of the action corresponding to the behavior state.

[0036] Secondly, this application also provides a control instruction generation device based on a language model, comprising:

[0037] The text acquisition module is used to acquire the text corresponding to natural language commands;

[0038] The text parsing module is used to parse text using a lightweight language model deployed on the companion robot and output an intent string. The intent string is an intermediate semantic result before generating control commands for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot.

[0039] The determination module is used to determine the control field type and control field content corresponding to any output semantic value after obtaining the intent string, based on the position of the output semantic value in the intent string and the output semantic value, combined with the target mapping relationship.

[0040] The generation module is used to generate control instructions for the companion robot based on the control field type and control field content.

[0041] Thirdly, this application also provides a companion robot, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps:

[0042] Retrieve the text corresponding to natural language commands;

[0043] The text is parsed using a lightweight language model deployed on the companion robot after training, and an intent string is output. The intent string is an intermediate semantic result before generating control commands for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot.

[0044] After obtaining the intent string, for any output semantic value, based on the position of the output semantic value in the intent string and the output semantic value itself, and in conjunction with the target mapping relationship, determine the control field type and control field content corresponding to the output semantic value;

[0045] Based on the control field type and control field content, control instructions are generated for the companion robot.

[0046] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0047] Retrieve the text corresponding to natural language commands;

[0048] The text is parsed using a lightweight language model deployed on the companion robot after training, and an intent string is output. The intent string is an intermediate semantic result before generating control commands for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot.

[0049] After obtaining the intent string, for any output semantic value, based on the position of the output semantic value in the intent string and the output semantic value itself, and in conjunction with the target mapping relationship, determine the control field type and control field content corresponding to the output semantic value;

[0050] Based on the control field type and control field content, control instructions are generated for the companion robot.

[0051] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0052] Retrieve the text corresponding to natural language commands;

[0053] The text is parsed using a lightweight language model deployed on the companion robot after training, and an intent string is output. The intent string is an intermediate semantic result before generating control commands for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot.

[0054] After obtaining the intent string, for any output semantic value, based on the position of the output semantic value in the intent string and the output semantic value itself, and in conjunction with the target mapping relationship, determine the control field type and control field content corresponding to the output semantic value;

[0055] Based on the control field type and control field content, control instructions are generated for the companion robot.

[0056] The aforementioned language model-based control command generation method and companion robot parse the text corresponding to natural language commands using a trained lightweight language model, and output an intent string including one or more output semantic values. This intent string serves as an independent intermediate semantic result before generating control commands for the companion robot, eliminating the need for the lightweight language model to directly generate structured commands containing field names, parameter names, and format symbols. After the lightweight language model independently outputs the intent string, a separate mapping and parsing step is performed. That is, based on the output semantic values ​​in the intent string and their positions within the intent string, the control field type and content are determined in conjunction with the target mapping relationship, and control commands for the companion robot are generated based on the control field type and content.

[0057] Compared to methods that directly generate structured control commands from a language model, this application allows a lightweight language model to focus on the intent parsing of natural language commands. This reduces the amount of formatted content the model needs to learn and output, lowers the model's output length, and reduces the local computational burden on the companion robot. This improves the accuracy and generation efficiency of intent strings, thereby enhancing the companion robot's command response efficiency. Since companion robots typically need to promptly display corresponding emotional or behavioral states after a user issues a natural language command, this application shortens the processing time between the user issuing the command and the companion robot generating the control command. This enables the companion robot to provide more timely feedback through facial expressions, lighting effects, voice, or actions, improving the efficiency of command response and enhancing the continuity and naturalness of human-computer interaction.

[0058] In addition, this application decouples the intent string generation process from the control command generation process. When adding new emotional or behavioral states that the companion robot can present, or adjusting the control command format, adaptation can be achieved by adjusting the target mapping relationship, control field content, or control parameters. This reduces the need to retrain the lightweight language model and improves the efficiency of expanding and maintaining the companion robot's control functions. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating a language model-based control instruction generation method in one embodiment;

[0061] Figure 2This is a flowchart illustrating the steps for determining the control field type and control field content in one embodiment;

[0062] Figure 3 This is a flowchart illustrating the control instruction generation steps in one embodiment;

[0063] Figure 4 This is a flowchart illustrating the steps for determining the target mapping relationship in one embodiment;

[0064] Figure 5 This is a flowchart illustrating a language model-based control instruction generation method in another embodiment;

[0065] Figure 6 This is a structural block diagram of a language model-based control instruction generation device in one embodiment;

[0066] Figure 7 This is an internal structural diagram of a companion robot in one embodiment. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0069] In recent years, artificial intelligence and robotics technologies have gradually become practical, and companion robots are increasingly equipped with natural language interaction, emotional expression, and action feedback capabilities. During interaction between companion robots and users, the robots typically need to understand the user's intentions based on voice or text commands, and then generate corresponding control commands to drive the robot to perform interactive actions such as facial expression display, voice feedback, movement, and posture adjustment.

[0070] Currently, to improve natural language understanding capabilities, some companion robots employ cloud-based large language models to parse user-input natural language commands and generate structured control commands for controlling the robot. However, structured commands typically contain numerous formatting characters such as parentheses, quotation marks, and separators. The model needs to output these symbols additionally when generating such structured control commands, which increases the command generation time. Furthermore, when the format of the structured control commands is complex, the model output may suffer from missing fields, incorrect field order, or omitted formatting symbols, affecting the parsing and execution of control commands. On the other hand, companion robots rely on the network to upload user commands to the cloud and receive control commands returned from the cloud. The control response is easily affected by network conditions, making it difficult to meet the low-latency response requirements of companion robots in continuous interaction scenarios. This application is proposed to solve the above-mentioned technical problems.

[0071] In one exemplary embodiment, such as Figure 1 As shown, a method for generating control commands based on a language model is provided. Taking the application of this method to a companion robot as an example, the method includes the following steps:

[0072] S110, obtain the text corresponding to the natural language instruction.

[0073] Natural language instructions can be understood as voice or text commands input by the user. The text can be obtained by speech recognition of the voice command or by preprocessing the text command.

[0074] In one alternative implementation, the natural language command is a voice command. The companion robot can collect the user's voice through a microphone, and accordingly, the voice command can be converted into corresponding text based on the companion robot's local voice recognition module or a voice recognition module connected to the companion robot.

[0075] In another alternative implementation, the companion robot can acquire text content input by the user through an input interface, mobile terminal, or communication interface, and perform at least one preprocessing step, such as word segmentation, noise reduction, format normalization, and encoding conversion, on the text content to obtain the text corresponding to the natural language command.

[0076] The S120 parses text and outputs intent strings using a lightweight language model trained and deployed on the companion robot.

[0077] The trained lightweight language model can be understood as a model specifically designed to understand the intent corresponding to natural language instructions. Its output is not a structured control instruction that is directly used to control the actions of the companion robot, but rather an intent string corresponding to the text.

[0078] It is understandable that the trained lightweight language model does not need to integrate intent strings to generate structured instructions. The model's generation task focuses on the generation of intent strings, which can improve both the efficiency and accuracy of intent string generation. In other words, the aforementioned lightweight language model can improve the efficiency and accuracy of intent understanding. The training process of the lightweight language model will be described in subsequent embodiments.

[0079] An intent string can be understood as an intermediate semantic result generated before control commands are generated for a companion robot. It is not a structured instruction, nor is it a control command that can be directly issued. Instead, it is an intermediate expression representing the control intent corresponding to a natural language command. The intent string carries the control information or control intent corresponding to the natural language command and does not itself carry formatting characters such as parentheses, quotation marks, or separators. The intent string includes one or more output semantic values, which represent the emotional state and / or behavioral state that the companion robot is to exhibit.

[0080] For example, if there are two semantic values ​​in the output, and the intent string is happy / come_over, then it represents that the emotional state to be presented by the companion robot is "happy" and the behavioral state to be presented is "walking towards the user". That is, the user wants the companion robot to show a happy emotion and walk towards the user.

[0081] Optionally, to distinguish different output semantic values, when the model outputs an intent string containing multiple output semantic values, a preset delimiter can be used to separate each output semantic value, so that the intent string can be split subsequently according to the preset delimiter. In some embodiments, the preset delimiter can be a character or string that does not belong to the set of output semantic value values. The preset delimiter can be a symbol such as " / " or "|". By using a preset delimiter that does not belong to the set of output semantic value values, the probability of confusion between the delimiter and the output semantic value itself can be reduced, improving the accuracy of subsequent splitting and parsing of the intent string.

[0082] Optionally, the trained lightweight language model can be deployed on the companion robot's local computing unit. The local computing unit may include at least one of a Neural Processing Unit (NPU), a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Microcontroller Unit (MCU), and an Artificial Intelligence (AI) accelerator. By deploying the trained lightweight language model on the companion robot's local computing unit, reliance on cloud model calls and network communication status can be reduced, improving the companion robot's real-time response to natural language commands.

[0083] To refine the performance of the companion robot's actions and facilitate the generation of subsequent structured instructions, in some embodiments, such as the example above where the intent string is happy / come_over, the intent string includes at least two output semantic values ​​arranged in a preset order. The output semantic value at the first position represents the emotional state to be presented by the companion robot, and the output semantic value at the second position represents the behavioral state to be presented by the companion robot. The preset order can be determined based on human experience or set as needed according to the requirements of the action execution; this application does not impose any limitations on this.

[0084] Optionally, if the intent string is `neutral / stay`, then the emotional state to be presented by the companion robot is "calm," and the behavioral state to be presented is "staying in place." That is, the user wants the companion robot to maintain a calm mood and remain in place. Optionally, if the intent string is `excited / fetch_ball`, then the emotional state to be presented by the companion robot is "excited," and the behavioral state to be presented is "picking up a ball." That is, the user wants the companion robot to show excitement and pick up a ball.

[0085] To further refine the performance of the companion robot's actions, in some embodiments, the intent string also includes an output semantic value at a third position, which is used to characterize the intensity of the action corresponding to the behavior state.

[0086] Optionally, if the intent string is happy / come_over / high, then the companion robot's emotional state to be presented is "happy," the behavioral state to be presented is "approaching the user," and the action intensity is high. That is, the user wants the companion robot to approach the user quickly with a happy emotion. Optionally, if the intent string is calm / rock / mild, then the companion robot's emotional state to be presented is "mild," the behavioral state to be presented is "gentle rocking," and the action intensity is moderate. That is, the user wants the companion robot to maintain a mild emotion and rock slowly and gently.

[0087] To improve the precision of companion robots' actions and enhance their efficiency when performing complex actions, in some embodiments, the intent string may also include other output semantic values, such as action duration, action priority, and action safety level. Specifically, action duration limits the execution time of the corresponding behavioral state; action priority distinguishes the order of action execution when multiple instructions are executed concurrently; and action safety level defines the safety constraints during action execution.

[0088] To further reduce the model output length, in some embodiments, a correspondence between semantic values ​​and encoded identifiers can be pre-established, and the encoded identifiers can be used as the output semantic values. For example, the semantic value "happy," which represents the emotion of happiness, can be encoded as H or 01; the semantic value "come_over," which represents the initiation of user behavior, can be encoded as CO or 03. Accordingly, happy / come_over can be simplified to H / CO and 01 / 03. Subsequent steps can map the encoded identifiers to the corresponding control field content based on the content mapping relationship, thereby reducing the model output length without changing the subsequent control instruction generation logic.

[0089] S130: After obtaining the intent string, for any output semantic value, based on the position of the output semantic value in the intent string and the output semantic value itself, and in conjunction with the target mapping relationship, determine the control field type and control field content corresponding to the output semantic value.

[0090] The target mapping relationship is used to characterize the mapping rules that determine the corresponding control field types and control field contents based on the position of the output semantic value in the intent string and the output semantic value. The control field type characterizes the control dimension or control category corresponding to the output semantic value when generating control instructions. Examples include control field types characterizing emotional states, behavioral states, action intensity, duration, and target objects. For ease of description, in the following text, the control field types characterizing emotional states, behavioral states, action intensity, duration, or target objects can be referred to as emotional state fields, behavioral state fields, action intensity fields, duration fields, or target object fields, respectively.

[0091] The content of the control field is used to characterize the specific control meaning or control parameter corresponding to the output semantic value under the corresponding control field type. For example, the content of the control field under the emotion state field may include a happy expression animation icon, a lighting effect parameter, or a voice tone parameter; for the behavior state field, the corresponding control field content may include at least one of the following: a motion control icon, a target point parameter, or a movement speed parameter.

[0092] In one alternative implementation, the intent string can be split to obtain one or more output semantic values, and the position of each output semantic value in the intent string can be determined. Then, based on the position of each output semantic value in the intent string and each output semantic value, a matching is performed in conjunction with a pre-stored target mapping relationship to determine the control field type and control field content corresponding to each output semantic value.

[0093] For example, when the intent string is happy / come_over, the control field type corresponding to the output semantic value "happy" at the first position can be determined to be the emotion state field "mood," and the content of the control field can be an expression animation identifier, lighting effect parameter, or voice tone parameter corresponding to the happy emotion; the control field type corresponding to the output semantic value "come_over" at the second position can be the behavior state field "instruction," and the content of the control field can be a motion control identifier, target point parameter, or movement speed parameter corresponding to the user's movement. For example, the control field content corresponding to the output semantic value "happy" may include at least one of a happy expression animation identifier, warm-colored lighting effect parameter, and pleasant voice tone parameter; the control field content corresponding to the output semantic value "come_over" may include at least one of a motion control identifier for the user's movement, a target point parameter corresponding to the user's current position, and a preset movement speed parameter.

[0094] S140 generates control instructions for the companion robot based on the control field type and control field content.

[0095] For example, in this embodiment, each control field type and its corresponding control field content can be encapsulated into control instructions that the companion robot can recognize. The control instructions can be at least one of the following: JSON format instructions, binary communication frames, serial port instructions, Robot Operating System (ROS) messages, or hardware interface commands.

[0096] In some embodiments, when the intent string includes an output semantic value, control instructions for the companion robot can be generated based on the control field type and control field content corresponding to the output semantic value; when the intent string includes multiple output semantic values, control instructions for the companion robot can be generated based on the control field type and control field content corresponding to each output semantic value.

[0097] Continuing with the example of an intent string containing two output semantic values, happy and come_over, assuming that the control field type for the output semantic value happy is the emotion state field "mood," and the control field content includes a happy expression animation identifier and warm-colored lighting effect parameters, and the control field type for the output semantic value come_over is the behavior state field "instruction," and the control field content includes a motion control identifier for the user, target point parameters, movement speed parameters, and obstacle avoidance activation parameters, the following JSON format control instruction can be generated:

[0098] {mood:{animation:"smile",light:"warm"},instruction:{action:"move_to_user",speed:"normal",avoidance:true}}.

[0099] The companion robot can send control commands to corresponding functional modules via an internal bus or control interface. For example, the expression interaction module can play happy expression animations based on the animation field, the lighting effect module can display warm lighting effects based on the light field, and the motion drive module can control the companion robot to move towards the user's location at a preset speed based on the action, speed, and avoidance fields, and activate obstacle avoidance detection during the movement.

[0100] Optionally, to clarify the device execution logic corresponding to each field and unify the standard for reproducing actions and emotions, in some embodiments, different control field contents are pre-set with different control logic or control parameters so that the control field contents can be recognized and executed by the corresponding functional modules. For example, the control field content mapped to the output semantic value "happy" may include playing a happy animation for 3 seconds; the control field content mapped to "come_over" may include using the user's current position as the target, moving at a normal speed, and enabling obstacle avoidance detection throughout.

[0101] In the aforementioned language model-based control command generation method, a trained lightweight language model parses the text corresponding to natural language commands and outputs an intent string containing one or more semantic values. This intent string serves as an independent intermediate semantic result before generating control commands for the companion robot, eliminating the need for the lightweight language model to directly generate structured commands containing field names, parameter names, and format symbols. After the lightweight language model independently outputs the intent string, a separate mapping and parsing step is performed. That is, based on the semantic values ​​in the intent string and their positions within the string, the control field type and content are determined in conjunction with the target mapping relationship. Control commands for the companion robot are then generated based on these control field types and content.

[0102] Compared to methods that directly generate structured control commands from a language model, this application allows a lightweight language model to focus on the intent parsing of natural language commands. This reduces the amount of formatted content the model needs to learn and output, lowers the model's output length, and reduces the local computational burden on the companion robot. This improves the accuracy and generation efficiency of intent strings, thereby enhancing the companion robot's command response efficiency. Since companion robots typically need to promptly display corresponding emotional or behavioral states after a user issues a natural language command, this application shortens the processing time between the user issuing the command and the companion robot generating the control command. This enables the companion robot to provide more timely feedback through facial expressions, lighting effects, voice, or actions, improving the efficiency of command response and enhancing the continuity and naturalness of human-computer interaction.

[0103] In addition, this application decouples the intent string generation process from the control command generation process. When adding new emotional or behavioral states that the companion robot can present, or adjusting the control command format, adaptation can be achieved by adjusting the target mapping relationship, control field content, or control parameters. This reduces the need to retrain the lightweight language model and improves the efficiency of expanding and maintaining the companion robot's control functions.

[0104] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the target mapping relationship includes a position mapping relationship and a content mapping relationship. Under this premise, the process of determining the control field type and control field content corresponding to the output semantic value based on the position of the output semantic value in the intent string and the output semantic value, combined with the target mapping relationship, is refined.

[0105] The positional mapping relationship defines the correspondence between the order of output semantic values ​​in the intent string and the control field types. For example, the first output semantic value in the intent string corresponds to the emotion state field; the second output semantic value corresponds to the behavior state field.

[0106] The content mapping relationship defines the correspondence between output semantic values ​​and control field content, given a defined field type. For example, for the emotion state field, the control field content corresponding to the output semantic value "happy" may include a happy emoji animation identifier, warm lighting effect parameters, or pleasant voice tone parameters; for the behavior state field, the control field content corresponding to the output semantic value "come_over" may include a motion control identifier for moving towards the user, target point parameters, movement speed parameters, or obstacle avoidance activation parameters.

[0107] It should be noted that the location mapping relationship and content mapping relationship can be pre-configured based on the interaction needs of the companion robot, the type of control field, the set of presentable emotions, the set of executable behaviors and / or the format of control instructions, or can be adjusted according to the update of the control function of the companion robot. This application does not impose any limitations on this.

[0108] See Figure 2 The steps for determining the control field type and control field content shown include:

[0109] S210, Based on the position of the output semantic value in the intent string and in conjunction with the position mapping relationship, determine the control field type corresponding to the output semantic value.

[0110] In one optional implementation, the split intent string can be traversed to record the permutation position number corresponding to each output semantic value; a pre-stored position mapping relationship can be retrieved, and the control field type corresponding to the output semantic value can be determined by matching the position number of the output semantic value.

[0111] For example, taking the intention string happy / come_over as an example, happy is in the first position, and its corresponding emotion state field mood is determined by the position mapping relationship; come_over is in the second position, and its corresponding behavior state field instruction is determined by the position mapping relationship.

[0112] S220, based on the output semantic value and its corresponding control field type, and in conjunction with the content mapping relationship, determine the content of the control field corresponding to the output semantic value.

[0113] In an optional implementation, after determining the control field type corresponding to each output semantic value through the above steps, the pre-stored content mapping relationship can be retrieved by combining the control field type and the corresponding output semantic value to match and obtain the corresponding control field content.

[0114] Continuing with the example above, under the mood field, the control field content obtained by matching the semantic value happy can include a happy emoji animation identifier, warm light effect parameters, or pleasant voice tone parameters; under the instruction field, the control field content obtained by matching the output semantic value come_over can include a motion control identifier for moving towards the user, target point parameters, movement speed parameters, or obstacle avoidance activation parameters.

[0115] To avoid incomplete or invalid instructions due to semantic output anomalies or semantic value matching failures, in some embodiments, the set of legal values ​​corresponding to the control field type can be determined based on the control field type; if the output semantic value belongs to the set of legal values, the output semantic value is used as the target semantic value; if the output semantic value does not belong to the set of legal values, the default semantic value corresponding to the control field type is used as the target semantic value; based on the target semantic value and its corresponding control field type, combined with the content mapping relationship, the control field content corresponding to the output semantic value is determined.

[0116] The legal value set and default semantic value corresponding to different control field types can be determined based on human experience or the control requirements of the companion robot, and this application does not impose any limitations on this. For example, the legal value set corresponding to the emotion state field "mood" includes happy, sad, angry, neutral, excited, and curious, with a corresponding default semantic value of neutral. This is used to generate the control field content using the default emotion when there is no corresponding output semantic value or the output semantic value does not belong to the legal value set. Under the behavior state field "instruction", the legal value set includes come_over, sit, wag_tail, nod, spin, lie_down, and none, with a corresponding default value of none. This is used to generate the control field content of not performing movement actions or not performing dangerous actions when there is no corresponding output semantic value or the output semantic value does not belong to the legal value set.

[0117] In this embodiment, by setting a set of legal values ​​and a default semantic value, on the one hand, the legality of the output semantic value of the lightweight language model can be verified, and if the output semantic value does not belong to the corresponding set of legal values, the default semantic value corresponding to the corresponding control field type can be used as the target semantic value, thereby avoiding the generation of illegal or unexecutable control instructions due to abnormal output semantic values; on the other hand, by using the default semantic value as a fallback output, even if the lightweight language model mistakenly generates abnormal output semantic values, or if some control field types lack corresponding output semantic values, the control field content that meets the control requirements of the companion robot can still be generated, ensuring that the human-computer interaction process is continuous and uninterrupted.

[0118] In the above embodiments, by first determining the control field type and then the control field content in a hierarchical parsing manner, the confusion of field meanings caused by directly matching the control field content based on the output semantic value can be avoided, the matching errors caused by the repetition or similarity of semantic values ​​under different control field types can be reduced, and the accuracy of intent string parsing can be improved.

[0119] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the process of generating control instructions for a companion robot based on the control field type and control field content is described in detail.

[0120] See Figure 3 The control command generation steps shown include:

[0121] S310, determine the controllable subsystem of the companion robot based on the control field type.

[0122] The subsystem to be controlled can be understood as an independent functional module in the companion robot responsible for performing the corresponding interactive functions. Different control field types can correspond to different subsystems to be controlled, or multiple subsystems to be controlled.

[0123] In some embodiments, different control field types correspond to different subsystems to be controlled. For example, a correspondence between control field types and reference subsystems can be pre-established, and the reference subsystem corresponding to the control field type determination result can be used as the subsystem to be controlled. For instance, if the identified control field type is an emotion state field, the corresponding subsystem to be controlled is determined to be an expression subsystem, a lighting effect subsystem, and / or a voice interaction subsystem, used to complete the display of emotional images, changes in lighting atmosphere, or adjustments to voice tone; if the identified control field type is a behavior state field, the corresponding subsystem to be controlled is determined to be a motion control subsystem and / or a posture control subsystem, used to complete behavioral control such as movement, turning, posture adjustment, or obstacle avoidance.

[0124] S320, determine the control parameters of the subsystem to be controlled based on the content of the control field.

[0125] In this context, control parameters can be understood as parameters or execution rules that drive the controlled subsystem to perform corresponding interactive functions. For example, control parameters may include, but are not limited to, specific configurations such as execution duration, movement speed, operating mode, display effects, and obstacle avoidance switches.

[0126] In some embodiments, after determining the subsystem to be controlled, the control parameters corresponding to the subsystem can be obtained by matching the parsed control field content.

[0127] For example, for the expression subsystem corresponding to the emotion state field, if the control field content includes a happy expression animation identifier, control parameters such as playing a happy expression animation and displaying it continuously for 3 seconds can be matched; for the motion control subsystem corresponding to the behavior state field, if the control field content includes a motion control identifier for moving towards the user, control parameters such as taking the user's current location as the target point, moving at a normal speed, and enabling obstacle avoidance function throughout can be matched, thereby matching the corresponding execution parameters according to the different control field contents.

[0128] S330 generates control commands for the companion robot based on the subsystem to be controlled and the control parameters.

[0129] For example, the identifier of the subsystem to be controlled and the corresponding control parameters can be uniformly encapsulated according to a preset communication protocol or instruction encapsulation format to generate standardized control instructions for the companion robot. These control instructions can be sent to the corresponding subsystem to be controlled via the companion robot's internal bus or control interface.

[0130] In the above embodiments, the method of first locating the subsystem to be controlled, then matching the corresponding control parameters, and finally uniformly encapsulating and generating control instructions is adopted. On the one hand, it can call only the functional modules related to the current natural language instructions to work, avoiding the ineffective scheduling of other functional modules in the companion robot. On the other hand, it can match the control instructions with the emotional state and / or behavioral state to be presented, improving the execution accuracy of the companion robot's actions and emotional interactions.

[0131] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the method for selecting the target mapping relationship is further explained.

[0132] See Figure 4 The steps for determining the target mapping relationship shown include:

[0133] S410, Determine mapping selection information.

[0134] The mapping selection information can be understood as reference information used to select the target mapping relationship from multiple candidate mapping relationships. For example, the mapping selection information can be the number of output semantic values ​​in the intent string, or the interaction scenario to which the natural language instruction belongs.

[0135] In some embodiments, the number of output semantic values ​​in the intent string can be counted, and the number of output semantic values ​​can be used as mapping selection information.

[0136] In other embodiments, the interaction scenario to which the natural language command belongs can be determined based on the current working mode of the companion robot, the currently enabled interactive functions, and / or the scene recognition results obtained by scene recognition of the text corresponding to the natural language command. The similarity threshold can be determined based on human experience or through extensive experimentation; this application does not impose any limitations on it.

[0137] Optionally, the interaction scenarios may include at least one of the following: daily companionship scenarios, game interaction scenarios, sleep companionship scenarios, and comforting companionship scenarios.

[0138] S420: Select the target mapping relationship from multiple candidate mapping relationships based on the mapping selection information.

[0139] Specifically, when the mapping selection information is the number of output semantic values, different candidate mapping relationships correspond to different numbers of output semantic values; when the mapping selection information is the interaction scenario, different candidate mapping relationships correspond to different interaction scenarios.

[0140] In one alternative implementation, a first correspondence between different output semantic values ​​and candidate mapping relationships can be predetermined; when the mapping selection information is the number of output semantic values, a target mapping relationship is selected from multiple candidate mapping relationships based on the number of output semantic values ​​and the first correspondence.

[0141] For example, candidate mapping relationships corresponding to single-output semantic values, dual-output semantic values, and multiple-output semantic values ​​can be pre-configured. When the intent string contains only a single output semantic value, the candidate mapping relationship adapted for single-field parsing is invoked to simplify the matching logic and improve parsing speed. When the intent string includes two output semantic values ​​(e.g., output semantic values ​​used to represent emotional states and output semantic values ​​used to represent behavioral states), the candidate mapping relationship adapted for multi-field parsing is invoked to improve the accuracy of multi-dimensional instruction matching.

[0142] For example, candidate mappings corresponding to a single output semantic value can include: a positional mapping where the output semantic value at the first position corresponds to the emotion state field; and a content mapping where the output semantic value "happy" corresponds to playing a happy animation for 3 seconds and the light effect changing to a warm yellow flashing; and "sad" corresponds to playing a sad animation for 5 seconds and the light effect changing to a blue slow flashing. Candidate mappings corresponding to two semantic values ​​can be as follows: a positional mapping where the output semantic value at the first position corresponds to the emotion state field, and the output semantic value at the second position corresponds to the behavior state field; and a content mapping, in addition to the content corresponding to the emotion state field, also includes the output semantic value "come_over" corresponding to moving at a normal speed with obstacle avoidance enabled, and the output semantic value "wave" corresponding to waving once in place. Candidate mappings corresponding to multiple output semantic values ​​can include: a positional mapping where the output semantic value at the first position corresponds to the emotion state field, the output semantic value at the second position corresponds to the behavior state field, and the output semantic value at the third position corresponds to the action intensity field; and a content mapping, in addition to the content corresponding to two output semantic values, also includes the output semantic value "high" corresponding to a higher action intensity parameter and the output semantic value "low" corresponding to a lower action intensity parameter.

[0143] In another alternative implementation, a second correspondence between different interaction scenarios and candidate mapping relationships can be predetermined. When the mapping selection information is an interaction scenario, the target mapping relationship is selected from multiple candidate mapping relationships by combining the interaction scenario and the second correspondence.

[0144] For example, specific mapping relationships can be pre-configured for different interaction scenarios such as daily companionship, game interaction, sleep companionship, and comfort companionship. The number, order, control field type, or control field content of the output semantic values ​​in the intent string can differ in different interaction scenarios. By configuring corresponding candidate mapping relationships for different interaction scenarios, the control needs of the companion robot in different interaction scenarios can be adapted.

[0145] For example, to improve the interaction response speed in daily companionship scenarios, candidate mapping relationships can be adapted to intent strings with single or dual output semantic values ​​to improve daily interaction response speed; in game interaction scenarios, candidate mapping relationships can include mapping rules corresponding to target object fields or action intensity fields to support game interaction behaviors such as picking up balls and following; in sleep companionship scenarios, candidate mapping relationships can include mapping rules corresponding to duration fields, volume fields, or lighting effect fields to support sleep companionship, soft feedback, or lighting adjustment; in comfort companionship scenarios, candidate mapping relationships can include mapping rules corresponding to emotional state fields and behavioral state fields to support interactive behaviors such as comforting voices, approaching the user, or gentle movements.

[0146] It is understandable that the training process of the corresponding model will be different when there are different numbers of output semantic values, and when the interaction requirements and instruction output format requirements are different for different interaction scenarios.

[0147] Accordingly, the lightweight language model is trained in the following way: determining the model generation protocol corresponding to different interaction scenarios; constructing training samples corresponding to each interaction scenario based on the model generation protocol corresponding to each interaction scenario; and performing supervised fine-tuning of the lightweight language model based on the training samples to obtain the trained lightweight language model.

[0148] The model generation protocol specifies the format of the intent string output by the lightweight language model in a corresponding interactive scenario. The format includes at least one of the following: the number of output semantic values, the control field types corresponding to the output semantic values, the order of the output semantic values, and the separation method. Each model generation protocol can be pre-configured based on the control field types, the set of presentable emotions, the set of executable behaviors, and / or the training sample annotation rules for the corresponding interactive scenario; this application does not impose any limitations on this. For example, if the model generation protocol specifies that the intent string includes two output semantic values, the model generation protocol can further specify that the output semantic value at the first position represents the emotional state, the output semantic value at the second position represents the behavioral state, and the two output semantic values ​​are connected by a preset separator.

[0149] The training samples include sample text and corresponding sample intent strings; the sample text is the text corresponding to the sample natural language instruction; the sample intent string includes one or more sample semantic values. In some embodiments, the sample intent string can be annotated according to the model generation protocol corresponding to the corresponding interaction scenario, so the sample intent string conforms to the model generation protocol corresponding to the corresponding interaction scenario.

[0150] Optionally, training samples can be constructed in at least one of the following ways:

[0151] In one alternative implementation, the real interaction process between the user and the companion robot can be simulated for each interaction scenario, and spoken natural language text can be collected as the original sample text. According to the model generation protocol of the corresponding interaction scenario, the original sample text can be labeled with the corresponding sample intent string to generate training samples.

[0152] In another alternative implementation, with user authorization, the historical interaction records of the companion robot can be read, and valid user command texts can be selected from the historical interaction records as candidate sample texts. Garbled texts, invalid statements, or texts that are outside the scope of the corresponding interaction scenario can be removed. The candidate sample texts can be labeled with sample intent strings according to the model generation protocol of the corresponding interaction scenario to generate training samples.

[0153] In some embodiments, the training samples further include sample scene prompts. These prompts characterize the interaction scene to which the sample text belongs and the format information of the intent string within that scene. When constructing the training samples, the sample scene prompts and sample text can be used as training inputs, and the sample intent strings can be used as the corresponding supervision labels.

[0154] Accordingly, in order to enable the model to fully distinguish the output specifications of different scenarios and improve the regularity of the intent string output format during model inference, in some embodiments, training samples corresponding to each interaction scenario can be constructed based on the following method: determining the sample scenario prompt words corresponding to the corresponding interaction scenario based on the model generation protocol corresponding to each interaction scenario; constructing training samples corresponding to the corresponding interaction scenario based on the sample scenario prompt words, sample text and sample intent string.

[0155] In some embodiments, the model generation protocol may include an interaction scenario identifier, the number of sample semantic values ​​in the intent string, the position of each sample semantic value in the sample intent string, the semantic meaning corresponding to each position, and the separation method between adjacent sample semantic values. Accordingly, based on the above information in the model generation protocol, sample scenario prompts corresponding to the corresponding interaction scenario can be determined.

[0156] For example, a prompt word template can be pre-set, which includes placeholder content such as the interaction scenario, the number of output semantic values, the semantic meaning of each output position, and the separation method; then, the protocol content in the model generation protocol corresponding to the corresponding interaction scenario is filled into the prompt word template to obtain the sample scenario prompt words corresponding to the interaction scenario.

[0157] For example, in a comforting and companionship scenario, the model generation protocol can specify that the intent string includes two output semantic values. The output semantic value at the first position represents the emotional state, and the output semantic value at the second position represents the behavioral state. The emotional state precedes the behavioral state, and the two output semantic values ​​are separated by a preset separator. Accordingly, the interaction scenario, the number of output semantic values, the semantic meaning of each output position, and the separation method in the above model generation protocol can be filled into a preset prompt word template to obtain sample scenario prompt words. For example, the sample scenario prompt word can be determined as "Currently a comforting and companionship scenario, please parse the input text into an intent string; the intent string includes two semantic values, the first one represents the emotional state, and the second one represents the behavioral state, separated by ' / '". When constructing training samples, the sample scenario prompt word and the collected sample text can be used as training input, and the sample intent string labeled according to the model generation protocol can be used as the supervision label corresponding to the training input to form training samples in this interaction scenario.

[0158] In this embodiment, by configuring corresponding model generation protocols for different interaction scenarios and performing supervised fine-tuning of the lightweight language model based on corresponding training samples, the lightweight language model can learn the format requirements of intent strings in different interaction scenarios, reducing the probability of semantic value errors, incorrect ordering, or incorrect separation in the intent strings output by the model, thereby improving the accuracy of subsequent semantic parsing.

[0159] In the above embodiments, by using two types of mapping selection information—the number of semantic values ​​and the interaction scenario—to adaptively match the target mapping relationship, this control instruction generation method is not limited to a single mapping rule, and can adapt to different instruction complexities and usage scenarios.

[0160] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment. In this optional embodiment, the language model-based control instruction generation method provided by this application will be described in detail.

[0161] See Figure 5 The language model-based control instruction generation method shown includes:

[0162] S510 retrieves the text corresponding to natural language commands.

[0163] The S520 parses text and outputs intent strings using a lightweight language model trained and deployed on the companion robot.

[0164] The intent string is an intermediate semantic result before generating control instructions for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot.

[0165] For example, the intent string includes at least two output semantic values ​​arranged in a preset order; wherein, the output semantic value located at the first position is used to characterize the emotional state to be presented by the companion robot; and the output semantic value located at the second position is used to characterize the behavioral state to be presented by the companion robot.

[0166] For example, the intent string also includes an output semantic value located in a third position, which is used to characterize the action intensity corresponding to the behavior state.

[0167] Optionally, the lightweight language model is trained in the following way: determining the model generation protocol corresponding to different interaction scenarios; constructing training samples corresponding to each interaction scenario based on the model generation protocol corresponding to each interaction scenario; and performing supervised fine-tuning of the lightweight language model based on the training samples to obtain the trained lightweight language model.

[0168] The model generation protocol specifies the format of the intent string output by the lightweight language model in the corresponding interactive scenario. The format includes at least one of the following: the number of output semantic values, the control field type corresponding to the output semantic values, the order of the output semantic values, and the separation method. The training samples include sample text and sample intent strings corresponding to the sample text. The sample text is the text corresponding to the sample natural language instruction. The sample intent string includes one or more sample semantic values.

[0169] Optionally, the training samples also include scene prompts. Accordingly, training samples for each interaction scene can be constructed in the following way:

[0170] Based on the model generation protocol corresponding to each interaction scenario, determine the sample scenario prompt words corresponding to the corresponding interaction scenario; based on the sample scenario prompt words, sample text and sample intent string, construct the training samples corresponding to the corresponding interaction scenario.

[0171] S530, determine the mapping selection information, and select the target mapping relationship from multiple candidate mapping relationships based on the mapping selection information; the target mapping relationship includes position mapping relationship and content mapping relationship.

[0172] The mapping selection information is either the number of output semantic values ​​in the intent string or the interaction scenario to which the natural language instruction belongs.

[0173] For example, when the mapping selection information is the number of output semantic values, different candidate mapping relationships correspond to different numbers of output semantic values; when the mapping selection information is the interaction scenario, different candidate mapping relationships correspond to different interaction scenarios.

[0174] S540: After obtaining the intent string, for any output semantic value, determine the control field type corresponding to the output semantic value based on the position of the output semantic value in the intent string and the position mapping relationship.

[0175] S550: Based on the output semantic value and its corresponding control field type, and in conjunction with the content mapping relationship, determine the content of the control field corresponding to the output semantic value.

[0176] For example, the set of legal values ​​corresponding to the control field type can be determined based on the control field type; if the output semantic value belongs to the set of legal values, the output semantic value is used as the target semantic value; if the output semantic value does not belong to the set of legal values, the default semantic value corresponding to the control field type is used as the target semantic value; and the content of the control field corresponding to the output semantic value is determined based on the target semantic value and its corresponding control field type, combined with the content mapping relationship.

[0177] S560 determines the controllable subsystem of the companion robot based on the control field type.

[0178] S570, determine the control parameters of the subsystem to be controlled based on the content of the control field.

[0179] The S580 generates control commands for the companion robot based on the subsystem to be controlled and the control parameters.

[0180] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0181] Based on the same inventive concept, this application also provides a language model-based control instruction generation device for implementing the language model-based control instruction generation method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more language model-based control instruction generation device embodiments provided below can be found in the limitations of the language model-based control instruction generation method described above, and will not be repeated here.

[0182] In one exemplary embodiment, such as Figure 6 As shown, a control instruction generation device based on a language model is provided, comprising: a text acquisition module 610, a text parsing module 620, a determination module 630, and a generation module 640, wherein:

[0183] The text acquisition module 610 is used to acquire the text corresponding to natural language instructions;

[0184] The text parsing module 620 is used to parse the text using a lightweight language model deployed on the companion robot and output an intent string. The intent string is an intermediate semantic result before generating control commands for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot.

[0185] The determination module 630 is used to determine the control field type and control field content corresponding to any output semantic value after obtaining the intent string, based on the position of the output semantic value in the intent string and the output semantic value, combined with the target mapping relationship.

[0186] The generation module 640 is used to generate control instructions for the companion robot based on the control field type and control field content.

[0187] In one embodiment, the target mapping relationship includes a position mapping relationship and a content mapping relationship; correspondingly, the determining module includes a first determining unit, used to determine the control field type corresponding to the output semantic value based on the position of the output semantic value in the intent string and in combination with the position mapping relationship; and a second determining unit, used to determine the control field content corresponding to the output semantic value based on the output semantic value and its corresponding control field type and in combination with the content mapping relationship.

[0188] In one embodiment, the second determining unit is specifically used to determine the set of legal values ​​corresponding to the control field type based on the control field type; if the output semantic value belongs to the set of legal values, the output semantic value is used as the target semantic value; if the output semantic value does not belong to the set of legal values, the default semantic value corresponding to the control field type is used as the target semantic value; and based on the target semantic value and its corresponding control field type, combined with the content mapping relationship, the content of the control field corresponding to the output semantic value is determined.

[0189] In one embodiment, the generation module includes a third determining unit for determining the control subsystem of the companion robot based on the control field type; a fourth determining unit for determining the control parameters of the control subsystem based on the control field content; and a generation unit for generating control instructions for the companion robot based on the control subsystem and the control parameters.

[0190] In one embodiment, the language model-based control instruction generation device further includes a mapping relationship selection module, used to determine mapping selection information; the mapping selection information is the number of output semantic values ​​in the intent string, or the interaction scenario to which the natural language instruction belongs; according to the mapping selection information, a target mapping relationship is selected from multiple candidate mapping relationships; wherein, when the mapping selection information is the number of output semantic values, different candidate mapping relationships correspond to different numbers of output semantic values; when the mapping selection information is an interaction scenario, different candidate mapping relationships correspond to different interaction scenarios.

[0191] In one embodiment, the language model-based control instruction generation device further includes a model training module for determining model generation protocols corresponding to different interaction scenarios. The model generation protocol specifies the format of the intent string output by the lightweight language model in the corresponding interaction scenario. The format includes at least one of the following: the number of output semantic values, the control field type corresponding to the output semantic values, the order of the output semantic values, and the separation method. Based on the model generation protocol corresponding to each interaction scenario, training samples corresponding to each interaction scenario are constructed. The training samples include sample text and sample intent strings corresponding to the sample text. The sample text is the text corresponding to the sample natural language instruction. The sample intent string includes one or more sample semantic values. Based on the training samples, the lightweight language model is subjected to supervised fine-tuning to obtain the trained lightweight language model.

[0192] In one embodiment, the training samples further include sample scene prompts; the model training module is specifically used to determine the sample scene prompts corresponding to the corresponding interaction scenario based on the model generation protocol corresponding to each interaction scenario; and to construct the training samples corresponding to the corresponding interaction scenario based on the sample scene prompts, sample text, and sample intent strings.

[0193] In one embodiment, the intent string output by the text parsing module includes at least two output semantic values ​​arranged in a preset order; wherein, the output semantic value located at the first position is used to characterize the emotional state to be presented by the companion robot; and the output semantic value located at the second position is used to characterize the behavioral state to be presented by the companion robot.

[0194] In one embodiment, the intent string output by the text parsing module also includes an output semantic value located in a third position, which is used to characterize the action intensity corresponding to the behavior state.

[0195] Each module in the aforementioned language model-based control instruction generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0196] In one exemplary embodiment, a companion robot is provided, the internal structure of which can be shown in the following diagram. Figure 7 As shown, the companion robot includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a language model-based control instruction generation method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0197] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the companion robot to which the present application is applied. A specific companion robot may include more or fewer parts than shown in the figure, or combine certain parts, or have different part arrangements.

[0198] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0199] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0200] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0201] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0202] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0203] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0204] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating control instructions based on a language model, applied to a companion robot, characterized in that, The method includes: Retrieve the text corresponding to natural language commands; The text is parsed using a lightweight language model deployed on the companion robot after training, and an intent string is output. The intent string is an intermediate semantic result before generating control commands for the companion robot. The intent string includes one or more output semantic values, which are used to characterize the emotional state and / or behavioral state to be presented by the companion robot. After obtaining the intent string, for any output semantic value, based on the position of the output semantic value in the intent string and the output semantic value, and in conjunction with the target mapping relationship, the control field type and control field content corresponding to the output semantic value are determined; Based on the control field type and the control field content, control instructions are generated for the companion robot.

2. The method according to claim 1, characterized in that, The target mapping relationship includes a position mapping relationship and a content mapping relationship; correspondingly, determining the control field type and control field content corresponding to the output semantic value based on the position of the output semantic value in the intent string and the output semantic value, combined with the target mapping relationship, includes: Based on the position of the output semantic value in the intent string, and in conjunction with the position mapping relationship, the control field type corresponding to the output semantic value is determined; Based on the output semantic value and its corresponding control field type, and in conjunction with the content mapping relationship, the content of the control field corresponding to the output semantic value is determined.

3. The method according to claim 2, characterized in that, The step of determining the content of the control field corresponding to the output semantic value based on the output semantic value and its corresponding control field type, combined with the content mapping relationship, includes: Based on the control field type, determine the set of legal values ​​corresponding to the control field type; If the output semantic value belongs to the set of legal values, the output semantic value is taken as the target semantic value; If the output semantic value does not belong to the set of legal values, the default semantic value corresponding to the control field type shall be used as the target semantic value. Based on the target semantic value and its corresponding control field type, and in conjunction with the content mapping relationship, the content of the control field corresponding to the output semantic value is determined.

4. The method according to claim 1, characterized in that, The step of generating control instructions for the companion robot based on the control field type and the control field content includes: The control subsystem of the companion robot is determined based on the control field type; The control parameters of the subsystem to be controlled are determined based on the contents of the control field. Based on the subsystem to be controlled and the control parameters, control commands are generated for the companion robot.

5. The method according to any one of claims 1 to 4, characterized in that, Before determining the control field type and control field content corresponding to the output semantic value by combining the target mapping relationship, the method further includes: Determine mapping selection information; the mapping selection information is the number of output semantic values ​​in the intent string, or the interaction scenario to which the natural language instruction belongs; Based on the mapping selection information, the target mapping relationship is selected from multiple candidate mapping relationships; Specifically, when the mapping selection information is the number of output semantic values, different candidate mapping relationships correspond to different numbers of output semantic values; when the mapping selection information is the interaction scenario, different candidate mapping relationships correspond to different interaction scenarios.

6. The method according to any one of claims 1 to 4, characterized in that, The lightweight language model is trained using the following method: A model generation protocol is determined for different interaction scenarios; wherein, the model generation protocol is used to specify the format of the intent string output by the lightweight language model in the corresponding interaction scenario; the format includes at least one of the following: the number of output semantic values, the control field type corresponding to the output semantic values, the arrangement order of the output semantic values, and the separation method; Based on the model generation protocol corresponding to each of the interaction scenarios, training samples corresponding to each of the interaction scenarios are constructed; wherein, the training samples include sample text and sample intent strings corresponding to the sample text; the sample text is the text corresponding to the sample natural language instruction; the sample intent string includes one or more sample semantic values; Based on the training samples, the lightweight language model is subjected to supervised fine-tuning to obtain the trained lightweight language model.

7. The method according to claim 6, characterized in that, The training samples also include sample scene prompts; correspondingly, the step of constructing training samples corresponding to each interaction scene based on the model generation protocol for each interaction scene includes: Based on the model generation protocol corresponding to each interaction scenario, determine the sample scenario prompt words corresponding to the respective interaction scenario; Based on the sample scene prompts, the sample text, and the sample intent string, the training samples corresponding to the respective interaction scenarios are constructed.

8. The method according to any one of claims 1 to 4, characterized in that, The intent string includes at least two output semantic values ​​arranged in a preset order; The output semantic value located at the first position is used to characterize the emotional state to be presented by the companion robot; the output semantic value located at the second position is used to characterize the behavioral state to be presented by the companion robot.

9. The method according to claim 8, characterized in that, The intent string also includes an output semantic value located at a third position, which is used to characterize the action intensity corresponding to the behavior state.

10. A companion robot, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-9.

Images