Model adjustment method and device, electronic equipment and readable storage medium

Abstract

The application is suitable for the technical field of robots, and provides a model adjustment method and device, electronic equipment and a readable storage medium. The model adjustment method comprises: in response to a control instruction, inputting environment information corresponding to the control instruction into a large language model to obtain a text label output by the large language model, wherein the text label comprises an action label of a to-be-executed action; inputting the action label into a control model of a mechanical arm to control the mechanical arm of the robot to execute the to-be-executed action through the control model; and in response to feedback information of the to-be-executed action, adjusting model parameters of the large language model and the control model according to the feedback information. The embodiment of the application can improve the understanding of the large language model for instructions, so that the robot can better execute the control command.

Description

Technical Field

[0001] This application belongs to the field of robotics technology, and in particular relates to a method, apparatus, electronic device, and readable storage medium for adjusting a model. Background Technology

[0002] Large language models (LLMs) are deep neural network models based on the Transformer architecture. They possess multi-turn dialogue capabilities, can generate natural language text or understand the meaning of language text, and handle various natural language tasks. However, when applying LLMs to the field of robotics, because the responses of LLMs are usually based on instruction text, the answers they provide are prone to inaccuracy or irrelevant to the user's control intentions. Summary of the Invention

[0003] This application provides a model adjustment method, apparatus, electronic device, and readable storage medium, which can solve the problem in related technologies where large speech models have insufficient understanding of instructions, resulting in robots being unable to control commands well.

[0004] The first aspect of this application provides a method for adjusting a model, comprising: responding to a control command, inputting environmental information corresponding to the control command into a large language model to obtain text labels output by the large language model, wherein the text labels include action labels of an action to be performed; inputting the action labels into a control model of a robotic arm to control the robotic arm of a robot to perform the action to be performed through the control model; and adjusting the model parameters of the large language model and the control model according to the feedback information on the action to be performed.

[0005] In some embodiments of the first aspect, the environmental information is multimodal information, which includes at least image information and voice information; before inputting the environmental information corresponding to the control command into the large language model, the method for adjusting the model further includes: acquiring the image information collected by the robot's image sensor and the voice information collected by the robot's microphone.

[0006] In some embodiments of the first aspect, inputting the environmental information corresponding to the control command into a large language model includes: encoding the image information to obtain a text vector corresponding to the image information; converting the speech information into text information; and inputting the text vector and the text information into the large language model.

[0007] In some embodiments of the first aspect, the control model is a reinforcement learning model, and the loss function of the reinforcement learning model is the Bellman equation; adjusting the model parameters of the control model according to the feedback information includes: when the feedback information is positive feedback information, generating a model reward, so as to adjust the model parameters of the control model based on the model reward and the Bellman equation.

[0008] In some embodiments of the first aspect, the text tag further includes a voice tag for the speech to be output; after obtaining the text tag output by the large language model, the adjustment method of the model further includes: inputting the voice tag into a speech processing model to control the robot's speaker to output the speech to be output through the speech processing model.

[0009] In some embodiments of the first aspect, after obtaining the text labels output by the large language model, the model adjustment method further includes: inputting the action labels into a driving model of the self-moving system to control the robot's self-moving system through the driving model to control the robot to perform pose adjustment.

[0010] In some embodiments of the first aspect, after adjusting the model parameters of the large language model and the control model according to the feedback information, the model adjustment method further includes: in response to the suggestion information, using the suggestion information as a prompt instruction to perform a secondary adjustment of the model parameters of the large language model and the control model.

[0011] A model adjustment device provided in the second aspect of this application includes: a large model processing unit, configured to, in response to a control command, input environmental information corresponding to the control command into a large language model to obtain text tags output by the large language model, wherein the text tags include action tags for an action to be performed; a control model processing unit, configured to input the action tags into a control model of a robotic arm to control the robotic arm of the robot to perform the action to be performed; and a model adjustment unit, configured to, in response to feedback information on the action to be performed, adjust the model parameters of the large language model and the control model according to the feedback information.

[0012] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the adjustment method of the above-described model.

[0013] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the adjustment method for the above-described model.

[0014] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the steps of the adjustment method of the above-described model.

[0015] In the embodiments of this application, in response to control commands, environmental information corresponding to the control commands is input into a large language model to obtain text labels output by the large language model. The action labels included in the text labels are then input into the control model of the robotic arm, so that the robotic arm of the robot can be controlled by the control model to perform the action to be performed. Then, in response to the feedback information of the action to be performed, the model parameters of the large language model and the control model are adjusted according to the feedback information. On the one hand, the environmental information can be used as input to enable the large language model to have a higher understanding of the commands. On the other hand, the model can be optimized according to the feedback information of the user on the mechanical execution of the action, so that the model is more in line with the user's control intention, thereby enabling the robot to better execute control commands. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the implementation process of a model adjustment method provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram illustrating the specific implementation process of a model adjustment method provided in an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of the structure of a model adjustment device provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.

[0022] Large language models (or simply large models) refer to deep neural network models based on the Transformer architecture. They possess multi-turn dialogue capabilities, can generate natural language text or understand the meaning of language text, and handle various natural language tasks. However, when applying large language models to the field of robotics, because their responses are usually based on instruction text, the answers they provide are prone to inaccuracy or irrelevant to the user's control intentions.

[0023] Therefore, this application improves the understanding of instructions and the reliability of robot execution of instructions by adjusting the model input and optimizing the large language model based on the action of a robotic arm.

[0024] To illustrate the technical solution of this application, specific embodiments are described below.

[0025] Figure 1 This illustration shows a schematic flowchart of a model adjustment method provided in an embodiment of this application. This method can be applied to electronic devices and is suitable for situations where it is necessary to improve the understanding of instructions by a large language model, enabling the robot to better execute control commands. The aforementioned electronic device can be a robot, an unmanned outdoor vehicle, a computer, a smartphone, or other intelligent device; this application does not limit its scope.

[0026] Specifically, the adjustment method of the above model may include the following steps S101 to S103.

[0027] Step S101: In response to the control command, the environmental information corresponding to the control command is input into the large language model to obtain the text label output by the large language model.

[0028] Among them, control commands can be user-triggered commands that instruct the robot to perform specific actions or complete specific tasks.

[0029] In the embodiments of this application, the triggering method and content of the control commands can be adjusted according to actual conditions. Command content may include, but is not limited to: voice, text, actions, facial expressions, etc. Triggering methods may include, but are not limited to: touchscreen operation, button operation, remote control operation, etc.

[0030] The environmental information corresponding to control commands refers to the information collected by the robot from its surrounding environment after receiving the control commands. This environmental information helps the large language model understand the user's situation, enabling it to better comprehend the control commands. By processing the environmental information, the large language model can output text labels in natural language, which describe the requirements of the control commands.

[0031] Specifically, text tags can include action tags for the actions to be performed, which can describe the type and specific content of the actions to be performed. In addition, text tags can also include speech tags for the speech to be output, which can describe the content of the speech to be output.

[0032] Step S102: Input the action label into the control model of the robotic arm so that the robotic arm of the robot can be controlled by the control model to perform the action to be performed.

[0033] In the embodiments of this application, the control model of the robotic arm is also an artificial intelligence model, which can be used to analyze and process action tags, and output instruction information that the robotic arm understands and corresponds to the action to be executed, so as to control the robotic arm of the robot to execute the action to be executed. The instruction information may include the posture angle, coordinates, bending angle, grasping range, etc. of the robotic arm. The action to be executed includes, but is not limited to, gripping and extending.

[0034] Step S103: In response to the feedback information of the action to be executed, adjust the model parameters of the large language model and the control model according to the feedback information.

[0035] Specifically, feedback information is the information output by the user regarding the robotic arm's execution results, which can be positive or negative. When the robotic arm executes the command correctly, the user can provide positive feedback, such as by nodding or confirming via touchscreen on the robot's display. When the robotic arm does not execute the command correctly, the user can provide negative feedback, such as by shaking their head or inputting a voice message containing negative keywords.

[0036] In response to feedback information on the actions to be performed, the electronic device can adjust the model parameters of the large language model and the control model based on the feedback information to optimize the large language model and the control model, so that when the large language model and the control model receive control commands again, the actions performed by the robotic arm are more in line with the user's control intentions.

[0037] In the embodiments of this application, in response to control commands, environmental information corresponding to the control commands is input into a large language model to obtain text labels output by the large language model. The action labels included in the text labels are then input into the control model of the robotic arm, so that the robotic arm of the robot can be controlled by the control model to perform the action to be performed. Then, in response to the feedback information of the action to be performed, the model parameters of the large language model and the control model are adjusted according to the feedback information. On the one hand, the environmental information can be used as input to enable the large language model to have a higher understanding of the commands. On the other hand, the model can be optimized according to the feedback information of the user on the mechanical execution of the action, so that the model is more in line with the user's control intention, thereby enabling the robot to better execute control commands.

[0038] Specifically, to improve the understanding capability of the large language model, the aforementioned environmental information can be multimodal information, which can include at least image information and speech information. Before inputting the environmental information corresponding to the control commands into the large language model, the electronic device can acquire image information collected by the robot's image sensor and speech information collected by the robot's microphone.

[0039] Next, the electronic device can encode the image information to obtain the corresponding text vector, and convert the speech information into text information. Then, the text vector and text information are input into the large language model. In this way, the input information can be made to meet the requirements of the large language model.

[0040] Specifically, image information encoding can be implemented using an encoder model. The encoder model can extract features from the image information and generate embedding text vectors based on these features. For speech information, a speech recognition model can be used to convert the speech information into corresponding text information.

[0041] Subsequently, the large speech model can process text information and text vectors to obtain text labels. Action labels within the text information can be input into the control model for processing.

[0042] The control model of the robotic arm can be a reinforcement learning model, and the loss function of the reinforcement learning model is the Bellman equation. During the adjustment of the model parameters of the control model based on feedback information, when the feedback information is positive, a model reward can be generated, and the model parameters of the control model can be adjusted based on the model reward and the Bellman equation.

[0043] Specifically, the Bellman equation can be expressed as:

[0044] V(s) = max a (R(s,a)+γΣ s ,P(s,a,s')V(s'));

[0045] In this model, V() represents the loss value, R() represents the model reward, s represents the current state of the robotic arm (e.g., current pose), a represents the current action of the robotic arm, γ is a preset coefficient, s′ represents the state of the robotic arm at the previous moment, P() represents the probability of each executable action of the robotic arm based on a, s, and s′, and max() represents taking the maximum value. When adjusting the model parameters, P() can be adjusted to change the probability of each executable action, thereby changing the resulting loss value. When optimizing the model using gradient descent, the highest probability executable action can be more closely matched to the action to be executed, thus making the model result more consistent with the user's control intent.

[0046] In other embodiments of this application, the text tag may further include a voice tag for the speech to be output. After obtaining the text tag output by the large language model, the electronic device can input the voice tag into the speech processing model to control the robot's audio output of the speech to be output through the speech processing model.

[0047] In other embodiments of this application, after obtaining the text labels output by the large language model, the electronic device can also input the action labels into the driving model of the self-mobilization system, so as to control the robot's self-mobilization system through the driving model, thereby controlling the robot to perform pose adjustments. The self-mobilization system may include a chassis, a navigation system, and a Simultaneous Localization and Mapping (SLAM) system.

[0048] The speech processing model and the driving model can be pre-trained deep learning models, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), etc., and this application does not impose any restrictions on them.

[0049] After adjusting the model parameters of the large language model and the control model based on the feedback information, the electronic device can also respond to the suggestion information and use the suggestion information as a prompt instruction to make a second adjustment to the model parameters of the large language model and the control model.

[0050] The suggestion information describes the optimization suggestions given by the user for the robotic arm's movements, such as a smaller arm bend, a larger arm bend, a smaller gripping range, etc. This suggestion information can serve as a prompt to further adjust the model parameters of the large language model and the control model, thereby optimizing the processing results of both models. Specifically, the suggestion information can be used as a new control command, correspondingly executing the aforementioned steps S101 to S103, causing the model parameters of the large language model and the control model to be adjusted again.

[0051] For ease of understanding, Figure 2 This diagram illustrates the interaction between a user and a robot. The user can issue control commands to the robot, which uses image sensors, microphones, and other components to collect environmental information. The collected speech information is then fed into a speech processing model to convert it into text, and the image information is encoded into text vectors. A large language model processes the text information and text vectors, outputting action and speech labels. The robot control and drive models output specific commands to the self-movement system and robotic arm based on the action labels, while the speech processing model outputs specific commands to the speakers based on the speech labels. The user monitors the execution results of the large language model, providing positive or negative feedback. Based on the feedback, the large language model and control model can update their parameters. Furthermore, the user can provide suggestions based on the robotic arm's movements as new prompts to further optimize the large language model and control model.

[0052] In the embodiments of this application, a large language model is deployed to the robot, which can process user-triggered actions in different forms. During the robot's execution of instructions, the user provides phased feedback and suggestions for model optimization, thereby improving the robot's execution of control instructions.

[0053] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders.

[0054] like Figure 3 The diagram shown is a schematic diagram of a model adjustment device 300 provided in an embodiment of this application. The model adjustment device 300 is disposed on an electronic device.

[0055] The large model processing unit 301 is used to respond to control commands by inputting environmental information corresponding to the control commands into the large language model to obtain text tags output by the large language model, wherein the text tags include action tags of actions to be performed.

[0056] The control model processing unit 302 is used to input the action label into the control model of the robotic arm, so as to control the robotic arm of the robot to perform the action to be performed through the control model;

[0057] The model adjustment unit 303 is used to adjust the model parameters of the large language model and the control model in response to feedback information on the action to be executed.

[0058] In some embodiments of this application, the environmental information is multimodal information, which includes at least image information and voice information; the adjustment device 300 of the above model may further include an acquisition unit, specifically used to: acquire the image information collected by the robot's image sensor and the voice information collected by the robot's microphone.

[0059] In some embodiments of this application, the large model processing unit 301 described above can be specifically used to: encode the image information to obtain a text vector corresponding to the image information; convert the speech information into text information; and input the text vector and the text information into the large language model.

[0060] In some embodiments of this application, the control model is a reinforcement learning model, and the loss function of the reinforcement learning model is the Bellman equation; the model adjustment unit 303 can be specifically used to: generate a model reward when the feedback information is positive feedback information, so as to adjust the model parameters of the control model based on the model reward and the Bellman equation.

[0061] In some embodiments of this application, the text tag also includes a voice tag for the speech to be output; the adjustment device 300 of the above model may also include a voice model processing unit, specifically used to: input the voice tag to the voice processing model, so as to control the robot's speaker to output the speech to be output through the voice processing model.

[0062] In some embodiments of this application, the model adjustment device 300 may further include a drive model processing unit, specifically used to: input the motion label into the drive model of the self-moving system, so as to control the self-moving system of the robot through the drive model, so as to control the robot to perform pose adjustment.

[0063] In some embodiments of this application, the model adjustment device 300 may further include a secondary adjustment unit, specifically used to: in response to suggestion information, use the suggestion information as a prompting instruction to perform secondary adjustments on the model parameters of the large language model and the control model.

[0064] It should be noted that, for the sake of convenience and brevity, the specific working process of the adjustment device 300 in the above model can be found in the following reference: Figures 1 to 2 The corresponding process of the method will not be described in detail here.

[0065] like Figure 4The diagram shown is a schematic representation of an electronic device provided in an embodiment of this application. Specifically, the electronic device 4 may include: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40, such as a model adjustment program. When the processor 40 executes the computer program 42, it implements the steps in the various model adjustment method embodiments described above, for example... Figure 1 The steps S101 to S103 are shown. Alternatively, when the processor 40 executes the computer program 42, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 3 The functions of the large model processing unit 301, the control model processing unit 302, and the model adjustment unit 303 are shown.

[0066] The computer program can be divided into one or more modules / units, which are stored in the memory 41 and executed by the processor 40 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0067] For example, the computer program can be divided into: a large model processing unit, a control model processing unit, and a model adjustment unit. The specific functions of each unit are as follows: The large model processing unit, in response to control commands, inputs the environmental information corresponding to the control commands into the large language model to obtain text tags output by the large language model, the text tags including action tags for the action to be executed; the control model processing unit, in response to feedback information on the action to be executed, adjusts the model parameters of the large language model and the control model according to the feedback information.

[0068] The electronic device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that... Figure 4 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0069] The processor 40 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0070] The memory 41 can be an internal storage unit of the electronic device, such as a hard drive or memory. The memory 41 can also be an external storage device of the electronic device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 41 can include both internal and external storage units. The memory 41 is used to store the computer program and other programs and data required by the electronic device. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0071] It should be noted that, for the sake of convenience and brevity, the structure of the above-mentioned electronic device can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.

[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0073] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0074] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.

[0075] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0076] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0077] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0078] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0079] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for adjusting a model, characterized in that, include: In response to a control command, the environmental information corresponding to the control command is input into the large language model to obtain text tags output by the large language model, wherein the text tags include action tags of the action to be performed; The action label is input into the control model of the robotic arm, so that the robotic arm of the robot can be controlled by the control model to perform the action to be performed; The motion labels are input into the driving model of the self-movement system, so as to control the robot's self-movement system through the driving model, and control the robot to perform pose adjustment; In response to feedback information regarding the action to be performed, the model parameters of the large language model and the control model are adjusted based on the feedback information. During the process of adjusting the model parameters of the control model based on feedback information, when the feedback information is positive feedback information, a model reward is generated. The model parameters of the control model are then adjusted based on the model reward and the Bellman equation, so that the probability of each executable action changes. The model is then optimized using the gradient descent method to make the executable action with the highest probability closer to the action to be executed. The Bellman equation is expressed as: ； in, Indicates the loss value. Indicates the model reward. This indicates the current state of the robotic arm. This indicates the current action of the robotic arm. The preset coefficients, This indicates the state of the robotic arm at the previous moment. Indicates based on , , The probabilities of each executable action of the robotic arm are obtained, and max() represents taking the maximum value. The current state includes the current pose.

2. The method for adjusting the model as described in claim 1, characterized in that, The environmental information is multimodal information, which includes at least image information and voice information; Before inputting the environmental information corresponding to the control command into the large language model, the model adjustment method further includes: The image information collected by the robot's image sensor and the voice information collected by the robot's microphone are acquired.

3. The model adjustment method as described in claim 2, characterized in that, The step of inputting the environmental information corresponding to the control command into the large language model includes: The image information is encoded to obtain the text vector corresponding to the image information; Convert the voice information into text information; The text vector and the text information are input into the large language model.

4. The method for adjusting the model as described in any one of claims 1 to 3, characterized in that, The text tags also include the voice tags for the speech to be output; After obtaining the text labels output by the large language model, the model adjustment method further includes: The voice tags are input into the voice processing model, which then controls the robot's speakers to output the voice to be output.

5. The method for adjusting the model as described in any one of claims 1 to 3, characterized in that, After adjusting the model parameters of the large language model and the control model based on the feedback information, the model adjustment method further includes: In response to the suggestion information, the suggestion information is used as a prompt instruction to make secondary adjustments to the model parameters of the large language model and the control model.

6. A model adjustment device, characterized in that, include: A large model processing unit is used to respond to control commands by inputting the environmental information corresponding to the control commands into a large language model to obtain text tags output by the large language model, wherein the text tags include action tags of actions to be performed. A control model processing unit is used to input the action label into the control model of the robotic arm, so as to control the robotic arm of the robot to perform the action to be performed through the control model; The driving model processing unit is used to: input the action label into the driving model of the self-movement system, so as to control the self-movement system of the robot through the driving model, and control the robot to perform pose adjustment; A model adjustment unit is used to adjust the model parameters of the large language model and the control model in response to feedback information on the action to be executed, based on the feedback information. The model adjustment unit is used to generate a model reward when the feedback information is positive feedback information during the process of adjusting the model parameters of the control model based on the feedback information. The model parameters of the control model are then adjusted based on the model reward and the Bellman equation, so that the probability of each executable action changes. The model is then optimized by gradient descent so that the executable action with the highest probability is closer to the action to be executed. The Bellman equation is expressed as: ； in, Indicates the loss value. Indicates the model reward. This indicates the current state of the robotic arm. This indicates the current action of the robotic arm. The preset coefficients, This indicates the state of the robotic arm at the previous moment. Indicates based on , , The probabilities of each executable action of the robotic arm are obtained, and max() represents taking the maximum value. The current state includes the current pose.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the adjustment method for the model as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the adjustment method for the model as described in any one of claims 1 to 5.

Images