Action generation method and device, electronic product, storage medium and program product

Abstract

The embodiment of the invention discloses an action generation method and device, an electronic product, a computer readable storage medium and a computer program product, and the method comprises the steps: generating thinking process content, a discrete action lexical element sequence and a lexical element coding sequence of each lexical element through an action language model according to prompt words and action description data; and then decoding processing is carried out on the lexical element coding sequence, and finally corresponding action data is generated. Therefore, under the control and constraint of the cue words and the action description data, the action language model is used for thinking and reasoning the complex action description and decomposing the complex action description into the basic actions in sequence, so that the complex semantic understanding ability of the action language model is enhanced, the action generation process of the action language model can be output, and the user experience is improved. The method includes thinking process content, semantic logic, lexical element coding sequences and the like, more accurate control over the generation process of action semantics is achieved, and therefore high-quality action data is generated.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence technology, and in particular to an action generation method, device, electronic product, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, the field of AI generation has made significant progress in recent years. Three-dimensional human motion generation has gradually become a key research direction in computer vision and graphics. It has broad applications in virtual humans, intelligent interaction, animation production, intelligent robotics, and other scenarios, and is crucial for enhancing the naturalness of human-computer interaction and the intelligence of task execution.

[0003] The goal of 3D human motion generation technology is to generate structurally complete, semantically coherent, and natural human motion sequences. Existing text-driven human motion generation techniques typically rely on generative model structures such as language models or diffusion models. They first represent the action in a discrete manner, then encode the input text into a context vector, which serves as a conditional input to guide the motion generation network to output the action sequence. While these methods perform well on closed datasets, they do not explicitly model the structural constraints, semantic logic, or reasoning process in the motion generation process. Instead, they rely solely on data-driven generative networks for prediction, lacking high-level control over the semantics of the action. Summary of the Invention

[0004] In order to solve the technical problems in the related art, the present disclosure provides an action generation method, device, electronic product, computer-readable storage medium and computer program product.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an action generation method, the method comprising: receiving a first prompt word and first action description data input by a user; Based on the first prompt word and the first action description data, generating, using an action language model, first thought process content, a first action lemma sequence, and a first lemma encoding sequence for each action lemma in the first action lemma sequence, wherein the first action lemma sequence is a series of discrete action lemmas obtained by decomposing the first action description data according to the first prompt word; The first action coding sequence is decoded by a preset coding tool to generate target action data.

[0006] As an optional embodiment, the method further includes: Constructing training data for the action language model, the training data including a preset number of triples, the triples including second action description data, second thought process content, a second action lemma sequence, and a lemma code corresponding to each action lemma in the second action lemma sequence; Constructing a vocabulary table for the action language model, wherein the vocabulary table includes a correspondence between word unit codes and action numbers; Based on the training data and the vocabulary, a large model supervised fine-tuning technology is used to train a preset action generation algorithm to obtain the action language model.

[0007] As an optional embodiment, the acquiring of training data for the action language model, the training data including a preset number of triples, the triples including the second action description data, the second thought process content, the second action lemma sequence, and a lemma code corresponding to each action lemma in the second action lemma sequence, includes: Acquire a preset number of source data pairs, any one of which includes the second prompt word and the second action description data; Based on the source data pair, the training data is generated by a preset large language model and the preset encoding and decoding tool, and the triple data includes second action description data, second thinking process content and second action word sequence, and word encoding corresponding to each action word in the second action word sequence.

[0008] As an optional embodiment, the training data is generated based on the source data pair using a preset large language model and the preset codec tool, wherein the triple data includes the second action description data, the second thought process content, and the second action word-gram sequence, and the word-gram encoding corresponding to each action word-gram in the second action word-gram sequence, including: generating a second thought process content and a second action word-gram sequence using the preset large language model, wherein the second action word-gram sequence is a series of discrete action word-grams obtained by decomposing the second action description data according to the second prompt word, and the second thought process content is text corresponding to the analysis process of the preset large language model generating the second action word-gram sequence; Encoding the second action word-gram sequence using a preset encoding and decoding tool to obtain a word-gram code corresponding to each action word-gram in the second action word-gram sequence; The second action description data, the second thinking process content, the second action word-gram sequence, and the word-gram encoding corresponding to each action word-gram in the second action word-gram sequence are combined into triple data, and the triple data is the training data.

[0009] As an optional embodiment, the constructing of a vocabulary for the action language model includes a correspondence between word unit codes and action numbers, including: Using a preset encoding and decoding tool to encode each action word in the second action word sequence to obtain a word code corresponding to each action word; The actions corresponding to the word unit codes are numbered to obtain corresponding action numbers, so that the corresponding relationship between the word unit codes and the action numbers is determined as the vocabulary.

[0010] As an optional embodiment, the method further includes: performing a reward evaluation on an output data set outputted by the action language model and the preset codec tool to obtain a composite reward score, wherein the output data set includes the first thought process content, the first action word-gram sequence, a first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data; The network parameters of the action language model are optimized according to the compound reward score.

[0011] As an optional embodiment, the output data group output by the action language model and the preset codec tool is rewarded and evaluated to obtain a composite reward score, wherein the output data group includes the first thought process content, the first action word-gram sequence, the first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data, including: For the same set of the first prompt word and the first action description data, collecting a preset number of output data groups; Performing reward evaluation on any group of the output data groups according to the preset evaluation dimension to obtain the same number of evaluation scores as the preset number of groups; Performing a compound calculation on the evaluation scores of the same number as the preset number of groups using a preset compound function to obtain the compound reward score; The output data group includes the first thought process content, the first action word-gram sequence, the first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data.

[0012] As an optional embodiment, the optimizing the network parameters of the action language model according to the composite reward score includes: For the same set of the first prompt word and the first action description data, comparing the evaluation score of each of the output data groups with the composite reward score to obtain a corresponding advantage comparison value; The network parameters of the action language model are optimized according to the advantage comparison value corresponding to each group of the output data groups.

[0013] As an optional embodiment, the optimizing the network parameters of the action language model according to the advantage comparison value corresponding to each group of the output data groups includes: Performing weighted processing on the advantage comparison values ​​corresponding to each group of the output data groups to obtain a weighted advantage value; The network parameters of the action language model are optimized according to the weighted advantage value.

[0014] According to a second aspect of an embodiment of the present disclosure, there is provided an action generation device, the device comprising: A receiving module, configured to receive a first prompt word and first action description data input by a user; an action thinking module, configured to generate, based on the first prompt word and the first action description data, a first thinking process content, a first action word sequence, and a first word encoding sequence of each action word in the first action word sequence using an action language model, wherein the first action word sequence is a series of discrete action words obtained by decomposing the first action description data according to the first prompt word; The encoding and decoding module is used to decode the first action coding sequence through a preset encoding and decoding tool to obtain target action data.

[0015] As an optional embodiment, the device further includes: a first construction module, configured to construct training data for the action language model, the training data comprising a preset number of triples, the triples comprising second action description data, second thought process content, a second action lemma sequence, and a lemma code corresponding to each action lemma in the second action lemma sequence; A second building module is used to build a vocabulary table for the action language model, wherein the vocabulary table includes a correspondence between word unit codes and action numbers; A model training module is used to train a preset action generation algorithm based on the training data and the vocabulary using a large model supervised fine-tuning technique to obtain the action language model.

[0016] As an optional embodiment, the first building module includes: a source data acquisition unit, configured to acquire a preset number of source data pairs, any one of the source data pairs comprising the second prompt word and the second action description data; A training data generation unit is used to generate the training data based on the source data pair through a preset large language model and the preset encoding and decoding tool, wherein the triple data includes second action description data, second thinking process content and second action word-gram sequence, and word-gram encoding corresponding to each action word-gram in the second action word-gram sequence.

[0017] As an optional embodiment, the training data generating unit is further configured to: generating a second thought process content and a second action word-gram sequence using the preset large language model, wherein the second action word-gram sequence is a series of discrete action word-grams obtained by decomposing the second action description data according to the second prompt word, and the second thought process content is text corresponding to the analysis process of the preset large language model generating the second action word-gram sequence; Encoding the second action word-gram sequence using a preset encoding and decoding tool to obtain a word-gram code corresponding to each action word-gram in the second action word-gram sequence; The second action description data, the second thinking process content, the second action word-gram sequence, and the word-gram encoding corresponding to each action word-gram in the second action word-gram sequence are combined into triple data, and the triple data is the training data.

[0018] As an optional embodiment, the second building module includes: an encoding unit, configured to encode each action word in the second action word sequence using a preset encoding and decoding tool to obtain a word code corresponding to each action word; The numbering unit is used to number the actions corresponding to the word unit codes to obtain corresponding action numbers, so as to determine the correspondence between the word unit codes and the action numbers as the vocabulary.

[0019] As an optional embodiment, the device further includes: a reward evaluation module, configured to perform reward evaluation on an output data set outputted by the action language model and the preset codec tool to obtain a composite reward score, wherein the output data set includes the first thought process content, the first action word sequence, a first word encoding sequence of each action word in the first action word sequence, and the target action data; An optimization training module is used to optimize the network parameters of the action language model according to the composite reward score.

[0020] As an optional embodiment, the reward evaluation module includes: an output data acquisition unit, configured to collect a preset number of output data groups for the same set of the first prompt word and the first action description data; A reward evaluation unit, configured to perform reward evaluation on any group of the output data groups according to a preset evaluation dimension, and obtain a number of evaluation scores equal to the preset number of groups; a compound calculation unit, configured to perform compound calculation on the evaluation scores of the same number as the preset number of groups using a preset compound function to obtain the compound reward score; The output data group includes the first thought process content, the first action word-gram sequence, the first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data.

[0021] As an optional embodiment, the optimization training module includes: a score comparison unit, for comparing, for the same set of the first prompt word and the first action description data, the evaluation score of each of the output data groups with the composite reward score to obtain a corresponding advantage comparison value; The optimization training unit is used to optimize the network parameters of the action language model according to the advantage comparison value corresponding to each group of output data groups.

[0022] As an optional embodiment, the optimization training unit is further configured to: Performing weighted processing on the advantage comparison values ​​corresponding to each group of the output data groups to obtain a weighted advantage value; The network parameters of the action language model are optimized according to the weighted advantage value.

[0023] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including: a memory for storing a computer program product; The processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in the first aspect above is implemented.

[0024] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect above is implemented.

[0025] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer program instructions, which implement the method described in the first aspect when executed by a processor.

[0026] In the embodiment of the present disclosure, the action language model can be used to generate the thought process content, discrete action word element sequence and word element encoding sequence of each word element according to the prompt words and action description data, and then the word element encoding sequence is decoded and processed to finally generate the corresponding action data. In this way, under the control and constraints of the prompt words and action description data, the action language model is used to think and reason (thinking process content) of the complex action description (action description data), and decompose it into basic actions (discrete action word element sequence) in sequence, thereby enhancing the complex semantic understanding ability of the action language model, and the embodiment of the present disclosure can output the action generation process of the action language model, including the thought process content (i.e., the reasoning process), semantic logic (i.e., the action word element sequence) and word element encoding sequence, etc., to achieve more accurate control of the generation process of the action semantics, thereby generating high-quality action data.

[0027] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0029] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which: Figure 1 This is one of the flowcharts of an action generation method according to an embodiment of the method disclosed herein.

[0030] Figure 2 This is a second flow chart of an action generation method according to an embodiment of the method disclosed herein.

[0031] Figure 3 This is a third flow chart of an action generation method according to an embodiment of the method disclosed herein.

[0032] Figure 4 A schematic diagram of the training process of the action language model according to an embodiment of the method disclosed herein.

[0033] Figure 5 A schematic diagram of an optimization training process of an action language model according to an embodiment of the disclosed method.

[0034] Figure 6 This is a structural block diagram of an action generation device according to an embodiment of the present invention.

[0035] Figure 7 The present invention is a structural block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] To address the technical issues in the related art, the present disclosure provides an action generation method, device, electronic product, computer-readable storage medium, and computer program product. To enable those skilled in the art to clearly and accurately understand the specific technical solutions of the present disclosure, the technical solutions of the present disclosure are described in detail below with reference to the accompanying drawings.

[0037] Figure 1 This is one of the flow charts of an action generation method according to an embodiment of the method disclosed herein. Figure 1 As shown, an action generation method according to an embodiment of the present disclosure may include the following steps: Step 101: Receive a first prompt word and first action description data input by a user.

[0038] In an embodiment of the present disclosure, a tool with input functions such as a smart display or a keyboard can be provided for the user to input a first prompt word and a first action description data. The first prompt word is a guiding question or instruction template used to guide the reasoning ability of the large language model and disassemble fuzzy instructions so that the large language model can output a detailed action decomposition, which may include chain-of-thought requirements (such as "please analyze step by step"), standardized formats (such as requiring steps to be listed in numerical sequence), etc.; and the action description data is the original action instruction or scenario description provided by the user, which is an unprocessed "demand statement". As the original input material of the large language model, it is mainly naturally generated by the user / scenario, and the terminology is unstructured (colloquial, fuzzy) data, so it needs to be processed by the prompt word before it can be effectively processed by the model. The first action description data refers to the action instructions or task goals expressed by the user in natural language. It is the original demand input that the model needs to understand and execute. Its form is mainly colloquial, unstructured text, used to describe physical actions (such as movement, operation) or abstract tasks (such as "handling emergencies"), such as making tea, making coffee, making breakfast, etc.

[0039] As an example, the first prompt word and the first action description data can be used as a combined input of the large language model.

[0040] The following describes the first prompt word and the first action description data by taking an example. Assume that the first prompt word and the first action description data input by the user are as follows: Action description data: "making tea"; Hint: "Please break down 'making tea' into 6 steps, using the '1. 2. 3.' format, with each step not exceeding 10 words."

[0041] Step 102, based on the first prompt word and the first action description data, generates the first thinking process content, the first action word sequence and the first word encoding sequence of each action word in the first action word sequence through the action language model, wherein the first action word sequence is a series of discrete action words obtained by decomposing the first action description data according to the first prompt word.

[0042] The action language model is a pre-trained language model capable of interacting with the environment. Its core functions are to understand natural language instructions, generate executable action sequences (such as calling APIs, operating software, and controlling physical devices), and dynamically adjust behavior based on environmental feedback. Its technical essence is an extension and enhancement of the Large Language Model (LLM). The action language model leverages the core capabilities of the LLM (such as understanding natural language and logical reasoning) but adds the ability to execute actions (such as calling tools, operating software, controlling robots, and generating specific actions).

[0043] In the disclosed embodiment, the action language model utilizes the reasoning and decomposition capabilities of the large language model to decompose the first action description data based on the first prompt word and the first action description data, obtaining a decomposed first action word-element sequence and providing the decomposed thought process content (reasoning process). For example, still using the action description data "making tea" as an example: Action description data: "making tea"; Prompt: "Please break down '<task>' into 6 physical steps, requiring: 1. Each step should start with a verb and should not exceed 8 words; 2. Mark the core operation objects (with []); 3. List one key security risk; 4. Output format: step [number]. [action]; According to an embodiment of the present disclosure, based on the aforementioned action description data and prompt words, the obtained thought process content and action word meta-sequence are as follows: Thinking process content: Phase 1: Task analysis.

[0044] - Main task: making tea → must cover the entire process from preparation to completion; - Constraints: 6 steps, labeled objects, safety warnings; Stage 2: Knowledge association.

[0045] - Recall the standard tea ceremony process: boiling water, warming utensils, adding tea leaves, brewing, serving tea, and cleaning; - Physical objects: kettle, teacup, tea leaves, hot water...; - Risk points: high temperature (100°C), fragile items, and liquid splashing; Phase 3: Step breakdown.

[0046] 1. Starting point: Prepare hot water (basic energy); 2. Key action: warm the cup (to avoid temperature difference and cracking); 3. Core operation: tea delivery; 4. Core operation: pouring water and brewing; 5. End point: separation of tea and soup; 6. Aftercare: basic cleaning; Phase 4: Risk mapping.

[0047] - Step 1 / 4 / 5 → High temperature burns; - Step 2 → Tea set broken; - Step 6 → Slip risk; Stage 5: Formatting output.

[0048] - Verb starting with: "burn", "inject", "hold"...; - Object label: [kettle] [tea cup]...; Action word sequence: Step 1. Boil water.

[0049] → Object: [kettle]; → Risk: Steam burns (distance ≥30cm); Step 2. Warm the teacup.

[0050] → Object: [Teacup]; → Risk: Temperature difference cracking (pour a small amount of hot water and rotate it first); Step 3. Add tea leaves.

[0051] → Object: [Teapot] + [Tea Leaves]; → Risk: Tea leaves lose their effectiveness due to moisture (sealed for use); Step 4. Pour water and brew.

[0052] → Object: [teapot] + [hot water]; → Risk: boiling water splashing (spout aimed at the center); Step 5. Pour tea into the cup.

[0053] → Object: [Teacup]; → Risk: Slippery handle (use scald-proof gloves); Step 6. Clean the countertop.

[0054] → Object: [Tea stain]; → Risk: Floor is slippery when wet (wipe it up immediately).

[0055] Through the above process, the action language model uses the large language model to decompose the action description data of "making tea" into six action words from steps 1 to 6, namely boiling water, warming the tea cup, adding tea leaves, pouring water to brew, pouring tea into the cup and cleaning the countertop.

[0056] Furthermore, the action language model can include a vocabulary that stores the correspondence between action token codes and action numbers, for example, [action 1] = 50001, [action 2] = 50002, and so on. This allows the action language model to understand not only text but also the codes representing these actions. Thus, upon receiving an action or its token code, it can output a series of token codes for the corresponding action or the corresponding action. For example, if the action "waving" has the action number 1000 and the corresponding token code sequence is [A12, B45, C78], then the vocabulary would store 1000 = [A12, B45, C78]. After learning the action language model, its vocabulary would store a correspondence, such as "1000 - [A12, B45, C78]." Thus, based on the action token "waving," it can be associated with the corresponding token code sequence (or tocken sequence) [A12, B45, C78].

[0057] Among them, the word encoding sequence can be implemented using an encoder (for example, variational autoencoder VQ-VAE, Vector Quantized Variational Autoencoder), so that the real action (such as "waving" in a video or "grasping" by a robot) is converted into a series of digital tokens (passwords or codes), and each token represents a small segment of the action, thereby processing the actions into a standardized digital format, which is convenient for processing and recognition by the action language model.

[0058] Step 103: Decode the first motion code sequence using a preset codec tool to generate target motion data.

[0059] In the disclosed embodiment, the preset codec tool may be a variational autoencoder (VQ-VAE), which can decode the action code sequence and thus "compile" the action code sequence into corresponding specific action data. For example, decoding [A12, B45, C78] yields the target action data of "waving".

[0060] exist Figure 1 Based on the embodiment shown, it is also possible to provide Figure 2 The embodiment shown. Figure 2This is a second flow chart of an action generation method according to an embodiment of the disclosed method. Figure 2 FIG. 1 shows the training process of the action language model, that is, an action generation method may further include the following steps: Step 104, constructing training data for the action language model, wherein the training data includes a preset number of triple data, the triple data includes the second action description data, the second thinking process content and the second action word sequence, and the word code corresponding to each action word in the second action word sequence.

[0061] As an optional embodiment, a preset number of source data pairs are obtained, any one of which includes a second prompt word and second action description data. Based on the source data pairs, training data is then generated using a preset large language model and a preset encoding and decoding tool. The triple data includes the second action description data, the second thought process content, the second action word sequence, and the word encoding corresponding to each action word in the second action word sequence. Furthermore, the second thought process content and the second action word sequence are generated using a preset large language model, wherein the second action word sequence is a series of discrete action words obtained by decomposing the second action description data according to the second prompt word, and the second thought process content is the text corresponding to the analysis process of the second action word sequence generated by the preset large language model. The second action word sequence is then encoded using a preset encoding and decoding tool to obtain the word encoding corresponding to each action word in the second action word sequence. The second action description data, the second thought process content, the second action word sequence, and the word encoding corresponding to each action word in the second action word sequence are combined into triple data, which serves as the training data.

[0062] It is understandable that the understanding and definition of the second prompt word and the second action description data can refer to the first prompt word and the second action description data. In the embodiment of the present disclosure, more sufficient training data is provided to the action language model, and a large amount of prompt words and action description data (i.e., second prompt words and second action description data) can be extracted from the historical data, and then the second action description data corresponding to each second prompt word is decomposed into actions based on the preset large language model (for details, please refer to Figure 1In the illustrated embodiment, the large language model performs reasoning and decomposition of the first action description data), thereby obtaining a series of corresponding action lemmas, namely, a second action lemma sequence, and further encoding each action lemma. The preset large language model also generates a reasoning process, namely, the second thinking process content. In this way, through the preset large language model and the preset encoding and decoding tool (such as the variational autoencoder VQ-VAE), the second action lemma sequence and the second thinking process content, as well as the lemma encoding of the second action lemma sequence, can be obtained. Combined with the second action description data, triple data is formed, and the triple data is combined into training data for training the action language model.

[0063] Step 105: construct a vocabulary table for the action language model, wherein the vocabulary table includes a correspondence between word unit codes and action numbers.

[0064] As an optional embodiment, a preset encoding and decoding tool is used to encode each action word in the second action word sequence to obtain the word code corresponding to each action word, and the action corresponding to the word code is numbered to obtain the corresponding action number, so as to determine the correspondence between the word code and the action number as a vocabulary. It can be understood that an action can be split into at least one word, and then each word is encoded. For example, the action word of "waving" is encoded as [A12, B45, C78]. In the embodiment of the present disclosure, the word code of the action word of "waving" can be further numbered, thereby associating the action word of "waving", its word code and the action number. In this way, the format can be unified, the action word is replaced by the action number, and the correspondence between the action number and the word code is established to obtain a vocabulary.

[0065] Step 106: Based on the training data and vocabulary, the preset action generation algorithm is trained using the large model supervised fine-tuning technology to obtain an action language model.

[0066] In the embodiment of the present disclosure, in the training of the preset action generation algorithm, since the preset codec tool is used to provide the word encoding and decoding conversion function, that is, the preset codec tool is used to realize the process of action word sequence data → action word encoding sequence → reconstructed action sequence (target action data), and the preset codec tool itself has this function of scripting, there is no need to train and learn. In this way, all parameters of the action language model can be involved in the training, and the parameters of the preset codec tool are frozen to avoid affecting the parameters of the preset codec tool during the training of the action language model.

[0067] Optionally, large-model supervised fine-tuning technology is used to implement dual-task model training for the action language model, that is, after training, the action language model can predict and output thought chains and action word meta-coding based on the input of prompt words and action description data. For example, after inputting a second prompt word and a second action description data, the action language model can output a predicted thought chain and action word meta-coding. Using the large-model supervised fine-tuning technology, the standard answer corresponding to the second prompt word and the second action description data (i.e., the standard thought chain and action word meta-coding) is also input to compare with the predicted thought chain and action word meta-coding to calculate the loss function. Then, through back propagation of the loss function, the action language model parameters are adjusted so that the action language model's prediction results approach the standard answer, ultimately obtaining a trained action language model.

[0068] like Figure 4 FIG. 1 is a schematic diagram of the training process of the action language model of an embodiment of the method disclosed herein. Figure 4 , input the prompt words and action description data into the large language model (preset large language model), and the large language model outputs the thinking process content (such as Figure 4 in <think>) and the decomposed action words (such as Figure 4 in <output>), and at the same time, the data corresponding to the action lemma is fed into a pre-set codec, which encodes it and produces a lemma encoding. During this process, the parameters of the pre-set codec are frozen, while only the parameters of the large language model are trained. Ultimately, an action language model is obtained that performs specific prediction tasks: predicting and generating thought chains (i.e., generating thought process content and action lemmas) and generating action lemma encodings.

[0069] exist Figure 1 and Figure 2 Based on the embodiment shown, the following is also provided: Figure 3 The embodiment shown. Figure 3 This is a flow chart of an action generation method according to an embodiment of the present disclosure. Figure 3 As shown in FIG, the optimization process of the action language model is shown, that is, an action generation method may also include the following steps: Step 107, performing a reward evaluation on the output data group output by the action language model and the preset encoding and decoding tool to obtain a composite reward score, wherein the output data group includes the first thinking process content, the first action word sequence, the first word encoding sequence of each action word in the first action word sequence, and the target action data.

[0070] As an optional embodiment of the present disclosure, for the same group of first prompt words and first action description data, a preset number of output data groups are collected, and a reward evaluation is performed on any group of output data groups according to a preset evaluation dimension to obtain the same number of evaluation scores as the preset number of groups, and a compound calculation is performed on the same number of evaluation scores as the preset number of groups through a preset compound function to obtain a compound reward score; wherein, the output data group includes the first thinking process content, the first action word sequence, the first word encoding sequence of each action word in the first action word sequence, and the target action data.

[0071] Step 108: Optimize the network parameters of the action language model according to the composite reward score.

[0072] Alternatively, for the same set of first prompt words and first action description data, the evaluation scores of each output data group may be compared with the composite reward score to obtain a corresponding advantage comparison value, and the network parameters of the action language model may be optimized based on the advantage comparison value corresponding to each output data group. Alternatively, the advantage comparison values ​​corresponding to each output data group may be weighted to obtain a weighted advantage value, and the network parameters of the action language model may be optimized based on the weighted advantage value.

[0073] For example, the optimization of the action language model by compound reward score can be achieved by using GRPO (Group Relative Policy Optimization) reinforcement learning algorithm. Specifically, first, Figure 2 The training process shown above obtains an action language model. By inputting prompt words and action description data into it, it can output the content of the thought process and action decomposition (action word sequence), as well as action tokens (i.e., word encoding sequence). The action tokens (i.e., word encoding sequence) are decoded by a preset encoding and decoding tool to obtain the target action data. Then, by pre-designing a composite reward function of output format, action similarity, and semantic similarity, the action language model is constrained to output actions that are formatted, authentic, reasonable, and semantically consistent.

[0074] As an optional embodiment, in each iteration of the GRPO training process, multiple samples can be sampled in parallel to obtain multiple compound reward scores. After group calculation, multiple advantages are obtained for optimizing the action language model.

[0075] Here, “parallel multi-sampling” refers to sampling the same prompt word and action description data multiple times (for example, Figure 5 G=8 times) sampling, the action language model outputs a set of 8 prediction results (output data groups), each of which can at least include the content of the thinking process, action decomposition (action word sequence) and action token sequence (word encoding sequence). Due to the randomness of the model, the prediction results of the action language model sampling will not be exactly the same. For the prediction results of each sampling, the format reward score can be obtained based on whether it matches the pre-designed output format. For the action token sequence (word encoding sequence), the reconstructed target action data is obtained after decoding, which can be used to calculate the similarity with the real action data, thereby obtaining the action similarity reward score and the semantic similarity reward score of the action description. The three reward scores of format reward score, action similarity reward score and semantic similarity reward score are added together to obtain a composite reward score for a single sampling, such as Figure 5 In the embodiment of the present disclosure, a set of 8 samplings is assumed, and the following can be obtained: Figure 5 r1, r2, ..., r G (G=8), a total of 8 compound reward points.

[0076] Furthermore, all the compound reward scores used above are grouped, that is, r = [r1, r2, ..., r8], and the advantage comparison value A is calculated according to the formula i , formulas such as: A i = [r i -mean(r)] / std(r), formula (1) Where i = 1, 2, ..., 8, mean(r) means the mean of r, and std(r) means the standard deviation of r.

[0077] In this way, the advantage comparison value of each adoption can be obtained, such as Figure 5 A1, A2, ..., A G (G=8). The advantage comparison values ​​can be used to obtain an average advantage value, which can then be used to determine whether the result is "positively advantageous" (for example, if the average advantage value is A, then assuming A1-A=+a is positive advantage) or "negatively advantageous" (for example, if the average advantage value is A, then assuming A2-A=-b is negative advantage). These "positive" and "negative" advantages are then used to optimize the training action language model. As an optional embodiment, the advantage comparison values ​​can be weighted to obtain weighted advantage values, thereby strengthening learning for output data sets with good advantage comparison values ​​and weakening learning for output data sets with poor advantage comparison values. Furthermore, the advantage comparison values ​​can be weighted using logarithmic probability weighting. The logarithmic function converts multiplication into addition. Compared to directly optimizing probabilities, this logarithmic probability weighting method results in smoother data gradient calculations and more stable numerical values. Furthermore, the optimization process will tend to increase the generation probability of samples that perform better than the average level (positive advantage) in the preset action model, and conversely reduce the generation probability of samples that perform poorly (negative advantage). In this way, the average advantage value can be dynamically and automatically increased, thereby increasing the generation probability of the model's positive advantage samples, thereby achieving the purpose of model optimization.

[0078] In short, the preset action model in the disclosed embodiments is not learned simply based on the absolute quality of each output, but rather adjusts its generation tendency by comparing the relative performance of multiple outputs under the same input. This effectively guides the preset action model to more stably converge to a more optimal strategy, while avoiding issues such as sensitivity to reward scale or training instability caused by individual sample noise, and also reduces the number of ineffective training times.

[0079] In summary, the action language model can be used to generate the thought process content, discrete action word-element sequences, and word-element encoding sequences of each word-element based on the prompt words and action description data, and then the word-element encoding sequence is decoded to finally generate the corresponding action data. In this way, under the control and constraints of the prompt words and action description data, the action language model is used to think and reason (thought process content) on the complex action description (action description data), and decompose it into basic actions (discrete action word-element sequences) in sequence, thereby enhancing the complex semantic understanding ability of the action language model. In addition, the embodiment of the present disclosure can output the action generation process of the action language model, including the thought process content (i.e., the reasoning process), semantic logic (i.e., the action word-element sequence), and word-element encoding sequence, etc., to achieve more accurate control of the generation process of the action semantics, thereby generating high-quality action data.

[0080] Furthermore, the model leverages the prior knowledge and chain-of-thought understanding of a large language model to enhance its reasoning and reasoning about complex action descriptions. The model then sequentially decomposes these into basic actions (action tokens), enhancing the action language model's ability to understand complex semantics. By integrating efficient reinforcement learning algorithms (such as GRPO), a composite reward function specifically designed for action generation (e.g., human motion) is designed to generate a composite reward score, improving the quality and diversity of action generation.

[0081] Correspondingly, the embodiments of the present disclosure also provide device embodiments corresponding to the aforementioned method embodiments. The device embodiments of the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. Figure 6 This is a structural block diagram of an action generation device according to an embodiment of the present invention. Figure 6 As shown, an action generation device according to an embodiment of the present disclosure may include: Receiving module 601, configured to receive a first prompt word and first action description data input by a user; an action thinking module 602 for generating, based on the first prompt word and the first action description data, a first thought process content, a first action word sequence, and a first word encoding sequence of each action word in the first action word sequence using an action language model, wherein the first action word sequence is a series of discrete action words obtained by decomposing the first action description data according to the first prompt word; The encoding and decoding module 603 is configured to decode the first action coding sequence using a preset encoding and decoding tool to obtain target action data.

[0082] In summary, the embodiment of the present disclosure can use the action language model to generate the thinking process content, discrete action word element sequence and word element encoding sequence of each word element according to the prompt words and action description data, and then decode the word element encoding sequence to finally generate the corresponding action data. In this way, under the control and constraints of the prompt words and action description data, the action language model is used to think and reason (thinking process content) of the complex action description (action description data), and decompose it into basic actions (discrete action word element sequence) in sequence, thereby enhancing the complex semantic understanding ability of the action language model, and the embodiment of the present disclosure can output the action generation process of the action language model, including the thinking process content (i.e., the reasoning process), semantic logic (i.e., the action word element sequence) and word element encoding sequence, etc., to achieve more accurate control of the generation process of the action semantics, thereby generating high-quality action data.

[0083] As an optional technical solution of the embodiment of the present disclosure, the device further includes: a first construction module, configured to construct training data for the action language model, the training data comprising a preset number of triples, the triples comprising second action description data, second thought process content, a second action lemma sequence, and a lemma code corresponding to each action lemma in the second action lemma sequence; A second building module is used to build a vocabulary table for the action language model, wherein the vocabulary table includes a correspondence between word unit codes and action numbers; A model training module is used to train a preset action generation algorithm based on the training data and the vocabulary using a large model supervised fine-tuning technique to obtain the action language model.

[0084] As an optional embodiment, the first building module includes: a source data acquisition unit, configured to acquire a preset number of source data pairs, any one of the source data pairs comprising the second prompt word and the second action description data; A training data generation unit is used to generate the training data based on the source data pair through a preset large language model and the preset encoding and decoding tool, wherein the triple data includes second action description data, second thinking process content and second action word-gram sequence, and word-gram encoding corresponding to each action word-gram in the second action word-gram sequence.

[0085] As an optional embodiment, the training data generating unit is further configured to: generating a second thought process content and a second action word-gram sequence using the preset large language model, wherein the second action word-gram sequence is a series of discrete action word-grams obtained by decomposing the second action description data according to the second prompt word, and the second thought process content is text corresponding to the analysis process of the preset large language model generating the second action word-gram sequence; Encoding the second action word-gram sequence using a preset encoding and decoding tool to obtain a word-gram code corresponding to each action word-gram in the second action word-gram sequence; The second action description data, the second thinking process content, the second action word-gram sequence, and the word-gram encoding corresponding to each action word-gram in the second action word-gram sequence are combined into triple data, and the triple data is the training data.

[0086] As an optional embodiment, the second building module includes: an encoding unit, configured to encode each action word in the second action word sequence using a preset encoding and decoding tool to obtain a word code corresponding to each action word; The numbering unit is used to number the actions corresponding to the word unit codes to obtain corresponding action numbers, so as to determine the correspondence between the word unit codes and the action numbers as the vocabulary.

[0087] As an optional embodiment, the device further includes: a reward evaluation module, configured to perform reward evaluation on an output data set outputted by the action language model and the preset codec tool to obtain a composite reward score, wherein the output data set includes the first thought process content, the first action word sequence, a first word encoding sequence of each action word in the first action word sequence, and the target action data; An optimization training module is used to optimize the network parameters of the action language model according to the composite reward score.

[0088] As an optional embodiment, the reward evaluation module includes: an output data acquisition unit, configured to collect a preset number of output data groups for the same set of the first prompt word and the first action description data; A reward evaluation unit, configured to perform reward evaluation on any group of the output data groups according to a preset evaluation dimension, and obtain a number of evaluation scores equal to the preset number of groups; a compound calculation unit, configured to perform compound calculation on the evaluation scores of the same number as the preset number of groups using a preset compound function to obtain the compound reward score; The output data group includes the first thought process content, the first action word-gram sequence, the first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data.

[0089] As an optional embodiment, the optimization training module includes: a score comparison unit, for comparing, for the same set of the first prompt word and the first action description data, the evaluation score of each of the output data groups with the composite reward score to obtain a corresponding advantage comparison value; The optimization training unit is used to optimize the network parameters of the action language model according to the advantage comparison value corresponding to each group of the output data groups.

[0090] As an optional embodiment, the optimization training unit is further configured to: Performing weighted processing on the advantage comparison values ​​corresponding to each group of the output data groups to obtain a weighted advantage value; The network parameters of the action language model are optimized according to the weighted advantage value.

[0091] It can be understood that the device embodiments provided by the embodiments of the present disclosure, their specific implementation processes and their functions and technical effects correspond one-to-one with the aforementioned method embodiments. Please refer to the relevant descriptions of the specific embodiments in the aforementioned method embodiments. Here, for the sake of brevity, they will not be repeated one by one.

[0092] Below, reference Figure 7 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be either or both of the first device and the second device, or a standalone device independent of them, and the standalone device may communicate with the first device and the second device to receive collected input signals from them.

[0093] Figure 7 The figure illustrates a structural block diagram of an electronic device according to an embodiment of the present disclosure.

[0094] like Figure 7 As shown, the electronic device includes one or more processors and memory.

[0095] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0096] The memory may store one or more computer program products. The memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may execute the computer program products to implement the action generation methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0097] In one example, the electronic device may further include an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0098] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0099] The output device can output various information to the outside, including determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0100] Of course, to simplify, Figure 7 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0101] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the action generation method according to various embodiments of the present disclosure described in the above part of this specification.

[0102] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0103] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the action generation method according to various embodiments of the present disclosure described in the above part of this specification.

[0104] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0105] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0106] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0107] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are intended to be illustrative examples only and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including but not limited to," and can be used interchangeably therewith. The words "or," "and," and "as used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as," as used herein, refers to the phrase "such as, but not limited to," and can be used interchangeably therewith.

[0108] The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0109] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0110] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0111] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.< / output> < / think>

Claims

1. An action generation method, characterized in that: The method comprises: receiving a first prompt word and first action description data input by a user; Based on the first prompt word and the first action description data, generating, using an action language model, first thought process content, a first action lemma sequence, and a first lemma encoding sequence for each action lemma in the first action lemma sequence, wherein the first action lemma sequence is a series of discrete action lemmas obtained by decomposing the first action description data according to the first prompt word; The first action coding sequence is decoded by a preset coding tool to generate target action data.

2. The method according to claim 1, characterized in that The method further comprises: Constructing training data for the action language model, the training data including a preset number of triples, the triples including second action description data, second thought process content, a second action lemma sequence, and a lemma code corresponding to each action lemma in the second action lemma sequence; Constructing a vocabulary table for the action language model, wherein the vocabulary table includes a correspondence between word unit codes and action numbers; Based on the training data and the vocabulary, a large model supervised fine-tuning technology is used to train a preset action generation algorithm to obtain the action language model.

3. The method according to claim 2, characterized in that The acquiring of training data for the action language model, wherein the training data includes a preset number of triples, wherein the triples include second action description data, second thought process content, a second action lemma sequence, and a lemma code corresponding to each action lemma in the second action lemma sequence, includes: Acquire a preset number of source data pairs, any one of which includes the second prompt word and the second action description data; Based on the source data pair, the training data is generated by a preset large language model and the preset encoding and decoding tool, and the triple data includes second action description data, second thinking process content and second action word sequence, and word encoding corresponding to each action word in the second action word sequence.

4. The method according to claim 3, characterized in that The training data is generated based on the source data pair using a preset large language model and the preset codec tool, wherein the triple data includes the second action description data, the second thought process content, the second action word-gram sequence, and the word-gram encoding corresponding to each action word-gram in the second action word-gram sequence, including: generating a second thought process content and a second action word-gram sequence using the preset large language model, wherein the second action word-gram sequence is a series of discrete action word-grams obtained by decomposing the second action description data according to the second prompt word, and the second thought process content is text corresponding to the analysis process of the preset large language model generating the second action word-gram sequence; Encoding the second action word-gram sequence using a preset encoding and decoding tool to obtain a word-gram code corresponding to each action word-gram in the second action word-gram sequence; The second action description data, the second thinking process content, the second action word-gram sequence, and the word-gram encoding corresponding to each action word-gram in the second action word-gram sequence are combined into triple data, and the triple data is the training data.

5. The method according to claim 3, characterized in that The constructing of a vocabulary for the action language model, wherein the vocabulary includes a correspondence between word unit codes and action numbers, includes: Using a preset encoding and decoding tool to encode each action word in the second action word sequence to obtain a word code corresponding to each action word; The actions corresponding to the word unit codes are numbered to obtain corresponding action numbers, so that the corresponding relationship between the word unit codes and the action numbers is determined as the vocabulary.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: performing a reward evaluation on an output data set outputted by the action language model and the preset codec tool to obtain a composite reward score, wherein the output data set includes the first thought process content, the first action word-gram sequence, a first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data; The network parameters of the action language model are optimized according to the compound reward score.

7. The method according to claim 6, characterized in that The reward evaluation is performed on the output data group output by the action language model and the preset codec tool to obtain a composite reward score, wherein the output data group includes the first thought process content, the first action word-gram sequence, the first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data, including: For the same set of the first prompt word and the first action description data, collecting a preset number of output data groups; Performing reward evaluation on any group of the output data groups according to the preset evaluation dimension to obtain the same number of evaluation scores as the preset number of groups; Performing a compound calculation on the evaluation scores of the same number as the preset number of groups using a preset compound function to obtain the compound reward score; The output data group includes the first thought process content, the first action word-gram sequence, the first word-gram encoding sequence of each action word-gram in the first action word-gram sequence, and the target action data.

8. The method according to claim 7, characterized in that The optimizing the network parameters of the action language model according to the composite reward score includes: For the same set of the first prompt word and the first action description data, comparing the evaluation score of each of the output data groups with the composite reward score to obtain a corresponding advantage comparison value; The network parameters of the action language model are optimized according to the advantage comparison value corresponding to each group of the output data groups.

9. The method according to claim 8, characterized in that The optimizing process of the network parameters of the action language model according to the advantage comparison value corresponding to each group of the output data groups includes: Performing weighted processing on the advantage comparison values ​​corresponding to each group of the output data groups to obtain a weighted advantage value; The network parameters of the action language model are optimized according to the weighted advantage value.

10. An action generating device, characterized in that: The device comprises: A receiving module, configured to receive a first prompt word and first action description data input by a user; an action thinking module, configured to generate, based on the first prompt word and the first action description data, a first thinking process content, a first action word sequence, and a first word encoding sequence of each action word in the first action word sequence using an action language model, wherein the first action word sequence is a series of discrete action words obtained by decomposing the first action description data according to the first prompt word; The encoding and decoding module is used to decode the first action coding sequence through a preset encoding and decoding tool to obtain target action data.

11. An electronic device, characterized in that: include: a memory for storing a computer program product; A processor is configured to execute a computer program product stored in the memory, and when the computer program product is executed, implements the method described in any one of claims 1 to 9.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 9 is implemented.

13. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images