A method and system for generalized intelligent generation of digital human actions

Abstract

The present application relates to the field of digital human technology, and specifically discloses a method and system for generalized intelligent generation of digital human actions, which obtains natural language descriptions of digital human actions and uses natural language processing technology based on artificial intelligence to perform text cleaning, word segmentation and semantic encoding on the natural language descriptions of digital human actions, so as to automatically obtain a sequence of target actions based on significant semantic aggregation representations between semantic features of each digital human text, and screen each target action to obtain a sequence of target action generation results, thereby generating a sequence of digital human actions, which can significantly reduce the time and labor costs required for manual script writing, and can dynamically adjust the action sequence according to the natural language description, so that the digital human can quickly adapt to new action requirements or scene changes without rewriting and testing the code.

Description

Technical Field

[0001] The present application relates to the field of digital human technology, and more specifically, to a method and system for intelligently generating generalized digital human actions. Background Art

[0002] Digital humans, also known as virtual digital characters, are virtual beings carefully constructed through advanced digital technology. They can not only imitate the appearance and movements of real people, but also simulate language communication and even cognitive processes. With the development of science and technology, the application of digital humans and mobile phone unmanned live broadcasts has rapidly expanded to industries such as life, public services, media, and the Internet, meeting the needs of many fields. In order to improve the user experience, users expect the interaction with digital humans to be more natural and seamless, which requires digital humans to not only be able to understand the user's instructions, but also be able to execute a series of complex action instructions.

[0003] However, traditional digital human motion generation often relies on manually written scripts or programs to define motion sequences. This method is not only time-consuming and labor-intensive, but also difficult to cope with complex changes in motion instructions. Specifically, manual writing usually requires professional programmers and designers to invest a lot of time and energy in development and debugging. Once the requirements change or new actions are required, the corresponding code must be rewritten and tested, which not only increases the development cycle, but may also cause project delays, making it difficult to quickly adapt to new motion requirements or scene changes. In addition, the manual writing method is difficult to ensure the consistency and naturalness of the action, especially for action sequences that require high coordination and complex interactions. Manual writing often fails to achieve the desired effect.

[0004] Therefore, a generalized intelligent generation solution for digital human actions is needed to solve the above technical problems. Summary of the invention

[0005] The present application provides a method and system for generalized intelligent generation of digital human actions, which can significantly reduce the time and labor cost required for manual script writing, and can dynamically adjust the action sequence according to natural language descriptions, so that the digital human can quickly adapt to new action requirements or scene changes without rewriting and testing the code. At the same time, it can better simulate human actions, improve the consistency and naturalness of actions, and achieve high automation and intelligence in the generation of digital human actions.

[0006] In a first aspect, a method for intelligently generating generalized digital human actions is provided, comprising:

[0007] Obtain the digital human's actions to generate natural language descriptions;

[0008] Performing text cleaning on the natural language description of digital human action generation to obtain the cleaned natural language description of digital human action generation, and then performing word segmentation processing to obtain a sequence of digital human action generation text;

[0009] Semantically encoding the sequence of text generated by the digital human action to obtain a sequence of context semantic encoding feature vectors of the text generated by the digital human action;

[0010] The sequence of digital human action generated text context semantic coding feature vectors is subjected to significant aggregation of digital human action generated text features to obtain significant semantic aggregation coding features of digital human target generated actions, including: cluster analysis is performed on the sequence of digital human action generated text context semantic coding feature vectors to obtain digital human action generated text context semantic self-supervised cluster representation; based on the digital human action generated text context semantic self-supervised cluster representation, contribution field analysis is performed on the sequence of digital human action generated text context semantic coding feature vectors to obtain significant semantic aggregation coding features of digital human target generated actions;

[0011] Extracting target actions from the salient semantic aggregation coding features of the target generated actions of the digital human to obtain a sequence of target actions;

[0012] Filtering the target action generation results corresponding to each target action in the sequence of target actions in a preset action library to obtain a sequence of target action generation results;

[0013] Based on the sequence of target action generation results, a sequence of digital human actions is generated.

[0014] In a possible implementation, the digital human action generation natural language description is subjected to text cleaning to obtain a cleaned digital human action generation natural language description, and then subjected to word segmentation processing to obtain a sequence of digital human action generation texts, including:

[0015] Performing text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation;

[0016] The cleaned natural language description of the digital human action generation is segmented to obtain a sequence of the digital human action generation text.

[0017] In a possible implementation, the sequence of text generated by the digital human actions is semantically encoded to obtain a sequence of contextual semantic encoding feature vectors of the text generated by the digital human actions, including: inputting the sequence of text generated by the digital human actions into a semantic encoder containing a Bert model to obtain a sequence of contextual semantic encoding feature vectors of the text generated by the digital human actions.

[0018] In a possible implementation, a cluster analysis is performed on the sequence of semantic encoding feature vectors of the text context generated by the digital human action to obtain a self-supervised cluster representation of the text context semantics generated by the digital human action, including: calculating the mean vector of the sequence of semantic encoding feature vectors of the text context generated by the digital human action to obtain a self-supervised cluster representation vector of the text context semantics generated by the digital human action as the self-supervised cluster representation of the text context semantics generated by the digital human action.

[0019] In a possible implementation, based on the digital human action generated text context semantic self-supervised clustering representation, a contribution field analysis is performed on the sequence of the digital human action generated text context semantic encoding feature vectors to obtain the digital human target generated action significant semantic aggregation encoding features, including:

[0020] Calculating the implicit clustering contribution factor of each digital human action generated text context semantic encoding feature vector in the sequence of digital human action generated text context semantic encoding feature vector relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain a sequence of digital human action generated text context semantic implicit clustering contribution factors;

[0021] Arranging the sequence of implicit clustering contribution factors of the digital human action generated text context semantics into a digital human action generated text context semantics clustering contribution field distribution vector;

[0022] Inputting the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action;

[0023] Taking each feature value in the modulation weight vector of the text context semantic clustering contribution field of the digital human action generation as the weight, the position-weighted sum of the sequence of the text context semantic coding feature vectors of the digital human action generation is calculated to obtain the digital human target generation action significant semantic aggregation coding feature vector as the digital human target generation action significant semantic aggregation coding feature.

[0024] In a possible implementation, calculating the implicit clustering contribution factor of each digital human action generated text context semantic encoding feature vector in the sequence of digital human action generated text context semantic encoding feature vectors relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain a sequence of digital human action generated text context semantic implicit clustering contribution factors, including:

[0025] Dividing the feature value of each position in the digital human action generated text context semantic encoding feature vector by the feature value of the corresponding position in the digital human action generated text context semantic self-supervised clustering representation vector by position to obtain the digital human action generated text context semantic contribution vector;

[0026] Calculating the logarithmic function value with base 2 of the absolute value of each eigenvalue of the digital human action generated text context semantic contribution vector to obtain the digital human action generated text context semantic contribution logarithmic vector;

[0027] After calculating the positional dot product of the semantic coding feature vector of the text context generated by the digital human action and the logarithmic vector of the semantic contribution to the text context generated by the digital human action, the sum of the values ​​of each position in the dot product vector is calculated to obtain the semantic clustering contribution value of the text context generated by the digital human action;

[0028] An exponential function with the natural constant e as the base and the semantic clustering contribution value of the text context generated by the digital human action as the exponent is calculated to obtain the semantic implicit clustering contribution factor of the text context generated by the digital human action.

[0029] In a possible implementation, the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action is input into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action, including:

[0030] Respectively performing matrix-vector multiplication of the query weight matrix, the key weight matrix and the value weight matrix with the digital human action generated text context semantic clustering contribution field distribution vector to obtain the digital human action generated text context semantic query vector, the digital human action generated text context semantic key vector and the digital human action generated text context semantic value vector;

[0031] After calculating the product between the digital human action generated text context semantic query vector and the transposed vector of the digital human action generated text context semantic key vector to obtain the digital human action generated text context semantic association matrix, the digital human action generated text context semantic association matrix and the square root of the length of the digital human action generated text context semantic key vector are divided by position to obtain the digital human action generated text context semantic association modulation matrix;

[0032] Inputting the digital human action generated text context semantic association modulation matrix into a softmax function to obtain a digital human action generated text context semantic association activation matrix;

[0033] The digital human action generated text context semantic value vector is multiplied by the digital human action generated text context semantic association activation matrix to obtain the digital human action generated text context semantic clustering contribution field modulation weight vector.

[0034] In a possible implementation, target action extraction is performed on the significant semantic aggregate coding features of the digital human target generation action to obtain a sequence of target actions, including: inputting the significant semantic aggregate coding feature vector of the digital human target generation action into a target digital human text action model to obtain the sequence of target actions.

[0035] In the second aspect, a generalized intelligent generation system for digital human actions is provided, comprising:

[0036] A natural language description acquisition module, used to acquire the digital human's actions and generate natural language descriptions;

[0037] A word segmentation processing module is used to perform text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation, and then perform word segmentation processing to obtain a sequence of digital human action generation texts;

[0038] A semantic encoding module, used for semantically encoding the sequence of text generated by the digital human action to obtain a sequence of context semantic encoding feature vectors of the text generated by the digital human action;

[0039] The text feature significant aggregation processing module is used to perform significant aggregation of digital human action generated text features on the sequence of digital human action generated text context semantic coding feature vectors to obtain significant semantic aggregation coding features of digital human target generated actions, including: a cluster analysis processing unit, used to perform cluster analysis on the sequence of digital human action generated text context semantic coding feature vectors to obtain digital human action generated text context semantic self-supervised cluster representation; a contribution field analysis unit, used to perform contribution field analysis on the sequence of digital human action generated text context semantic coding feature vectors based on the digital human action generated text context semantic self-supervised cluster representation to obtain significant semantic aggregation coding features of digital human target generated actions;

[0040] A target action extraction module, used for extracting target actions from the significant semantic aggregation coding features of the target generated actions of the digital human to obtain a sequence of target actions;

[0041] A target action generation result screening module, used for screening the target action generation results corresponding to each target action in the sequence of target actions in a preset action library to obtain a sequence of target action generation results;

[0042] The digital human action generation module is used to generate a sequence of digital human actions based on the sequence of target action generation results.

[0043] In a possible implementation, the word segmentation processing module includes:

[0044] A text cleaning unit, used for performing text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation;

[0045] The natural language description word segmentation processing unit is used to perform word segmentation processing on the cleaned natural language description of the digital human action generation to obtain a sequence of the digital human action generation text.

[0046] The present application has at least the following technical effects: Compared with the prior art, the present application provides a generalized intelligent generation method and system for digital human actions, which obtains the natural language description of digital human actions and uses artificial intelligence-based natural language processing technology to perform text cleaning, word segmentation and semantic encoding on the natural language description of digital human actions, so as to automatically obtain the sequence of target actions based on the significant semantic aggregation representation between the semantic features of each digital hot text, and screen each target action to obtain the sequence of target action generation results, and then generate the sequence of digital human actions, which can significantly reduce the time and labor cost required for manual script writing, and can dynamically adjust the action sequence according to the natural language description, so that the digital human can quickly adapt to new action requirements or scene changes without rewriting and testing the code. At the same time, it can better simulate human actions, improve the consistency and naturalness of actions, and achieve high automation and intelligence of digital human action generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0048] Figure 1 This is a schematic flowchart of the method for intelligently generating generalized digital human actions according to an embodiment of the present application.

[0049] Figure 2 Schematic diagram of data flow of the method for intelligently generating generalized digital human actions according to an embodiment of the present application.

[0050] Figure 3 This is a schematic flowchart of a sequence of performing text cleaning on the natural language description of digital human action generation to obtain the cleaned natural language description of digital human action generation, and then performing word segmentation processing to obtain the digital human action generation text in the generalized intelligent generation method of digital human action in an embodiment of the present application.

[0051] Figure 4 This is a schematic flowchart of performing significant aggregation of digital human action generation text features on a sequence of digital human action generation text context semantic coding feature vectors in the generalized intelligent generation method of digital human actions in an embodiment of the present application to obtain significant semantic aggregation coding features of digital human target generation actions.

[0052] Figure 5 In the generalized intelligent generation method of digital human actions in an embodiment of the present application, based on the self-supervised clustering representation of the text context semantics generated by the digital human actions, a contribution field analysis is performed on the sequence of semantic encoding feature vectors of the text context semantics generated by the digital human actions to obtain a schematic flowchart of the significant semantic aggregation encoding features of the digital human target generation actions.

[0053] Figure 6 This is a schematic block diagram of a system for intelligently generating generalized digital human actions according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0055] In the prior art, the generation of digital human actions relies on manually writing scripts to define action sequences. This method not only consumes a lot of time and human resources, but also seems to be unable to cope with the changes in complex action instructions. Specifically, manual scripting requires professional programmers and designers to invest a lot of energy in development and debugging. When the action requirements change or new actions need to be added, the code must be rewritten and tested, which prolongs the development cycle and sometimes even causes project delays. In addition, the manual scripting method performs poorly in ensuring the continuity and natural fluency of actions, especially when dealing with action sequences that require high coordination and complex interactions. It is difficult to achieve the expected ideal effect.

[0056] In response to the above technical problems, the technical concept of this application is to obtain a natural language description of the digital human action generation, and use artificial intelligence-based natural language processing technology to perform text cleaning, word segmentation and semantic encoding on the natural language description of the digital human action generation, so as to automatically obtain a sequence of target actions based on the significant semantic aggregation representation between the semantic features of each digital hot text, and screen each target action to obtain a sequence of target action generation results, and then generate a sequence of digital human actions, which can significantly reduce the time and labor cost required for manual script writing, and can dynamically adjust the action sequence according to the natural language description, so that the digital human can quickly adapt to new action requirements or scene changes without rewriting and testing the code. At the same time, it can better simulate human actions, improve the consistency and naturalness of actions, and achieve a high degree of automation and intelligence in the generation of digital human actions.

[0057] Figure 1 This is a schematic flowchart of the method for intelligently generating generalized digital human actions according to an embodiment of the present application. Figure 2 This is a data flow diagram of the method for intelligently generating generalized digital human actions according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the generalized intelligent generation method of digital human actions includes: S1, obtaining a natural language description of digital human action generation; S2, performing text cleaning on the natural language description of digital human action generation to obtain the cleaned natural language description of digital human action generation, and then performing word segmentation processing to obtain a sequence of digital human action generation texts; S3, semantically encoding the sequence of digital human action generation texts to obtain a sequence of digital human action generation text context semantic encoding feature vectors; S4, performing significant aggregation of digital human action generation text features on the sequence of digital human action generation text context semantic encoding feature vectors to obtain significant semantic aggregation encoding features of digital human target generation actions; S5, performing target action extraction on the significant semantic aggregation encoding features of digital human target generation actions to obtain a sequence of target actions; S6, screening the target action generation results corresponding to each target action in the sequence of target actions in a preset action library to obtain a sequence of target action generation results; S7, generating a sequence of digital human actions based on the sequence of target action generation results.

[0058] In the above-mentioned generalized intelligent generation method for digital human actions, S1, obtains the natural language description of the digital human action. Optionally, in one embodiment of the present application, obtaining the natural language description of the digital human action includes the following methods: 1. The natural language description of the digital human action can be received through multiple channels, such as text input, text converted by voice recognition, or pre-prepared script files. 2. If the description is directly provided by the user, it needs to be accurately recorded. Ensure that the recorded information is detailed enough so that the subsequent processing can correctly parse the user's true intention. 3. If the digital human action is designed according to a specific story line or task, then a detailed script or instruction set needs to be written in advance, which includes the action description of the digital human in different scenarios. The natural language description should cover all the action details that the digital human needs to complete, including but not limited to the action type, action sequence, duration, and interaction method with other objects or environments. Finally, in order to make the description more standardized, a set of fixed templates or formats can be formulated, and all natural language descriptions are written according to this template, which is conducive to subsequent automated processing.

[0059] In the above-mentioned generalized intelligent generation method for digital human actions, the S2 performs text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation, and then performs word segmentation processing to obtain a sequence of digital human action generation text. Specifically, considering that the original natural language description of the digital human action generation may contain irrelevant characters, symbols, stop words or other non-standard tags, and there may also be spelling errors or grammatical errors, which will interfere with subsequent processing. Based on this, in order to ensure the quality of input data and enable the natural language processing system to better understand and perform subsequent tasks, in the technical solution of the present application, the natural language description of the digital human action generation is subjected to text cleaning to obtain the cleaned natural language description of the digital human action generation.

[0060] Optionally, in one embodiment of the present application, Figure 3 This is a schematic flow chart of a sequence of performing text cleaning on the natural language description of digital human action generation to obtain the cleaned natural language description of digital human action generation and then performing word segmentation processing to obtain the digital human action generation text in the generalized intelligent generation method of digital human action in the embodiment of the present application. Figure 3As shown, the text of the digital human action generation natural language description is cleaned to obtain the cleaned digital human action generation natural language description, and then word segmentation processing is performed to obtain a sequence of digital human action generation texts, including: S21, text cleaning of the digital human action generation natural language description to obtain the cleaned digital human action generation natural language description; S22, word segmentation processing of the cleaned digital human action generation natural language description to obtain a sequence of digital human action generation texts. Specifically, considering that the cleaned digital human action generation natural language description contains a large number of action generation words, such as action type, direction, etc. Based on this, in the technical solution of the present application, in order to more carefully analyze and understand the semantics of each word, so as to improve the subsequent analysis of the digital human action, the cleaned digital human action generation natural language description is subjected to word segmentation processing to obtain a sequence of digital human action generation texts.

[0061] In the above-mentioned generalized intelligent generation method for digital human actions, the S3 performs semantic encoding on the sequence of digital human action generated text to obtain a sequence of contextual semantic encoding feature vectors of digital human action generated text. Specifically, in order to more accurately clarify the semantic information between the contexts contained in each word, in the technical solution of the present application, the sequence of digital human action generated text is input into a semantic encoder containing a Bert model to effectively capture the contextual relevance of words in the text, and obtain a sequence of contextual semantic encoding feature vectors of digital human action generated text containing rich semantic information.

[0062] Optionally, in one embodiment of the present application, the sequence of text generated by the digital human action is semantically encoded to obtain a sequence of contextual semantic encoding feature vectors of the text generated by the digital human action, including: inputting the sequence of text generated by the digital human action into a semantic encoder containing a Bert model to obtain a sequence of contextual semantic encoding feature vectors of the text generated by the digital human action.

[0063] In the above-mentioned generalized intelligent generation method for digital human actions, the S4 performs significant aggregation of digital human action generation text features on the sequence of digital human action generation text context semantic coding feature vectors to obtain significant semantic aggregation coding features of digital human target generation actions. Specifically, considering that the sequence of digital human action generation text context semantic coding feature vectors expresses the semantic understanding of a natural language text describing the digital human action, different digital human action generation text context semantic coding features have different contributions and importance in the entire context. Therefore, in order to evaluate the importance of each feature and thus highlight those features that are most critical for action generation, in the technical solution of the present application, the sequence of digital human action generation text context semantic coding feature vectors is subjected to significant aggregation of digital human action generation text features to obtain significant semantic aggregation coding feature vectors of digital human target generation actions and significant semantic aggregation coding features of digital human target generation actions.

[0064] Optionally, in one embodiment of the present application, Figure 4 This is a schematic flow chart of the method for generalizing intelligent generation of digital human actions in the embodiment of the present application, in which the sequence of digital human action generation text context semantic coding feature vectors is subjected to significant aggregation of digital human action generation text features to obtain significant semantic aggregation coding features of digital human target generation action. Figure 4 As shown, the sequence of digital human action generated text context semantic coding feature vectors is significantly aggregated to obtain the significant semantic aggregation coding features of digital human target generated actions, including: S41, clustering analysis is performed on the sequence of digital human action generated text context semantic coding feature vectors to obtain the digital human action generated text context semantic self-supervised clustering representation; S42, based on the digital human action generated text context semantic self-supervised clustering representation, the sequence of digital human action generated text context semantic coding feature vectors is subjected to contribution field analysis to obtain the significant semantic aggregation coding features of digital human target generated actions.

[0065] Optionally, in one embodiment of the present application, a cluster analysis is performed on the sequence of semantic encoding feature vectors of the text context generated by the digital human action to obtain a self-supervised cluster representation of the text context semantics generated by the digital human action, including: calculating the mean vector of the sequence of semantic encoding feature vectors of the text context generated by the digital human action to obtain a self-supervised cluster representation vector of the text context semantics generated by the digital human action as the self-supervised cluster representation of the text context semantics generated by the digital human action.

[0066] Optionally, in one embodiment of the present application, Figure 5This is a schematic flow chart of the generalized intelligent generation method of digital human actions in the embodiment of the present application, based on the self-supervised clustering representation of the semantic context of the digital human action generation text, the sequence of the semantic encoding feature vector of the digital human action generation text context is analyzed for contribution degree to obtain the significant semantic aggregation encoding feature of the digital human target generation action. Figure 5 As shown, based on the digital human action generated text context semantic self-supervised clustering representation, the sequence of digital human action generated text context semantic coding feature vectors is subjected to contribution field analysis to obtain the significant semantic aggregation coding features of the digital human target generated action, including: S421, calculating the implicit clustering contribution factor of each digital human action generated text context semantic coding feature vector in the sequence of digital human action generated text context semantic coding feature vectors relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain the sequence of digital human action generated text context semantic implicit clustering contribution factors; S422, calculating the implicit clustering contribution factor of the digital human action generated text context semantic The sub-sequence is arranged as a distribution vector of the semantic clustering contribution field of the text context generated by the digital human action; S423, the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action is input into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action; S424, using each eigenvalue in the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action as a weight, the position-weighted sum of the sequence of the semantic encoding feature vector of the text context generated by the digital human action is calculated to obtain the significant semantic aggregation encoding feature vector of the digital human target generation action as the significant semantic aggregation encoding feature of the digital human target generation action.

[0067] Optionally, in one embodiment of the present application, the implicit clustering contribution factor of each digital human action generated text context semantic encoding feature vector in the sequence of the digital human action generated text context semantic encoding feature vector is calculated relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain a sequence of digital human action generated text context semantic implicit clustering contribution factors, including: dividing the feature value of each position in the digital human action generated text context semantic encoding feature vector by the feature value of the corresponding position in the digital human action generated text context semantic self-supervised clustering representation vector by position to obtain the digital human action generated text context semantic contribution factor. Quantity; calculate the logarithmic function value with base two of the absolute value of each eigenvalue of the digital human action generated text context semantic contribution vector to obtain the logarithmic vector of the digital human action generated text context semantic contribution; calculate the positional dot product of the digital human action generated text context semantic encoding feature vector and the logarithmic vector of the digital human action generated text context semantic contribution, and then calculate the sum of each position in the dot product vector to obtain the digital human action generated text context semantic clustering contribution value; calculate the exponential function with the natural constant e as the base, with the digital human action generated text context semantic clustering contribution value as the exponent, to obtain the digital human action generated text context semantic implicit clustering contribution factor.

[0068] Optionally, in one embodiment of the present application, the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action is input into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action, including: respectively performing matrix-vector multiplication on the query weight matrix, key weight matrix and value weight matrix with the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action to obtain the semantic query vector of the text context generated by the digital human action, the semantic key vector of the text context generated by the digital human action and the semantic value vector of the text context generated by the digital human action; calculating the semantic query vector of the text context generated by the digital human action and the semantic key vector of the text context generated by the digital human action After obtaining the digital human action generated text context semantic association matrix by multiplying the transposed vectors of the vectors, the digital human action generated text context semantic association matrix and the square root of the length of the digital human action generated text context semantic key vector are divided by position to obtain the digital human action generated text context semantic association modulation matrix; the digital human action generated text context semantic association modulation matrix is ​​input into the softmax function to obtain the digital human action generated text context semantic association activation matrix; the digital human action generated text context semantic value vector is multiplied by the digital human action generated text context semantic association activation matrix to obtain the digital human action generated text context semantic clustering contribution field modulation weight vector.

[0069] In summary, in the embodiment of the present application, the sequence of the digital human action generated text context semantic encoding feature vectors is subjected to significant aggregation of digital human action generated text features, which can be expressed by the formula:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077]

[0078]

[0079] in, generating a sequence of text context semantic encoding feature vectors for the digital human action, The first, second, ..., and third sequences of text context semantic encoding feature vectors are generated for the digital human actions respectively. ,..., The digital human action generates a text context semantic encoding feature vector, the number of feature vectors in a sequence of text context semantic encoding feature vectors generated for the digital human action, The first sequence of generating a text context semantic encoding feature vector for the digital human action The digital human action generates a text context semantic encoding feature vector, generating a textual context semantic self-supervised clustering representation vector for the digital human action, For the The first one in the semantic encoding feature vector of the text context generated by the digital human action The eigenvalues ​​at the positions, The first one represents the self-supervised clustering representation vector of the text context semantics generated by the digital human action The eigenvalues ​​at the positions, represents the logarithmic function value with base 2, Generate the number of feature values ​​in the text context semantic encoding feature vector for each digital human action, represents an exponential function with the natural constant e as the base, for Relative to The contribution factor of implicit clustering of text context semantics for digital human action generation, The first, second, ..., and third vectors of the text context semantic clustering contribution field distribution vectors are generated for digital human actions respectively. ,..., The contribution factor of implicit clustering of text context semantics generated by digital human actions, Generate a text context semantic clustering contribution field distribution vector for the digital human action, , and They are the semantic query vector of the text context generated by the digital human action, the semantic key vector of the text context generated by the digital human action, and the semantic value vector of the text context generated by the digital human action. yes The transposed vector of yes Length, yes function, represents vector-matrix multiplication, is the domain modulation weight vector of the semantic clustering contribution to the text context generated by the digital human action, , and They are query weight matrix, key weight matrix and value weight matrix, is the first in the modulation weight vector of the text context semantic clustering contribution field generated by the digital human action The eigenvalues ​​at the positions, It is the salient semantic aggregated encoding feature vector of the digital human target generated action.

[0080] Specifically, first, the sequence of the semantic encoding feature vectors of the text context generated by the digital human action is clustered to find the internal patterns between the feature vectors, and the self-supervised clustering representation vector of the semantics of the text context generated by the digital human action is obtained. Then, the implicit clustering contribution factor between each semantic encoding feature vector of the text context generated by the digital human action and the self-supervised clustering representation vector is calculated to obtain a sequence of implicit clustering contribution factors of the semantics of the text context generated by the digital human action. Specifically, the implicit clustering contribution factor can indicate the influence of each semantic encoding feature of the text context generated by the digital human action on the self-supervised clustering, which helps to find which semantic encoding features of the text context generated by the digital human action have a significant effect on the clustering results, and provides a basis for subsequent feature selection and importance evaluation, so that the model can focus more on those features that have a decisive influence on the clustering effect. Then, the calculated semantic encoding features of the text context generated by the digital human action are arranged to be summarized into a vector field, and the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action is obtained. Subsequently, the self-attention mechanism is used to explicitly model the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action to generate the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action. Specifically, by using the self-attention mechanism to perform a full-field autocorrelation analysis on the fine-grained factors in the cluster contribution field, an explicit model of the feature vector with significant clustering contribution can be clearly constructed, so as to enhance the model's attention to key features and reduce the interference of non-key features. It can be understood that by using the self-attention mechanism to perform a full-field autocorrelation analysis on the fine-grained factors in the cluster contribution metric, a model of feature vectors that contribute significantly to clustering can be clearly constructed, which helps to improve the model's attention to key features and reduce the noise impact caused by non-key features. Finally, the digital human action generates text context semantic clustering contribution field modulated weight vector to weightedly fuse the sequence of text context semantic coding feature vectors generated by the digital human action to obtain the digital human target generated action salient semantic aggregate coding feature vector. Specifically, the final aggregate coding feature obtained not only contains the information of the original feature, but also integrates the key features in the clustering process to obtain a comprehensive feature representation.

[0081] In the above-mentioned generalized intelligent generation method of digital human actions, the S5 extracts the target action from the significant semantic aggregation coding features of the digital human target generation action to obtain a sequence of target actions. Specifically, considering that in order to convert semantic information into specific executable action instructions, in the technical solution of the present application, the significant semantic aggregation coding feature vectors of the digital human target generation action are input into the target digital human text action model to obtain a sequence of target actions. Specifically, these feature vectors already contain rich semantic information, but they still need to be converted into specific actions that the digital human can understand and execute. The target digital human text action model is responsible for mapping these vectors to the actual action sequence, so that the digital human can perform corresponding actions according to these instructions.

[0082] Optionally, in one embodiment of the present application, target action extraction is performed on the significant semantic aggregate coding features of the digital human target generation action to obtain a sequence of target actions, including: inputting the significant semantic aggregate coding feature vector of the digital human target generation action into the target digital human text action model to obtain the sequence of target actions.

[0083] Preferably, inputting the salient semantic aggregated coded feature vector of the digital human target generated action into the target digital human text action model to obtain a sequence of target actions comprises:

[0084] Calculate the sum of the absolute values ​​of each eigenvalue of the digital human target generated action significant semantic aggregate coding feature vector to obtain a first digital human target generated action significant semantic aggregate coding and modulation value, and calculate the square root of the sum of the squares of each eigenvalue of the digital human target generated action significant semantic aggregate coding feature vector to obtain a second digital human target generated action significant semantic aggregate coding and modulation value;

[0085] After performing dot-wise subtraction on the feature vector of the digital human target generated action-significant semantic aggregate coding and the second digital human target generated action-significant semantic aggregate coding and modulation value, perform dot-wise multiplication on the number of feature values ​​of the feature vector of the digital human target generated action-significant semantic aggregate coding and the reciprocal of the first digital human target generated action-significant semantic aggregate coding and modulation value, and take the reciprocal of each feature value to obtain the first digital human target generated action-significant semantic aggregate coding phase conversion vector;

[0086] After performing dot-wise subtraction on the digital human target generated action significant semantic aggregate coding feature vector and the first digital human target generated action significant semantic aggregate coding and modulation value, perform dot-wise multiplication with the square root of the number of eigenvalues ​​of the digital human target generated action significant semantic aggregate coding feature vector and the reciprocal of the second digital human target generated action significant semantic aggregate coding and modulation value, and take the reciprocal of each eigenvalue to obtain the second digital human target generated action significant semantic aggregate coding phase conversion vector;

[0087] Subtract the dot product vector of the second digital human target generation action-significant semantic aggregation coding phase conversion vector from the first digital human target generation action-significant semantic aggregation coding phase conversion vector and the weighted hyperparameter to obtain an optimized digital human target generation action-significant semantic aggregation coding feature vector;

[0088] The optimized digital human target generated action salient semantic aggregated encoding feature vector is input into the target digital human text action model to obtain a sequence of target actions.

[0089] Among them, the digital human target generates an action-significant semantic aggregation encoding feature vector, which is recorded as The optimization expression is:

[0090]

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] in, Generate an action-significant semantic aggregate encoding feature vector for the digital human target, represents the set of real numbers, The first one represents the salient semantic aggregated encoding feature vector of the digital human target generated action The eigenvalues ​​at the positions, represents the number of eigenvalues ​​of the salient semantic aggregated encoding feature vector of the digital human target generated action, Indicates the first digital human target generates action-significant semantic aggregate coding and modulation value, Indicates the second digital human target generates action-significant semantic aggregate encoding and modulation value, It means point multiplication by position. It means subtracting by position. To take the inverse of each eigenvalue in the vector, Represents the first digital human target generation action salient semantic aggregation encoding phase transition vector, Represents the second digital human target generation action salient semantic aggregation encoding phase transition vector, represents the weighted hyperparameter, Represents the optimized digital human target generation action salient semantic aggregate encoding feature vector.

[0097] Specifically, considering that each digital human action generated text context semantic coding feature vector in the sequence of the digital human action generated text context semantic coding feature vectors respectively represents the context semantic coding features of each text word in the natural language description of the digital human action generation, when it is input into the feature significant aggregation network based on cluster contribution field analysis, the semantic difference of the coding feature based on the source semantic difference will cause the cluster contribution field analysis difference, so that the digital human target generated action significant semantic aggregation coding feature vector will also have the fused significant difference based on the cluster contribution calculation difference. Therefore, it is expected to improve the detail semantic aggregation expression effect based on the fused significant difference of the digital human target generated action significant semantic aggregation coding feature vector.

[0098] Based on this, in this preferred embodiment, the difference of the feature value of the salient semantic aggregate coding feature vector of the digital human target generated action relative to the feature set of the vector as a whole of the salient semantic aggregate coding feature vector of the digital human target generated action is used as the semantic change intensity information, and a phase-like conversion corresponding to the position-based intensity modulation is performed through different and modulation representation forms, so that the aggregation enhancement of the semantic change phase perception can improve the axial aggregation receptive field along the feature aggregation direction by performing a spatial translation operation based on alternating stacking under the scale balance of the vector set of the salient semantic aggregate coding feature vector of the digital human target generated action, thereby improving the perception effect of the aggregate semantics of the salient semantic aggregate coding feature vector of the digital human target generated action on the semantic changes of the time sequence of details, so as to improve the expression effect of the salient semantic aggregate coding feature vector of the digital human target generated action, and improve the accuracy of the sequence of target actions obtained by inputting the target digital human text action model, which can significantly reduce the time and labor cost required for manual script writing, and can dynamically adjust the action sequence according to the natural language description, so that the digital human can quickly adapt to new action requirements or scene changes without rewriting and testing the code. At the same time, it can better simulate human movements, improve the consistency and naturalness of movements, and achieve high automation and intelligence in the generation of digital human movements.

[0099] In the above-mentioned generalized intelligent generation method for digital human actions, the S6, in the preset action library, filters the target action generation results corresponding to each target action in the sequence of the target action to obtain a sequence of target action generation results. Specifically, in order to ensure that the generated action instructions not only conform to the user's description, but are also practical and can be effectively executed by the digital human. In the technical solution of the present application, the target action generation results corresponding to each target action in the sequence of the target action are filtered in the preset action library to obtain a sequence of target action generation results. Specifically, the preset action library contains specific actions that the digital human can perform, and by filtering in the action library, it can be ensured that the generated action instructions are actually executable by the digital human, rather than unrealizable actions. Specifically, it is ensured that the generated action instructions follow these standards, thereby improving the consistency and predictability of the actions.

[0100] In the above-mentioned generalized intelligent generation method for digital human actions, the S7 generates a sequence of digital human actions based on the sequence of target action generation results. Specifically, the sequence of target action generation results is screened and optimized to ensure that all actions are feasible. By generating a sequence of digital human actions based on these results, it can be ensured that the digital human can actually perform these actions. Specifically, the target action generation results contain detailed parameters and instructions, which are screened and optimized by a preset action library. Therefore, the action sequence generated based on these results is more accurate and can better reflect the user's intention, thereby generating more accurate digital human actions. In this way, the time and labor cost required for manual script writing can be significantly reduced, and the action sequence can be dynamically adjusted according to the natural language description, so that the digital human can quickly adapt to new action requirements or scene changes without rewriting and testing the code. At the same time, it can better simulate human actions, improve the consistency and naturalness of actions, and achieve a high degree of automation and intelligence in the generation of digital human actions.

[0101] In summary, the generalized intelligent generation method of digital human actions according to the embodiment of the present application is explained, which obtains the natural language description of digital human actions and uses artificial intelligence-based natural language processing technology to perform text cleaning, word segmentation and semantic encoding on the natural language description of digital human actions, so as to automatically obtain the sequence of target actions based on the significant semantic aggregation representation between the semantic features of each digital hot text, and screen each target action to obtain the sequence of target action generation results, and then generate the sequence of digital human actions, which can significantly reduce the time and labor cost required for manual script writing, and can dynamically adjust the action sequence according to the natural language description, so that the digital human can quickly adapt to new action requirements or scene changes without rewriting and testing the code. At the same time, it can better simulate human actions, improve the consistency and naturalness of actions, and achieve a high degree of automation and intelligence in the generation of digital human actions.

[0102] Figure 6 Schematic block diagram of the generalized intelligent generation system of digital human actions according to the embodiment of the present application. Figure 6As shown, the generalized intelligent generation system 100 of digital human actions includes: a natural language description acquisition module 110, which is used to acquire the natural language description of digital human action generation; a word segmentation processing module 120, which is used to perform text cleaning on the natural language description of digital human action generation to obtain the cleaned natural language description of digital human action generation, and then perform word segmentation processing to obtain a sequence of digital human action generation text; a semantic encoding module 130, which is used to perform semantic encoding on the sequence of digital human action generation text to obtain a sequence of contextual semantic encoding feature vectors of digital human action generation text; a text feature significant aggregation processing module 140, which is used to perform significant aggregation of digital human action generation text features on the sequence of contextual semantic encoding feature vectors of digital human action generation text to obtain significant semantic aggregation encoding features of digital human target generation action, including: a clustering analysis processing unit 141, which is used to perform contextual semantic encoding on the text generated by the digital human action The method comprises the following steps: performing cluster analysis on a sequence of text semantic coding feature vectors to obtain a self-supervised cluster representation of the text context semantics generated by digital human actions; a contribution field analysis unit 142 is used to perform contribution field analysis on a sequence of text context semantic coding feature vectors generated by digital human actions based on the self-supervised cluster representation of the text context semantics generated by the digital human actions to obtain significant semantic aggregation coding features of digital human target generation actions; a target action extraction module 150 is used to extract target actions from the significant semantic aggregation coding features of the digital human target generation actions to obtain a sequence of target actions; a target action generation result screening module 160 is used to screen the target action generation results corresponding to each target action in the sequence of target actions in a preset action library to obtain a sequence of target action generation results; a digital human action generation module 170 is used to generate a sequence of digital human actions based on the sequence of target action generation results.

[0103] Optionally, in one embodiment of the present application, the word segmentation processing module includes: a text cleaning unit, which is used to perform text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation; a natural language description word segmentation processing unit, which is used to perform word segmentation processing on the cleaned natural language description of the digital human action generation to obtain a sequence of the digital human action generation text.

[0104] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0105] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0106] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0107] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0108] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the device claim can also be implemented by one unit through software or hardware.

[0109] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for intelligently generating generalized digital human actions, characterized in that: include: Obtain the digital human's actions to generate natural language descriptions; Performing text cleaning on the natural language description of digital human action generation to obtain the cleaned natural language description of digital human action generation, and then performing word segmentation processing to obtain a sequence of digital human action generation text; Semantically encoding the sequence of text generated by the digital human action to obtain a sequence of context semantic encoding feature vectors of the text generated by the digital human action; The sequence of digital human action generated text context semantic coding feature vectors is subjected to significant aggregation of digital human action generated text features to obtain significant semantic aggregation coding features of digital human target generated actions, including: cluster analysis is performed on the sequence of digital human action generated text context semantic coding feature vectors to obtain digital human action generated text context semantic self-supervised cluster representation; based on the digital human action generated text context semantic self-supervised cluster representation, contribution field analysis is performed on the sequence of digital human action generated text context semantic coding feature vectors to obtain significant semantic aggregation coding features of digital human target generated actions; Extracting target actions from the salient semantic aggregation coding features of the target generated actions of the digital human to obtain a sequence of target actions; Filtering the target action generation results corresponding to each target action in the sequence of target actions in a preset action library to obtain a sequence of target action generation results; Based on the sequence of target action generation results, generating a sequence of digital human actions; The method comprises: performing cluster analysis on the sequence of the semantic encoding feature vectors of the text context generated by the digital human action to obtain the self-supervised cluster representation of the semantics of the text context generated by the digital human action, comprising: calculating the mean vector of the sequence of the semantic encoding feature vectors of the text context generated by the digital human action to obtain the self-supervised cluster representation vector of the semantics of the text context generated by the digital human action as the self-supervised cluster representation of the semantics of the text context generated by the digital human action; Among them, based on the self-supervised clustering representation of the digital human action generated text context semantics, the sequence of the digital human action generated text context semantic encoding feature vectors is subjected to contribution field analysis to obtain the significant semantic aggregation encoding features of the digital human target generated action, including: Calculating the implicit clustering contribution factor of each digital human action generated text context semantic encoding feature vector in the sequence of digital human action generated text context semantic encoding feature vector relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain a sequence of digital human action generated text context semantic implicit clustering contribution factors; Arranging the sequence of implicit clustering contribution factors of the digital human action generated text context semantics into a digital human action generated text context semantics clustering contribution field distribution vector; Inputting the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action; Taking each feature value in the modulation weight vector of the semantic clustering contribution field of the digital human action generated text context as a weight, calculating the position-weighted sum of the sequence of the semantic coding feature vector of the digital human action generated text context to obtain the significant semantic aggregation coding feature vector of the digital human target generated action as the significant semantic aggregation coding feature of the digital human target generated action; The step of calculating the implicit clustering contribution factor of each digital human action generated text context semantic encoding feature vector in the sequence of digital human action generated text context semantic encoding feature vectors relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain a sequence of digital human action generated text context semantic implicit clustering contribution factors includes: Dividing the feature value of each position in the digital human action generated text context semantic encoding feature vector by the feature value of the corresponding position in the digital human action generated text context semantic self-supervised clustering representation vector by position to obtain the digital human action generated text context semantic contribution vector; Calculating the logarithmic function value with base 2 of the absolute value of each eigenvalue of the digital human action generated text context semantic contribution vector to obtain the digital human action generated text context semantic contribution logarithmic vector; After calculating the positional dot product of the semantic coding feature vector of the text context generated by the digital human action and the logarithmic vector of the semantic contribution to the text context generated by the digital human action, the sum of the values ​​of each position in the dot product vector is calculated to obtain the semantic clustering contribution value of the text context generated by the digital human action; Calculate an exponential function with the natural constant e as the base and the semantic clustering contribution value of the text context generated by the digital human action as the exponent to obtain the semantic implicit clustering contribution factor of the text context generated by the digital human action; The distribution vector of the semantic clustering contribution field of the text context generated by the digital human action is input into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action, including: Respectively performing matrix-vector multiplication of the query weight matrix, the key weight matrix and the value weight matrix with the digital human action generated text context semantic clustering contribution field distribution vector to obtain the digital human action generated text context semantic query vector, the digital human action generated text context semantic key vector and the digital human action generated text context semantic value vector; After calculating the product between the digital human action generated text context semantic query vector and the transposed vector of the digital human action generated text context semantic key vector to obtain the digital human action generated text context semantic association matrix, the digital human action generated text context semantic association matrix and the square root of the length of the digital human action generated text context semantic key vector are divided by position to obtain the digital human action generated text context semantic association modulation matrix; Inputting the digital human action generated text context semantic association modulation matrix into a softmax function to obtain a digital human action generated text context semantic association activation matrix; The digital human action generated text context semantic value vector is multiplied by the digital human action generated text context semantic association activation matrix to obtain the digital human action generated text context semantic clustering contribution field modulation weight vector.

2. The method for intelligently generating generalized digital human actions according to claim 1, characterized in that: The digital human action generation natural language description is subjected to text cleaning to obtain the cleaned digital human action generation natural language description, and then subjected to word segmentation processing to obtain a sequence of digital human action generation text, including: Performing text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation; The cleaned natural language description of the digital human action generation is segmented to obtain a sequence of the digital human action generation text.

3. The method for intelligently generating generalized digital human actions according to claim 2, characterized in that: The sequence of text generated by the digital human action is semantically encoded to obtain a sequence of contextual semantic encoding feature vectors of the text generated by the digital human action, including: inputting the sequence of text generated by the digital human action into a semantic encoder containing a Bert model to obtain a sequence of contextual semantic encoding feature vectors of the text generated by the digital human action.

4. The method for intelligently generating generalized digital human actions according to claim 3, characterized in that: The target action is extracted from the significant semantic aggregate coding features of the digital human target generation action to obtain a sequence of target actions, including: inputting the significant semantic aggregate coding feature vector of the digital human target generation action into a target digital human text action model to obtain the sequence of target actions.

5. A digital human action generalization intelligent generation system, characterized in that: include: A natural language description acquisition module, used to acquire the digital human's actions and generate natural language descriptions; A word segmentation processing module is used to perform text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation, and then perform word segmentation processing to obtain a sequence of digital human action generation texts; A semantic encoding module, used for semantically encoding the sequence of text generated by the digital human action to obtain a sequence of context semantic encoding feature vectors of the text generated by the digital human action; The text feature significant aggregation processing module is used to perform significant aggregation of digital human action generated text features on the sequence of digital human action generated text context semantic coding feature vectors to obtain significant semantic aggregation coding features of digital human target generated actions, including: a cluster analysis processing unit, used to perform cluster analysis on the sequence of digital human action generated text context semantic coding feature vectors to obtain digital human action generated text context semantic self-supervised cluster representation; a contribution field analysis unit, used to perform contribution field analysis on the sequence of digital human action generated text context semantic coding feature vectors based on the digital human action generated text context semantic self-supervised cluster representation to obtain significant semantic aggregation coding features of digital human target generated actions; A target action extraction module, used for extracting target actions from the significant semantic aggregation coding features of the target generated actions of the digital human to obtain a sequence of target actions; A target action generation result screening module, used for screening the target action generation results corresponding to each target action in the sequence of target actions in a preset action library to obtain a sequence of target action generation results; A digital human action generation module, used to generate a sequence of digital human actions based on the sequence of target action generation results; The method comprises: performing cluster analysis on the sequence of the semantic encoding feature vectors of the text context generated by the digital human action to obtain the self-supervised cluster representation of the semantics of the text context generated by the digital human action, comprising: calculating the mean vector of the sequence of the semantic encoding feature vectors of the text context generated by the digital human action to obtain the self-supervised cluster representation vector of the semantics of the text context generated by the digital human action as the self-supervised cluster representation of the semantics of the text context generated by the digital human action; Among them, based on the self-supervised clustering representation of the digital human action generated text context semantics, the sequence of the digital human action generated text context semantic encoding feature vectors is subjected to contribution field analysis to obtain the significant semantic aggregation encoding features of the digital human target generated action, including: Calculating the implicit clustering contribution factor of each digital human action generated text context semantic encoding feature vector in the sequence of digital human action generated text context semantic encoding feature vector relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain a sequence of digital human action generated text context semantic implicit clustering contribution factors; Arranging the sequence of implicit clustering contribution factors of the digital human action generated text context semantics into a digital human action generated text context semantics clustering contribution field distribution vector; Inputting the distribution vector of the semantic clustering contribution field of the text context generated by the digital human action into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action; Taking each feature value in the modulation weight vector of the semantic clustering contribution field of the digital human action generated text context as a weight, calculating the position-weighted sum of the sequence of the semantic coding feature vector of the digital human action generated text context to obtain the significant semantic aggregation coding feature vector of the digital human target generated action as the significant semantic aggregation coding feature of the digital human target generated action; The step of calculating the implicit clustering contribution factor of each digital human action generated text context semantic encoding feature vector in the sequence of digital human action generated text context semantic encoding feature vectors relative to the digital human action generated text context semantic self-supervised clustering representation vector to obtain a sequence of digital human action generated text context semantic implicit clustering contribution factors includes: Dividing the feature value of each position in the digital human action generated text context semantic encoding feature vector by the feature value of the corresponding position in the digital human action generated text context semantic self-supervised clustering representation vector by position to obtain the digital human action generated text context semantic contribution vector; Calculating the logarithmic function value with base 2 of the absolute value of each eigenvalue of the digital human action generated text context semantic contribution vector to obtain the digital human action generated text context semantic contribution logarithmic vector; After calculating the positional dot product of the semantic coding feature vector of the text context generated by the digital human action and the logarithmic vector of the semantic contribution to the text context generated by the digital human action, the sum of the values ​​of each position in the dot product vector is calculated to obtain the semantic clustering contribution value of the text context generated by the digital human action; Calculate an exponential function with the natural constant e as the base and the semantic clustering contribution value of the text context generated by the digital human action as the exponent to obtain the semantic implicit clustering contribution factor of the text context generated by the digital human action; The distribution vector of the semantic clustering contribution field of the text context generated by the digital human action is input into the clustering contribution field explicit modeling module based on the self-attention mechanism to obtain the modulation weight vector of the semantic clustering contribution field of the text context generated by the digital human action, including: Respectively performing matrix-vector multiplication of the query weight matrix, the key weight matrix and the value weight matrix with the digital human action generated text context semantic clustering contribution field distribution vector to obtain the digital human action generated text context semantic query vector, the digital human action generated text context semantic key vector and the digital human action generated text context semantic value vector; After calculating the product between the digital human action generated text context semantic query vector and the transposed vector of the digital human action generated text context semantic key vector to obtain the digital human action generated text context semantic association matrix, the digital human action generated text context semantic association matrix and the square root of the length of the digital human action generated text context semantic key vector are divided by position to obtain the digital human action generated text context semantic association modulation matrix; Inputting the digital human action generated text context semantic association modulation matrix into a softmax function to obtain a digital human action generated text context semantic association activation matrix; The digital human action generated text context semantic value vector is multiplied by the digital human action generated text context semantic association activation matrix to obtain the digital human action generated text context semantic clustering contribution field modulation weight vector.

6. The generalized intelligent generation system for digital human actions according to claim 5 is characterized in that: The word segmentation processing module includes: A text cleaning unit, used for performing text cleaning on the natural language description of the digital human action generation to obtain the cleaned natural language description of the digital human action generation; The natural language description word segmentation processing unit is used to perform word segmentation processing on the cleaned natural language description of the digital human action generation to obtain a sequence of the digital human action generation text.

Images