Training method, device, equipment and storage medium for text feature generation model

Abstract

The present disclosure relates to a training method, apparatus, device, and storage medium for a text feature generation model, and relates to the field of artificial intelligence technology. The method comprises: obtaining a training data set, the training data set comprising multiple sets of training data, each set of training data comprising a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text, wherein the positive text corresponding to the training text is a text semantically related to the training text; performing masking processing on the multiple training texts to obtain masked texts for each training text; training a text feature generation model based on the masked texts of each training text and each training text to obtain an intermediate text feature generation model; and training the intermediate text feature generation model based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text. The text feature generation model trained by this method has a greater depth of understanding and representational capabilities.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, device, and storage medium for a text feature generation model. Background Art

[0002] Vincent video technology is one of the important topics in the current field of artificial intelligence technology. The goal of Vincent video is to generate a video based on the natural language text input by the user that is highly consistent with the natural language text in content, action, scene and style.

[0003] In related technologies, the Wensheng video model includes a text feature generation model and a video generation model. The natural language text input by the user is fed into the text feature generation model to obtain text features corresponding to the natural speech text. The text features corresponding to the natural language text are then fed into the video generation model to generate a video corresponding to the natural language text.

[0004] However, the depth of understanding and representation capabilities of natural language text generated by the text feature generation model fundamentally determine the quality of the generated video. Therefore, a training method for text feature generation models with deep understanding and strong representation capabilities is urgently needed. Summary of the Invention

[0005] The present disclosure provides a training method, apparatus, device, and storage medium for a text feature generation model, which can train a text feature generation model with a high depth of understanding and strong representation capabilities. The technical solution of the present disclosure is as follows.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for training a text feature generation model is provided, the method comprising: obtaining a training data set, the training data set comprising multiple groups of training data, each group of training data comprising a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text, the positive text corresponding to the training text being a text semantically relevant to the training text, and the negative text corresponding to the training text being a text semantically irrelevant to the training text; performing masking on multiple training texts to obtain masked texts of respective training texts, wherein at least one word in the masked text of any training text is masked; training a text feature generation model based on the masked texts of the respective training texts and the respective training texts to obtain an intermediate text feature generation model; training the intermediate text feature generation model based on the respective training texts, the positive texts corresponding to the respective training texts, and the negative texts corresponding to the respective training texts.

[0007] According to a second aspect of an embodiment of the present disclosure, a text feature generation method is provided, the method comprising: obtaining a target text for which text features are to be generated; calling a trained text feature generation model to process the target text to obtain text features of the target text, wherein the trained text feature generation model is trained by the training method of the text feature generation model described in the first aspect.

[0008] According to a third aspect of an embodiment of the present disclosure, a training device for a text feature generation model is provided, and the device includes the following contents.

[0009] The acquisition module is configured to execute acquisition of a training data set, wherein the training data set includes multiple groups of training data, each group of training data includes a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text, wherein the positive text corresponding to the training text is a text that is semantically related to the training text, and the negative text corresponding to the training text is a text that is semantically irrelevant to the training text.

[0010] The processing module is configured to perform masking processing on a plurality of training texts to obtain masked texts of each training text, wherein at least one word in the masked text of any training text is masked.

[0011] The training module is configured to train the text feature generation model based on the mask text of each training text and each training text to obtain an intermediate text feature generation model.

[0012] The training module is further configured to train the intermediate text feature generation model based on the training texts, the positive texts corresponding to the training texts, and the negative texts corresponding to the training texts.

[0013] In some embodiments, the training module is configured to perform grouping of the masked texts of the respective training texts to obtain at least two text groups, each text group including at least one masked text; determine the first loss value corresponding to each text group based on the masked texts included in each text group and the training texts corresponding to the masked texts included in each text group; and train the text feature generation model based on the first loss values ​​corresponding to each text group to obtain the intermediate text feature generation model.

[0014] In some embodiments, the training module is configured to execute, for any one of the text groups, calling the text feature generation model to process the masked text included in the any one text group respectively, to obtain the predicted content of the masked text included in the any one text group; obtaining the original content of the masked text included in the any one text group based on the masked text included in the any one text group and the training text corresponding to the masked text included in the any one text group; determining the first loss value corresponding to the any one text group based on the predicted content of the masked text included in the any one text group and the original content of the masked text included in the any one text group.

[0015] In some embodiments, the training module is configured to execute, for any masked text included in any one of the masked texts, determine the first feature vector of the predicted content of the any one masked text, and determine the second feature vector of the original content of the any one masked text; determine the loss value of the any one masked text based on the first feature vector and the second feature vector, the loss value of the any one masked text being the similarity between the predicted content of the any one masked text and the original content of the any one masked text; determine the first loss value corresponding to the any one text group based on the loss value of the masked text included in the any one text group.

[0016] In some embodiments, the training module is configured to determine a dot product result of the first feature vector and the second feature vector as a loss value of any masked text.

[0017] In some embodiments, the training module is configured to determine an average of the loss values ​​of the masked texts included in any one of the text groups as a first loss value corresponding to the any one of the text groups.

[0018] In some embodiments, the training module is configured to determine the loss value corresponding to each training text based on the each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text; determine a second loss value based on the loss value corresponding to each training text; and train the intermediate text feature generation model based on the second loss value.

[0019] In some embodiments, the training module is configured to execute, for any of the training texts, calling the intermediate text feature generation model to process the any training text, the positive text corresponding to the any training text, and the negative text corresponding to the any training text, to obtain the third feature vector corresponding to the any training text, the fourth feature vector of the positive text corresponding to the any training text, and the fifth feature vector of the negative text corresponding to the any training text; and determine the loss value corresponding to the any training text based on the third feature vector, the fourth feature vector, and the fifth feature vector.

[0020] In some embodiments, the training module is configured to determine a first similarity based on the third eigenvector and the fourth eigenvector, wherein the first similarity is used to characterize the similarity between any training text and the positive text corresponding to any training text; determine a second similarity based on the third eigenvector and the fifth eigenvector, wherein the second similarity is used to characterize the similarity between any training text and the negative text corresponding to any training text; and determine a loss value corresponding to any training text based on the first similarity and the second similarity.

[0021] In some embodiments, the acquisition module is further configured to acquire a text video dataset, where the text video dataset includes a plurality of texts and a first video corresponding to each text.

[0022] The processing module is further configured to execute calling the trained text feature generation model to process the multiple texts respectively to obtain text features of each text.

[0023] The processing module is further configured to call a video generation model to process text features of each text to obtain a second video corresponding to each text.

[0024] The training module is further configured to train the video generation model based on the first video corresponding to each text and the second video corresponding to each text.

[0025] According to a fourth aspect of an embodiment of the present disclosure, a text feature generation device is provided, and the device includes the following contents.

[0026] The acquisition module is configured to execute the acquisition of the target text of the text features to be generated.

[0027] The processing module is configured to execute the call of the trained text feature generation model to process the target text to obtain the text features of the target text. The trained text feature generation model is trained by the training method of the text feature generation model described in the first aspect.

[0028] In some embodiments, the processing module is further configured to call a trained video generation model to process text features of the target text to obtain a target video corresponding to the target text.

[0029] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing program code executable by the processor; wherein the processor is configured to execute the program code to implement the above-mentioned training method of the text feature generation model, or to implement the above-mentioned text feature generation method.

[0030] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the program code in the computer-readable storage medium is executed by a processor of a terminal, the electronic device is enabled to execute the training method of the above-mentioned text feature generation model, or to implement the above-mentioned text feature generation method.

[0031] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the training method of the above-mentioned text feature generation model, or implements the above-mentioned text feature generation method.

[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0033] The disclosed embodiment provides a method for training a text feature generation model, which performs masking on the training text to train the text feature generation model using the masked text and the training text, thereby strengthening the text feature generation model's reliance on text semantics and its ability to capture contextual relationships. The intermediate text feature generation model is then trained using the training text and its positive and negative texts, which can guide the intermediate text feature generation model to accurately distinguish semantic similarities and differences, enabling the intermediate text feature generation model to not only mine the semantic structure of the text, but also extract more discriminative semantic features. Ultimately, the trained text feature generation model can not only accurately represent the semantics of the text itself, but also extract more discriminative features from semantic association comparisons, thereby enhancing the depth of understanding and representation capabilities of the trained text feature generation model. When the trained text feature generation model is used to obtain text features of a text, the obtained text features have a higher degree of match and accuracy with the text, and are more capable of representing the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0035] Figure 1 It is a schematic diagram showing an implementation environment according to an exemplary embodiment.

[0036] Figure 2 The figure is a flowchart of a method for training a text feature generation model according to an exemplary embodiment.

[0037] Figure 3 The figure is a flowchart of a method for generating text features according to an exemplary embodiment.

[0038] Figure 4 The figure is a flowchart of another method for training a text feature generation model according to an exemplary embodiment.

[0039] Figure 5 The figure is a flowchart of obtaining text features of a target text according to an exemplary embodiment.

[0040] Figure 6 The figure is a block diagram of a training device for a text feature generation model according to an exemplary embodiment.

[0041] Figure 7 It is a block diagram of a device for generating text features according to an exemplary embodiment.

[0042] Figure 8 It is a block diagram of a terminal according to an exemplary embodiment.

[0043] Figure 9 The figure is a block diagram of a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0044] In order to enable ordinary people in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0045] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0046] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, and display, etc.), and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the videos and texts involved in this disclosure were obtained with full authorization.

[0047] This disclosure relates to a training method for a text feature generation model for a Vincent video model. Vincent video technology is a key topic in the current field of artificial intelligence. Its goal is to automatically synthesize a video based on user-entered natural language text, ensuring that the content, action, scene, and style are highly consistent with the original text. Vincent video technology demonstrates significant potential for application in areas such as content creation, film and television previews, educational simulations, and advertising marketing. It significantly lowers the barrier to entry for video production and enables the rapid transformation of creative ideas into video.

[0048] In the technical architecture of the Wensheng video model, the text feature generation model plays an indispensable role. Its task is to convert human natural language text into a high-dimensional, machine-understandable mathematical representation, namely, text features. These text features serve as guiding conditions for the subsequent video generation model and directly determine the effectiveness of the generated video. Therefore, the depth of the text feature generation model's understanding of the input text and its ability to represent it fundamentally determine the performance ceiling and expressiveness of the Wensheng video model. A powerful text feature generation model can accurately capture subtle nuances in text, while an inadequate one will become a bottleneck in the effectiveness of the Wensheng video model.

[0049] The present disclosure provides a training method for a text feature generation model to train the text feature generation model and improve the understanding depth and representation ability of the text feature generation model.

[0050] The training method for the text feature generation model provided by the embodiments of the present disclosure can be executed by an electronic device. Figure 1 This is a schematic diagram of an implementation environment provided by the embodiment of the present disclosure, see Figure 1, the implementation environment includes: an electronic device 101. The electronic device 101 can be a terminal or a server, which is not limited in the embodiment of the present disclosure. In the embodiment of the present disclosure, the electronic device 101 obtains a training data set, which includes multiple groups of training data, each group of training data includes a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text; the multiple training texts are masked to obtain masked texts of each training text, and at least one word in the masked text of any training text is masked; according to the masked text of each training text and each training text, a text feature generation model is trained to obtain an intermediate text feature generation model; according to each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text, the intermediate text feature generation model is trained.

[0051] Optionally, electronic device 101 is a terminal, which can be at least one of a smartphone, smartwatch, desktop computer, laptop, virtual reality terminal, augmented reality terminal, wireless terminal, and laptop computer. The terminal has communication capabilities and can access a wired or wireless network. The term "terminal" can generally refer to one of multiple terminals, and those skilled in the art will appreciate that the number of terminals can be greater or lesser. Electronic device 101 is a server, which can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed file system. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some embodiments, the server and the terminal are directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment. Optionally, the number of servers can be greater or lesser, which is not limited in this embodiment. Of course, the server can also include other functional servers to provide more comprehensive and diverse services. Among them, the server undertakes the main computing work and the terminal undertakes the secondary computing work; or, the server undertakes the secondary computing work and the terminal undertakes the main computing work; or, the server or the terminal can undertake the computing work independently, which is not limited in the embodiments of the present disclosure.

[0052] Figure 2 FIG. 1 is a flowchart of a method for training a text feature generation model according to an exemplary embodiment. Figure 2 As shown, the method is executed by an electronic device, and the method includes the following steps.

[0053] In step 201 , the electronic device obtains a training data set, where the training data set includes multiple sets of training data, and each set of training data includes a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text.

[0054] In the embodiment of the present disclosure, the positive text corresponding to the training text is text that is semantically relevant to the training text, and the negative text corresponding to the training text is text that is semantically irrelevant to the training text.

[0055] The disclosed embodiments do not limit the method for obtaining the training dataset. Optionally, the training dataset is stored in the storage space of the electronic device, and the training dataset is obtained from the storage space of the electronic device. Alternatively, the training dataset is stored in the server, and the electronic device obtains the training dataset by interacting with the server.

[0056] The embodiment of the present disclosure does not limit the number of groups of training data included in the training data set. Each group of training data includes a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text.

[0057] For example, the training text is “The plot of the movie is so exciting”, the positive text of the training text is “The plot of the movie is really so exciting”, and the negative text of the training text is “The plot of the movie is not exciting at all”.

[0058] In step 202, the electronic device performs masking processing on a plurality of training texts to obtain masked texts of the respective training texts.

[0059] In the embodiment of the present disclosure, at least one word in the mask text of any training text is masked.

[0060] In some embodiments, in natural language processing, masking is a technique that guides a model to learn semantic associations by masking part of the content in the text, and is often used in pre-training models or text generation tasks.

[0061] In some embodiments, for any training text among the plurality of training texts, at least one word in any training text is replaced by a special symbol to obtain a masked text of any training text.

[0062] The special symbol is any symbol, which is not limited in the present disclosure. For example, the special symbol is [mask].

[0063] For example, the training text is “I like to go for a walk in the park in spring”, and the mask text of the training text is “[mask]I like to go for a walk in the park in spring”.

[0064] In step 203, the electronic device trains a text feature generation model based on the mask text of each training text and each training text to obtain an intermediate text feature generation model.

[0065] In some embodiments, the text feature generation model is trained based on the mask text of each training text and each training text to obtain an intermediate text feature generation model. The process includes: grouping the mask text of each training text to obtain at least two text groups, each text group including at least one mask text; determining the first loss value corresponding to each text group based on the mask text included in each text group and the training text corresponding to the mask text included in each text group; and training the text feature generation model based on the first loss value corresponding to each text group to obtain an intermediate text feature generation model.

[0066] In this embodiment, by grouping and processing masked texts, the text feature generation model can focus on the feature differences of different text groups during training, and use the loss values ​​of each group to guide the text feature generation model to learn more representative text features, thereby enhancing the text feature generation model's ability to understand and capture different texts, thereby optimizing the training process of the text feature generation model, improving the accuracy and generalization of the generated text features, and laying a solid model foundation for subsequent text processing tasks.

[0067] In some embodiments, the process of determining the first loss value corresponding to each text group based on the masked text included in each text group and the training text corresponding to the masked text included in each text group includes: for any text group in each text group, calling the text feature generation model to process the masked text included in any text group separately to obtain the predicted content of the masked text included in any text group; obtaining the original content of the masked text included in any text group based on the masked text included in any text group and the training text corresponding to the masked text included in any text group; determining the first loss value corresponding to any text group based on the predicted content of the masked text included in any text group and the original content of the masked text included in any text group.

[0068] In this embodiment, by using the text feature generation model to predict the masked content in the masked text and comparing it with the original content to determine the first loss value, the difference between the content predicted by the text feature generation model and the actual content can be accurately measured, and the masked text is combined with the corresponding training text to obtain the original content, ensuring the accuracy of the comparison standard, so that the loss value can effectively reflect the performance defects of the text feature generation model when processing the text group, thereby providing a clear direction for the targeted optimization of the text feature generation model, which helps to improve the accuracy and reliability of the text feature generation model in the masked text prediction task.

[0069] In some embodiments, the process of determining the first loss value corresponding to any text group based on the predicted content of the masked text included in any text group and the original content of the masked text included in any text group includes: for any masked text included in any text group, determining the first feature vector of the predicted content of any masked text, and determining the second feature vector of the original content of any masked text; determining the loss value of any masked text based on the first feature vector and the second feature vector, the loss value of any masked text being the similarity between the predicted content of any masked text and the original content of any masked text; determining the first loss value corresponding to any text group based on the loss value of the masked text included in any text group.

[0070] In this embodiment, the predicted content and original content of the masked text are converted into feature vectors, the loss value is measured by vector similarity, and then aggregated to obtain the first loss value of the text group. This method can quantify the difference between the model prediction and the real content from the feature level. The similarity measurement based on vector operation is more objective and accurate, and can keenly capture subtle deviations in text semantics and structure. The loss value calculation process from a single masked text to a text group can comprehensively evaluate the performance of the model at different granularities, provide multi-level and accurate feedback for model optimization, and effectively improve the model's text understanding and generation capabilities.

[0071] In some embodiments, the process of determining the loss value of any masked text according to the first feature vector and the second feature vector includes: determining the dot product result of the first feature vector and the second feature vector as the loss value of any masked text.

[0072] In this embodiment, by using the dot product result of the first eigenvector and the second eigenvector as the loss value of any masked text, the semantic similarity between the predicted content and the real content can be quantified using the geometric relationship in the vector space. The dot product operation can efficiently measure the consistency of the vector in direction, and the numerical value directly reflects the degree of semantic association between the two. It can not only capture the matching degree of the deep semantic features of the text, but also has the characteristic of simple calculation, which facilitates the model to perform gradient updates based on the loss value during training, thereby accurately optimizing the feature representation capability and improving the accuracy of masked text prediction.

[0073] In some embodiments, the process of determining the first loss value corresponding to any text group based on the loss value of the masked text included in any text group includes: determining the average value of the loss value of the masked text included in any text group as the first loss value corresponding to any text group.

[0074] In this embodiment, the randomness and abnormal fluctuations of the loss value of a single masked text are eliminated by statistical averaging, so that the loss value can more stably reflect the overall prediction error level of the text group, avoiding the interference of individual extreme samples on the optimization direction of the model. At the same time, the overall training effect of the text group is represented by a concise numerical value, which facilitates the balanced optimization of the prediction performance of each text group in batch training, ensuring the smoothness of the training process and the overall improvement of model performance.

[0075] In step 204 , the electronic device trains an intermediate text feature generation model based on each training text, a positive text corresponding to each training text, and a negative text corresponding to each training text.

[0076] The disclosed embodiment provides a method for training a text feature generation model, which performs masking on the training text to train the text feature generation model using the masked text and the training text, thereby strengthening the text feature generation model's reliance on text semantics and its ability to capture contextual relationships. The intermediate text feature generation model is then trained using the training text and its positive and negative texts, which can guide the intermediate text feature generation model to accurately distinguish semantic similarities and differences, enabling the intermediate text feature generation model to not only mine the semantic structure of the text, but also extract more discriminative semantic features. Ultimately, the trained text feature generation model can not only accurately represent the semantics of the text itself, but also extract more discriminative features from semantic association comparisons, thereby enhancing the depth of understanding and representation capabilities of the trained text feature generation model. When the trained text feature generation model is used to obtain text features of a text, the obtained text features have a higher degree of match and accuracy with the text, and are more capable of representing the text.

[0077] In some embodiments, the process of training the intermediate text feature generation model based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text includes: determining the loss value corresponding to each training text based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text; determining a second loss value based on the loss value corresponding to each training text; and training the intermediate text feature generation model based on the second loss value.

[0078] In this embodiment, the loss value of each training text is determined by combining each training text and its positive text and negative text, and then the overall second loss value is determined based on the loss value of each training text. The intermediate text feature generation model is trained with the second loss value, which can enhance the feature differentiation ability with the help of comparative learning of positive and negative texts. The positive text is used to constrain the model to capture semantic similarity, and the negative text promotes the model to identify semantic differences, thereby guiding the intermediate text feature generation model to learn more discriminative text feature representations, effectively improving the generalization ability and accuracy of the intermediate text feature generation model in tasks such as semantic matching and text similarity calculation.

[0079] In some embodiments, the process of determining the loss value corresponding to each training text based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text includes: for any training text among the training texts, calling the intermediate text feature generation model to process any training text, the positive text corresponding to any training text, and the negative text corresponding to any training text, to obtain the third feature vector corresponding to any training text, the fourth feature vector of the positive text corresponding to any training text, and the fifth feature vector of the negative text corresponding to any training text; based on the third feature vector, the fourth feature vector, and the fifth feature vector, determining the loss value corresponding to any training text.

[0080] In this embodiment, the intermediate text feature generation model extracts feature vectors for the training text, the corresponding positive text, and the corresponding negative text, and determines the loss value for the training text based on these vectors. This strengthens the model's understanding of the semantic associations and differences between texts at the feature level. Comparisons between different vectors encourage the model to learn more discriminative feature representations, allowing it to capture the semantic similarities between similar texts (training text and positive text) while also identifying the semantic differences between dissimilar texts (training text and negative text). This effectively improves the model's ability to determine text semantic relationships and generalizes its performance, providing a more accurate feature foundation for subsequent tasks such as text matching and classification.

[0081] In some embodiments, the process of determining the loss value corresponding to any training text based on the third eigenvector, the fourth eigenvector and the fifth eigenvector includes: determining a first similarity based on the third eigenvector and the fourth eigenvector, the first similarity being used to characterize the similarity between any training text and the positive text corresponding to any training text; determining a second similarity based on the third eigenvector and the fifth eigenvector, the second similarity being used to characterize the similarity between any training text and the negative text corresponding to any training text; and determining the loss value corresponding to any training text based on the first similarity and the second similarity.

[0082] In this embodiment, by calculating a first similarity between the training text and the positive text, and a second similarity between the training text and the negative text, and determining the loss value of the training text based on these two, the model can be used to strengthen the clustering of similar texts and the separation of different texts in the semantic space through comparative learning. The first similarity constrains the model to capture the semantic association between the training text and the positive text, while the second similarity drives the model to increase the semantic distance between the training text and the negative text, guiding the model to learn more discriminative feature representations, effectively improving the distinguishability of texts in the semantic space, and thus enhancing the accuracy and robustness of the model in tasks such as text matching and retrieval.

[0083] In some embodiments, after the intermediate text feature generation model is trained based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text, the method further includes: obtaining a text video dataset, the text video dataset including multiple texts and a first video corresponding to each text; calling the trained text feature generation model to process the multiple texts separately to obtain text features of each text; calling the video generation model to process the text features of each text to obtain a second video corresponding to each text; and training the video generation model based on the first video corresponding to each text and the second video corresponding to each text.

[0084] In this embodiment, a trained text feature generation model is used to extract text features from a text-video dataset. A video generation model then generates a second video based on the text features. The video generation model is then trained using the first and second videos. This approach enables the trained video generation model to generate videos that closely match the text, resulting in better quality videos.

[0085] Figure 3 is a flowchart of a method for generating text features according to an exemplary embodiment. Figure 3 As shown, the method is executed by an electronic device, and the method includes the following steps.

[0086] In step 301, the electronic device obtains a target text for which text features are to be generated.

[0087] In some embodiments, the target text for generating text features is any text, which is not limited in the embodiments of the present disclosure.

[0088] The embodiment of the present disclosure does not limit the method for obtaining the target text. Optionally, the text input by the user is used as the target text for generating text features.

[0089] In step 302, the electronic device calls the trained text feature generation model to process the target text to obtain text features of the target text.

[0090] Among them, the trained text feature generation model is based on Figure 2 The text feature generation model is trained using the training method shown.

[0091] In this embodiment, a trained text feature generation model is used to process the target text to obtain text features. Leveraging the semantic representation capabilities learned by the model from large-scale data, the target text is mapped into a high-dimensional semantic space, enabling the text features to capture both surface information at the lexical level and deep structural features at the syntactic and semantic levels. The trained model possesses generalization capabilities, adapting to text input in diverse contexts. The generated text features possess strong semantic representation capabilities and a higher degree of match with the target text.

[0092] In some embodiments, the method further includes: calling a trained video generation model to process text features of the target text to obtain a target video corresponding to the target text.

[0093] In this embodiment, since the text features of the target text obtained after processing the target text through the trained text feature generation model can better represent the target text, the target video obtained through the text features of the target text has a higher matching degree with the target text.

[0094] The above is based on Figure 2 The embodiment of the text feature generation model training process is briefly introduced below. Figure 4 The embodiment of the present invention further introduces the training process of the text feature generation model. Figure 4 , Figure 4 The present invention is a flowchart of a method for training a text feature generation model according to an exemplary embodiment. The method includes the following steps.

[0095] In step 401 , the electronic device obtains a training data set, where the training data set includes multiple sets of training data, and each set of training data includes a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text.

[0096] In the embodiment of the present disclosure, the positive text corresponding to the training text is text that is semantically relevant to the training text, and the negative text corresponding to the training text is text that is semantically irrelevant to the training text.

[0097] The disclosed embodiments do not limit the method for obtaining the training dataset. Optionally, the training dataset is stored in the storage space of the electronic device, and the training dataset is obtained from the storage space of the electronic device. Alternatively, the training dataset is stored in the server, and the electronic device obtains the training dataset by interacting with the server.

[0098] The embodiment of the present disclosure does not limit the number of groups of training data included in the training data set. Each group of training data includes a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text.

[0099] For example, the training text is “The plot of the movie is so exciting”, the positive text of the training text is “The plot of the movie is really so exciting”, and the negative text of the training text is “The plot of the movie is not exciting at all”.

[0100] In step 402, the electronic device performs masking processing on a plurality of training texts to obtain masked texts of the respective training texts, wherein at least one word in the masked text of any training text is masked.

[0101] In some embodiments, performing masking on a plurality of training texts to obtain masked texts of each training text includes: performing masking on the plurality of training texts through a Masked Next Token Prediction task to obtain masked texts of each training text.

[0102] Optionally, masking any training text through the masked next word prediction task to obtain the masked text of any training text includes: replacing at least one word in any training text with a special symbol through the masked next word prediction task to obtain the masked text of any training text.

[0103] The special symbol is any symbol, which is not limited in the present disclosure. For example, the special symbol is [mask].

[0104] For example, the training text is “I like to go for a walk in the park in spring”, and the mask text of the training text is “[mask]I like to go for a walk in the park in spring”.

[0105] In step 403, the electronic device groups the masked texts of the training texts to obtain at least two text groups, each text group including at least one masked text.

[0106] In some embodiments, after obtaining the mask texts of multiple training texts in the above step 302, the mask texts of the multiple training texts are grouped to obtain at least two text groups, each text group including at least one mask text, and the number of mask texts included in each text group can be the same or different, and the embodiments of the present disclosure do not limit this.

[0107] The embodiment of the present disclosure does not limit the manner in which the masked texts of each training text are grouped. Optionally, the masked texts of each training text are grouped based on text length to obtain at least two text groups.

[0108] In some embodiments, the process of grouping the masked texts of each training text based on the text length to obtain at least two text groups includes: determining the number of words included in the masked text of each training text, and grouping the masked texts with the same number of words into one group to obtain at least two text groups.

[0109] For example, there are 10 training texts, the masked text of the first training text includes 8 words, the masked text of the second training text includes 10 words, the masked text of the third training text includes 12 words, the masked text of the fourth training text includes 8 words, the masked text of the fifth training text includes 10 words, the masked text of the sixth training text includes 12 words, the masked text of the seventh training text includes 8 words, the masked text of the eighth training text includes 10 words, the masked text of the ninth training text includes 12 words, and the masked text of the tenth training text includes 8 words. Therefore, the masked text of the first training text, the masked text of the fourth training text, the masked text of the seventh training text, and the masked text of the tenth training text are grouped together; the masked text of the second training text, the masked text of the fifth training text, and the masked text of the eighth training text are grouped together; and the masked text of the third training text, the masked text of the sixth training text, and the masked text of the ninth training text are grouped together.

[0110] In step 404, the electronic device determines a first loss value corresponding to each text group based on the masked text included in each text group and the training text corresponding to the masked text included in each text group.

[0111] In some embodiments, the process of determining the first loss value corresponding to each text group based on the masked text included in each text group and the training text corresponding to the masked text included in each text group includes: for any text group in each text group, calling the text feature generation model to process the masked text included in any text group separately to obtain the predicted content of the masked text included in any text group; obtaining the original content of the masked text included in any text group based on the masked text included in any text group and the training text corresponding to the masked text included in any text group; determining the first loss value corresponding to any text group based on the predicted content of the masked text included in any text group and the original content of the masked text included in any text group.

[0112] In this embodiment, by using the text feature generation model to predict the masked content in the masked text and comparing it with the original content to determine the first loss value, the difference between the content predicted by the text feature generation model and the actual content can be accurately measured, and the masked text is combined with the corresponding training text to obtain the original content, ensuring the accuracy of the comparison standard, so that the loss value can effectively reflect the performance defects of the text feature generation model when processing the text group, thereby providing a clear direction for the targeted optimization of the text feature generation model, which helps to improve the accuracy and reliability of the text feature generation model in the masked text prediction task.

[0113] In some embodiments, the process of calling the text feature generation model to process the masked text included in any text group separately and obtaining the predicted content of the masked text included in any text group includes: inputting the masked text included in any text group into the text feature generation model separately and obtaining the predicted content of the masked text included in any text group.

[0114] The text feature generation model is a large language model that has been pre-trained on large-scale text data and replaces the unidirectional attention mechanism with a bidirectional attention mechanism. Due to the large language model's powerful language representation capabilities, it has inherent advantages as a text feature generation model. However, the large language model's structure and training objectives are not fully adapted to the text encoding requirements of the heavy text and video tasks, so improvements have been made to the large language model. Large language models typically use a unidirectional attention mechanism, which to some extent limits the model's overall understanding of the input text. Therefore, the unidirectional attention mechanism in the large language model is replaced with a bidirectional attention mechanism to obtain the text feature generation model.

[0115] In some embodiments, the process of obtaining the original content of the masked text included in any text group based on the masked text included in any text group and the training text corresponding to the masked text included in any text group includes: for any masked text in the masked text included in any text group, taking the different words in any masked text and the training text corresponding to any masked text as the original content of any masked text.

[0116] For example, if any mask text is “[mask]I like to go for a walk in the park in spring”, and the training text corresponding to any mask text is “I like to go for a walk in the park in spring”, then the original content of any mask text is “I”.

[0117] In some embodiments, the process of determining the first loss value corresponding to any text group based on the predicted content of the masked text included in any text group and the original content of the masked text included in any text group includes: for any masked text included in any text group, determining the first feature vector of the predicted content of any masked text, and determining the second feature vector of the original content of any masked text; determining the loss value of any masked text based on the first feature vector and the second feature vector, the loss value of any masked text being the similarity between the predicted content of any masked text and the original content of any masked text; determining the first loss value corresponding to any text group based on the loss value of the masked text included in any text group.

[0118] In this embodiment, the predicted content and original content of the masked text are converted into feature vectors, the loss value is measured by vector similarity, and then aggregated to obtain the first loss value of the text group. This method can quantify the difference between the model prediction and the real content from the feature level. The similarity measurement based on vector operation is more objective and accurate, and can keenly capture subtle deviations in text semantics and structure. The loss value calculation process from a single masked text to a text group can comprehensively evaluate the performance of the model at different granularities, provide multi-level and accurate feedback for model optimization, and effectively improve the model's text understanding and generation capabilities.

[0119] In some embodiments, the process of determining the first feature vector of the predicted content of any masked text is similar to the process of determining the second feature vector of the original content of any masked text. The disclosed embodiments illustrate this process using only the process of determining the first feature vector of the predicted content of any masked text as an example. Alternatively, the process of determining the first feature vector of the predicted content of any masked text includes: inputting the predicted content of any masked text into a text feature generation model to obtain the first feature vector of the predicted content of any masked text.

[0120] In some embodiments, the process of determining the loss value of any masked text according to the first feature vector and the second feature vector includes: determining the dot product result of the first feature vector and the second feature vector as the loss value of any masked text.

[0121] In this embodiment, by using the dot product result of the first eigenvector and the second eigenvector as the loss value of any masked text, the semantic similarity between the predicted content and the real content can be quantified using the geometric relationship in the vector space. The dot product operation can efficiently measure the consistency of the vector in direction, and the numerical value directly reflects the degree of semantic association between the two. It can not only capture the matching degree of the deep semantic features of the text, but also has the characteristic of simple calculation, which facilitates the model to perform gradient updates based on the loss value during training, thereby accurately optimizing the feature representation capability and improving the accuracy of masked text prediction.

[0122] The dimension of the first eigenvector is the same as the dimension of the second eigenvector. For example, the dimension of the first eigenvector is 1×3, and the dimension of the second eigenvector is also 1×3.

[0123] In some embodiments, the process of determining the first loss value corresponding to any text group based on the loss value of the masked text included in any text group includes: determining the average value of the loss value of the masked text included in any text group as the first loss value corresponding to any text group.

[0124] In this embodiment, the randomness and abnormal fluctuations of the loss value of a single masked text are eliminated by statistical averaging, so that the loss value can more stably reflect the overall prediction error level of the text group, avoiding the interference of individual extreme samples on the optimization direction of the model. At the same time, the overall training effect of the text group is represented by a concise numerical value, which facilitates the balanced optimization of the prediction performance of each text group in batch training, ensuring the smoothness of the training process and the overall improvement of model performance.

[0125] Exemplarily, any text group includes 3 masked texts, where the loss value of the first masked text is A, the loss value of the second masked text is B, and the loss value of the third masked text is C. Then the first loss value corresponding to any text group is (A+B+C) / 3.

[0126] In step 405, the electronic device trains the text feature generation model according to the first loss value corresponding to each text group to obtain an intermediate text feature generation model.

[0127] In some embodiments, the text feature generation model is trained according to the first loss value corresponding to each text group to obtain an intermediate text feature generation model. The process includes: determining the adjustment gradient corresponding to each text group based on the first loss value corresponding to each text group; and adjusting the model parameters of the text feature generation model in turn according to the adjustment gradient corresponding to each text group to obtain an intermediate text feature generation model.

[0128] Optionally, the electronic device stores a correspondence between loss values ​​and adjustment gradients, and determines the adjustment gradient corresponding to each text group according to the first loss value corresponding to each text group and the correspondence between the loss value and the adjustment gradient.

[0129] For example, there are three text groups. Based on the first loss value corresponding to the first text group, the adjustment gradient corresponding to the first text group is determined to be adjustment gradient 1. Based on the first loss value corresponding to the second text group, the adjustment gradient corresponding to the second text group is determined to be adjustment gradient 2. Based on the first loss value corresponding to the third text group, the adjustment gradient corresponding to the third text group is determined to be adjustment gradient 3. The model parameters of the text feature generation model are first adjusted according to adjustment gradient 1, then adjusted according to adjustment gradient 2, and finally adjusted according to adjustment gradient 3 to obtain an intermediate text feature generation model. Of course, the model parameters of the text feature generation model can also be adjusted first according to adjustment gradient 2, then adjusted according to adjustment gradient 1, and finally adjusted according to adjustment gradient 3 to obtain an intermediate text feature generation model. Alternatively, the model parameters of the text feature generation model can be adjusted first according to adjustment gradient 3, then adjusted according to adjustment gradient 2, and finally adjusted according to adjustment gradient 1 to obtain an intermediate text feature generation model. The embodiment of the present disclosure does not limit the adjustment order.

[0130] In step 406, the electronic device determines a loss value corresponding to each training text according to each training text, a positive text corresponding to each training text, and a negative text corresponding to each training text.

[0131] In some embodiments, the process of determining the loss value corresponding to each training text based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text includes: for any training text among the training texts, calling the intermediate text feature generation model to process any training text, the positive text corresponding to any training text, and the negative text corresponding to any training text, to obtain the third feature vector corresponding to any training text, the fourth feature vector of the positive text corresponding to any training text, and the fifth feature vector of the negative text corresponding to any training text; based on the third feature vector, the fourth feature vector, and the fifth feature vector, determining the loss value corresponding to any training text.

[0132] In this embodiment, the intermediate text feature generation model extracts feature vectors for the training text, the corresponding positive text, and the corresponding negative text, and determines the loss value for the training text based on these vectors. This strengthens the model's understanding of the semantic associations and differences between texts at the feature level. Comparisons between different vectors encourage the model to learn more discriminative feature representations, allowing it to capture the semantic similarities between similar texts (training text and positive text) while also identifying the semantic differences between dissimilar texts (training text and negative text). This effectively improves the model's ability to determine text semantic relationships and generalizes its performance, providing a more accurate feature foundation for subsequent tasks such as text matching and classification.

[0133] In some embodiments, the process of calling the intermediate text feature generation model to process any training text to obtain the third feature vector corresponding to any training text is similar to the process of calling the intermediate text feature generation model to process the positive text corresponding to any training text to obtain the fourth feature vector of the positive text corresponding to any training text, and the process of calling the intermediate text feature generation model to process the negative text corresponding to any training text to obtain the fifth feature vector of the negative text corresponding to any training text. The embodiments of the present application only illustrate the process of calling the intermediate text feature generation model to process any training text to obtain the third feature vector corresponding to any training text as an example.

[0134] Optionally, the process of calling the intermediate text feature generation model to process any training text and obtaining the third feature vector corresponding to any training text includes: inputting any training text into the intermediate text feature generation model, and using the output result of the intermediate text feature generation model as the third feature vector corresponding to any training text.

[0135] In some embodiments, the process of determining the loss value corresponding to any training text based on the third eigenvector, the fourth eigenvector and the fifth eigenvector includes: determining a first similarity based on the third eigenvector and the fourth eigenvector, the first similarity being used to characterize the similarity between any training text and the positive text corresponding to any training text; determining a second similarity based on the third eigenvector and the fifth eigenvector, the second similarity being used to characterize the similarity between any training text and the negative text corresponding to any training text; and determining the loss value corresponding to any training text based on the first similarity and the second similarity.

[0136] In this embodiment, by calculating a first similarity between the training text and the positive text, and a second similarity between the training text and the negative text, and determining the loss value of the training text based on these two, the model can be used to strengthen the clustering of similar texts and the separation of different texts in the semantic space through comparative learning. The first similarity constrains the model to capture the semantic association between the training text and the positive text, while the second similarity drives the model to increase the semantic distance between the training text and the negative text, guiding the model to learn more discriminative feature representations, effectively improving the distinguishability of texts in the semantic space, and thus enhancing the accuracy and robustness of the model in tasks such as text matching and retrieval.

[0137] In some embodiments, the process of determining the first similarity based on the third eigenvector and the fourth eigenvector includes: determining a dot product result of the third eigenvector and the fourth eigenvector as the first similarity.

[0138] The dimension of the third eigenvector is the same as the dimension of the fourth eigenvector.

[0139] In some embodiments, the process of determining the second similarity based on the third eigenvector and the fifth eigenvector includes: determining a dot product result of the third eigenvector and the fifth eigenvector as the second similarity.

[0140] The dimension of the third eigenvector is the same as the dimension of the fifth eigenvector.

[0141] In some embodiments, the process of determining the loss value corresponding to any training text according to the first similarity and the second similarity includes: determining an average of the first similarity and the second similarity as the loss value corresponding to any training text.

[0142] In step 407, the electronic device determines a second loss value according to the loss value corresponding to each training text.

[0143] In some embodiments, the process of determining the second loss value according to the loss values ​​corresponding to the training texts includes: determining an average value of the loss values ​​corresponding to the training texts as the second loss value.

[0144] For example, there are 10 training texts, and the average of the loss values ​​corresponding to the 10 training texts is used as the second loss value.

[0145] In step 408, the electronic device trains the intermediate text feature generation model based on the second loss value.

[0146] In some embodiments, the process of training the intermediate text feature generation model according to the second loss value includes: determining the adjustment gradient corresponding to the second loss value; and adjusting the model parameters of the intermediate text feature generation model according to the adjustment gradient corresponding to the second loss value.

[0147] In some embodiments, the intermediate text feature generation model is iteratively trained according to steps 406 to 408 until an iteration stop condition is reached, and the intermediate text feature generation model when the iteration stop condition is reached is used as the trained text feature generation model.

[0148] The iteration stopping condition can be set as needed, and is not limited in the present embodiment. For example, the iteration stopping condition is when the number of iterations reaches the target number, the second loss value reaches the target loss value, or the second loss value reaches convergence.

[0149] The target number of times and the target loss value are both set based on experience, or flexibly adjusted according to the implementation environment, and the embodiments of the present disclosure do not limit this.

[0150] In some embodiments, after training the intermediate text feature generation model based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text, text features can also be obtained using the trained text feature generation model. Optionally, this process includes: obtaining a target text for which text features are to be generated; and invoking the trained text feature generation model to process the target text to obtain text features for the target text.

[0151] In this embodiment, a trained text feature generation model is used to process the target text to obtain text features. Leveraging the semantic representation capabilities learned by the model from large-scale data, the target text is mapped into a high-dimensional semantic space, enabling the text features to capture both surface information at the lexical level and deep structural features at the syntactic and semantic levels. The trained model possesses generalization capabilities, adapting to text input in diverse contexts. The generated text features possess strong semantic representation capabilities and a higher degree of match with the target text.

[0152] The embodiment of the present disclosure does not limit the method for obtaining the target text. For example, the target text is the text input by the user.

[0153] In some embodiments, the process of calling the trained text feature generation model to process the target text and obtaining the text features of the target text includes: inputting the target text into the trained text feature generation model, and using the output results of the trained text feature generation model as the text features of the target text.

[0154] The target text features are the weighted fusion of the outputs of each layer of the trained text feature generation model. The input of each layer is the output of the previous layer. This makes the text features more expressive, enabling the text feature generation model to better capture complex semantics.

[0155] Take the trained text feature generation model as an example, which includes three layers, such as Figure 5 This is a flowchart for obtaining text features of a target text provided by an embodiment of the present application. The target text is input into the first layer of the trained text feature generation model to obtain the output result of the first layer; the output result of the first layer is input into the second layer to obtain the output result of the second layer; the output result of the second layer is input into the third layer to obtain the output result of the third layer; based on the output results of the first layer, the output results of the second layer, and the output results of the third layer, the text features of the target text are obtained. That is, the output results of the first layer, the output results of the second layer, and the output results of the third layer are weightedly fused to obtain the text features of the target text.

[0156] In some embodiments, after the intermediate text feature generation model is trained based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text, a text video dataset can also be obtained, where the text video dataset includes multiple texts and a first video corresponding to each text; the trained text feature generation model is called to process the multiple texts separately to obtain the text features of each text; the video generation model is called to process the text features of each text to obtain a second video corresponding to each text; and the video generation model is trained based on the first video corresponding to each text and the second video corresponding to each text.

[0157] In this embodiment, a trained text feature generation model is used to extract text features from a text-video dataset. A video generation model then generates a second video based on the text features. The video generation model is then trained using the first and second videos. This approach enables the trained video generation model to generate videos that closely match the text, resulting in better quality videos.

[0158] The embodiment of the present disclosure does not limit the method for obtaining the text video dataset. Optionally, the method for obtaining the text video dataset is similar to the method for obtaining the training dataset.

[0159] In some embodiments, the process of calling the trained text feature generation model to process multiple texts separately to obtain the text features of each text includes: for any text among the texts, any text is input into the trained text feature generation model, and the output result of the trained text feature generation model is used as the text feature of any text.

[0160] In some embodiments, the process of training a video generation model based on the first video corresponding to each text and the second video corresponding to each text includes: determining the loss value of each text based on the first video corresponding to each text and the second video corresponding to each text, the loss value of any text being used to characterize the similarity between the first video corresponding to any text and the second video corresponding to any text; determining a third loss value based on the loss value of each text; and training the video generation model based on the third loss value.

[0161] In some embodiments, the process of determining the loss value of each text based on the first video corresponding to each text and the second video corresponding to each text includes: for any text among the texts, determining the sixth feature vector of the first video corresponding to any text and the seventh feature vector of the second video corresponding to any text; determining the loss value of any text based on the sixth feature vector and the seventh feature vector.

[0162] Optionally, the process of determining the loss value of any text according to the sixth eigenvector and the seventh eigenvector includes: determining a dot product result of the sixth eigenvector and the seventh eigenvector as the loss value of any text.

[0163] In some embodiments, the process of determining the third loss value according to the loss values ​​of the respective texts includes: taking an average of the loss values ​​of the respective texts as the third loss value.

[0164] In some embodiments, the process of training the video generation model according to the third loss value includes: determining an adjustment gradient corresponding to the third loss value; and adjusting the model parameters of the video generation model according to the adjustment gradient corresponding to the third loss value.

[0165] Optionally, the video generation model includes a text-adapter module and a cross-attention module. The text-adapter module is a module for processing text data, typically used in multimodal models or cross-modal tasks. Its core function is to convert text information into a format suitable for the target task or other modal processing, acting as a bridge. The cross-attention module is a variant of the attention mechanism, primarily used to process interactions between different modalities or sequences. Unlike the self-attention mechanism, which focuses on the internal associations of the same input sequence, the cross-attention module allows the model to refer to information from another sequence when processing a certain sequence.

[0166] In some embodiments, the process of adjusting the model parameters of the video generation model according to the adjustment gradient corresponding to the third loss value includes: adjusting the parameters of the text adapter module and the cross-attention module in the video generation model according to the adjustment gradient corresponding to the third loss value.

[0167] In some embodiments, the video generation model is iteratively trained according to the method of determining the third loss value as described above and training the video generation model according to the third loss value until the iteration stop condition is reached, and the video generation model when the iteration stop condition is reached is used as the trained video generation model.

[0168] In some embodiments, after obtaining the trained video generation model, the trained video generation model is called to process the text features of the target text to obtain a target video corresponding to the target text.

[0169] In this embodiment, since the text features of the target text obtained after processing the target text through the trained text feature generation model can better represent the target text, the target video obtained through the text features of the target text has a higher matching degree with the target text.

[0170] In some embodiments, the process of calling the trained video generation model to process the text features of the target text to obtain the target video corresponding to the target text includes: inputting the text features of the target text into the trained video generation model, and using the output content of the trained video generation model as the target video corresponding to the target text.

[0171] In some embodiments, after obtaining the trained text feature generation model and the trained video generation model, a Vincent video model is obtained based on the trained text feature generation model and the trained video generation model. The Vincent video model includes the trained text feature generation model and the trained video generation model. The Vincent video model is used to generate corresponding videos based on the text.

[0172] In some embodiments, after obtaining the Vincent video model, a video can be generated using the Vincent video model. The process of generating a video using the Vincent video model includes: obtaining a first text, the first text being used to generate a video; inputting the first text into the Vincent video model, and using the video output by the Vincent video model as the video corresponding to the first text.

[0173] The disclosed embodiment provides a method for training a text feature generation model, which performs masking on the training text to train the text feature generation model using the masked text and the training text, thereby strengthening the text feature generation model's reliance on text semantics and its ability to capture contextual relationships. The intermediate text feature generation model is then trained using the training text and its positive and negative texts, which can guide the intermediate text feature generation model to accurately distinguish semantic similarities and differences, enabling the intermediate text feature generation model to not only mine the semantic structure of the text, but also extract more discriminative semantic features. Ultimately, the trained text feature generation model can not only accurately represent the semantics of the text itself, but also extract more discriminative features from semantic association comparisons, thereby enhancing the depth of understanding and representation capabilities of the trained text feature generation model. When the trained text feature generation model is used to obtain text features of a text, the obtained text features have a higher degree of match and accuracy with the text, and are more capable of representing the text.

[0174] Moreover, applying the text feature generation model obtained by this method to the text-generated video model can effectively improve the consistency of the generated video with the text in terms of temporal and spatial information, style, etc.

[0175] In addition, when the text feature generation model obtained by this method is used to train the video generation model, the training speed of the video generation model can be accelerated.

[0176] Figure 6 1 is a block diagram of a training device for a text feature generation model according to an exemplary embodiment. Figure 6 , the device includes the following contents.

[0177] The acquisition module 601 is configured to execute acquisition of a training data set, where the training data set includes multiple groups of training data, each group of training data includes a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text. The positive text corresponding to the training text is a text that is semantically related to the training text, and the negative text corresponding to the training text is a text that is semantically irrelevant to the training text.

[0178] The processing module 602 is configured to perform masking processing on a plurality of training texts to obtain masked texts of respective training texts, wherein at least one word in the masked text of any training text is masked.

[0179] The training module 603 is configured to train the text feature generation model based on the mask text of each training text and each training text to obtain an intermediate text feature generation model.

[0180] The training module 603 is further configured to train the intermediate text feature generation model based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text.

[0181] In some embodiments, the training module 603 is configured to perform grouping of the masked texts of each training text to obtain at least two text groups, each text group including at least one masked text; determine the first loss value corresponding to each text group based on the masked texts included in each text group and the training texts corresponding to the masked texts included in each text group; and train the text feature generation model based on the first loss value corresponding to each text group to obtain an intermediate text feature generation model.

[0182] In some embodiments, the training module 603 is configured to execute, for any text group in each text group, calling the text feature generation model to process the masked text included in any text group separately, and obtain the predicted content of the masked text included in any text group; obtain the original content of the masked text included in any text group based on the masked text included in any text group and the training text corresponding to the masked text included in any text group; determine the first loss value corresponding to any text group based on the predicted content of the masked text included in any text group and the original content of the masked text included in any text group.

[0183] In some embodiments, the training module 603 is configured to determine the first feature vector of the predicted content of any masked text and the second feature vector of the original content of any masked text for any masked text included in any text group; determine the loss value of any masked text based on the first feature vector and the second feature vector, and the loss value of any masked text is the similarity between the predicted content of any masked text and the original content of any masked text; and determine the first loss value corresponding to any text group based on the loss value of the masked text included in any text group.

[0184] In some embodiments, the training module 603 is configured to determine a dot product result of the first feature vector and the second feature vector as a loss value of any mask text.

[0185] In some embodiments, the training module 603 is configured to determine an average of the loss values ​​of the masked texts included in any text group as a first loss value corresponding to any text group.

[0186] In some embodiments, the training module 603 is configured to determine the loss value corresponding to each training text based on each training text, the positive text corresponding to each training text, and the negative text corresponding to each training text; determine a second loss value based on the loss value corresponding to each training text; and train the intermediate text feature generation model based on the second loss value.

[0187] In some embodiments, the training module 603 is configured to execute, for any training text among the training texts, calling the intermediate text feature generation model to process any training text, the positive text corresponding to any training text, and the negative text corresponding to any training text, to obtain the third feature vector corresponding to any training text, the fourth feature vector of the positive text corresponding to any training text, and the fifth feature vector of the negative text corresponding to any training text; and determine the loss value corresponding to any training text based on the third feature vector, the fourth feature vector, and the fifth feature vector.

[0188] In some embodiments, the training module 603 is configured to determine a first similarity based on the third eigenvector and the fourth eigenvector, where the first similarity is used to characterize the similarity between any training text and the positive text corresponding to any training text; determine a second similarity based on the third eigenvector and the fifth eigenvector, where the second similarity is used to characterize the similarity between any training text and the negative text corresponding to any training text; and determine a loss value corresponding to any training text based on the first similarity and the second similarity.

[0189] In some embodiments, the acquisition module 601 is further configured to acquire a text video dataset, where the text video dataset includes multiple texts and a first video corresponding to each text.

[0190] The processing module 602 is further configured to execute calling the trained text feature generation model to process the multiple texts respectively to obtain text features of each text.

[0191] The processing module 602 is further configured to call the video generation model to process the text features of each text to obtain a second video corresponding to each text.

[0192] The training module 603 is further configured to train the video generation model based on the first video corresponding to each text and the second video corresponding to each text.

[0193] The disclosed embodiment provides a training device for a text feature generation model, which performs masking on the training text to train the text feature generation model using the masked text and the training text, thereby strengthening the text feature generation model's reliance on text semantics and its ability to capture contextual relationships. The intermediate text feature generation model is then trained using the training text and its positive and negative texts, which can guide the intermediate text feature generation model to accurately distinguish semantic similarities and differences, enabling the intermediate text feature generation model to not only mine the semantic structure of the text, but also extract more discriminative semantic features. Ultimately, the trained text feature generation model can not only accurately represent the semantics of the text itself, but also extract more discriminative features from semantic association comparisons, thereby enhancing the depth of understanding and representation capabilities of the trained text feature generation model. When the trained text feature generation model is used to obtain text features of a text, the obtained text features have a higher degree of match with the text, higher accuracy, and are more capable of representing the text.

[0194] Regarding the apparatus in the above embodiment, the specific manner in which each unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0195] Figure 7 FIG. 1 is a block diagram of a text feature generation device according to an exemplary embodiment. Figure 7 , the device includes the following contents.

[0196] The acquisition module 701 is configured to execute acquisition of a target text with text features to be generated.

[0197] The processing module 702 is configured to execute the call of the trained text feature generation model to process the target text to obtain the text features of the target text. The trained text feature generation model is trained by the training method of the text feature generation model described in the first aspect.

[0198] In some embodiments, the processing module 702 is further configured to call a trained video generation model to process text features of the target text to obtain a target video corresponding to the target text.

[0199] The disclosed embodiments provide a text feature generation device that processes a target text to obtain text features by invoking a trained text feature generation model. Leveraging the semantic representation capabilities learned by the model from large-scale data, the device maps the target text into a high-dimensional semantic space, enabling the text features to capture both surface information at the lexical level and deep structural features at the syntactic and semantic levels. The trained model possesses generalization capabilities, adapting to text input in diverse contexts. The generated text features possess strong semantic representation capabilities and a higher degree of match with the target text.

[0200] Regarding the apparatus in the above embodiment, the specific manner in which each unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0201] Figure 8 The following is a block diagram illustrating the structure of a terminal 800 provided by an exemplary embodiment of the present disclosure. Terminal 800 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Terminal 800 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0202] Typically, the terminal 800 includes a processor 801 and a memory 802 .

[0203] Processor 801 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 801 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 801 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content required for display. In some embodiments, processor 801 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0204] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one program code, which is used to be executed by the processor 801 to implement the training method of the text feature generation model provided in the method embodiment of the present disclosure, or to implement the text feature generation method provided in the method embodiment of the present disclosure.

[0205] In some embodiments, terminal 800 may optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 803 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, and a power supply 808.

[0206] The peripheral device interface 803 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0207] The RF circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 804 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 804 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 804 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 804 may also include circuitry related to Near Field Communication (NFC), although this disclosure does not limit this.

[0208] Display screen 805 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, or any combination thereof. If display screen 805 is a touchscreen display, it is also capable of detecting touch signals on or above the surface of display screen 805. These touch signals can be input as control signals to processor 801 for processing. Display screen 805 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be a single display screen 805, located on the front panel of terminal 800. In other embodiments, there can be at least two display screens 805, located on different surfaces of terminal 800 or in a foldable design. In still other embodiments, display screen 805 can be a flexible display, located on a curved or foldable surface of terminal 800. Display screen 805 can also be configured as a non-rectangular, irregular shape, also known as a special-shaped screen. Display screen 805 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0209] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0210] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 801 for processing, or input into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0211] Power supply 808 is used to power various components in terminal 800. Power supply 808 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 808 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0212] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0213] Figure 9The structural block diagram of a server 900 provided by an exemplary embodiment of the present disclosure is shown. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 901 and one or more memories 902, wherein the one or more memories 902 store at least one program code, and the at least one program code is loaded and executed by the one or more processors 901 to implement the training method of the text feature generation model provided by the above-mentioned various method embodiments, or to implement the text feature generation method provided by the above-mentioned various method embodiments. Of course, the server 900 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 900 may also include other components for implementing device functions, which will not be described in detail here.

[0214] In an exemplary embodiment, a computer-readable storage medium including instructions, such as a memory including instructions, is also provided. The instructions can be executed by a processor of a terminal to implement the above-described method for training a text feature generation model, or to implement the above-described method for generating text features. Alternatively, the computer-readable storage medium can be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, or the like.

[0215] In an exemplary embodiment, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the training method of the above-mentioned text feature generation model, or implements the above-mentioned text feature generation method. In some embodiments, the computer program product involved in the embodiments of the present disclosure can be deployed and executed on a terminal, or on multiple terminals located in a single location, or on multiple terminals distributed in multiple locations and interconnected by a communication network. Multiple terminals distributed in multiple locations and interconnected by a communication network can constitute a blockchain system.

[0216] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims. All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0217] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A training method for a text feature generation model, characterized in that: The method comprises: Obtaining a training data set, the training data set including multiple sets of training data, each set of training data including a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text, wherein the positive text corresponding to the training text is a text semantically related to the training text, and the negative text corresponding to the training text is a text semantically irrelevant to the training text; Performing masking processing on a plurality of training texts to obtain masked texts of each training text, wherein at least one word in the masked text of any training text is masked; Training a text feature generation model based on the mask texts of the respective training texts and the respective training texts to obtain an intermediate text feature generation model; Training the intermediate text feature generation model according to the respective training texts, the positive texts corresponding to the respective training texts, and the negative texts corresponding to the respective training texts; Acquire a text video dataset, where the text video dataset includes a plurality of texts and a first video corresponding to each text; Calling the trained text feature generation model to process the multiple texts respectively to obtain text features of each text; Calling a video generation model to process text features of each text to obtain a second video corresponding to each text; The video generation model is trained according to the first video corresponding to each text and the second video corresponding to each text.

2. The method according to claim 1, characterized in that The step of training a text feature generation model based on the mask texts of the respective training texts and the respective training texts to obtain an intermediate text feature generation model includes: Grouping the masked texts of the respective training texts to obtain at least two text groups, each text group including at least one masked text; Determining a first loss value corresponding to each text group according to the masked text included in each text group and the training text corresponding to the masked text included in each text group; The text feature generation model is trained according to the first loss values ​​corresponding to the respective text groups to obtain the intermediate text feature generation model.

3. The method according to claim 2, characterized in that The determining, based on the masked texts included in each text group and the training texts corresponding to the masked texts included in each text group, a first loss value corresponding to each text group includes: For any one of the text groups, calling the text feature generation model to process the masked text included in the any one of the text groups, and obtaining predicted content of the masked text included in the any one of the text groups; Obtaining original content of the masked text included in any one of the text groups according to the masked text included in any one of the text groups and the training text corresponding to the masked text included in any one of the text groups; A first loss value corresponding to any one of the text groups is determined according to the predicted content of the masked text included in any one of the text groups and the original content of the masked text included in any one of the text groups.

4. The method according to claim 3, characterized in that The determining, based on the predicted content of the masked text included in any one of the text groups and the original content of the masked text included in any one of the text groups, a first loss value corresponding to the any one of the text groups, includes: For any masked text in the masked texts included in any one of the text groups, determining a first feature vector of predicted content of the any masked text, and determining a second feature vector of original content of the any masked text; Determining a loss value of any masked text according to the first feature vector and the second feature vector, where the loss value of any masked text is a similarity between a predicted content of any masked text and an original content of any masked text; A first loss value corresponding to any one of the text groups is determined according to the loss value of the mask text included in any one of the text groups.

5. The method according to claim 4, characterized in that The determining, based on the first feature vector and the second feature vector, a loss value of any masked text includes: Determine a dot product result of the first feature vector and the second feature vector as a loss value of any masked text.

6. The method according to claim 4, characterized in that The determining, based on the loss value of the masked text included in any one of the text groups, a first loss value corresponding to the any one of the text groups, includes: An average of the loss values ​​of the masked texts included in any one of the text groups is determined as a first loss value corresponding to the any one of the text groups.

7. The method according to any one of claims 1 to 6, characterized in that: The training of the intermediate text feature generation model according to the training texts, the positive texts corresponding to the training texts, and the negative texts corresponding to the training texts includes: Determining a loss value corresponding to each training text according to each training text, a positive text corresponding to each training text, and a negative text corresponding to each training text; Determining a second loss value according to the loss values ​​corresponding to the respective training texts; The intermediate text feature generation model is trained according to the second loss value.

8. The method according to claim 7, characterized in that The determining, based on the respective training texts, the positive texts corresponding to the respective training texts, and the negative texts corresponding to the respective training texts, a loss value corresponding to the respective training texts includes: For any training text among the training texts, calling the intermediate text feature generation model to process the any training text, the positive text corresponding to the any training text, and the negative text corresponding to the any training text, to obtain a third feature vector corresponding to the any training text, a fourth feature vector of the positive text corresponding to the any training text, and a fifth feature vector of the negative text corresponding to the any training text; A loss value corresponding to any one of the training texts is determined according to the third eigenvector, the fourth eigenvector, and the fifth eigenvector.

9. The method according to claim 8, characterized in that The determining, based on the third eigenvector, the fourth eigenvector, and the fifth eigenvector, a loss value corresponding to any one of the training texts includes: determining a first similarity based on the third eigenvector and the fourth eigenvector, wherein the first similarity is used to represent a similarity between the any training text and a normal text corresponding to the any training text; determining a second similarity based on the third eigenvector and the fifth eigenvector, wherein the second similarity is used to represent a similarity between the any training text and a negative text corresponding to the any training text; Determine a loss value corresponding to any one of the training texts according to the first similarity and the second similarity.

10. A method for generating text features, characterized in that: The method comprises: Obtain the target text to be generated text features; Calling a trained text feature generation model to process the target text to obtain text features of the target text, wherein the trained text feature generation model is trained by the training method of the text feature generation model according to any one of claims 1 to 9; The trained video generation model is called to process the text features of the target text to obtain the target video corresponding to the target text. The trained video generation model is obtained by training the first video corresponding to each text and the second video corresponding to each text. The second video corresponding to each text is obtained by processing the text features corresponding to each text through the video generation model. The text features corresponding to each text are obtained through the trained text feature generation model.

11. A training device for a text feature generation model, characterized in that: The device comprises: an acquisition module configured to acquire a training data set, the training data set comprising a plurality of sets of training data, each set of training data comprising a training text, a positive text corresponding to the training text, and a negative text corresponding to the training text, the positive text corresponding to the training text being a text semantically related to the training text, and the negative text corresponding to the training text being a text semantically irrelevant to the training text; a processing module configured to perform masking processing on a plurality of training texts to obtain masked texts of respective training texts, wherein at least one word in the masked text of any training text is masked; A training module is configured to train a text feature generation model based on the mask text of each training text and each training text to obtain an intermediate text feature generation model; The training module is further configured to train the intermediate text feature generation model based on the training texts, the positive texts corresponding to the training texts, and the negative texts corresponding to the training texts; The acquisition module is further configured to execute acquisition of a text video dataset, wherein the text video dataset includes a plurality of texts and a first video corresponding to each text; The processing module is further configured to execute calling the trained text feature generation model to process the multiple texts respectively to obtain text features of each text; The processing module is further configured to execute calling a video generation model to process text features of each text to obtain a second video corresponding to each text; The training module is further configured to train the video generation model based on the first video corresponding to each text and the second video corresponding to each text.

12. A text feature generation device, characterized in that: The device comprises: An acquisition module is configured to execute the acquisition of a target text of a text feature to be generated; a processing module configured to execute calling a trained text feature generation model to process the target text to obtain text features of the target text, wherein the trained text feature generation model is trained by the training method of the text feature generation model according to any one of claims 1 to 9; The processing module is further configured to execute a call to the trained video generation model to process the text features of the target text to obtain a target video corresponding to the target text. The trained video generation model is obtained by training the first video corresponding to each text and the second video corresponding to each text. The second video corresponding to each text is obtained by processing the text features corresponding to each text through the video generation model. The text features corresponding to each text are obtained through the trained text feature generation model.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the training method of the text feature generation model according to any one of claims 1 to 9, or to implement the text feature generation method according to claim 10.

14. A computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the training method of the text feature generation model described in any one of claims 1 to 9, or to implement the text feature generation method described in claim 10.

15. A computer program product, characterized in that The computer program product includes a computer program, which, when executed by a processor, implements the training method for the text feature generation model described in any one of claims 1 to 9, or implements the text feature generation method described in claim 10.

Images