Text expansion model training method and text expansion method

Abstract

The present disclosure relates to a text expansion model training method and a text expansion method. The training method includes expanding a first sample text based on an initial model to obtain a first expanded text; adjusting the parameters of the initial model based on the first sample text and the first expanded text; expanding a second sample text based on the adjusted initial model to obtain a second expanded text; adjusting the parameters of the adjusted initial model based on the second sample text and the second expanded text until an intermediate model is obtained; for each target sample text, expanding the target sample text based on the intermediate model to obtain multiple target expanded texts; inputting each target expanded text into a video generation model to obtain a video generation result corresponding to the target sample text; and training the intermediate model based on video verification based on each target sample text and the corresponding video generation result until a text expansion model is obtained. The present disclosure can improve the quality of the text expansion model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a text expansion model training method and a text expansion method. Background Art

[0002] The video generation model in the related art can generate videos based on text. The quality of the text has a significant impact on the quality of the generated video. For example, if the text is short and the content it describes is simple and empty, the generated video may lack details and richness, appear monotonous and low-quality. On the contrary, if the text is long and the content it describes is detailed and vivid and has clear scenes and plot settings, the generated video is more likely to present the expected picture and effect and have higher quality. However, since the text provided by the user is usually relatively short, in order to improve the video quality, the text provided by the user can be expanded first to obtain the expanded text, and then the video can be generated based on the expanded text.

[0003] However, the expansion model relied on by related technologies has problems such as the training process being out of sync with the video generation model and the low semantic quality of the expansion results. As a result, the expanded text cannot well meet the requirements of the video generation model, thus affecting the quality of the final generated video. Summary of the Invention

[0004] The present disclosure provides a text expansion model training method and a text expansion method to solve the problems in the related art. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a text expansion model training method is provided, the method comprising:

[0006] Get the first sample text;

[0007] performing expansion processing on the first sample text based on an initial model to obtain a first expanded text, wherein the initial model is a large-scale generative language model;

[0008] Adjusting parameters of the initial model based on the first sample text and the first expanded text;

[0009] Get the second sample text;

[0010] Expanding the second sample text based on the adjusted initial model to obtain a second expanded text;

[0011] Adjusting the parameters of the adjusted initial model based on the second sample text and the second expanded text until a first preset condition is satisfied, thereby obtaining an intermediate model;

[0012] Obtain at least one target sample text;

[0013] For each target sample text, the target sample text is expanded based on the intermediate model to obtain a plurality of target expanded texts; and a video generation result corresponding to the target sample text is obtained by inputting each target expanded text into a video generation model;

[0014] Based on each of the target sample texts and the corresponding video generation results, the intermediate model is trained based on video verification until the second preset condition is met, thereby obtaining a text expansion model.

[0015] In an exemplary embodiment, the expanding the first sample text based on the initial model to obtain the first expanded text includes:

[0016] Expanding the first sample text based on the initial model and outputting a plurality of candidate expanded texts;

[0017] Performing accuracy analysis on each candidate expanded text to obtain a corresponding accuracy parameter, wherein the accuracy parameter is used to indicate the degree of consistency between the candidate expanded text and the first sample text;

[0018] Performing consistency analysis on each candidate expansion text to obtain a corresponding consistency parameter, wherein the consistency parameter is used to indicate the degree of semantic consistency of the content of the candidate expansion text itself;

[0019] Obtaining a corresponding expansion quality score based on the accuracy parameter and the consistency parameter corresponding to each candidate expansion text;

[0020] Based on the expansion quality scores corresponding to the candidate expansion texts, the first expanded text is determined from the candidate expansion texts.

[0021] In an exemplary embodiment, adjusting the parameters of the initial model based on the first sample text and the first expanded text includes:

[0022] Inputting the first sample text into the initial model to obtain a predicted expanded text;

[0023] Based on the difference between the predicted expanded text and the first expanded text, the parameters of the initial model are adjusted.

[0024] In an exemplary embodiment, the step of inputting each target expanded text into a video generation model to obtain a video generation result corresponding to the target sample text includes:

[0025] Inputting each of the target expanded texts into the video generation model to generate corresponding candidate videos;

[0026] Performing a video quality assessment on each candidate video to obtain a corresponding video quality score, wherein the video quality score includes a semantic alignment parameter, and the semantic alignment parameter is used to indicate the degree of semantic alignment between the candidate video and the corresponding target sample text;

[0027] The candidate video with the highest video quality score is used as the first target video, and the candidate video with the lowest video quality score is used as the second target video. The video generation result includes the first target video and the second target video.

[0028] In an exemplary embodiment, the training of the intermediate model based on video verification based on each target sample text and the corresponding video generation result includes:

[0029] For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0030] Based on the positive and negative sample pairs corresponding to each target sample text, the intermediate model is directly optimized in terms of preference.

[0031] In an exemplary embodiment, the training of the intermediate model based on video verification based on each target sample text and the corresponding video generation result includes:

[0032] For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0033] Based on the positive and negative sample pairs corresponding to each target sample text, the corresponding first target video and the corresponding second target video, comparative learning is performed on the intermediate model.

[0034] In an exemplary embodiment, performing video quality assessment on each candidate video to obtain a corresponding video quality score includes:

[0035] Performing motion rationality detection on the candidate video to obtain motion rationality parameters;

[0036] Performing a picture consistency test on the candidate video to obtain a picture consistency parameter;

[0037] Calculating semantic similarity between the candidate video and the corresponding target sample text to obtain the semantic alignment parameter;

[0038] The motion rationality parameter, the picture consistency parameter, and the semantic alignment parameter are integrated to obtain the video quality score.

[0039] In an exemplary embodiment, the method further comprises:

[0040] Acquire a sample text set, the sample text set including the first sample text and the second sample text;

[0041] The obtaining of at least one target sample text includes:

[0042] Performing a text quality assessment on each sample text in the sample text set to obtain a corresponding text quality score;

[0043] The sample text whose text quality score meets the preset text quality requirement is used as the target sample text.

[0044] In an exemplary embodiment, obtaining a corresponding expansion quality score based on the accuracy parameter and the consistency parameter corresponding to each candidate expansion text includes:

[0045] For each candidate expanded text, input the candidate expanded text into the video generation model to obtain a corresponding plurality of predicted videos; and determine a corresponding alignment quality score based on an expected semantic similarity between each predicted video and the candidate expanded text;

[0046] The accuracy parameter, the consistency parameter and the alignment quality score corresponding to each candidate expanded text are integrated to obtain a corresponding expanded quality score.

[0047] According to a second aspect of an embodiment of the present disclosure, a text expansion method is provided, the method comprising:

[0048] Get the target text;

[0049] Inputting the target text into a text expansion model to obtain a text expansion result;

[0050] The text expansion model is trained using the text expansion model training method described in any one of the first aspects.

[0051] According to a third aspect of an embodiment of the present disclosure, a text expansion model training device is provided, the device comprising:

[0052] The first training module is configured to perform the following operations:

[0053] Get the first sample text;

[0054] performing expansion processing on the first sample text based on an initial model to obtain a first expanded text, wherein the initial model is a large-scale generative language model;

[0055] Adjusting parameters of the initial model based on the first sample text and the first expanded text;

[0056] Get the second sample text;

[0057] Expanding the second sample text based on the adjusted initial model to obtain a second expanded text;

[0058] Adjusting the parameters of the adjusted initial model based on the second sample text and the second expanded text until a first preset condition is satisfied, thereby obtaining an intermediate model;

[0059] The second training module is configured to perform the following operations:

[0060] Obtain at least one target sample text;

[0061] For each target sample text, the target sample text is expanded based on the intermediate model to obtain a plurality of target expanded texts; and a video generation result corresponding to the target sample text is obtained by inputting each target expanded text into a video generation model;

[0062] Based on each of the target sample texts and the corresponding video generation results, the intermediate model is trained based on video verification until the second preset condition is met, thereby obtaining a text expansion model.

[0063] In an exemplary embodiment, the first training module is configured to perform the following operations:

[0064] Expanding the first sample text based on the initial model and outputting a plurality of candidate expanded texts;

[0065] Performing accuracy analysis on each candidate expanded text to obtain a corresponding accuracy parameter, wherein the accuracy parameter is used to indicate the degree of consistency between the candidate expanded text and the first sample text;

[0066] Performing consistency analysis on each candidate expansion text to obtain a corresponding consistency parameter, wherein the consistency parameter is used to indicate the degree of semantic consistency of the content of the candidate expansion text itself;

[0067] Obtaining a corresponding expansion quality score based on the accuracy parameter and the consistency parameter corresponding to each candidate expansion text;

[0068] Based on the expansion quality scores corresponding to the candidate expansion texts, the first expanded text is determined from the candidate expansion texts.

[0069] In an exemplary embodiment, the first training module is configured to perform the following operations:

[0070] Inputting the first sample text into the initial model to obtain a predicted expanded text;

[0071] Based on the difference between the predicted expanded text and the first expanded text, the parameters of the initial model are adjusted.

[0072] In an exemplary embodiment, the second training module is configured to perform the following operations:

[0073] Inputting each of the target expanded texts into the video generation model to generate corresponding candidate videos;

[0074] Performing a video quality assessment on each candidate video to obtain a corresponding video quality score, wherein the video quality score includes a semantic alignment parameter, and the semantic alignment parameter is used to indicate the degree of semantic alignment between the candidate video and the corresponding target sample text;

[0075] The candidate video with the highest video quality score is used as the first target video, and the candidate video with the lowest video quality score is used as the second target video. The video generation result includes the first target video and the second target video.

[0076] In an exemplary embodiment, the second training module is configured to perform the following operations:

[0077] For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0078] Based on the positive and negative sample pairs corresponding to each target sample text, the intermediate model is directly optimized in terms of preference.

[0079] In an exemplary embodiment, the second training module is configured to perform the following operations:

[0080] For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0081] Based on the positive and negative sample pairs corresponding to each target sample text, the corresponding first target video and the corresponding second target video, comparative learning is performed on the intermediate model.

[0082] In an exemplary embodiment, the second training module is configured to perform the following operations:

[0083] Performing motion rationality detection on the candidate video to obtain motion rationality parameters;

[0084] Performing a picture consistency test on the candidate video to obtain a picture consistency parameter;

[0085] Calculating semantic similarity between the candidate video and the corresponding target sample text to obtain the semantic alignment parameter;

[0086] The motion rationality parameter, the picture consistency parameter, and the semantic alignment parameter are integrated to obtain the video quality score.

[0087] In an exemplary embodiment, the first training module is configured to perform the following operations:

[0088] Acquire a sample text set, the sample text set including the first sample text and the second sample text;

[0089] The second training module is configured to perform the following operations:

[0090] Performing a text quality assessment on each sample text in the sample text set to obtain a corresponding text quality score;

[0091] The sample text whose text quality score meets the preset text quality requirement is used as the target sample text.

[0092] In an exemplary embodiment, the first training module is configured to perform the following operations:

[0093] For each candidate expanded text, input the candidate expanded text into the video generation model to obtain a corresponding plurality of predicted videos; and determine a corresponding alignment quality score based on an expected semantic similarity between each predicted video and the candidate expanded text;

[0094] The accuracy parameter, the consistency parameter and the alignment quality score corresponding to each candidate expanded text are integrated to obtain a corresponding expanded quality score.

[0095] According to a fourth aspect of an embodiment of the present disclosure, a text expansion device is provided, the device comprising:

[0096] A target text acquisition module is configured to execute acquisition of the target text;

[0097] an expansion module configured to input the target text into a text expansion model to obtain a text expansion result;

[0098] The text expansion model is trained using the text expansion model training method described in any one of the first aspects.

[0099] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0100] processor;

[0101] a memory for storing instructions executable by the processor;

[0102] The processor is configured to execute the instructions to implement the text expansion model training method or text expansion method as described in any of the above embodiments.

[0103] According to a sixth aspect of an embodiment of the present disclosure, a computer storage medium is provided. When the instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device executes the text expansion model training method or the text expansion method described in any of the above embodiments.

[0104] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, it implements the text expansion model training method or text expansion method described in any of the above embodiments.

[0105] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0106] The text expansion model training method provided by the embodiment of the present disclosure trains a text expansion model through two training stages. In the first training stage, the initial model is trained based on the first sample text to obtain the initial model after parameter adjustment. The initial model after parameter adjustment is trained based on the second sample text. This is essentially an iterative training method. This training method enables the model to use the data generated by itself during the training and iteration process to retrain and optimize itself, continuously perform self-distillation, and gradually refine and sublimate the learned knowledge, improve learning efficiency, and continuously break through its own limitations to achieve more powerful learning effects. Therefore, the expansion quality of the intermediate model trained in the first training stage is significantly improved compared with related technologies.

[0107] In order to make the expanded text expanded by the expansion model trained in this application make greater contributions to improving the videos generated by the video generation model, the second training stage optimizes the intermediate model from the perspective of video verification, so that the expanded text expanded by the model can be used by the video generation model to generate high-quality videos. During this training process, the model will be more focused on improving its expansion capabilities in the field of video generation. For example, the expanded text can contain more details that improve the expressiveness of the video, thereby obtaining a text expansion model.

[0108] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] The accompanying drawings herein are incorporated into the specification and constitute a part of the present disclosure, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation to the present disclosure.

[0110] Figure 1 is a flowchart of a text expansion model training method according to an exemplary embodiment;

[0111] Figure 2 is a schematic diagram of a first expanded text screening method according to an exemplary embodiment;

[0112] Figure 3 is a schematic diagram of a method for obtaining a video generation result according to an exemplary embodiment;

[0113] Figure 4 is a schematic diagram of a method for obtaining a video quality score according to an exemplary embodiment;

[0114] Figure 5 A video verification-based training method according to an exemplary embodiment is shown. Figure 1 ;

[0115] Figure 6 A video verification-based training method according to an exemplary embodiment is shown. Figure 2 ;

[0116] Figure 7 is a flow chart of a text expansion method according to an exemplary embodiment;

[0117] Figure 8 is a block diagram of a text expansion model training device according to an exemplary embodiment;

[0118] Figure 9 is a block diagram of a text expansion device according to an exemplary embodiment;

[0119] Figure 10 A structural block diagram of a computer device according to an exemplary embodiment is shown. Figure 1 ;

[0120] Figure 11 A structural block diagram of a computer device according to an exemplary embodiment is shown. Figure 2 . DETAILED DESCRIPTION

[0121] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0122] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0123] Figure 1 It is a flowchart of a text expansion model training method according to an exemplary embodiment. The text expansion model training method can be applied to electronic devices, which can be implemented by a server or a terminal alone, or can be implemented by a terminal and a server in collaboration. The terminal can be, but is not limited to, a physical device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart wearable device, a digital assistant, an augmented reality device, a virtual reality device, etc., and can also include software such as an application running in the physical device. The server can be, but is not limited to, an independent server, or a server cluster or distributed system composed of multiple physical servers, or one or more cloud servers that provide cloud services, cloud databases, cloud computing, cloud, cloud storage, network services, cloud communications, intermediate services, domain name services, security services, and basic cloud computing services such as big data and artificial intelligence platforms. Reference Figure 1 As shown, the method includes the following steps.

[0124] In step S110 , a first sample text is obtained.

[0125] The present disclosure can obtain a sample text set, wherein the sample text set includes the first sample text and the second sample text described below. The first sample text and the second sample text can be any sample text in the sample text set. The present disclosure does not limit the length of the sample text; for example, the sample text can be less than a first preset length. The present disclosure does not limit the first preset length; for example, the sample text can be 50 characters.

[0126] The present disclosure does not limit the content of the sample text, which can be any content that can be visually presented. For example, it can be a news report, a poem, an essay, a story, etc. Different types of sample texts can provide rich and diverse language materials for model training to enhance the model's understanding and expansion capabilities of various contents. For example, using news reports as sample texts can allow the model to learn formal, rigorous and timely expressions; using poetry as sample texts can expose the model to a language style that is rhythmic, emotional and imaginative. Sample texts can also come from instructions input by users to the video generation model in order to generate videos. Such instructions can include the user's specific requirements for video content, style, theme, etc., and may have a clearer goal orientation, which can be used as sample text.

[0127] In step S120 , the first sample text is expanded based on an initial model to obtain a first expanded text, wherein the initial model is a large-scale generative language model.

[0128] The Large Language Model (LLM) mentioned in this disclosure refers to a generative artificial intelligence model based on deep learning that can understand and generate natural language text. LLM learns the structure and patterns of language by training on large amounts of text data, thereby being able to perform various language tasks such as translation, question answering, and text generation. Represents the initial model. The present disclosure can select any LLM in the related art as the initial model , for example, the initial model This model is one of the LLaMA-7B models developed by Meta, which has 7 billion parameters. LLaMA-7B has demonstrated excellent performance in natural language processing tasks and can understand and process a variety of texts.

[0129] The present disclosure may also be configured to input the initial model The prompt instruction is used to provide the initial model Issue specific instructions to trigger the initial model Perform specific tasks. For this disclosure, the prompt instruction is used to guide the initial model The input text is expanded, and the number of expanded texts to be output can also be specified. For example, the prompt instruction can be as follows: "Please expand the input text and output 5 expanded texts."

[0130] By setting the prompt instruction and inputting the first sample text and the prompt instruction into the initial model , an expansion result that meets the prompt instruction can be obtained, and the expansion result can include the first expanded text. The first expanded text can be an expanded text with better expansion quality in the expansion result, that is, the first expanded text is obtained by screening the expanded texts in the expansion result.

[0131] In step S130 , the parameters of the initial model are adjusted based on the first sample text and the first expanded text.

[0132] The present disclosure does not limit the method for adjusting the parameters of the initial model based on the first sample text and the first expanded text. In an exemplary embodiment, adjusting the parameters of the initial model based on the first sample text and the first expanded text includes: inputting the first sample text into the initial model to obtain a predicted expanded text; and adjusting the parameters of the initial model based on the difference between the predicted expanded text and the first expanded text.

[0133] For this initial model For example, even if the same sample text is input, the expansion results obtained each time can be different. This is due to the initial model The first sample text is input into the initial model again. , and obtain the predicted expanded text. Since the predicted expanded text is different from the first expanded text, the parameters of the initial model can be adjusted based on the difference, and the predicted expanded text is obtained. The advantage of this design method is that it can fully utilize the randomness of the model, allowing the model to learn richer expansion patterns through multiple attempts.

[0134] The present disclosure does not limit the difference quantification method. For example, the text prediction loss can be used to quantify the difference between the predicted expanded text and the first expanded text. The calculation process of the text prediction loss can be performed by treating each word or character in the predicted expanded text and the first expanded text as a classification label and calculating the probability distribution difference between them. There is no limitation on this specific process and it does not constitute an implementation obstacle. The present disclosure also does not limit the parameter adjustment method. For example, the gradient descent method can be used for parameter adjustment. The gradient descent algorithm will update the model parameters along the negative gradient direction of the loss to obtain the adjusted model parameters. .

[0135] In step S140 , a second sample text is obtained.

[0136] The second sample text is any other sample text in the aforementioned sample text set that is different from the first sample text.

[0137] In step S150, the second sample text is expanded based on the adjusted initial model to obtain a second expanded text; the parameters of the adjusted initial model are adjusted based on the second sample text and the second expanded text until the first preset condition is met to obtain an intermediate model.

[0138] In the present disclosure, the execution process of "expanding the second sample text based on the adjusted initial model to obtain the second expanded text; adjusting the parameters of the adjusted initial model based on the second sample text and the second expanded text" is consistent with the execution process of "expanding the first sample text based on the initial model to obtain the first expanded text, the initial model being a large generative language model; adjusting the parameters of the initial model based on the first sample text and the first expanded text". Train to model Then, in steps S140-S150, based on the second sample text, the model Train to model Obviously, this process can be iterated. For example, after getting the model Afterwards, if the first preset condition is still not met, the third sample text can be obtained from the sample text set, and the model The third sample text is expanded to obtain a third expanded text; the parameters of the adjusted initial model are adjusted based on the third sample text and the third expanded text to obtain a model , of course, the third sample text is a sample that has not been used in the previous two iterations. For each round i, the i-th sample text used is a sample that has not been used in the previous (i-1) iterations, where i is an integer greater than 1. This process is iterated until the first preset condition is met, and the model is obtained. , m is the number of iterative parameter adjustment rounds. This disclosure does not limit the first preset condition. For example, the number of iteration rounds can be used as a standard. When the number of iteration rounds reaches a set upper limit, the first preset condition is considered to have been met. This set upper limit can be set as needed and does not constitute an implementation obstacle.

[0139] In step S160 , at least one target sample text is obtained.

[0140] This disclosure does not limit the target sample text, which may be the same as or different from any of the aforementioned sample texts. In an exemplary embodiment, obtaining at least one target sample text includes: performing a text quality assessment on each sample text in the sample text set to obtain a corresponding text quality score; and selecting the sample text whose text quality score meets the preset text quality requirements as the target sample text.

[0141] This disclosure does not limit the content of text quality assessment. For example, text quality can be assessed from the following aspects:

[0142] (1) Contradictions: Evaluate whether there are logical contradictions or inconsistent statements in the text. If there are self-contradictory content, the text quality score will be reduced. For example, in a text about a product introduction, it is mentioned at the beginning that the product has good waterproof performance, but later it says that the product cannot be exposed to water. This is an obvious contradiction.

[0143] (2) Whether the narrative description is rich: Evaluate whether the text has sufficient details and richness in terms of narrative. If the text simply lists events or information, lacking vivid descriptions and specific plots, the narrative will appear thin, thus affecting the text quality score. For example, when describing a travel experience, simply saying "I went to a certain place and had a great time" is relatively empty and the quality score will be lowered.

[0144] (3) Whether there is camera description: Evaluate whether the text contains dynamic descriptions similar to those in film and television. Camera descriptions can make the text more visual and rhythmic, and enhance the text's appeal and readability. If the text can describe scenes or events with descriptions such as camera movement, focus, and switching, it will make the text more vivid and vivid.

[0145] By evaluating the above (1)-(3), a text quality score can be obtained. The present disclosure does not limit the method for determining the text quality score. For example, target prompt information can be generated based on (1)-(3). The target prompt information is used to instruct the LLM to comprehensively score the content quality of the input text based on (1)-(3). The target prompt information and the sample text are input into the LLM together to obtain the text quality score corresponding to the sample text output by the LLM.

[0146] The present disclosure does not limit the preset text quality requirements. For example, a sample text with a text quality score higher than a certain set threshold can be used as a target sample text that meets the requirements. The present disclosure does not limit the size of the set threshold and does not constitute an implementation obstacle. By evaluating the above (1)-(3), a text quality score can be obtained, and at least one target sample can be screened out based on the text quality score. High-quality target sample texts can be screened out for the next stage of training. These high-quality target sample texts have high quality in terms of content logic, narrative richness, and camera description, which is conducive to enabling the text expansion model trained in the next stage to have better effects in the field of video generation, that is, the text expanded by the text expansion model can better trigger the video generation model to generate videos with rich details, correct logic, and beautiful pictures.

[0147] In step S170, for each target sample text, the target sample text is expanded based on the intermediate model to obtain multiple target expanded texts; and by inputting each target expanded text into a video generation model, a video generation result corresponding to the target sample text is obtained.

[0148] In step S180, based on each of the target sample texts and the corresponding video generation results, the intermediate model is trained based on video verification until a second preset condition is met, thereby obtaining a text expansion model.

[0149] This disclosure does not limit the second preset condition, which can be flexibly set according to actual needs. For example, when the number of parameter adjustment rounds reaches a set value, it can be determined that the second preset condition is met, thereby obtaining a text expansion model. This disclosure does not limit the set value and does not constitute an implementation obstacle.

[0150] The text expansion model training method provided by the embodiment of the present disclosure trains a text expansion model through two training stages. In the first training stage, the initial model is trained based on the first sample text to obtain the initial model after parameter adjustment. The initial model after parameter adjustment is trained based on the second sample text. This is essentially an iterative training method. This training method enables the model to use the data generated by itself during the training and iteration process to retrain and optimize itself, continuously perform self-distillation, and gradually refine and sublimate the learned knowledge, improve learning efficiency, and continuously break through its own limitations to achieve more powerful learning effects. Therefore, the expansion quality of the intermediate model trained in the first training stage is significantly improved compared with related technologies.

[0151] In order to make the expanded text expanded by the expansion model trained in this application make greater contributions to improving the videos generated by the video generation model, the second training stage optimizes the intermediate model from the perspective of video verification, so that the expanded text expanded by the model can be used by the video generation model to generate high-quality videos. During this training process, the model will be more focused on improving its expansion capabilities in the field of video generation. For example, the expanded text can contain more details that improve the expressiveness of the video, thereby obtaining a text expansion model.

[0152] In an exemplary embodiment, please refer to Figure 2 , which shows a schematic diagram of the first expanded text screening method in the present disclosure. The method of expanding the first sample text based on the initial model to obtain the first expanded text includes:

[0153] S210. Expand the first sample text based on the initial model and output a plurality of candidate expanded texts.

[0154] Continuing with the previous example, the initial model Five candidate expansion texts may be output for the first sample text.

[0155] S220. Perform accuracy analysis on each candidate expanded text to obtain a corresponding accuracy parameter, where the accuracy parameter is used to indicate the degree of consistency between the candidate expanded text and the first sample text.

[0156] The accuracy parameter is used to indicate the degree of consistency between the candidate expanded text and the content of the first sample text. For example, if the first sample text describes "a white cat sleeping on the sofa in the living room", a candidate expanded text is "a snow-white cat is quietly sleeping on the soft sofa in the living room". This candidate expanded text describes the color and sleeping state of the cat in more detail, but the core content is consistent with the first sample text, and its accuracy parameter is higher; if the candidate expanded text is "a black dog running in the park", it is very different from the core content of the first sample text, and its accuracy parameter is low. The present disclosure does not limit the accuracy quantification method. For example, LLM can be used to directly score each candidate expanded text based on accuracy to obtain the corresponding accuracy parameter.

[0157] S230. Perform consistency analysis on each candidate expansion text to obtain a corresponding consistency parameter, where the consistency parameter is used to indicate the degree of semantic consistency of the content of the candidate expansion text itself.

[0158] The consistency parameter is used to indicate the degree of semantic consistency of the content of the candidate expanded text itself. For example, if the candidate expanded text is "Today is a sunny day, but at the same time it is overcast and raining heavily", the weather conditions described before and after are contradictory, its semantic consistency is poor, and the consistency parameter is low; while if the candidate expanded text is "Today is a sunny day, the sun shines warmly on the earth, and the birds sing happily on the branches", the content is semantically coherent and consistent, and its consistency parameter is high. The present disclosure also does not limit the consistency quantification method. For example, LLM can be used to directly score each candidate expanded text based on consistency to obtain the corresponding consistency parameter.

[0159] S240. Based on the accuracy parameter and the consistency parameter corresponding to each candidate expanded text, obtain a corresponding expansion quality score.

[0160] The present disclosure does not limit the method for obtaining the corresponding expansion quality score based on the accuracy parameter and the corresponding consistency parameter corresponding to each candidate expansion text. For example, a weighted method can be used, and the weight value is not limited.

[0161] S250. Determine the first expanded text from the candidate expanded texts based on the expansion quality scores corresponding to the candidate expanded texts.

[0162] The present disclosure does not limit the method of determining the first expanded text in each of the candidate expanded texts based on the expansion quality score corresponding to each of the candidate expanded texts. Exemplarily, the candidate expanded texts can be sorted from high to low according to the expansion quality score, and the candidate expanded text ranked first is selected as the first expanded text. Based on steps S210-S250, the first expanded text that meets the requirements can be screened out. Specifically, by fully considering its performance in accuracy and consistency when screening the first expanded text, the high quality of the first expanded text in semantic expression and logical structure can be effectively guaranteed.

[0163] In an exemplary embodiment, for each candidate expanded text, the candidate expanded text can be input into the video generation model to obtain a plurality of corresponding predicted videos; based on the expected semantic similarity between each predicted video and the candidate expanded text, the corresponding alignment quality score is determined; and the accuracy parameter, the corresponding consistency parameter, and the corresponding alignment quality score corresponding to each candidate expanded text are fused to obtain the corresponding expansion quality score. The present disclosure does not limit the method of fusing the accuracy parameter, the corresponding consistency parameter, and the corresponding alignment quality score corresponding to each candidate expanded text to obtain the corresponding expansion quality score. For example, a weighted method can be used, and the weight value is not limited.

[0164] The present disclosure does not limit the calculation method of semantic similarity, and the CLIP model of related technology can be used for calculation. The CLIP model refers to a multimodal model that can map images and texts into the same feature space, thereby realizing the semantic association between images and texts. Through the CLIP model, the similarity between the predicted video and the candidate expanded text in the feature space can be easily calculated, and then the semantic similarity can be obtained. The present disclosure obtains the corresponding expansion quality score by fusing the accuracy parameter, the corresponding consistency parameter and the corresponding alignment quality score corresponding to each of the candidate expanded texts, and can more comprehensively and objectively evaluate the quality of the candidate expanded text. The expansion quality score comprehensively considers accuracy, consistency and the semantic expression ability in the scene of generating the video, so that texts with higher quality and more in line with the requirements of the video generation scene can be screened out from among the many candidate expanded texts, thereby having a better targeted effect in the video generation scene.

[0165] In an exemplary embodiment, please refer to Figure 3 , which shows a schematic diagram of the method for obtaining the video generation result in the present disclosure. The method of obtaining the video generation result corresponding to the target sample text by inputting each target expanded text into the video generation model includes:

[0166] S310. Input each of the target expanded texts into the video generation model to generate corresponding candidate videos.

[0167] This disclosure does not limit the video generation model. You can choose from related technologies. For example, the DiT model can be used as the video generation model. DiT (Diffusion Transformer) is a new diffusion model that combines the advantages of the diffusion model and the Transformer architecture. DiT can generate high-quality, realistic video content by simulating the diffusion process from noise to data. Its design is inspired by Vision Transformers (ViTs) and replaces and optimizes the U-Net backbone network commonly used in traditional diffusion models.

[0168] S320. Perform video quality evaluation on each candidate video to obtain a corresponding video quality score, where the video quality score includes a semantic alignment parameter, and the semantic alignment parameter is used to indicate the degree of semantic alignment between the candidate video and the corresponding target sample text.

[0169] The present disclosure can utilize the CLIP model from related art to map candidate videos and target sample text into the same feature space, thereby achieving semantic association between the two. The CLIP model can conveniently calculate the cosine similarity between the two and use it as the semantic alignment parameter. In an exemplary embodiment, this semantic alignment parameter can be directly used as the video quality score.

[0170] In another exemplary embodiment, please refer to Figure 4 , which shows a schematic diagram of a method for obtaining a video quality score in the present disclosure. The video quality evaluation is performed on each candidate video to obtain a corresponding video quality score, including:

[0171] S410. Perform motion rationality detection on the candidate video to obtain motion rationality parameters.

[0172] The motion rationality parameter is used to evaluate whether the content of the candidate video itself conforms to the laws of physics, for example, whether the movement of objects therein conforms to the laws of physics. For example, if the candidate video shows a motion situation that does not conform to the laws of physics, such as the object is suspended for no reason, the movement speed is contrary to common sense, etc., the value of the motion rationality parameter should be reduced accordingly. At the same time, it is also necessary to consider whether the interaction between different objects in the video follows the rules of physics, such as whether the change in the motion state after the collision is reasonable. By comprehensively considering these factors, the motion rationality of the candidate video can be more accurately evaluated, thereby providing a reliable basis for video quality scoring. The present disclosure does not limit the method of quantifying motion rationality. For example, LLM can be used to directly score each of the candidate videos based on motion rationality to obtain the corresponding motion rationality parameter.

[0173] S420. Perform picture consistency detection on the candidate video to obtain a picture consistency parameter.

[0174] The picture consistency parameter is used to evaluate the coherence and logic of the candidate video pictures between different frames. Picture consistency is mainly reflected in many aspects, such as whether the transition of scenes is natural and smooth, and whether the position, shape and color of objects in different frames maintain reasonable changes. If there are sudden jumps in the scene, instantaneous deformation of objects or sudden changes in color in the video, the value of the picture consistency parameter will be reduced. In addition, the light and shadow effects in the picture should also remain consistent between different frames. If the light and shadow change suddenly and there is no reasonable explanation, it will also affect the consistency of the picture. Through careful detection of these picture consistency-related factors, the picture consistency parameters can be accurately obtained, which provides an important reference indicator for the comprehensive evaluation of the quality of the candidate video. The present disclosure does not limit the method of quantifying the rationality of motion. For example, LLM can be used to directly score each of the candidate videos based on picture consistency to obtain the corresponding picture consistency parameters.

[0175] S430. Calculate the semantic similarity between the candidate video and the corresponding target sample text to obtain the semantic alignment parameter; fuse the motion rationality parameter, the picture consistency parameter, and the semantic alignment parameter to obtain the video quality score.

[0176] The present disclosure does not limit the method of fusing the motion rationality parameter, the picture consistency parameter, and the semantic alignment parameter to obtain the video quality score. For example, a weighted summation method can be used to assign different weights to the motion rationality parameter, the picture consistency parameter, and the semantic alignment parameter, and each parameter is multiplied by the corresponding weight and then added to obtain the video quality score. The video quality score obtained in this way can more comprehensively and objectively reflect the comprehensive quality of the candidate video. It not only takes into account the rationality of the movement of objects in the video, avoiding the impact of unreasonable movement on the video quality, but also takes into account the consistency of the picture, ensuring the visual coherence and stability of the video, and also pays attention to the semantic similarity between the video content and the target sample text, so that the video content can accurately match the corresponding target sample text. Through such a comprehensive video quality score, it is possible to screen out videos with higher quality and more consistent with the target sample text from many candidate videos, thereby providing high-quality data for subsequent model training.

[0177] S330. The candidate video with the highest video quality score is used as the first target video, and the candidate video with the lowest video quality score is used as the second target video. The video generation result includes the first target video and the second target video.

[0178] This method can be used to screen out a first target video and a second target video. These videos can be used to generate positive and negative samples, providing effective data for training the text expansion model. The first target video, as a high-quality representative, can provide the model with positive learning examples, encouraging the model to maximize semantic alignment with it. The second target video, as a low-quality example, can help the model avoid semantic alignment with it. This allows the trained text expansion model to generate high-quality expanded text that is semantically aligned with the high-quality video, optimizing the expansion effect for video generation scenarios.

[0179] Please refer to Figure 5 , which shows the method of training based on video verification in this disclosure Figure 1 The step of training the intermediate model based on video verification based on each target sample text and the corresponding video generation result includes:

[0180] S510. For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0181] S520. Based on the positive and negative sample pairs corresponding to each target sample text, directly perform preference optimization on the intermediate model.

[0182] Direct Preference Optimization (DPO) is a method for preference modeling that does not rely on reinforcement learning. This disclosure does not limit the specific method of using each target sample text and the corresponding positive and negative sample pairs to perform direct preference optimization on the intermediate model. For example, the following formula (1) can be used as the optimization target: , optimize the intermediate model:

[0183] (1)

[0184] Specifically, It represents a functional relationship, which is represented by the following formula (2):

[0185] (2) In this function relationship Corresponding to the contents within the curly brackets at the bottom of formula (1).

[0186] and, Refers to any target sample text, Refers to the target expanded text corresponding to the corresponding first target video, Refers to the target expanded text corresponding to the corresponding second target video, Refers to the initial model of the second training phase, specifically the intermediate model obtained after the first phase of training. Refers to the current model in the current iteration of the second training phase, Refers to the positive sample obtained after the target sample text is input into the current model. Refers to the negative sample obtained after the target sample text is input into the current model. Refers to the positive sample obtained after the target sample text is input into the intermediate model. Refers to the negative sample obtained after the target sample text is input into the intermediate model. It can be understood as an expectation. is the preference dataset, which includes the dataset formed by each target sample text in the second training phase. It is a hyperparameter, and its size can be set according to actual conditions.

[0187] Based on the determination of positive and negative sample pairs, the present invention directly learns preferences based on DPO training. This training method bridges the text-video semantic gap, allowing the intermediate model to quickly learn knowledge about semantic alignment with the video, so that the expanded text expanded by the model can be used by the video generation model to generate high-quality videos. During this training process, the intermediate model will be more focused on improving its expansion capabilities in the field of video generation. For example, the expanded text can contain more details that enhance the expressiveness of the video, thereby obtaining a text expansion model.

[0188] Please refer to Figure 6 , which shows a schematic diagram of the training method based on video verification in this disclosure Figure 2 The step of training the intermediate model based on video verification based on each target sample text and the corresponding video generation result includes:

[0189] S610. For each target sample text, generate a positive sample based on the target sample text and the target expanded text corresponding to the corresponding first target video; generate a negative sample based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0190] S620. Based on the positive and negative sample pairs corresponding to each target sample text, the corresponding first target video and the corresponding second target video, comparative learning is performed on the intermediate model.

[0191] Contrastive learning involves comparing the differences between positive and negative samples, allowing the intermediate model to learn an expansion pattern that better suits the needs of video generation. During the contrastive learning process, the model analyzes the detailed information in the positive samples that can effectively improve the expressiveness of the video, as well as the key elements lacking in the negative samples, to adjust its own parameters and weights. Specifically, the model extracts and compares the text features in the positive and negative samples, strengthening the learning of the positive sample features and suppressing the focus on the negative sample features. In this way, through continuous contrastive learning, the intermediate model can more accurately grasp how to expand the text, thereby generating expanded text that is highly aligned with the video semantics and contains rich details that enhance the video's expressiveness, further improving the application effect of the text expansion model in the field of video generation.

[0192] The present disclosure does not limit the specific method of using each target sample text and the corresponding positive and negative sample pairs to perform comparative learning on the intermediate model. For example, the following formula (3) can be used as the optimization target: , optimize the intermediate model:

[0193] (3)

[0194] Specifically, TextEncoder refers to text information encoding, and VideoEncoder refers to video information encoding. The two can be mapped to the same space through cross-modal alignment. Refers to the target sample text, i represents the corresponding subscript, Refers to the target expanded text corresponding to the positive sample, Refers to the target expanded text corresponding to the negative sample, Refers to the first target video generated based on the target expanded text corresponding to the positive sample, Refers to the second target video generated based on the target expanded text corresponding to the negative sample, and max means taking the maximum value. , indicating the calculation of logical distance, It is a hyperparameter and can be set according to actual needs.

[0195] The text expansion models trained in the related art often do not use the self-distillation training process of the first training stage in the present disclosure, nor do they use the expansion quality score, video quality score, and text quality score in the present disclosure to screen high-quality expanded texts, candidate videos, or sample texts. Therefore, the related art has a significant disadvantage compared to the text expansion model generated by the present disclosure.

[0196] Specifically, the present disclosure has at least the following advantages:

[0197] (1) The related art does not perform semantic alignment between text and video, which results in semantic drift in the expansion model. However, the present disclosure establishes a joint training mechanism for the text expansion model and the video generation model, thereby eliminating the semantic drift phenomenon.

[0198] (2) Related technologies lack automated quality evaluation methods for videos generated from text expanded by the expansion model, and instead rely heavily on manual work. This makes it difficult to feed back the video evaluation results to the training process of the expansion model in real time, limiting the iterative efficiency and scalability of the training process. Experiments show that its single-round optimization cycle is as long as 48 hours, making it difficult to support large-scale training and applications. This disclosure proposes an automated video quality evaluation method that can quickly and accurately evaluate the quality of generated videos. This method can feed back the video generation effect to the training process of the expansion model in real time, greatly improving the iterative efficiency and scalability of the training process.

[0199] (3) The related technology does not use the self-distillation mechanism, which leads to the error accumulation effect and reduces the quality of the expansion model.

[0200] This paper implements multiple rounds of iterative optimization through self-distillation in the first training phase, and feeds the video generation effect back to the training process in the second training phase. This can achieve end-to-end training collaboration and significantly improve the training effect. Its quantitative performance indicators are shown in the table below:

[0201]

[0202] Furthermore, this paper is the first to use self-distillation to train a text expansion model, and can also feed back the video generation effect into the text expansion model training. This exponentially improves the quality of the text expansion model through multiple rounds of self-distillation iterations, and can reduce manual annotation costs by 90%. Furthermore, by feeding back the video generation effect into the text expansion model training, end-to-end optimization and cross-model semantic alignment are achieved, which can improve the video generation quality by 41.2% (FVD indicator) and avoid deviation in the generated content.

[0203] Please refer to Figure 7 , which shows a flow chart of the text expansion method disclosed in the present invention. The text expansion method includes:

[0204] S710. Get target text;

[0205] S720. Input the target text into the text expansion model to obtain a text expansion result;

[0206] The text expansion model is trained using the text expansion model training method described above.

[0207] The target text is any input text that is expanded using a text expansion model. By expanding the text expansion model obtained by the training disclosed herein, the obtained text expansion results are significantly improved in terms of accuracy, comprehensiveness, expansion rationality and logical consistency. Compared with traditional expansion models, the text expansion model disclosed herein can generate more accurate, comprehensive, reasonable and logically coherent text expansion results. At the same time, due to the combination of self-distillation and video generation effect feedback in the training process, the video generated by the video generation model generated by the text expansion result also has higher quality and better coherence, and has obvious advantages in terms of the logic of the picture, the richness of the content and the matching degree with the text. This end-to-end optimization method makes the entire text-to-video generation process more efficient and accurate, providing users with a better audio-visual experience, and can be widely used in multiple fields that require text expansion and video generation, such as film and television production, advertising, online education, etc.

[0208] Figure 8 1 is a block diagram of a text expansion model training device according to an exemplary embodiment. Figure 8 , the device comprises:

[0209] The first training module 810 is configured to perform the following operations:

[0210] Get the first sample text;

[0211] performing expansion processing on the first sample text based on an initial model to obtain a first expanded text, wherein the initial model is a large-scale generative language model;

[0212] Adjusting parameters of the initial model based on the first sample text and the first expanded text;

[0213] Get the second sample text;

[0214] Expanding the second sample text based on the adjusted initial model to obtain a second expanded text;

[0215] Adjusting the parameters of the adjusted initial model based on the second sample text and the second expanded text until a first preset condition is satisfied, thereby obtaining an intermediate model;

[0216] The second training module 820 is configured to perform the following operations:

[0217] Obtain at least one target sample text;

[0218] For each target sample text, the target sample text is expanded based on the intermediate model to obtain a plurality of target expanded texts; and a video generation result corresponding to the target sample text is obtained by inputting each target expanded text into a video generation model;

[0219] Based on each of the target sample texts and the corresponding video generation results, the intermediate model is trained based on video verification until the second preset condition is met, thereby obtaining a text expansion model.

[0220] In an exemplary embodiment, the first training module 810 is configured to perform the following operations:

[0221] Expanding the first sample text based on the initial model and outputting a plurality of candidate expanded texts;

[0222] Performing accuracy analysis on each candidate expanded text to obtain a corresponding accuracy parameter, wherein the accuracy parameter is used to indicate the degree of consistency between the candidate expanded text and the first sample text;

[0223] Performing consistency analysis on each candidate expansion text to obtain a corresponding consistency parameter, wherein the consistency parameter is used to indicate the degree of semantic consistency of the content of the candidate expansion text itself;

[0224] Obtaining a corresponding expansion quality score based on the accuracy parameter and the consistency parameter corresponding to each candidate expansion text;

[0225] Based on the expansion quality scores corresponding to the candidate expansion texts, the first expanded text is determined from the candidate expansion texts.

[0226] In an exemplary embodiment, the first training module 810 is configured to perform the following operations:

[0227] Inputting the first sample text into the initial model to obtain a predicted expanded text;

[0228] Based on the difference between the predicted expanded text and the first expanded text, the parameters of the initial model are adjusted.

[0229] In an exemplary embodiment, the second training module 820 is configured to perform the following operations:

[0230] Inputting each of the target expanded texts into the video generation model to generate corresponding candidate videos;

[0231] Performing a video quality assessment on each candidate video to obtain a corresponding video quality score, wherein the video quality score includes a semantic alignment parameter, and the semantic alignment parameter is used to indicate the degree of semantic alignment between the candidate video and the corresponding target sample text;

[0232] The candidate video with the highest video quality score is used as the first target video, and the candidate video with the lowest video quality score is used as the second target video. The video generation result includes the first target video and the second target video.

[0233] In an exemplary embodiment, the second training module 820 is configured to perform the following operations:

[0234] For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0235] Based on the positive and negative sample pairs corresponding to each target sample text, the intermediate model is directly optimized in terms of preference.

[0236] In an exemplary embodiment, the second training module 820 is configured to perform the following operations:

[0237] For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video;

[0238] Based on the positive and negative sample pairs corresponding to each target sample text, the corresponding first target video and the corresponding second target video, comparative learning is performed on the intermediate model.

[0239] In an exemplary embodiment, the second training module 820 is configured to perform the following operations:

[0240] Performing motion rationality detection on the candidate video to obtain motion rationality parameters;

[0241] Performing a picture consistency test on the candidate video to obtain a picture consistency parameter;

[0242] Calculating semantic similarity between the candidate video and the corresponding target sample text to obtain the semantic alignment parameter;

[0243] The motion rationality parameter, the picture consistency parameter, and the semantic alignment parameter are integrated to obtain the video quality score.

[0244] In an exemplary embodiment, the first training module 810 is configured to perform the following operations:

[0245] Acquire a sample text set, the sample text set including the first sample text and the second sample text;

[0246] The second training module 820 is configured to perform the following operations:

[0247] Performing a text quality assessment on each sample text in the sample text set to obtain a corresponding text quality score;

[0248] The sample text whose text quality score meets the preset text quality requirement is used as the target sample text.

[0249] In an exemplary embodiment, the first training module 810 is configured to perform the following operations:

[0250] For each candidate expanded text, input the candidate expanded text into the video generation model to obtain a corresponding plurality of predicted videos; and determine a corresponding alignment quality score based on an expected semantic similarity between each predicted video and the candidate expanded text;

[0251] The accuracy parameter, the consistency parameter and the alignment quality score corresponding to each candidate expanded text are integrated to obtain a corresponding expanded quality score.

[0252] Regarding the device in the above embodiment, the specific manner of each step has been described in detail in the embodiment of the aforementioned method and will not be elaborated here.

[0253] Figure 9 FIG. 1 is a block diagram of a text expansion device according to an exemplary embodiment. Figure 9 , the device comprises:

[0254] The target text acquisition module 910 is configured to execute acquisition of the target text;

[0255] The expansion module 920 is configured to input the target text into a text expansion model to obtain a text expansion result;

[0256] The text expansion model is trained using the text expansion model training method described above.

[0257] Regarding the device in the above embodiment, the specific manner of each step has been described in detail in the embodiment of the aforementioned method and will not be elaborated here.

[0258] Please refer to Figure 10 , which shows a structural block diagram of a computer device provided by an exemplary embodiment of the present disclosure Figure 1 The computer device may be a terminal. The computer device is used to implement the text expansion model training method or text expansion method provided in the above embodiments. Specifically:

[0259] Typically, the computer device 1000 includes a processor 1001 and a memory 1002 .

[0260] Processor 1001 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 1001 may be implemented in hardware using at least one of a DSP (Digital Signal Processing), an FPGA (Field Programmable Gate Array), and a PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In one exemplary embodiment, processor 1001 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content displayed on the display screen. In one exemplary embodiment, processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0261] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In an exemplary embodiment, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is configured to be executed by one or more processors to implement the above-mentioned text expansion model training method or text expansion method.

[0262] In an exemplary embodiment, computer device 1000 may optionally include a peripheral device interface 1003 and at least one peripheral device. Processor 1001, memory 1002, and peripheral device interface 1003 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1003 via a bus, signal lines, or circuit boards. Specifically, the peripheral device includes at least one of a radio frequency circuit 1004, a touchscreen display 1005, a camera assembly 1006, an audio circuit 1007, a positioning assembly 1008, and a power supply 1009.

[0263] Those skilled in the art will understand that Figure 10 The structure shown in the figure does not constitute a limitation on the computer device 1000, and the computer device 1000 may include more or fewer components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0264] Please refer to Figure 11 It shows a structural block diagram of a computer device provided by another exemplary embodiment of the present disclosure. Figure 2 The computer device may be a server for executing the above-mentioned text expansion model training method or text expansion method. Specifically:

[0265] Computer device 1100 includes a central processing unit (CPU) 1101, a system memory 1104 including random access memory (RAM) 1102 and read-only memory (ROM) 1103, and a system bus 1105 connecting system memory 1104 and CPU 1101. Computer device 1100 also includes a basic input / output system (I / O) 1106 that facilitates information transfer between various components within the computer, and a mass storage device 1107 for storing an operating system 1113, application programs 1114, and other program modules 1111.

[0266] The basic input / output system 1106 includes a display 1108 for displaying information and an input device 1109, such as a mouse and keyboard, for user input. Both the display 1108 and the input device 1109 are connected to the central processing unit 1101 via an input / output controller 1110 connected to the system bus 1105. The basic input / output system 1106 may also include an input / output controller 1110 for receiving and processing input from a variety of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1110 also provides output to a display screen, printer, or other types of output devices.

[0267] Mass storage device 1107 is connected to central processing unit 1101 via a mass storage controller (not shown) connected to system bus 1105. Mass storage device 1107 and its associated computer-readable media provide non-volatile storage for computer device 1100. In other words, mass storage device 1107 may include computer-readable media (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0268] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the aforementioned types. The aforementioned system memory 1104 and mass storage device 1107 may be collectively referred to as memory.

[0269] According to various embodiments of the present disclosure, the computer device 1100 may also be connected to a remote computer on a network such as the Internet for operation. That is, the computer device 1100 may be connected to the network 1112 via the network interface unit 1111 connected to the system bus 1105. Alternatively, the network interface unit 1111 may be used to connect to other types of networks or remote computer systems (not shown).

[0270] The above-mentioned memory also includes a computer program, which is stored in the memory and configured to be executed by one or more processors to implement the above-mentioned text expansion model training method or text expansion method.

[0271] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored. When the at least one instruction, the at least one program, the code set or the instruction set is executed by a processor, the text expansion model training method or the text expansion method is implemented.

[0272] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or an optical disk. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0273] In an exemplary embodiment, a computer-readable storage medium including program code is also provided, such as a memory including the program code. The program code can be executed by a processor to perform the above-described text expansion model training method or text expansion method. Alternatively, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0274] In an exemplary embodiment, a computer program product is also provided, including a computer program, which implements the above-mentioned text expansion model training method or text expansion method when executed by a processor.

[0275] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0276] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A text expansion model training method, characterized in that: The method comprises: Get the first sample text; performing expansion processing on the first sample text based on an initial model to obtain a first expanded text, wherein the initial model is a large-scale generative language model; Adjusting parameters of the initial model based on the first sample text and the first expanded text; Get the second sample text; Expanding the second sample text based on the adjusted initial model to obtain a second expanded text; Adjusting the parameters of the adjusted initial model based on the second sample text and the second expanded text until a first preset condition is satisfied, thereby obtaining an intermediate model; Obtain at least one target sample text; For each target sample text, the target sample text is expanded based on the intermediate model to obtain multiple target expanded texts; each target expanded text is input into a video generation model to generate a corresponding candidate video; a video quality assessment is performed on each candidate video to obtain a corresponding video quality score, wherein the video quality score includes a semantic alignment parameter, and the semantic alignment parameter is used to indicate the degree of semantic alignment between the candidate video and the corresponding target sample text; the candidate video with the highest video quality score is used as the first target video, and the candidate video with the lowest video quality score is used as the second target video; Based on each of the target sample texts and the corresponding video generation results, the intermediate model is trained based on video verification until a second preset condition is met to obtain a text expansion model, and the video generation results include the first target video and the second target video.

2. The method according to claim 1, characterized in that The step of expanding the first sample text based on the initial model to obtain a first expanded text includes: Expanding the first sample text based on the initial model and outputting a plurality of candidate expanded texts; Performing accuracy analysis on each candidate expanded text to obtain a corresponding accuracy parameter, wherein the accuracy parameter is used to indicate the degree of consistency between the candidate expanded text and the first sample text; Performing consistency analysis on each candidate expansion text to obtain a corresponding consistency parameter, wherein the consistency parameter is used to indicate the degree of semantic consistency of the content of the candidate expansion text itself; Obtaining a corresponding expansion quality score based on the accuracy parameter and the consistency parameter corresponding to each candidate expansion text; Based on the expansion quality scores corresponding to the candidate expansion texts, the first expanded text is determined from the candidate expansion texts.

3. The method according to claim 1 or 2, characterized in that The adjusting the parameters of the initial model based on the first sample text and the first expanded text includes: Inputting the first sample text into the initial model to obtain a predicted expanded text; Based on the difference between the predicted expanded text and the first expanded text, the parameters of the initial model are adjusted.

4. The method according to claim 1, wherein The training of the intermediate model based on video verification based on each target sample text and the corresponding video generation result includes: For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video; Based on the positive and negative sample pairs corresponding to each target sample text, the intermediate model is directly optimized in terms of preference.

5. The method according to claim 1, wherein The training of the intermediate model based on video verification based on each target sample text and the corresponding video generation result includes: For each target sample text, a positive sample is generated based on the target sample text and the target expanded text corresponding to the corresponding first target video; a negative sample is generated based on the target sample text and the target expanded text corresponding to the corresponding second target video; Based on the positive and negative sample pairs corresponding to each target sample text, the corresponding first target video and the corresponding second target video, comparative learning is performed on the intermediate model.

6. The method according to claim 1, wherein The performing video quality assessment on each candidate video to obtain a corresponding video quality score includes: Performing motion rationality detection on the candidate video to obtain motion rationality parameters; Performing a picture consistency test on the candidate video to obtain a picture consistency parameter; Calculating semantic similarity between the candidate video and the corresponding target sample text to obtain the semantic alignment parameter; The motion rationality parameter, the picture consistency parameter, and the semantic alignment parameter are integrated to obtain the video quality score.

7. The method according to claim 1, characterized in that The method further comprises: Acquire a sample text set, the sample text set including the first sample text and the second sample text; The obtaining of at least one target sample text includes: Performing a text quality assessment on each sample text in the sample text set to obtain a corresponding text quality score; The sample text whose text quality score meets the preset text quality requirement is used as the target sample text.

8. The method according to claim 2, characterized in that The step of obtaining a corresponding expansion quality score based on the accuracy parameter and the consistency parameter corresponding to each candidate expansion text includes: For each candidate expanded text, input the candidate expanded text into the video generation model to obtain a corresponding plurality of predicted videos; and determine a corresponding alignment quality score based on an expected semantic similarity between each predicted video and the candidate expanded text; The accuracy parameter, the consistency parameter and the alignment quality score corresponding to each candidate expanded text are integrated to obtain a corresponding expanded quality score.

9. A text expansion method, characterized in that: The method comprises: Get the target text; Inputting the target text into a text expansion model to obtain a text expansion result; The text expansion model is trained using the text expansion model training method described in any one of claims 1 to 8.

10. A text expansion model training device, characterized in that: The device comprises: The first training module is configured to perform the following operations: Get the first sample text; performing expansion processing on the first sample text based on an initial model to obtain a first expanded text, wherein the initial model is a large-scale generative language model; Adjusting parameters of the initial model based on the first sample text and the first expanded text; Get the second sample text; Expanding the second sample text based on the adjusted initial model to obtain a second expanded text; Adjusting the parameters of the adjusted initial model based on the second sample text and the second expanded text until a first preset condition is satisfied, thereby obtaining an intermediate model; The second training module is configured to perform the following operations: Obtain at least one target sample text; For each target sample text, the target sample text is expanded based on the intermediate model to obtain multiple target expanded texts; each target expanded text is input into a video generation model to generate a corresponding candidate video; a video quality assessment is performed on each candidate video to obtain a corresponding video quality score, wherein the video quality score includes a semantic alignment parameter, and the semantic alignment parameter is used to indicate the degree of semantic alignment between the candidate video and the corresponding target sample text; the candidate video with the highest video quality score is used as the first target video, and the candidate video with the lowest video quality score is used as the second target video; Based on each of the target sample texts and the corresponding video generation results, the intermediate model is trained based on video verification until a second preset condition is met to obtain a text expansion model, and the video generation results include the first target video and the second target video.

11. A text expansion device, characterized in that: The device comprises: A target text acquisition module is configured to execute acquisition of the target text; an expansion module configured to input the target text into a text expansion model to obtain a text expansion result; The text expansion model is trained using the text expansion model training method described in any one of claims 1 to 8.

12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the text expansion model training method according to any one of claims 1 to 8, or the text expansion method according to claim 9.

13. A computer-readable storage medium, characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device executes the text expansion model training method as described in any one of claims 1 to 8, or the text expansion method as described in claim 9.

14. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a readable storage medium. At least one processor of a computer device reads and executes the computer program from the readable storage medium, so that the device executes the text expansion model training method according to any one of claims 1 to 8, or the text expansion method according to claim 9.

Images