A method and system for generating a movie story that fuses movie character features

Abstract

The application discloses a movie story generation method and system fusing movie character features, and comprises the following steps: constructing a movie story resource library, respectively adopting a movie character set and a normal distribution to randomly initialize movie character vector representation and movie character verb vector representation; adopting a second adaptive feature to train the movie character verb vector representation, and combining a first and a third adaptive feature, a behavior predictor, an encoder, a CVAE model and a decoder to jointly train the movie character vector representation to calculate the last distribution of words; and determining the content of movie story generation through the last distribution of words, so as to realize the creation of a movie story around a given movie character. The movie story generated in combination with the movie character features can obviously capture movie character information and strengthen the connection between a movie plot and movie characters, so that the explainability and consistency are improved, and the creation of a movie story around a given movie character is realized.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for story generation using a conditional variational autoencoder that integrates the characteristics of movie characters. Background Technology

[0002] Film story generation involves creating a film story that is relevant and innovative, based on prompts or referenced information. Traditional film story generation requires creators to collect and organize knowledge, combining inspiration and writing skills to create a story. This approach suffers from long creation cycles, low efficiency, and difficulty in meeting the ever-evolving needs of film audiences. Exploring a new method for automated film story generation to assist in and complete film story creation has become a demand from both audiences and creators.

[0003] The core elements of film story generation include the film title, time period, scene keywords, and characters. Among these, the characters' background information, their behavioral reactions in different film scenes, and the relationships between characters play crucial roles in the film story. To improve the interpretability of film story generation and align it with human thought, a character-centric storytelling model can create innovative film stories around given characters in different contexts. However, current film story creation relies on creators with specific professional knowledge and experience, resulting in a lengthy creation cycle and limitations imposed by the creators' knowledge and experience. Furthermore, film story creation needs to constantly adapt to the evolving tastes and preferences of the audience. Therefore, building a film story resource library and integrating story generation methods that incorporate the characteristics of film characters have become critical issues that urgently need to be addressed in the film and television industry. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing a method and system for generating film stories that integrates the characteristics of film characters. It enables the creation of film stories around given film characters. First, film knowledge is collected and organized to construct a film story resource library. Then, combining film titles and relevant information about film characters, a behavior predictor is used to predict the behavioral responses of film characters in different scenarios. Next, three adaptive features are incorporated, and a conditional variational autoencoder (CVAE), dual encoders, and a decoder are used to controllably generate the film story.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for generating film stories that integrates the characteristics of film characters includes the following steps:

[0007] Step 101: Build a film story resource library;

[0008] Step 101 includes the following sub-steps:

[0009] S11: Retrieve movie title, movie story text, and movie character background information;

[0010] S12: Based on the movie story text, obtain the movie time period and scene keywords;

[0011] S13: Combine the movie title, movie story text and movie character background information to construct a list of movie character verbs and a set of movie characters, and initialize the vector representation of the movie characters.

[0012] Step 102: Using the generated movie story segments or sentences as the context information for the currently generated movie story sentence, the first encoder is used to extract the semantic information of the context information.

[0013] Step 103: Based on the input movie title, obtain the movie characters related to the movie title as the core content of the story. Integrate the first adaptive feature, using the vector representation of the movie character obtained in S101, the movie character verb list, and the encoded context information, and employ a behavior predictor to predict the vector representation of the movie character's actions, that is, the operation that the movie character decides to perform in the currently generated sentence in the known context.

[0014] S104: Combining movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions, a second adaptive feature is incorporated, and a second encoder is used to further encode the preceding information;

[0015] Based on the input movie title and the movie characters selected by the user in step 103, the corresponding appearance time periods and scene keywords for each movie character are obtained from the movie story resource library in step 101. During the generation of each movie story, the corresponding time periods and scene keywords for the movie characters are used as auxiliary information to complete the background of the generated movie story.

[0016] Step 105: Based on the context information encoded by the second encoder, and combined with self-attention, the CVAE model is used to improve the quality of the generated movie story;

[0017] This approach combines film character features with CVAE (Continuous Video Image Processing). CVAE models the probability of generating target text given source text, taking both source and target text as input and sampling from the posterior distribution to train the decoder. Furthermore, a conditional prior distribution is provided for sampling generated text during the testing phase.

[0018] S106: Combining Pseudo self-attention with a third adaptive feature, a decoder is used to calculate the final distribution of words and generate the movie story.

[0019] Furthermore, step 103 includes the following sub-steps:

[0020] S31: Input a movie title, select movie characters from the movie character set in the movie story resource library, and obtain the vector representation of the corresponding movie characters;

[0021] S32: Incorporating the first adaptive feature, a behavior predictor is used to predict the actions of movie characters based on the vector representation of the corresponding movie characters, the movie characters' verb list, and the encoded context information.

[0022] Furthermore, step 104 includes the following sub-steps:

[0023] S41: Combine the input movie title and corresponding movie characters to obtain the corresponding movie time period and scene words for each movie character;

[0024] S42: Based on the input movie title, movie time period, movie scene words, movie character vector representations, and movie character action vector representations, a second adaptive feature is incorporated, and a second encoder is used to further encode the preceding information.

[0025] Furthermore, step 105 includes the following sub-steps:

[0026] S51: Uses the output of the second encoder to encode self-attention information;

[0027] S52: Encode information based on the self-attention mechanism and use the CVAE model to improve the quality of movie story generation.

[0028] Furthermore, step 106 includes the following sub-steps:

[0029] S61: Based on the vector representation of the corresponding movie character, concatenate it with the vector representation of the movie story already generated by the decoder in time step t to construct the input of the decoder in time step t;

[0030] S62: Combining the latent variable z and the decoder input, a third adaptive feature is incorporated to encode pseudo self-attention information;

[0031] S63: Based on pseudo self-attention information, the decoder is used to calculate the final distribution of words;

[0032] S64: The final distribution of words is used to generate the movie story.

[0033] This invention also discloses a film story generation system that integrates the characteristics of film characters, comprising:

[0034] Movie Story Resource Library Building Module: Used to build a movie story resource library;

[0035] First encoder module: Based on the generated movie story segments or sentences as the context information of the currently generated movie story sentence, the first encoder is used to extract the semantic information of the context.

[0036] Behavior prediction module: Based on the movie title, the vector representation of the movie characters, the verb list of the movie characters, and the encoded context information, the first adaptive feature is incorporated, and the behavior predictor is used to predict the actions of the movie characters.

[0037] The second encoder module combines the movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions, and incorporates a second adaptive feature to further encode the preceding information using a second encoder.

[0038] Self-attention module: Based on the information encoded by the second encoder, combined with self-attention, the CVAE model is used to improve the quality of movie story generation.

[0039] Movie story generation module: Combining Pseudo self-attention with a third adaptive feature, the decoder calculates the final distribution of words and generates the movie story.

[0040] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described movie story generation method.

[0041] The present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described movie story generation method.

[0042] Compared with the prior art, the advantages of the present invention are as follows:

[0043] By constructing a film story resource library, the vector representations of film characters and verbs are randomly initialized using a set of film characters and a normal distribution, respectively. A second adaptive feature is used to train the verb vector representations of film characters. The final word distribution is calculated by combining the first and third adaptive features, an action predictor, an encoder, a CVAE model, and a decoder to train the film character vector representations. The final word distribution then determines the content of the generated film story, thus enabling the creation of film stories around given film characters. Film stories generated by combining film character features can explicitly capture film character information and strengthen the connection between the film plot and the film characters, improving interpretability and consistency, and enabling the creation of film stories around given film characters. Attached Figure Description

[0044] Figure 1This is a flowchart of the conditional variational autoencoder story generation method that integrates movie character features according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of the operational structure of the conditional variational autoencoder story generation method that integrates movie character features according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the movie story generation model structure according to an embodiment of the present invention;

[0047] Figure 4 This is a flowchart of the calculation process of the first encoder and the second encoder in an embodiment of the present invention;

[0048] Figure 5 This is a flowchart of the calculation process of the CVAE model in an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.

[0050] like Figure 1 As shown, a method for generating film stories that integrates the characteristics of film characters includes the following steps:

[0051] Step 101: Build a film story resource library;

[0052] In practical applications, film stories are characterized by being segmented according to time and scene lines. A film story can consist of content that occurs in multiple time periods and scenes, and the content that occurs in a scene includes location, atmosphere, and plot. Regarding the film story generation process, it is divided into segments, generated sentence by sentence, with each segment using different time periods and scene keywords to support the background of the film story generation. Therefore, in this embodiment of the invention, a film story resource library is constructed. On the one hand, it collects relevant information about film characters, including film titles, film time periods and scene keywords, and background information of film characters; on the other hand, it initializes the vector representations of film characters and integrates film character features for subsequent story generation operations.

[0053] In one embodiment, step 101 may include the following sub-steps:

[0054] S11: Retrieve movie title, movie story text, and movie character background information;

[0055] S12: Based on the movie story text, obtain the movie time period and scene keywords;

[0056] S13: Combine the movie title, movie story text and movie character background information to construct a list of movie character verbs and a set of movie characters, and initialize the vector representation of the movie characters.

[0057] In its specific implementation, this embodiment of the invention constructs a film story resource library, the structure of which is as follows: Figure 2 As shown, the film resource library consists of film story text, film title, film time period and scene words, and film character set.

[0058] In one embodiment, S11: Obtain the movie title, movie story text, and movie character background information from the Wikipedia knowledge base.

[0059] S12: Based on the film's story text, the Stanford CoreNLP tool is first used to extract time periods, location words within scenes, and film characters from each paragraph. Scene words include not only location words but also atmospheric words and plot words. To obtain atmospheric and plot words, a graph-based keyword extraction algorithm, TextRank, is used. In TextRank, the film's story text is viewed as a network of word groups, where links between words represent semantic relationships. The extraction of atmospheric and plot words is achieved through the semantic relationships within the words in the document.

[0060] Then, for each film character, the time periods and scene keywords associated with that character are linked together to form the character's appearance time periods and scene keywords. Among these, the following methods are used: <eos>Special symbols are used to connect time periods and all scene words within those time periods to construct the time periods and scene words for a film character's appearance. Specifically, for example, "time period" <eoe>Location words <eoe>Scene atmosphere words <eoe>Plot details of the story.

[0061] S13: Constructing the film character set is primarily for training the vector representation of each film character during the story generation model training process. Each dimension of the vector representation represents a feature of the film character, including their birth date, personality, work status, job position, gender, and emotional state. By extracting the descriptive content of each film character in the film story, and using the Stanford CoreNLP tool to extract relevant verbs and attributes, a film character verb list and a film character set are constructed respectively.

[0062] The verbs include verbs related to the film characters as subjects or objects, while the attributes include adjectives or adjectives, appositives, compound words, etc. Finally, the relevant attribute characteristics of the film characters, the time periods and scene words of the film characters, and the relevant background information obtained from the knowledge base are stored together in the film character set.

[0063] To better train the vector representations of movie characters, the feature information related to each movie character is aggregated and the vector representations of the movie characters are initialized. Based on the pre-trained Clove vector vocabulary, the initial vector representation of each movie character is calculated using average summation, as shown in the formula:

[0064]

[0065] Among them, w i Let i represent the word vector, N represent the number of feature words of the movie character, C represent the vector representation of the movie character, and Glove(.) represent the pre-trained word vector table.

[0066] Step 102: Encode the preceding information using the first encoder;

[0067] In the specific implementation, based on the generated movie story segments or sentences as the context information of the currently generated movie story sentence, the first encoder is used to extract the semantic information of the context, and its structure is as follows: Figure 4 As shown, the encoder structure is similar to the pre-trained GPT-2 model. The only difference is that the encoder uses multi-head attention computation, while the GPT-2 model uses masked multi-head attention. This is because the GPT-2 model is an autoregressive model, and only considers the cases mentioned above during the decoding process. The GPT-2 model is shown in the paper "Language Models are Unsupervised MultitaskLearners" (Radford et al. (Corr 2019)). The computation process of the first encoder is shown in the formula:

[0068] S′ i-1 =Encoder-1(S i-1 )

[0069] Where Encoder-1(.) represents the first encoder, S′ i-1 S represents the information encoded by the first encoder. i-1 The vector representation of the generated movie story text is used, similar to GPT-2. The vector representation of the vocabulary is randomly initialized, and then the vector representation of the words is dynamically trained using a self-training method; i-1 represents the content generated before the i-th sentence is generated.

[0070] Step 103: Based on the movie title, the vector representation of the movie characters, the verb list of the movie characters, and the encoded context information, incorporate the first adaptive feature and use the behavior predictor to predict the actions of the movie characters;

[0071] In the specific implementation, based on the input movie title, movie characters related to the movie title are obtained as the core content of the story. Incorporating the first adaptive feature, through the vector representation of the movie characters (the initial vector representation of the movie characters is obtained from step S12), the movie character verb list, and the encoded contextual information, the movie characters can determine the sentence S to be generated based on the known context. i The operations performed in the process.

[0072] In one embodiment, step 103 may include the following sub-steps:

[0073] S31: Input a movie title, select movie characters from the movie character set in the movie story resource library, and obtain the vector representation of the corresponding movie characters;

[0074] S32: Incorporating the first adaptive feature, a behavior predictor is used to predict the actions of movie characters based on the vector representation of the corresponding movie characters, the movie characters' verb list, and the encoded context information.

[0075] In the specific implementation,

[0076] S31: Based on the input title, movie characters with similar titles can be selected from the set of movie characters. The structure is as follows: Figure 3 As shown. For example, if the title is "The Redemption of Prison," users can search for related movie characters using keywords such as "prison" or "redemption," including names like "Andy Dufresne," "Warden Norton," or "Heywood." Furthermore, if the title is too short or lacks sufficient information, a thesaurus can be used to broaden the search scope.

[0077] Furthermore, in this embodiment of the invention, besides generating movie stories around a single movie character, users can independently set up movie stories to be generated around multiple movie characters. Specifically, for example, if multiple movie characters are selected for movie story generation, settings need to be configured for each movie character, specifying which sentence in the movie story text the character appears in. For example, if the user selects two movie characters, "Andy Dufresne" and "Warden Norton," then the movie character "Andy Dufresne" will be positioned in the nth sentence of the movie story text. a The sentence appears, where n represents the number of sentences in the film story, and a represents the a-th sentence among the n sentences, where a can contain multiple constants. This indicates that the character "Andy Dufresne" appears in multiple locations within the film story; it also provides the nth sentence in the film story text for the character "Warden Norton". b There are n sentences, where n represents the number of sentences in the movie story, and b represents the b-th sentence among those n sentences. b can contain multiple constants, indicating that the movie character "Warden Norton" appears in multiple locations within the movie story. Therefore, the two movie characters may appear in the same sentence.

[0078] S32: Incorporate the first adaptive feature to train the vector representation C of the film character (the initial vector representation of the film character is obtained from step S12), which can describe the information of the film character from a more comprehensive perspective. a This represents the movie character vector after training with the first adaptive feature, and its calculation process is shown in the formula:

[0079] C a =W 2 ·ReLU(W 1 ·C+B 1 )+B 2

[0080] Where ReLU(.) represents the activation function, W 2 and W 1 B represents the trainable weight matrix. 1 and B 2 This represents a trainable bias.

[0081] To create a movie story around a given movie character, embodiments of this invention can predict the behavioral content of multiple movie characters. For ease of description, a movie character is first represented, and the probability p(V) of the movie character's actions is predicted based on the preceding information and the movie character information. i |S′ i-1 C a The calculation process is shown in the formula:

[0082] p(V i |S′ i-1 C a = softmax(MLP(S′) i-1 C a ))

[0083] Among them, S′ i-1 It is the representation of the above text after being encoded by the first encoder, C a It is a movie character vector representation that incorporates the first adaptive feature, V i This indicates that the movie character is generating the current sentence S. i The action to be performed at that time, MLP(.) is a two-layer perceptron, and softmax(.) represents the softmax function.

[0084] Then, in generating the current sentence S i During the process, the behavior content of multiple movie characters is predicted. During the training of the behavior predictor, the vector representation of the movie characters can be dynamically calculated and updated. The calculation process is shown in the formula:

[0085]

[0086] Where N represents the number of sentences in the generated movie story segments, M represents the number of movie characters whose actions are predicted by the behavior predictor, K represents the number of segments in the generated movie story, and S′ i-1 This represents the preceding text after it has been encoded by the first encoder. C represents the movie character vector after training by incorporating the first adaptive feature into the j-th movie character. a The superscript j represents the j-th movie character, and the subscript a represents the vector representation after incorporating the first adaptive feature. V i j Let j represent the action performed by the j-th movie character at time step i, where the superscript j indicates the action of the j-th movie character and the subscript i indicates time step i. The calculation process and p(V) i |S′ i-1 C a )Same.

[0087] S104: Combining movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions, a second adaptive feature is incorporated, and a second encoder is used to further encode the preceding information;

[0088] In the specific implementation, based on the input movie title and the movie character selected by the user in step 103, the corresponding appearance time period and scene words of each movie character are obtained from the movie story resource library in step 101. During the generation of each movie story, the corresponding time period and scene words of the movie characters are used as auxiliary information to improve the background of the generated movie story.

[0089] In one embodiment, step 104 may include the following sub-steps:

[0090] S41: Combine the input movie title and corresponding movie characters to obtain the corresponding movie time period and scene words for each movie character;

[0091] S42: Based on the input movie title, movie time period, movie scene words, movie character vector representations, and movie character action vector representations, a second adaptive feature is incorporated, and a second encoder is used to further encode the preceding information.

[0092] In the specific implementation,

[0093] S41: Based on the constructed set of movie characters, obtain the segments and scene keywords of the movie characters' appearance times in the movie. In the segments generated from the movie story, the time segments and scene keywords of each segment are different, which can clearly capture the connection between movie characters and movie plot and different scenes.

[0094] S42: In order to represent the action information of movie characters, this embodiment of the invention uses a normal distribution to randomly initialize the vector representation of movie character verbs. (The movie characters share a common verb list provided by S13), and then a second adaptive feature is incorporated to train the movie character action vector representation. V can describe the information of movie characters' actions from a more comprehensive perspective. a This represents the movie character motion vector after training with the second adaptive feature, where the subscript 'a' indicates the vector representation after training with the second adaptive feature. The calculation process is shown in the formula:

[0095]

[0096] Where ReLU(.) represents the activation function, W 3 and W 4 Let B represent the trainable weight matrix. 3 and B 4 This represents a trainable bias.

[0097] In this embodiment of the invention, the following is adopted: <eoc> 、 <eot> 、 <eop> 、 <eoe> 、 <eov>The special symbols are concatenated with five pieces of information: the preceding text encoded by the first encoder, the vector representation of the movie character incorporating the first adaptive feature, the input movie title, movie time period, movie scene words, and the motion vector representation of the movie character incorporating the second adaptive feature. This concatenation is then input into the second encoder. Figure 4 As shown, the specific calculation process is illustrated in the formula:

[0098]

[0099] Among them, S′ i-1 This represents the preceding information after it has been encoded by the first encoder. C represents the movie character vector after training by incorporating the first adaptive feature into the j-th movie character. a The superscript j represents the j-th movie character, and the subscript a represents the vector representation after incorporating the first adaptive feature. This represents the representation of the j-th movie character action vector after training with the second adaptive feature. The superscript i indicates that the current sentence S is generated. i The action performed at that time, the subscript 'a' indicates the vector representation after incorporating the second adaptive feature, and [.] indicates the concatenation operation. Special symbols used in Chinese <eoc>Concatenate the vector representations of j movie characters, and... Special symbols used in Chinese <eov>The action vectors of j movie characters are concatenated. B represents the vector representation of the movie title (obtained directly from pre-trained GloVe word vectors), P represents the movie duration (obtained directly from pre-trained GloVe word vectors), and E... n This represents n scene words (this vector is obtained directly from the pre-trained GloVe word vectors; there are multiple scene words, including location words, atmosphere words, and plot words of the story).

[0100] Step 105: Based on the information encoded by the second encoder, and combined with self-attention, the CVAE model is used to improve the quality of the generated movie story;

[0101] In its specific implementation, this invention combines film character features with CVAE. CVAE models the probability of generating target text given source text, while simultaneously inputting the source text (source text refers to the input information S of the first encoder). i-1 ) and target text (target text refers to the information S that the decoder wants to generate) i The posterior distribution is sampled for use in the training of the decoder. Additionally, a conditional prior distribution is provided for sampling generated text during the testing phase.

[0102] In one embodiment, step 105 may include the following sub-steps:

[0103] S51: Uses the output of the second encoder to encode self-attention information;

[0104] S52: Encode information based on the self-attention mechanism and use the CVAE model to improve the quality of movie story generation.

[0105] In the specific implementation,

[0106] S51: Based on the output h of the second encoder s The information is encoded using self-attention, and its calculation process is shown in the formula:

[0107]

[0108] in, It is a trainable weight matrix based on the output h of the second encoder. s h is processed through two independent linear layers respectively. s Mapped to two different matrix vectors, i.e., K = h s W 5 and V=h s W 6 W 5 and W 6 Both are trainable weight matrices. represents the scaling factor, and Att represents the information after self-attention encoding.

[0109] S52: This embodiment of the invention uses CVAE to improve the quality of movie story generation, and its structure is as follows: Figure 5 As shown, the vector representation Att obtained from self-attention computation is passed to the linear layer to predict the prior and posterior distributions, respectively. The prior probability distribution is represented as p = (z|x), and the posterior probability distribution is represented as q = (z|x, y), where x represents the current output information of the decoder, y represents the information Att after self-attention encoding, and z represents the latent variable of CVAE, i.e., the distribution to be trained. Here, y is used as a condition for CVAE. During training, the latent variable z trained by the model is required to be sufficiently close to the true z. During inference, the model relies entirely on the z sampled by the model to assist the encoder in generating the movie story. The prior and posterior sampling distributions are multidimensional Gaussian distributions, specifically... μ represents the mean, σ 2 Let I represent the variance, and let 1 represent the covariance.

[0110] S106: Combining Pseudo self-attention with a third adaptive feature, a decoder is used to calculate the final distribution of words and generate the movie story.

[0111] In the specific implementation, the decoder uses a pre-trained GPT-2 model, and completes a full decoding operation for each sentence generated. A full decoding operation is divided into multiple time steps t, 1≤t≤T. In each time step t, the decoder calculates the final distribution of words to obtain the generated content of the movie story.

[0112] In one embodiment, step 106 may include the following sub-steps:

[0113] S61: Based on the vector representation of the corresponding movie character, concatenate it with the vector representation of the movie story already generated by the decoder in time step t to construct the input of the decoder in time step t;

[0114] S62: Combining the latent variable z and the decoder input, a third adaptive feature is incorporated to encode pseudo self-attention information;

[0115] S63: Based on pseudo self-attention information, the decoder is used to calculate the final distribution of words;

[0116] S64: The final distribution of words is used to generate the movie story.

[0117] In the specific implementation,

[0118] S61: Generate a new sentence of movie story content based on the context information calculated by the CVAE model. In time step t, the first input O of the decoder... t The decoder structure is the same as that of the vector representation of the movie story generated before time step t. Figure 4 Similarly, the difference lies in replacing multi-head attention with masked multi-head attention. And the second input to the decoder... It consists of vector representations of the corresponding movie characters. C represents the movie character vector after training by incorporating the first adaptive feature into the j-th movie character. a The superscript j represents the j-th movie character, and the subscript a represents the vector representation after incorporating the first adaptive feature. In this embodiment of the invention, the first input and the second input are concatenated to form the input of the decoder at time step t.

[0119] S62: A pseudo self-attention mechanism is used to jointly encode the latent variable z and the decoder input. Specifically, the latent variable z is added to the Masked multi-head attention layer of the decoder for computation. First, based on the latent variable... A third adaptive feature is incorporated, which can be jointly learned with the first adaptive feature in step 103 and the second adaptive feature in step 104 to strengthen the connection between the behavior predictor in step 103, the encoder in step 104, and the decoder (the adaptive features in steps 103, 104, and 106 are essentially the same; to highlight the difference, they are named the first adaptive feature, the second adaptive feature, and the third adaptive feature). Specifically, this third adaptive feature utilizes a two-layer feedforward network to map z to z0. L ∈R dxL Where d represents the dimension of the vector, and L represents the number of decoder layers. Then, z is divided into L vectors, i.e., [z1, ..., z2]. L The purpose is to feed z into each layer for attention computation. Two independent linear layers are used to feed z... l Mapped to two independent spaces, l∈L represents the l-th layer of the decoder, thus obtaining z K and z V z K and z V spliced ​​to K respectively d and V d Generate K′ d and V′ d Perform Pseudo self-attention, K′ d Compared to K d There's an extra line of information after the z-mapping; similarly, V′ d V d An additional line of information after z-mapping has been added. The calculation process of Pseudo self-attention Pse-Att is shown in the formula:

[0120]

[0121] K′ d =[z K K d ]

[0122] V′ d =[z V V d ]

[0123] Among them, Q d K d V d all Where l represents the l-th layer of the decoder, and d represents the dimension of the vector. Q d K d V d Based on the input to the decoder at time step t, it is composed of three independent linear layers, namely W 7 W 8 W 9 All are trainable weight vectors, O i , All of them come from the input of the decoder time step t. [.] indicates a concatenation operation. This represents the scaling factor.

[0124] S63: Based on the output of the pseudo self-attention mechanism, the decoder calculates the final word distribution. The output vector representation of the pseudo self-attention mechanism is passed to the linear layer and the softmax layer to output the final word distribution. The word with the highest probability is selected as the current output. Therefore, the decoder's calculation process is as shown in the formula:

[0125]

[0126] Among them, h t O represents the output of the decoder at time step t. i This represents the vector representation of the movie story that has been generated before time step t. C represents the movie character vector after training by incorporating the first adaptive feature into the j-th movie character. a The superscript j represents the j-th movie character, and the subscript a represents the vector representation after incorporating the first adaptive feature. [.] indicates a concatenation operation. The j movie character representation vectors are directly concatenated.

[0127] S64: Finally, the final word distribution is used to generate the movie story. The loss function of this embodiment is shown in the formula:

[0128] L=αL1+βL2

[0129]

[0130] Here, α and β are trainable parameters used to balance the two loss functions. L1 represents the loss of the behavior predictor, L2 represents the loss of the codec, N represents the number of sentences in the generated movie story segments, M represents the number of movie characters whose behaviors are predicted by the behavior predictor, and K represents the number of movie story segments generated. C represents the movie character vector after training by incorporating the first adaptive feature into the j-th movie character. a The superscript j represents the j-th movie character, and the subscript a represents the vector representation after incorporating the first adaptive feature; The representation of the j-th movie character action vector after training with the second adaptive feature. The superscript i indicates that the current sentence S is generated. i The action performed at that time, with the subscript 'a' indicating the vector representation after incorporating the second adaptive feature; S i Let z represent the sentence of the movie story text generated by the target, and h represent the latent variables of the CVAE model. i S represents a complete output of the decoder. i-1 Sentences that represent the already generated movie story text.

[0131] In its implementation, a combination of automated evaluation metrics and manual screening is used to select high-quality film story texts. Automated evaluation metrics include BLUE and ROUGE metrics, while manual screening employs Intrinsic evaluation and Inter-evaluator Agreement (IA) methods. The Intrinsic evaluation method compares the attributes of the generated film story text with those in the knowledge base, including fluency, internal coherence, and accuracy. The IAA is designed to allow different evaluators to assess the degree of consensus among them.

[0132] Finally, the newly generated high-quality film story text will be stored in a semantic knowledge base, providing more material for generating new story texts later.

[0133] This invention also discloses a film story generation system that integrates film character features, comprising:

[0134] Movie Story Resource Library Building Module: Used to build a movie story resource library;

[0135] First encoder module: Based on the generated movie story segments or sentences as the context information of the currently generated movie story sentence, the first encoder is used to extract the semantic information of the context.

[0136] Behavior prediction module: Based on the movie title, the vector representation of the movie characters, the verb list of the movie characters, and the encoded context information, the first adaptive feature is incorporated, and the behavior predictor is used to predict the actions of the movie characters.

[0137] The second encoder module combines the movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions, and incorporates a second adaptive feature, using a second encoder to further encode the preceding information;

[0138] Self-attention module: Based on the information encoded by the second encoder, combined with self-attention, the CVAE model is used to improve the quality of movie story generation.

[0139] Specific limitations regarding the film story generation system that integrates film character features can be found in the limitations of the film story generation method that integrates film character features described above, and will not be repeated here. Each module in the aforementioned film story generation system that integrates film character features can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0140] This invention also provides a computer device, which can be a server. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data on equivalent resistance models and equivalent sub-models, as well as equivalent resistance, operating resistance, and contact resistance obtained during calculations. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for generating a film story that integrates the characteristics of film characters.

[0141] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: constructing a movie story resource library; extracting semantic information of the preceding text using a first encoder, based on the generated movie story segments or sentences as the context information of the currently generated movie story sentence; using a behavior predictor to predict the actions of movie characters by incorporating a first adaptive feature based on the movie title, vector representations of movie characters, a list of movie character verbs, and the encoded context information; further encoding the context information using a second encoder by incorporating a second adaptive feature, combining the movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions; and improving the quality of movie story generation using a CVAE model based on the information encoded by the second encoder and combined with self-attention.

[0142] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps: constructing a movie story resource library; extracting semantic information of the preceding text using a first encoder, based on the generated movie story paragraphs or sentences as the context information for the currently generated movie story sentence; incorporating a first adaptive feature based on the movie title, vector representations of movie characters, a list of movie character verbs, and the encoded context information, and using a behavior predictor to predict the actions of movie characters; further encoding the context information using a second encoder by incorporating a second adaptive feature, combining the movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions; and improving the quality of movie story generation using a CVAE model based on the information encoded by the second encoder, combined with self-attention.

[0143] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0144] The methods of the present invention described above can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the processing methods described herein. Furthermore, when a general-purpose computer accesses the code used to implement the processing shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for performing the processing shown herein.

[0145] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.< / eov> < / eoc> < / eov> < / eoe> < / eop> < / eot> < / eoc> < / eoe> < / eoe> < / eoe> < / eos>

Claims

1. A method for generating film stories that integrates the characteristics of film characters, characterized in that, Includes the following steps: Step 101: Build a film story resource library; Step 101 includes the following sub-steps: S11: Retrieve movie title, movie story text, and movie character background information; S12: Based on the movie story text, obtain the movie time period and scene keywords; S13: Combine the movie title, movie story text and movie character background information to construct a list of movie character verbs and a set of movie characters, and initialize the vector representation of movie characters; Step 102: Using the generated movie story segments or sentences as the context information for the currently generated movie story sentence, the first encoder is used to extract the semantic information of the context information; Step 103: Based on the input movie title, obtain the movie characters related to the movie title as the core content of the story; Includes the following sub-steps: S31: Input a movie title, select movie characters from the movie character set in the movie story resource library, and obtain the vector representation of the corresponding movie characters; S32: The enhanced vector representation of the movie character is obtained by calculating the vector representation C of the movie character as follows. : ,in , For trainable weight matrix, , For trainable bias; based on the enhanced movie character vector representation Using a list of verbs from movie characters and encoded contextual information, a behavior predictor is used to predict the actions of movie characters. S104: Combining movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions, a second adaptive feature is incorporated, and a second encoder is used to further encode the preceding information; Includes the following sub-steps: S41: Combine the input movie title and corresponding movie characters to obtain the corresponding movie time period and scene words for each movie character; S42: Based on the input movie title, movie time period, movie scene words, movie character vector representations, and movie character motion vector representations, the movie character motion vector representations are used to obtain the enhanced character motion vector representations through the following calculations. : ,in , For trainable weight matrix, , For trainable bias; based on the input movie title, movie time period, movie scene words, movie character vector representations, and the enhanced character action vector representation. The preceding information is further encoded using a second encoder. Vector representations of verbs used by movie characters; Step 105: Based on the context information encoded by the second encoder, and combined with self-attention, the CVAE model is used to improve the quality of the generated movie story; By combining movie character features with CVAE, CVAE models the probability of generating target text given source text. It takes both source and target text as input and samples from the posterior distribution to participate in the training of the decoder. In addition, a conditional prior distribution is provided for sampling generated text during the testing phase. S106: Combining Pseudo self-attention with a third adaptive feature, the decoder calculates the final distribution of words and generates the movie story, including the following sub-steps: S61: Based on the vector representations of the corresponding movie characters, concatenate them to multiple time steps. The decoder has generated vector representations of the film's story together, and the decoder is constructed in time steps. The input is given by the decoder, which completes a full decoding operation for each sentence generated. A full decoding operation is divided into multiple time steps. , At each time step In the process, the decoder calculates the final distribution of words to obtain the generated content of the movie story; S62: The latent variable z is mapped through a two-layer feedforward network and combined with the input of the decoder to encode pseudo self-attention information; S63: Based on pseudo self-attention information, the decoder is used to calculate the final distribution of words; S64: The final distribution of words is used to generate the movie story.

2. The method for generating a film story by integrating film character features according to claim 1, characterized in that: Step 105 includes the following sub-steps: S51: Uses the output of the second encoder to encode self-attention information; S52: Encode information based on the self-attention mechanism and use the CVAE model to improve the quality of movie story generation.

3. A film story generation system integrating film character features, used to execute the film story generation method according to claim 1 or 2, characterized in that, include: Movie Story Resource Library Building Module: Used to build a movie story resource library; First encoder module: Based on the generated movie story segments or sentences as the context information of the currently generated movie story sentence, the first encoder is used to extract the semantic information of the context. Behavior prediction module: Based on the movie title, the vector representations of movie characters, the verb list of movie characters, and the encoded preceding context, the enhanced vector representation of the movie character C is obtained through the following calculation. : ,in , For trainable weight matrix, , For trainable bias; a behavior predictor is used to predict the actions of movie characters; The second encoder module combines the movie title, time period, scene words, vector representations of movie characters, and vector representations of character actions to obtain the enhanced character action vector representation through the following calculations. : ,in , For trainable weight matrix, , To provide a trainable bias, a second encoder is used to further encode the preceding information. Self-attention module: Based on the information encoded by the second encoder, it combines self-attention and uses the CVAE model to improve the quality of movie story generation; Movie story generation module: Combining Pseudo self-attention, the latent variable z is mapped through a two-layer feedforward network and combined with the input of the decoder. The decoder calculates the final distribution of words and generates the movie story.

4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the movie story generation method of claim 1 or 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the movie story generation method as described in claim 1 or 2.

Images