Short drama generation method and system based on knowledge graph

Abstract

The invention relates to the technical field of script generation, and discloses a short drama script generation method and system based on a knowledge graph, and the method comprises the steps: a user selects a plurality of short drama scripts as materials; generating a material knowledge graph; inputting the knowledge graph and the entity features into a generation model, outputting more than one initial drama script, separating a split script from the initial drama script, then inputting the split script into an image generator to generate a split script image, and selecting a part of the split script image by a user for marking; constructing an imaginary object, inputting features of the imaginary object into the reconstructed neural network, and outputting a reconstructed drama script by the reconstructed neural network; the method can get rid of accurate plot definitions, and can generate a short drama script meeting the tendency of a user.

Description

Technical Field

[0001] The present invention relates to the technical field of script generation, and more specifically, to a short play script generation method and system based on a knowledge graph. Background Art

[0002] With the rise of short video platforms, the demand for short play content has increased sharply. The traditional script creation method is difficult to meet the needs of mass content production. There is an urgent need for efficient and intelligent script generation technology. Accordingly, many widely used script generation tools have been designed and the functions of scene division and shot division have been realized. However, these tools generally rely on accurate plot and tendency qualifiers. A small number of qualifiers will lead to fuzzy results, and a large number of qualifiers will lead to plot conflicts in the results. In addition, users are generally not professional scriptwriters, and it is a relatively difficult task for users to input an appropriate number and accurate qualifiers, which requires multiple repeated attempts. Summary of the Invention

[0003] The present invention provides a short play script generation method and system based on a knowledge graph to solve the technical problems in the related art.

[0004] The present invention provides a short play script generation method based on a knowledge graph, including the following steps:

[0005] A user selects multiple short play scripts as materials;

[0006] Extract named entities according to the word segmentation content of the materials, and then construct entity relationships according to the relationships between the word segments corresponding to the named entities to generate a material knowledge graph; and encode the scenes or dialogues associated with the named entities to obtain entity features;

[0007] Input the knowledge graph and entity features into a generation model. The generation model includes a first hidden layer, a second hidden layer, a feature fusion layer, a third hidden layer, a fourth hidden layer and a first output layer. The first hidden layer and the second hidden layer respectively input the material knowledge graph and entity features, and then output a first hidden feature and a second hidden feature to the feature fusion layer respectively. The feature fusion layer fuses random noise with the first hidden feature and the second hidden feature to obtain a fusion feature and outputs it to the third hidden layer. The third hidden layer outputs a reconstruction matrix to the fourth hidden layer. The fourth hidden layer inputs the reconstruction matrix and entity features, and then outputs semantic features to the first output layer. The first output layer outputs one or more initial short play scripts, and the initial short play scripts include scene division and shot division marks;

[0008] Separate the shot script from the initial short play script, and then input the shot script into an image generator to generate shot script images. The user selects some of the shot script images for marking;

[0009] Index to the sub-scene script in the initial short play script according to the marked storyboard image;

[0010] Sort the sub-scene scripts in chronological order, construct fictional objects, where the fictional objects include: storyboard images and sub-scene scripts, and establish connections between fictional objects with associations. The associations of fictional objects include: two adjacent sub-scene scripts in sorting, the storyboard image and the sub-scene script where the storyboard it comes from is located;

[0011] Input the fictional object features into the reconstruction neural network, where the reconstruction neural network includes a pattern recognition layer, a tensor layer, and a second output layer. Among them, the pattern recognition layer inputs the fictional object features and outputs the object recognition features of each fictional object. The tensor layer splices the object recognition features of all fictional objects to obtain a spliced vector and outputs it to the second output layer to output the reconstructed short play script.

[0012] Furthermore, perform cleaning, word segmentation, and vectorization operations on the materials.

[0013] Furthermore, the calculation formula of the first hidden layer is as follows:

[0014] O 1 =SReLU ( SXW 1 ) W 2

[0015]

[0016] where X represents the input feature matrix, and one row vector of it represents an entity feature, and O 1 represents the first hidden feature; represents the sum of the adjacency matrix of the entities in the material knowledge graph and the identity matrix, represents 's degree matrix; W 1 and W 2 represent trainable weight parameters.

[0017] Furthermore, the calculation formula of the second hidden layer is as follows:

[0018] O 2 =SReLU(SXW 1 )W 3

[0019] O 2 represents the second hidden feature, and W 3 represents a trainable weight parameter;

[0020] It should be noted that the element in the i-th row and j-th column of the adjacency matrix indicates whether there is an entity relationship between the i-th and j-th entities. If there is, it is assigned a value of 1; otherwise, it is 0.

[0021] Furthermore, the calculation formula of the feature fusion layer is as follows:

[0022] Z = O 1 + O 2 * ε, ε ~ N(0, I)

[0023] where ε represents multi-dimensional noise, and I represents the identity matrix of the multi-dimensional standard Gaussian distribution.

[0024] Furthermore, the calculation formula of the third hidden layer is as follows:

[0025] U = σ(ZZ T )

[0026] U represents the reconstruction matrix, which has the same size as the adjacency matrix of the entities in the material knowledge graph, and σ represents the sigmoid function.

[0027] Furthermore, the calculation formula of the fourth hidden layer is as follows:

[0028] U 1 = ReLU(HXW 4 )

[0029]

[0030] where U 1 represents the semantic feature, represents the sum of U and the identity matrix, represents 's degree matrix, and W 4 represents the trainable parameter, and ReLU represents the activation function;

[0031] The calculation formula of the first output layer is as follows:

[0032] Y = σ ( W 5 U 1 )

[0033] where Y represents the short play script feature, and one row vector of it is mapped to a word in the word library, and W 5 represents the trainable parameter.

[0034] Furthermore, the calculation formula of the pattern recognition layer is as follows:

[0035]

[0036] where The object recognition feature representing the v-th fictional object in the l-th layer, N ( v ) Represents the set of fictional objects associated with the fictional object v, Represents the first weight parameter in the l-th layer, Is the attention weight of the fictional object u to the fictional object v, calculated by the following formula:

[0037]

[0038] a is a learnable attention vector, ∥ represents the vector concatenation operation, Represents the second weight parameter in the l-th layer, T represents the transpose;

[0039] When l = 1 where x v and x u respectively represent the object features of the fictional objects indexed by the v-th and u-th fictional objects;

[0040] The calculation formula of the tensor layer is as follows:

[0041]

[0042] where H represents the concatenated vector, Represents the object recognition feature of the v-th fictional object in the last layer, CONCAT represents vector concatenation, M all Represents the set of all fictional objects;

[0043] The calculation formula of the second output layer is as follows:

[0044] Y new = σ ( W 6 H)

[0045] where Y new Represents the reconstructed short play script feature, and one row vector is mapped to a word in the vocabulary, W 6 Represents the trainable parameter.

[0046] The present invention provides a short play script generation system based on a knowledge graph, which can be run on a computer entity and can execute the aforementioned short play script generation method based on a knowledge graph when it is run.

[0047] The present invention provides a computer storage medium for storing computer-readable instructions, which can execute the aforementioned short play script generation method based on a knowledge graph when the computer-readable instructions are read.

[0048] The beneficial effects of the present invention are as follows:

[0049] The present invention does not need to rely on the guidance of the user to summarize keywords by himself to be incorporated into the original semantic vector for the operation of generating a script. It only needs to map the script that the user is interested in to the semantic space to obtain the corresponding semantic vector, and then randomly perturb the vector in the semantic space to obtain the initial script. Then, visual images are generated based on the initial script to guide the user to make further selections. According to the results of user interaction, the information of the semantic and image modalities is mixed to reconstruct the initial short play script again, which can get rid of the precise plot qualifiers and generate a short play script that meets the user's preferences. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flowchart of a method for generating a short play script based on a knowledge graph according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] Now, the subject matter described herein will be discussed with reference to example embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein. Without departing from the scope of protection of the content of this specification, changes can be made to the functions and arrangements of the elements discussed. Each example can omit, substitute, or add various processes or components as needed. Additionally, the features described in some examples can also be combined in other examples.

[0052] In at least one embodiment of the present invention, a method for generating a short play script based on a knowledge graph is disclosed. As Figure 1 shown, it includes the following steps:

[0053] Step 101, the user selects multiple short play scripts as materials, and then encodes the materials to perform preprocessing operations such as cleaning, word segmentation, and vectorization on the materials

[0054] Step 102, extract named entities according to the word segmentation content of the materials, and then construct entity relationships according to the relationships between the word segments corresponding to the named entities to generate a material knowledge graph;

[0055] And encode the scenes or dialogues associated with the named entities to obtain entity features;

[0056] Step 103: Input the knowledge graph and entity features into the generation model. The generation model includes a first hidden layer, a second hidden layer, a feature fusion layer, a third hidden layer, a fourth hidden layer, and a first output layer. The first hidden layer and the second hidden layer respectively input the material knowledge graph and entity features, and then output the first hidden feature and the second hidden feature to the feature fusion layer. The feature fusion layer fuses random noise with the first hidden feature and the second hidden feature to obtain a fused feature and outputs it to the third hidden layer. The third hidden layer outputs a reconstruction matrix to the fourth hidden layer. The fourth hidden layer inputs the reconstruction matrix and entity features, and then outputs semantic features to the first output layer. The first output layer outputs one or more initial short play scripts, and the initial short play scripts contain sub-scenes and shot marks.

[0057] In an embodiment of the present invention, the calculation formula of the first hidden layer is as follows:

[0058] O 1 =SReLU(SXW 1 )W 2

[0059]

[0060] where X represents the input feature matrix, and one row vector thereof represents one entity feature, and O 1 represents the first hidden feature; represents the sum of the adjacency matrix of the entities of the material knowledge graph and the identity matrix, represents 's degree matrix; W 1 and W 2 represent trainable weight parameters;

[0061] The calculation formula of the second hidden layer is as follows:

[0062] O 2 =SReLU ( SXW 1 ) W 3

[0063] O 2 represents the second hidden feature, and W 3 represents a trainable weight parameter;

[0064] It should be noted that the element in the i-th row and j-th column of the adjacency matrix indicates whether there is an entity relationship between the i-th and j-th entities. If there is, it is assigned 1, otherwise it is 0.

[0065] The calculation formula of the feature fusion layer is as follows:

[0066] Z=O 1 +O 2*ε, ε ~ N(0, I)

[0067] where ε represents multi-dimensional noise and I represents the identity matrix of the multi-dimensional standard Gaussian distribution.

[0068] The calculation formula of the third hidden layer is as follows:

[0069] U = σ ( ZZ T )

[0070] U represents the reconstruction matrix, whose size is the same as the adjacency matrix of the entities in the material knowledge graph, and σ represents the sigmoid function;

[0071] The calculation formula of the fourth hidden layer is as follows:

[0072] U 1 = ReLU ( HXW 4

[0073]

[0074] where U 1 represents the semantic feature, represents the sum of U and the identity matrix, represents the degree matrix of, W 4 represents the trainable parameter, and ReLU represents the activation function.

[0075] The calculation formula of the first output layer is as follows:

[0076] Y = σ ( W 5 U 1 )

[0077] where Y represents the short play script feature, and one row vector of it is mapped to a word in the vocabulary, and W 5 represents the trainable parameter.

[0078] In an embodiment of the present invention, the first output layer can also adopt a Transformer decoder.

[0079] Step 104: Separate the storyboard script from the initial short play script, then input the storyboard script into an image generator to generate storyboard script images, and display them to the user, and the user selects some of the storyboard script images for marking;

[0080] Step 105: Index to the sub-scene script in the initial short play script according to the marked storyboard script images;

[0081] Since the sub-scenario scripts obtained by indexing are only part of them and have lost the original context order, in order to make up for the loss of pattern recognition caused by the information fragmentation and loss due to the missing context, the following association rules are designed to reconstruct the context semantics:

[0082] Sort the sub-scenario scripts in chronological order, construct fictional objects, which include: storyboard images and sub-scenario scripts, and establish connections between fictional objects with associations. The associations of fictional objects include:

[0083] Two sub-scenario scripts are adjacent in sorting, the storyboard image and the sub-scenario script where the storyboard it comes from is located;

[0084] After that, encode the images or scripts corresponding to the fictional objects to obtain fictional object features;

[0085] Step 106, input the fictional object features into the reconstruction neural network. The reconstruction neural network includes a pattern recognition layer, a tensor layer, and a second output layer. Among them, the pattern recognition layer inputs the fictional object features and outputs the object recognition features of each fictional object. The tensor layer splices the object recognition features of all fictional objects to obtain a spliced vector and outputs it to the second output layer to output the reconstructed short play script.

[0086] In an embodiment of the present invention, the calculation formula of the pattern recognition layer is as follows:

[0087]

[0088] Where represents the object recognition feature of the vth fictional object in the lth layer, N ( v ) represents the set of fictional objects associated with the fictional object v, represents the first weight parameter of the lth layer, is the attention weight of the fictional object u to the fictional object v, and is calculated by the following formula:

[0089]

[0090] a is a learnable attention vector, ∥ represents the vector splicing operation, represents the second weight parameter of the lth layer, and T represents the transpose;

[0091] When l = 1 Where x v and x u respectively represent the fictional object features of the fictional objects indexed by the vth and uth fictional object indices.

[0092] The calculation formula of the tensor layer is as follows:

[0093]

[0094] Among them, H represents the concatenated vector, represents the object recognition feature of the v-th fictional object in the last layer, CONCAT represents vector concatenation, M all represents the set of all fictional objects.

[0095] The calculation formula of the second output layer is as follows:

[0096] Y new = σ ( W 6 H)

[0097] Among them, Y new represents the reconstructed short play script feature, and one row vector of it is mapped to a word in the thesaurus, W 6 represents the trainable parameter.

[0098] In at least one embodiment of the present invention, a short play script generation system based on a knowledge graph is provided, which can be run on a computer entity, and when it is run, it can execute the foregoing short play script generation method based on a knowledge graph.

[0099] In at least one embodiment of the present invention, a computer storage medium is provided, which is used to store computer-readable instructions, and when the computer-readable instructions are read, they can execute the foregoing short play script generation method based on a knowledge graph.

[0100] The embodiments of the present invention have been described above, but the embodiments are not limited to the above specific embodiments. The above specific embodiments are only illustrative and not restrictive. Under the inspiration of this embodiment, those of ordinary skill in the art can also make more equivalent embodiments in various forms, all of which fall within the protection scope of this embodiment.

Claims

1. A method for generating short play scripts based on knowledge graph, characterized in that: The following steps are involved: The user selects multiple short play scripts as materials; Extract named entities based on the word segmentation content of the material, and then construct entity relationships based on the relationships between the word segmentations corresponding to the named entities to generate a material knowledge graph; and encode the scenes or dialogues associated with the named entities to obtain entity features; The knowledge graph and entity features are input into the generation model, and the generation model includes a first hidden layer, a second hidden layer, a feature fusion layer, a third hidden layer, a fourth hidden layer and a first output layer, wherein the first hidden layer and the second hidden layer respectively input the material knowledge graph and entity features, and then respectively output the first hidden features and the second hidden features to the feature fusion layer, the feature fusion layer fuses random noise with the first hidden features and the second hidden features to obtain fusion features and outputs them to the third hidden layer, the third hidden layer outputs a reconstruction matrix to the fourth hidden layer, the fourth hidden layer inputs the reconstruction matrix and entity features, and then outputs semantic features to the first output layer, and the first output layer outputs more than one initial skit script, and the initial skit script contains scene and storyboard marks; Separating storyboards from the initial skit script, and then inputting the storyboards into an image generator to generate storyboard images, wherein a user selects part of the storyboard images for marking; Indexing the scene scripts in the initial short play script according to the marked storyboard script images; Sorting the sub-scene scripts in chronological order, constructing fictional objects, including: storyboard script images and sub-scene scripts, and establishing connections between related fictional objects, including: two sub-scene scripts are arranged adjacently, and the storyboard script image and the sub-scene script where the storyboard script from which it originated are located; The features of the fictional objects are input into a reconstruction neural network, which includes a pattern recognition layer, a tensor layer and a second output layer. The pattern recognition layer inputs the features of the fictional objects and outputs the object recognition features of each fictional object. The tensor layer concatenates the object recognition features of all fictional objects to obtain a concatenated vector and outputs it to the second output layer, which outputs the reconstructed skit script.

2. According to the method for generating short play scripts based on knowledge graph in claim 1, it is characterized in that: Clean, segment, and vectorize the materials.

3. A method for generating short play scripts based on knowledge graph according to claim 1, characterized in that: The calculation formula for the first hidden layer is as follows: O1=SReLU(SXW1_W2 Where X represents the input feature matrix, a row vector of which represents an entity feature, and O1 represents the first hidden feature; The sum of the adjacency matrix and the identity matrix representing the entity of the material knowledge graph, express The degree matrix of ; W1 and W2 represent the trainable weight parameters.

4. A method for generating short play scripts based on knowledge graph according to claim 3, characterized in that: The calculation formula for the second hidden layer is as follows: O2=SReLU(SXW1)W3 O2 represents the second hidden feature, and W3 represents the trainable weight parameter; It should be noted that the element in the i-th row and j-th column of the adjacency matrix indicates whether there is an entity relationship between the i-th and j-th entities. If so, the value is 1, otherwise it is 0.

5. A method for generating short play scripts based on knowledge graph according to claim 4, characterized in that: The calculation formula of the feature fusion layer is as follows: Z=O1+O2*ε,ε~N(0,I) Where ε represents multidimensional noise and I represents the identity matrix of the multidimensional standard Gaussian distribution.

6. A method for generating short play scripts based on knowledge graph according to claim 5, characterized in that: The calculation formula for the third hidden layer is as follows: U=σ(ZZ T ) U represents the reconstruction matrix, whose size is consistent with the adjacency matrix of the entity in the material knowledge graph, and σ represents the sigmoid function.

7. A method for generating short play scripts based on knowledge graph according to claim 6, characterized in that: The calculation formula for the fourth hidden layer is as follows: U1=ReLU(HXW4 Where U1 represents the semantic feature, represents the sum of U and the identity matrix, express The degree matrix, W4 represents the trainable parameters, and ReLU represents the activation function; The calculation formula of the first output layer is as follows: Y=σ(W5U1) Where Y represents the features of the skit script, a row vector of which is mapped to a word in the vocabulary, and W5 represents a trainable parameter.

8. The method for generating a short play script based on a knowledge graph according to claim 1, characterized in that: The calculation formula of the pattern recognition layer is as follows: in represents the object recognition feature of the vth fictional object in the lth layer, N(v) represents the set of fictional objects that are related to the fictional object v, represents the first weight parameter of the lth layer, is the attention weight of the imaginary object u to the imaginary object v, which is calculated by the following formula: a is a learnable attention vector, ∥ represents the vector concatenation operation, represents the second weight parameter of the lth layer, and T represents transposition; When l = 1 where x v and x u represent the imaginary object features of the vth and uth imaginary object indices respectively; The calculation formula of the tensor layer is as follows: Where H represents the concatenation vector, represents the object recognition feature of the vth imaginary object in the last layer, CONCAT represents vector concatenation, and M all represents the set of all imaginary objects; The calculation formula of the second output layer is as follows: AND new =σ(W6H) where Y new Represents the reconstructed skit script features, a row vector of which is mapped to a word in the vocabulary, and W6 represents a trainable parameter.

9. A short play script generation system based on knowledge graph, characterized in that: It can be run on a computer entity, and when it is run, it can execute a method for generating short play scripts based on a knowledge graph as described in any one of claims 1-8.

10. A computer storage medium, characterized in that: It is used to store computer-readable instructions, which, when read, can execute a method for generating short play scripts based on a knowledge graph as described in any one of claims 1-8.

Images