Method of processing script and apparatus thereof

Abstract

The present invention relates to a script processing method and apparatus. A script processing method according to one embodiment of the present invention comprises the steps of receiving a script from a script processing apparatus, performing preprocessing on the script on a scene-by-scene basis, generating a graph including nodes for scenes, nodes for actions, and nodes for characters based on the results of the preprocessing, performing learning on the graph, and obtaining scene embeddings based on the graph after the learning is performed. The step of performing learning on the graph may include the step of obtaining at least one metapath, and the step of performing learning on the graph based on the metapath and the initial embedding values ​​of the nodes for scenes, the initial embedding values ​​of the nodes for actions, and the initial embedding values ​​of the nodes for characters.

Description

Technology Field

[0001] The present invention relates to a script processing method and apparatus, and more particularly to a script processing method and apparatus for processing movie scripts. Background Technology

[0002] Artificial intelligence technology consists of machine learning (deep learning) and component technologies utilizing machine learning. Machine learning is an algorithmic technology that classifies and learns the characteristics of input data on its own, and component technologies are technologies that mimic the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms such as deep learning, and consist of technology fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control.

[0003] Because movie scripts consist of complex elements such as dialogue, actions, and scene descriptions, existing natural language processing models face limitations in analyzing them. Therefore, models are currently used by supervising them using movie-related information, such as genre, to analyze dialogue and scenes within the script. However, since this approach is trained on specific labels, it may have limitations when applied to general analysis. The problem to be solved

[0004] The objective of the present invention is to solve the aforementioned problems by providing a script processing method and apparatus capable of reflecting characters and their actions within a scene of a movie script. means of solving the problem

[0005] A script processing method according to one embodiment of the present invention comprises the steps of receiving a script from a script processing device according to one embodiment of the present invention, performing preprocessing on the script on a scene-by-scene basis, generating a graph including nodes for scenes, nodes for actions, and nodes for characters based on the results of the preprocessing, performing learning on the graph, and obtaining scene embeddings based on the graph after the learning is performed. The step of performing learning on the graph may include the step of obtaining at least one metapath, and the step of performing learning on the graph based on the metapath and the initial embedding values ​​of the nodes for scenes, the initial embedding values ​​of the nodes for actions, and the initial embedding values ​​of the nodes for characters.

[0006] The step of performing the above preprocessing may include the step of separating the script into scene units based on pre-set words and the step of extracting sentences corresponding to dialogue and sentences describing a character's actions from the script separated into scene units.

[0007] The above graph may further include scene-action edges and action-character edges.

[0008] The initial embedding value of the node for the above action may be a value obtained based on a sentence-transformer.

[0009] A script processing device according to one embodiment of the present invention may include a processor and a memory in which one or more instructions executed by the processor are stored. The one or more instructions may include: receiving a script from the script processing device; performing preprocessing on the script on a scene-by-scene basis; generating a graph including nodes for scenes, nodes for actions, and nodes for characters based on the results of the preprocessing; performing learning on the graph; and obtaining scene embeddings based on the graph in which the learning has been performed. The step of performing learning on the graph may include: obtaining at least one metapath; and performing learning on the graph based on the metapath and the initial embedding values ​​of the nodes for scenes, the initial embedding values ​​of the nodes for actions, and the initial embedding values ​​of the nodes for characters.

[0010] The step of performing the above preprocessing may include the step of separating the script into scene units based on pre-set words and the step of extracting sentences corresponding to dialogue and sentences describing a character's actions from the script separated into scene units.

[0011] The above graph may further include scene-action edges and action-character edges.

[0012] The initial embedding value of the node for the above action may be a value obtained based on a sentence-transformer. Effects of the invention

[0013] According to the present invention, it is possible to generate an embedding for a scene that reflects information about a character and his actions in a scene of a movie script. Brief explanation of the drawing

[0014] To further understand the present invention, the attached drawings, which are incorporated into and constitute part of the present application, illustrate embodiments of the present invention along with a detailed description explaining the principles of the present invention. FIG. 1 is a block diagram of an image processing system according to one embodiment of the present invention. FIG. 2 is a conceptual diagram illustrating a script according to one embodiment of the present invention. FIG. 3 is a conceptual diagram illustrating a preprocessing process according to one embodiment of the present invention. FIG. 4 is a conceptual diagram illustrating a graph according to one embodiment of the present invention. FIG. 5 is a conceptual diagram for explaining the learning of a graph according to one embodiment of the present invention. FIG. 6 is a flowchart of a script processing method according to one embodiment of the present invention. FIGS. 7 to 9 are conceptual diagrams illustrating the effects of a script processing method according to an embodiment of the present invention. FIG. 10 is a block diagram of a script processing device according to another embodiment of the present invention. Specific details for implementing the invention

[0015] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols will be assigned the same reference number, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not inherently possess distinct meanings or roles. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the concept and technical scope of this specification.

[0016] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0017] When it is stated that one component is "electrically connected" or "connected" to another component, it should be understood that it may be directly electrically connected or connected to that other component, or that there may be other components in between. Conversely, when it is stated that one component is "directly electrically connected" or "directly connected" to another component, it should be understood that there are no other components in between. A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0018] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0019] FIG. 1 is a block diagram of an image processing system according to one embodiment of the present invention.

[0020] Referring to FIG. 1, a script processing system (10) according to one embodiment of the present invention may include a script acquisition device (100) and a script processing device (200).

[0021] The script acquisition device (100) can acquire a script from a video. For example, the video may be a movie, and the script may correspond to a scene in the movie. For example, the script acquisition device (100) can acquire a script such as that shown in FIG. 2. The script acquisition device (100) can transmit the script to a script processing device (200).

[0022] The script processing device (200) can receive the script from the script acquisition device (100). The script processing device (200) can perform preprocessing on the script. The script processing device (200) can perform preprocessing on the script on a scene-by-scene basis. For example, the script processing device (200) can separate the script on a scene-by-scene basis based on words such as INT or EXT, and can perform preprocessing on the script by extracting sentences corresponding to dialogue and sentences describing the actions of a character from the script separated on a scene-by-scene basis. For example, the script processing device (200) can perform preprocessing on the script by extracting sentences describing the actions of a character and dialogue as shown in FIG. 3.

[0023] The script processing device (200) can generate a graph based on the results of preprocessing. The graph may include nodes for scenes, nodes for actions, and nodes for characters, and may include scene-action edges and action-character edges. Nodes for actions may include nodes for sentences describing character actions and nodes for dialogue. For example, the script processing device (200) can obtain a graph such as that of FIG. 4.

[0024] The script processing device (200) can perform learning on the graph. The script processing device (200) can perform learning on the graph based on Metapath2Vec. The script processing device (200) can set a metapath. The script processing device (200) can set a metapath such as Action-Scene-Action, Action-Character-Action, or Character-Action-Scene-Action-Character. The script processing device (200) can perform learning on the graph based on the metapath. The script processing device (200) can perform learning on the graph based on the initial embedding value of the node for Scene, the initial embedding value of the node for Action, and the initial embedding value of the node for Character. Here, the initial embedding value of the action node may be a sentence embedding value obtained through a Sentence-Transformer. The script processing device (200) can extract a path from a graph based on a metapath. The script processing device (200) can perform learning on the path based on Word2vec. Referring to FIG. 5, for example, if the script processing device (200) sets the metapath as Person-Action-Scene-Action-Person, the graph can extract a path such as julia (Person)-Act 3 (Action)-Scene 5 (Scene)-Act5 (Action)-kane (Person).

[0025] The script processing device (200) can obtain scene embeddings based on a graph on which learning has been performed. The script processing device (200) can input a preprocessed script into the graph on which learning has been performed, and can obtain embedding values ​​for the scene based thereon.

[0026] FIG. 6 is a flowchart of a script processing method according to one embodiment of the present invention.

[0027] Referring to FIG. 6, the script processing device can perform preprocessing on the script (S610). The script processing device (e.g., the script processing device (200) of FIG. 1) can receive the script from the script acquisition device (e.g., the script acquisition device (100) of FIG. 1). The script processing device can perform preprocessing on the script on a scene-by-scene basis.

[0028] The script processing device can generate a graph (S620). The script processing device can generate a graph based on the results of preprocessing. The graph may include nodes for scenes, nodes for actions, and nodes for characters. Additionally, the graph may include scene-action edges and action-character edges.

[0029] The script processing unit can perform learning on the graph (S630). The script processing unit can perform learning on the graph based on Metapath2vec. The script processing unit can set a metapath. The script processing unit can perform learning on the graph based on the metapath for the scene and the initial embedding values ​​of the nodes for the scene, the initial embedding values ​​of the nodes for the action, and the initial embedding values ​​of the nodes for the character. In this case, the initial embedding values ​​of the nodes for the action may be sentence embedding values ​​obtained through a sentence converter.

[0030] The script processing device can obtain an embedding value for the scene (S640). The script processing device can obtain an embedding value for the scene by inputting the script into the graph where training was performed in S630. Here, the script may have been preprocessed by S610.

[0031] FIGS. 7 to 9 are conceptual diagrams illustrating the effects of a script processing method according to an embodiment of the present invention.

[0032] In FIGS. 7 to 9, Doc2Vec may be a scene embedding obtained by inputting a script into Doc2vec on a scene-by-scene basis, the average embedding may be the average of embeddings obtained by inputting sentences corresponding to the scene into a sentence converter, the scene summary embedding may be a scene embedding obtained by inputting a scene into a summary model to obtain a summarized scene, and then inputting the summarized scene into a sentence converter, and Metapath2Vec may be a scene embedding obtained by a script processing method according to an embodiment of the present invention. Additionally, Longformer may be a scene embedding obtained by inputting a scene into Longformer.

[0033] Figure 7 is a major character identification test, and it may be a task to determine whether the characters appear when the scene embeddings are fed into a classification model by extracting characters 1, 2, and 3 that frequently appear in the movie script.

[0034] Referring to FIG. 7, it can be seen that in the case of a scene embedding obtained according to one embodiment of the present invention, Person 1, Person 2, and Person 3 appear more frequently compared to a scene embedding obtained by another method.

[0035] Figure 8 is a task that matches the sequentially connected scene embeddings when the sequentially connected and the reversely connected scene embeddings are each fed into a classification model.

[0036] Referring to FIG. 8, it can be seen that the scene embedding obtained according to one embodiment of the present invention is effective for scene arrangement compared to the scene embedding obtained by another method.

[0037] Figure 9 is a task that involves selecting four abstract keywords (twist ending, flashback, cult film, independent film) and using scene embeddings to determine whether the keywords exist in the film.

[0038] Referring to FIG. 9, it can be seen that in the case of a scene embedding obtained according to one embodiment of the present invention, the probability of matching abstract keywords increases compared to a scene embedding obtained by another method.

[0039] FIG. 10 is a block diagram of a script processing device according to another embodiment of the present invention.

[0040] The script processing device (300) of FIG. 10 may be identical to the script processing device (200) of FIG. 1. The script processing device (300) may include at least one processor (310), a memory (320), and a transceiver (330) that is connected to a network to perform communication. Additionally, the script processing device (300) may further include an input interface device (340), an output interface device (350), a storage device (360), etc. Each component included in the script processing device (300) may be connected by a bus (470) to perform communication with each other. However, each component included in the script processing device (300) may be connected through individual interfaces or individual buses centered around the processor (310), rather than through a common bus (370). For example, the processor (310) may be connected to at least one of a memory (320), a transmission / reception device (330), an input interface device (340), an output interface device (350), and a storage device (360) through a dedicated interface.

[0041] The processor (310) can execute a program command stored in at least one of the memory (320) and the storage device (360). The processor (310) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Each of the memory (320) and the storage device (360) may be composed of at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory (320) may be composed of at least one of read-only memory (ROM) and random access memory (RAM).

[0042] Most terms used in this invention are selected from those commonly used in the field, but some terms are chosen at the applicant's discretion, and their meanings are described in detail in the following description as necessary. Accordingly, this invention should be understood based on the intended meaning of the terms, rather than their mere names or meanings.

[0043] It is obvious to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential features of the invention. Accordingly, the foregoing detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

Claim 1 A script processing method performed by a processor, comprising: receiving a script from a script processing device in the processor; performing preprocessing on the script on a scene-by-scene basis in the processor; generating a graph including nodes for scenes, nodes for actions, and nodes for characters in the processor based on the results of the preprocessing in the processor; performing learning on the graph in the processor; and obtaining scene embeddings based on the graph in the processor, wherein the step of performing learning on the graph comprises: setting a metapath representing the relationship between the nodes for scenes, nodes for actions, and nodes for characters; extracting a path between nodes from the graph based on the metapath; and performing learning on the node embeddings of the graph based on the path and the initial embedding value of the node for the scene, the initial embedding value of the node for the action, and the initial embedding value of the node for characters. Claim 2 A script processing method according to claim 1, wherein the step of performing the preprocessing comprises: a step of separating the script into scene units based on a pre-set word; and a step of extracting sentences corresponding to a character's action and dialogue from the script separated into scene units. Claim 3 A script processing method according to claim 1, wherein the graph further includes scene-action edges and action-character edges. Claim 4 A script processing method according to claim 1, wherein the initial embedding value of the node for the above action is a value obtained based on a sentence-transformer. Claim 5 A script processing device comprising: a processor; and a memory connected to the processor and storing one or more commands performed by the processor, wherein the one or more commands include: receiving a script from a script processing device; performing preprocessing on the script on a scene-by-scene basis; generating a graph including nodes for scenes, nodes for actions, and nodes for characters based on the results of the preprocessing; performing learning on the graph; and obtaining scene embeddings based on the graph after the learning is performed, wherein the step of performing learning on the graph includes: setting a metapath representing the relationship between the nodes for scenes, the nodes for actions, and the nodes for characters; extracting a path between nodes from the graph based on the metapath; and performing learning on the node embeddings of the graph based on the path and the initial embedding value of the nodes for scenes, the initial embedding value of the nodes for actions, and the initial embedding value of the nodes for characters. Claim 6 A script processing method according to claim 5, wherein the step of performing the above preprocessing comprises: a step of separating the script into scene units based on a pre-set word; and a step of extracting sentences corresponding to a character's action and dialogue from the script separated into scene units. Claim 7 In paragraph 5, the above graph further includes scene-action edges and action-character edges, a script processing method. Claim 8 A script processing method according to claim 5, wherein the initial embedding value of the node for the above action is a value obtained based on a sentence-transformer.

Images