A method and system for automatically generating movie scripts and movie summaries

Abstract

This application provides a method and system for automatically generating movie scripts and movie summaries, relating to the field of computer technology. After acquiring a target movie video, it extracts a set of keyframes, speech-transcribed text, and character action recognition results. Visual feature vectors are then extracted from this set. Based on preset scene segmentation principles, a scene segmentation scheme with the lowest total bit cost is searched to divide the target movie video into multiple scenes. Target actor information images are extracted from video clips containing cast and crew information to generate a first correspondence and a second correspondence, thereby generating a target format script and scene list for each target scene. A tree diagram is generated based on the scenes, and after processing, a plot summary and database are obtained. Based on the target format script and database, a story outline is generated, and combined with the scene list, a movie script and movie summary are generated. This achieves the automatic generation of logically coherent, information-rich movie scripts and plot summaries that match the visual content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and system for automatically generating movie scripts and movie summaries. Background Technology

[0002] In the modern film industry, with the ever-increasing demand for high-quality content and intensifying market competition, the rapid and accurate generation of film scripts and summaries has become a crucial technological requirement. This not only helps improve production efficiency but also provides strong support for marketing teams, attracting more viewers through precise plot summaries. Especially in independent film production and online platform-produced series, the ability to efficiently utilize existing materials for creation is paramount, given limited time and resources.

[0003] Currently, a relatively advanced approach involves using deep learning algorithms to automatically analyze video content and extract key information to generate scripts and summaries. This primarily relies on computer vision technology to identify elements such as characters, scene transitions, and actions in the video, and combines this with natural language processing technology to convert this visual information into textual descriptions. However, existing solutions still have some limitations. For example, because they rely on pre-trained models, for some films with unique styles or a lot of special effects, the models may not be able to accurately identify the characters and scenes, resulting in inaccurate or even erroneous content. When dealing with complex and varied narrative structures, it is difficult to capture deep emotional threads and plot twists, making the generated summaries lack depth and coherence. Feature extraction based on a single dimension usually ignores the interaction of information between different modalities, limiting the quality of the final output. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for automatically generating movie scripts and movie summaries, in order to solve the problems in the existing technology that lead to inaccurate character and scene recognition, lack of depth and coherence in summaries, and limited output quality due to reliance on pre-trained models, single-modal extraction, and difficulty in handling complex narratives.

[0005] To address the aforementioned technical problems, in a first aspect, this application provides a method for automatically generating film scripts and film summaries, comprising:

[0006] The target movie video is acquired, and keyframe sets, speech-transcribed text, and character action recognition results are extracted from the target movie video. Multiple visual feature vectors are also extracted from the keyframe sets.

[0007] Based on the preset scene division principle and the visual feature vector, the scene segmentation scheme with the minimum total bit cost is determined to segment the target movie video into multiple scenes;

[0008] Based on the target movie video, a distance matrix is ​​constructed, and based on the distance matrix, a first correspondence between actor names and scenes and a second correspondence between actor names and speakers are generated;

[0009] Based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text and the character action recognition results, a target format script and scene list are generated for each target scene.

[0010] Based on the scenario, a tree diagram is generated, the tree diagram is processed to obtain a plot summary, and target movie video frames that match the plot summary are identified to form a database.

[0011] The character behavior elements and target plot elements corresponding to the second correspondence are extracted from the target format script and the database respectively to generate a story outline. Combined with the scene list, a movie script and a movie summary are generated.

[0012] Optionally, based on preset scene segmentation principles and the visual feature vectors, a scene segmentation scheme with the minimum total bit cost is determined, including:

[0013] Based on the preset scene division principle, the visual feature vector is divided into multiple vector segment combinations;

[0014] Calculate the probability density value of all visual feature vectors and the corresponding average feature vector within each vector segment in each vector segment combination, and calculate the total segmentation cost of each vector segment combination based on the probability density value.

[0015] Calculate the first number of bits required to encode all visual feature vectors within each vector segment in each vector segment combination and the second number of bits required to encode the corresponding average feature vector, and calculate the total bit cost for each vector segment combination;

[0016] Based on the second time sequence of the target movie video, the segmentation range of the vector segment combination is adjusted to form multiple adjusted vector segment combinations. The total bit cost of each adjusted vector segment combination is calculated, and the vector segment combination or adjusted vector segment combination with the minimum total bit cost is selected as the scene segmentation scheme.

[0017] Optionally, based on the target movie video, a distance matrix is ​​constructed, and based on the distance matrix, a first correspondence between actor names and scenes and a second correspondence between actor names and speakers are generated, including:

[0018] Multiple first actor information images and second actor information images in different scenes are extracted from the target movie video;

[0019] Facial feature information is extracted from the first actor information image to calculate the facial feature similarity between different first actor information images, and the first actor information images with facial feature similarity higher than a preset threshold are integrated into the target actor information image;

[0020] Based on the facial feature information in the target actor information image, calculate the feature difference value between any two target actor information images to construct a distance matrix;

[0021] Calculate the matching degree between each target actor information image and the second actor information image. Combined with the distance matrix, assign the actor name in the target actor information image with the highest matching degree to the corresponding scene to form a first correspondence between actor name and scene.

[0022] The speaker's voice features and text identifiers are extracted from the target movie video. Combined with the actor's name in the target actor information image and the distance matrix, a second correspondence between the actor's name and the speaker is determined.

[0023] Optionally, based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text, and the character action recognition results, a target format script and scene list corresponding to each target scene are generated, including:

[0024] The character position parameters are extracted from the keyframe set. Based on the temporal correlation between the scene and each keyframe in the keyframe set in the target movie video and the first correspondence, the target scene corresponding to each actor's name is determined.

[0025] Each actor's name is bound to the character's position parameters in the corresponding target scene to obtain the initial position information of each actor in the corresponding target scene;

[0026] Based on the first time sequence of the keyframes in the target movie video, and combined with the time interval and position parameter differences of the same actor in adjacent keyframes, the position change data of each actor between adjacent keyframes is calculated to update the initial position information of the corresponding actor, and the updated position information of each actor in the corresponding target scene is obtained.

[0027] Based on the preset script format, and combined with the actor's name, updated location information, the transcribed speech text, and the character action recognition results for each target scene, a target format script corresponding to each target scene is generated.

[0028] Based on the third time sequence of each target scene in the target movie video, arrange the target format scripts corresponding to all target scenes to form a scene list.

[0029] Secondly, this application provides a system for automatically generating film scripts and film summaries, including:

[0030] The acquisition module is used to acquire the target movie video, extract the key frame set, speech-transcribed text and character action recognition results from the target movie video, and extract multiple visual feature vectors from the key frame set.

[0031] The search module is used to determine the scene segmentation scheme with the minimum total bit cost based on the preset scene division principle and the visual feature vector, so as to segment the target movie video into multiple scenes.

[0032] The filtering module is used to extract an initial set of actor information images from video clips including the cast and crew list in the target movie video, filter target actor information images from the initial set of actor information images to construct a distance matrix, and generate a first correspondence between actor names and scenes and a second correspondence between actor names and speakers based on the distance matrix.

[0033] The generation module is used to generate a target format script and a scene list for each target scene based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text and the character action recognition results.

[0034] The identification module is used to generate a tree diagram based on the scene, process the tree diagram to obtain a plot summary, and identify target movie video frames that match the plot summary to form a database.

[0035] The extraction module is used to extract character behavior elements and target plot elements corresponding to the second correspondence from the target format script and the database, respectively, to generate a story outline, and combine it with the scene list to generate a movie script and a movie summary.

[0036] Thirdly, this application provides an electronic device, comprising:

[0037] Memory, used to store computer programs;

[0038] A processor, configured to execute the computer program to implement the steps of an automatic method for generating film scripts and film summaries as described in the first aspect above.

[0039] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can perform the steps of an automatic method for generating a movie script and movie summary as described in the first aspect above.

[0040] This application provides a method for automatically generating movie scripts and movie summaries. By acquiring target movie videos and extracting keyframe sets, speech-transcribed text, and character action recognition results, combined with visual feature vectors extracted from the keyframes, a scene segmentation scheme that minimizes the total bit cost is searched based on preset scene segmentation principles. This achieves semantic consistency segmentation of movie content along the timeline, improving the accuracy of scene boundaries. Subsequently, an initial set of actor information images is extracted from video clips including the cast and crew list. By filtering target actor information images and constructing a distance matrix, the stability and discriminative power of actor identification are enhanced. This results in the generation of a dual correspondence between actor names and scenes, and between actor names and speakers. This approach solves the problems of role confusion and speaker mismatch. By combining the correspondence, character position parameters, speech-transcribed text, and action recognition results, it generates target format scripts and scene lists for each scene, making the generated content more consistent in spatiotemporal logic, dialogue reconstruction, and behavioral description. Furthermore, by constructing a tree diagram and processing the generated plot summary, it identifies matching video frames to form a database, realizing the abstract expression of the narrative structure and the associated storage of key visual evidence. Finally, it extracts character behavior elements and target plot elements from the text script and database to generate a story outline, and outputs a movie script and summary by combining the scene list, improving the structural integrity, plot coherence, and information richness of the automatically generated content.

[0041] Furthermore, actor information images are extracted from video clips including the cast and crew list to form an initial set. Then, a target set is constructed by retaining images with complete facial regions and gender features that match the names, thus improving the reliability of the input data. On this basis, facial features are extracted and clustered based on similarity to form the largest clique subset. Combined with the distance matrix, high-confidence modeling of actor identities is achieved. By comparing the matching degree between actor images appearing in the scene and the largest clique subset, the attribution relationship between actor names and specific scenes is accurately established. At the same time, the speaker's voice features and text identifiers are fused for cross-modal association to determine the correspondence between actor names and speakers, thereby achieving accurate identity binding in complex multi-role interaction scenarios. This overcomes the problems of unstable role recognition, ambiguous scene attribution, and speaker matching errors caused by existing methods that rely on single-modal analysis. By introducing a structured image screening mechanism and distance matrix modeling based on the largest clique, the robustness of the system in the face of changes in lighting, pose differences, and occlusion is enhanced. At the same time, the multi-source information fusion strategy achieves dual alignment of actor identities in the spatiotemporal and semantic dimensions, providing a solid foundation for the logic and reconstruction of the subsequent script and summary. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart illustrating a method for automatically generating movie scripts and movie summaries provided in this application embodiment;

[0044] Figure 2 A schematic diagram illustrating a specific implementation of a method for automatically generating movie scripts and movie summaries provided in this application embodiment;

[0045] Figure 3 This is a schematic diagram of the structure of an automatic movie script and movie summary generation system provided in an embodiment of this application. Detailed Implementation

[0046] To address the problems of existing solutions relying on single-modal features when processing complex video content, leading to blurred scene boundaries, frequent misidentification of character identities, and difficulty in capturing non-linear plot development and emotional progression in narrative structure understanding, this application starts by acquiring the target movie video. First, it extracts a set of keyframes and then extracts visual feature vectors based on these keyframes. Using a pre-defined scene segmentation principle, it searches for a segmentation scheme that minimizes the total bit cost, achieving semantically consistent scene segmentation. Subsequently, it extracts an initial set of facial images from video clips including the cast and crew credits. By selecting the subset of the largest cluster of facial images and constructing a distance matrix, it establishes a dual correspondence between actor names and scenes, and between actor names and speakers. Combining this correspondence with character location information, it generates updated location information and, based on this, generates target format scripts for each scene, forming a structured scene list. Further, it constructs a tree diagram and processes the graph structure to generate a plot summary, identifies matching video frames to build a database, and finally extracts plot and character elements from the text script and database to generate a story outline. Combined with the scene list, it completes the generation of the movie script and summary, improving the accuracy of content generation and narrative completeness.

[0047] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The core of this application is to provide a method for automatically generating movie scripts and movie summaries, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes:

[0049] Step 101: Obtain the target movie video, extract the keyframe set, speech-transcribed text, and character action recognition results from the target movie video, and extract multiple visual feature vectors from the keyframe set.

[0050] In this step, the target movie video refers to the movie video file that is processed in the method of this invention. The visual feature vector refers to the vector form in which visually relevant data such as pixel distribution, color features, and contour features are extracted from each keyframe in the keyframe set.

[0051] In this embodiment, the target movie video is first acquired. Then, using computer vision technology, frames reflecting key content are extracted from the target movie video at fixed time intervals to form a keyframe set. Simultaneously, speech-to-text technology is used to convert all speech information in the target movie video into text, resulting in speech-transcribed text. Next, action recognition technology is used to identify the body movements and posture changes of characters in the target movie video, outputting the character action recognition results. Finally, visually relevant data such as pixel distribution, color features, and contour features are extracted from each keyframe in the keyframe set, and this data is converted into vector form to obtain multiple visual feature vectors.

[0052] Step 102: Based on the preset scene division principle and the visual feature vector, determine the scene segmentation scheme with the minimum total bit cost to segment the target movie video into multiple scenes.

[0053] In this step, the preset scene segmentation principle refers to the core principle used for scene segmentation. This principle requires that the visual features within a scene should have high consistency, and that the visual features between scenes should have significant differences. Total bit cost refers to a comprehensive metric used to evaluate the quality of a scene segmentation scheme. A scene segmentation scheme refers to a specific method for segmenting a target movie video into multiple scenes, obtained by searching for the combination of vector segments with the minimum total bit cost based on the preset scene segmentation principle and visual feature vectors.

[0054] In this embodiment of the application, step 102 specifically includes the following steps:

[0055] Step 201: Based on the preset scene division principle, the visual feature vector is divided into multiple vector segment combinations.

[0056] In this step, vector segment combination refers to multiple complete scene segmentation schemes formed by dividing visual feature vectors according to different segmentation granularities based on preset scene division principles. Each combination contains several continuous vector segments covering all visual feature vectors, and each vector segment corresponds to a candidate scene.

[0057] In this embodiment, the preset scene division principle adopts the visual feature consistency principle, that is, visual feature vectors with similar feature distribution patterns are grouped into the same vector segment. First, multiple segmentation granularities are determined, such as one vector segment for every 3 keyframes, one vector segment for every 6 keyframes, and one vector segment for every 9 keyframes; then, all visual feature vectors are continuously divided according to each segmentation granularity to obtain multiple sets of vector segments; all vector segments under the same segmentation granularity are sequentially associated according to the chronological order of their corresponding keyframes in the target movie video to form a complete set of vector segment combinations. Different segmentation granularities correspond to different vector segment combinations, ultimately resulting in multiple vector segment combinations.

[0058] Step 202: Calculate the probability density value of all visual feature vectors and the corresponding average feature vector within each vector segment in each vector segment combination, and calculate the total segmentation cost of each vector segment combination based on the probability density value.

[0059] In this step, the corresponding average feature vector refers to a vector obtained by summing the corresponding dimensional values ​​of all visual feature vectors within a single vector segment, and then dividing the sum of each dimension by the number of visual feature vectors within that segment. The probability density value refers to the density of the distribution of a single visual feature vector within a vector segment relative to the average feature vector of that segment. Segmentation cost is an indicator that measures the rationality of segmenting a single vector segment. Total segmentation cost is a comprehensive segmentation evaluation index corresponding to a set of vector segment combinations, obtained by summing the segmentation costs of all vector segments within that combination.

[0060] In this embodiment, the average feature vector of each vector segment in each vector segment combination is first calculated. This is done by summing the corresponding dimension values ​​of all visual feature vectors within the vector segment, and then dividing the sum of each dimension by the number of visual feature vectors within the vector segment to obtain the average feature vector of that vector segment. Next, the probability density values ​​of all visual feature vectors within the vector segment relative to the average feature vector are calculated: first, the Euclidean distance between each visual feature vector and the average feature vector is calculated using vector statistics, i.e., the difference between the corresponding dimension values ​​of the two vectors is calculated, the squares of each difference are summed, and then the sum is squared. The Euclidean distance is obtained; then, assuming that all visual feature vectors within a vector segment follow a normal distribution, the vectors with closer Euclidean distances are closer to the peak region of the normal distribution, and their corresponding probability density values ​​are higher. The probability density value of each vector relative to the average feature vector is calculated using the normal distribution probability density formula; finally, the segmentation cost of the corresponding vector segment is defined based on the probability density value. For example, the higher the probability density value, the lower the segmentation cost. The segmentation cost of each vector segment will be used to calculate the total bit cost of the vector segment combination in the subsequent calculation; the segmentation costs of all vector segments within the vector segment combination are summed to obtain the total segmentation cost of the vector segment combination.

[0061] Step 203: Calculate the first number of bits required to encode all visual feature vectors in each vector segment of each vector segment combination and the second number of bits required to encode the corresponding average feature vector, and calculate the total bit cost of each vector segment combination.

[0062] In this step, the first bit count refers to the number of bits required to encode all visual feature vectors within a vector segment using the data encoding method, reflecting the storage scale of the visual feature data within that vector segment. The second bit count refers to the number of bits required to encode the average feature vector of that vector segment using the same encoding method as the first bit count, reflecting the data storage scale of the average feature vector. The scene average feature vector refers to the global visual feature vector obtained by summing the values ​​of each dimension of all visual feature vectors corresponding to the target movie video and dividing by the total number of vectors. The total bit cost is a comprehensive indicator that measures the rationality of a set of vector segment combination segmentation and data storage efficiency, used to select the optimal scene segmentation scheme.

[0063] In this embodiment of the application, the first bit number and the second bit number of each vector segment within each vector segment combination are calculated. Specifically, a unified data encoding method is first determined. The calculation process of the first bit number is as follows: first, count the number of dimensions of a single visual feature vector within the vector segment, and then determine the number of binary encoding bits required for each dimension value. The encoding bit number of a single visual feature vector is the number of dimensions multiplied by the number of encoding bits for each dimension. Next, count the total number of visual feature vectors within the vector segment, and multiply the encoding bit number of a single vector by the total number of vectors. The result is the first bit number of the vector segment.

[0064] The calculation process for the second bit number is as follows: Since the same binary encoding method as the first bit number is used, the average feature vector of this vector segment has the same number of dimensions as a single visual feature vector. Each dimension value uses the same number of bits for binary encoding. The number of bits for encoding a single average feature vector is the number of dimensions multiplied by the number of bits for encoding each dimension. Since each vector segment corresponds to only one average feature vector, this number of bits is the second bit number of the vector segment. Then, based on the first bit number, the second bit number, and the total segmentation cost of all vector segments in each vector segment combination, the total bit cost is calculated in combination with the scene average feature vector: First, the similarity between the scene average feature vector and the average feature vector of each vector segment is calculated. The higher the similarity, the smaller the correction coefficient.

[0065] The total bit cost is obtained by calculating the sum of the total first bit count, the total second bit count, and the total segmentation cost, and then multiplying this sum by the correction coefficient. Here, the total first bit count refers to the sum of the first bit counts of all vector segments within the vector segment combination, the total second bit count refers to the sum of the second bit counts of all vector segments within the vector segment combination, the total segmentation cost refers to the sum of the segmentation costs of all vector segments within the vector segment combination, and the correction coefficient refers to the calibration parameter determined based on the similarity between the scene average feature vector and the average feature vector of each vector segment. It is used to correct the deviation in the calculation of the total bit cost. The total bit cost will serve as the core indicator for judging the quality of the scene segmentation scheme.

[0066] Step 204: According to the second time sequence of the target movie video, adjust the segmentation range of the vector segment combination to form multiple adjusted vector segment combinations, calculate the total bit cost of each adjusted vector segment combination, and select the vector segment combination or adjusted vector segment combination with the minimum total bit cost as the scene segmentation scheme.

[0067] In this step, the second temporal order refers to the chronological order of keyframes within the target movie video that is consistent with the first temporal order. The segmentation range refers to the starting and ending vector positions of a vector segment within an ordered set of visual feature vectors, reflecting the vector range included in the segment. Based on the initial segmentation granularity, the scene range can be optimized by adjusting the starting or ending positions. The adjusted vector segment combination refers to the complete scene segmentation scheme formed by re-associating the vector segments after adjusting their segmentation ranges.

[0068] In this embodiment, based on the second temporal order of the target movie video, the segmentation range of the vector segments in the vector segment combination is adjusted. For example, the starting vector position of a vector segment is moved forward or backward by one vector, or the ending vector position of the vector segment is adjusted. This adjustment forms multiple adjusted vector segment combinations. Then, the total bit cost of each adjusted vector segment combination is calculated. Finally, the total bit cost of all vector segment combinations is compared with that of the adjusted vector segment combinations, and the combination with the minimum total bit cost is selected as the scene segmentation scheme. This scheme can ensure that the segmented scene achieves optimal visual feature consistency and data encoding efficiency, providing a specific implementation path for segmenting the target movie video into multiple scenes.

[0069] This application's embodiments can avoid the subjective recognition bias of pre-trained models in terms of scene features. By judging the rationality of the scene through the consistency of the vectors themselves, the scene segmentation is made to better fit the visual feature rules of the video itself. It is especially suitable for films with unique styles or many special effects, and where the model is prone to misjudgment. The scene segmentation scheme achieves optimal visual feature consistency and data encoding efficiency, solving the problem of the lack of quantitative optimization standards in existing scene segmentation schemes. It provides a precise scene division basis for subsequent steps, avoiding deviations in subsequent character positioning and plot organization due to blurred scene boundaries. This improves the accuracy and efficiency of the overall film script and summary generation, meeting the needs of the modern film industry for efficient and accurate content processing.

[0070] Step 103: Extract an initial set of actor information images from the video clips including the cast and crew credits in the target movie video; filter out target actor information images from the initial set of actor information images to construct a distance matrix; and based on the distance matrix, generate a first correspondence between actor names and scenes and a second correspondence between actor names and speakers.

[0071] In this step, the initial actor information image set refers to the set of images containing actor facial images and corresponding actor names extracted from video clips including the cast and crew credits in the target movie video using computer vision technology. The target actor information image refers to the image selected from the initial actor information image set that has a complete facial region, matches the actor's gender with their name, and has a facial feature similarity higher than a preset threshold. The distance matrix is ​​a matrix constructed using the feature difference values ​​between any two target actor information images, based on the facial feature information in the target actor information images, and using these difference values ​​as elements. The first correspondence relationship refers to the relationship formed by calculating the matching degree between the target actor information image and actor information images in different scenes, combining it with the distance matrix, and assigning the actor's name corresponding to the target actor information image with the highest matching degree to the corresponding scene. The second correspondence relationship refers to the relationship determined by associating and comparing the speaker's voice features and text identifiers in the target movie video with the actor's name in the target actor information image, and combining it with the distance matrix.

[0072] In this embodiment of the application, step 103 specifically includes the following steps:

[0073] Step 301: Extract multiple first actor information images and second actor information images in different scenes from the target movie video.

[0074] In this step, the first actor information image refers to an image extracted from a video clip including the cast and crew credits of the target movie video, containing the actors' facial information and corresponding actor names. The second actor information image refers to an image extracted from video clips corresponding to each scene, containing the actors' facial information within that scene.

[0075] In this embodiment, a face detection algorithm is first used to locate the image region of the video segment including the cast and crew credits in the target movie video. Text information within this region is extracted using optical character recognition (OCR) technology, and face regions within the same region are detected simultaneously. The extracted face images are then associated with the corresponding actor names, ultimately forming multiple first actor information images containing actor face information and their corresponding names. Next, based on the segmented multiple scenes, face detection algorithms are used to extract the actor face images appearing in the video segments corresponding to each scene, resulting in second actor information images for different scenes.

[0076] Step 302: Extract facial feature information from the first actor information image to calculate the facial feature similarity between different first actor information images, and integrate the first actor information images with facial feature similarity higher than a preset threshold into a target actor information image.

[0077] In this step, facial feature information refers to high-dimensional vectors extracted from facial images using a facial feature extraction model, used to quantify the visual features of a face. Facial feature similarity refers to the cosine similarity between two facial feature vectors, reflecting the degree of similarity between two facial images. The preset threshold refers to a manually set similarity threshold used to determine whether two facial images belong to the same actor. The target actor information image refers to the image set formed by integrating multiple first actor information images whose facial feature similarity is higher than the preset threshold.

[0078] In this embodiment, facial feature information is extracted from each first actor information image. Specifically, the first actor information image is first preprocessed, adjusted to a preset model size, and pixel values ​​are normalized. Then, basic visual features such as image edge texture are extracted through shallow convolutional layers of the face feature extraction model. After key features are filtered by pooling layers, multiple sets of convolutional and pooling layers are used alternately to integrate the basic visual features into high-level semantic features that can distinguish faces. Finally, the features are mapped to a preset fixed dimension and standardized through a fully connected layer to obtain a fixed-dimensional facial feature vector. Next, the cosine similarity between the facial feature vectors corresponding to any two first actor information images is calculated by dividing the dot product of the facial feature vectors by the product of the magnitudes of the two facial feature vectors. If the cosine similarity between two first actor information images is higher than a preset threshold, the two images are grouped together. After traversing all first actor information images, the first actor information images in each group whose facial feature similarity is higher than the preset threshold are integrated to obtain the target actor information image.

[0079] Step 303: Based on the facial feature information in the target actor information image, calculate the feature difference value between any two target actor information images to construct a distance matrix.

[0080] In this step, the feature difference value is the value obtained by subtracting the facial feature similarity from 1, used to construct the distance matrix. The smaller the value, the more similar the two images are. The distance matrix is ​​a square matrix constructed with the feature difference values ​​between the target actor's information images as elements, used to characterize the degree of similarity between different target actor's information images.

[0081] In this embodiment, a distance matrix is ​​constructed based on facial feature information in the target actor's information images. Specifically, for each target actor's information image, the feature difference value of the corresponding feature vector of that image and all other target actor's information images is calculated. Then, these feature difference values ​​are arranged in rows and columns according to the order of the target actor's information images, forming a square matrix with the same dimension as the number of target actor's information images. If there are N target actor's information images, the matrix is ​​an N×N square matrix, and each element in the square matrix represents the feature difference value between two corresponding target actor's information images.

[0082] Step 304: Calculate the matching degree between each target actor information image and the second actor information image. Combined with the distance matrix, assign the actor name in the target actor information image with the highest matching degree to the corresponding scene to form the first correspondence between actor name and scene.

[0083] In this step, the matching degree refers to the overall similarity between the second actor's information image and the target actor's information image. The first correspondence refers to the association between the actor's name and the scene in which the actor appears.

[0084] In this embodiment, features are first extracted from each second actor's information image and converted into feature vectors, following the same extraction process as for the first actor's information image. Next, the cosine similarity between the feature vector of each second actor's information image and the feature vectors of each target actor's information image is calculated. Then, the cosine similarity is weighted using a distance matrix to obtain the matching degree between the second actor's information image and each target actor's information image. Finally, the actor's name corresponding to the target actor's information image with the highest matching degree is assigned to the scene to which the current second actor's information image belongs, thus forming the first correspondence between actor names and scenes.

[0085] Step 305: Extract the speaker's voice features and text identifiers from the target movie video, and combine them with the actor's name in the target actor information image and the distance matrix to determine the second correspondence between the actor's name and the speaker.

[0086] In this step, speaker voice features refer to acoustic features such as voiceprint and intonation extracted from the target film video audio. Text identifiers refer to the text in the subtitles used to identify the speaker. The second correspondence refers to the association between the actor's name and the corresponding speaker's identity.

[0087] In this embodiment, the speaker's speech features are first extracted from the target movie video, and text identifiers in the video are obtained using speech-to-text technology. The extracted speaker speech features are then converted into feature vectors. Specifically, the speech signal is first segmented into short frames at fixed time intervals, and the acoustic features of each frame are extracted. These acoustic features are then processed layer by layer using a speech feature extraction model, integrating them into high-level features that characterize the speaker's voiceprint. Finally, these features are mapped to a preset fixed dimension and standardized to obtain the speaker's speech feature vector. The cosine similarity between this feature vector and the feature vectors of each target actor's information image is calculated. Simultaneously, the role information in the text identifier is compared with the actor's name corresponding to each target actor's information image. Combining the distance matrix, the target actor's information image with the highest cosine similarity and whose text identifier matches the actor's name is selected. The actor's name corresponding to this target actor's information image is determined as the current speaker's identifier, thus forming a second correspondence between actor's name and speaker.

[0088] The embodiments of this application can avoid misjudgment of a single modality in complex scenes; ensure that different images of the same actor are included in the same subset to avoid missed detection due to changes in angle; reduce confusion caused by the same name but different genders through dual verification of name and gender; and bind the speaker in the voice to the actor's identity in the scene to ensure the consistency of dialogue and character actions when the script is generated.

[0089] Step 104: Based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text, and the character action recognition results, generate a target format script and scene list corresponding to each target scene.

[0090] In this step, the target format script refers to the standardized text script formed by integrating the actor names, updated location information, speech-transcribed text, and character motion recognition results corresponding to each target scene according to a preset script format. The scene list refers to the ordered set formed by arranging all target format scripts in the target movie video according to the chronological order of the target scenes.

[0091] In the embodiments of this application, such as Figure 2 As shown, step 104 specifically includes the following steps:

[0092] Step 401: Extract the character position parameters from the keyframe set, and determine the target scene corresponding to each actor's name based on the temporal correlation between the scene and each keyframe in the keyframe set in the target movie video and the first correspondence relationship.

[0093] In this step, the temporal relationship refers to the correspondence between a scene and a keyframe established through timestamps; the target scene refers to the actor's activity scene unit determined by combining the temporal relationship and the first correspondence.

[0094] In this embodiment, the outline of the character in the key frame is located by the target detection algorithm, and the horizontal and vertical coordinates of the outline center are read as the character position parameters; the start and end timestamps of each scene are obtained, and it is determined whether the key frame timestamp falls within the scene time interval to establish a time association relationship; combined with the first correspondence relationship, each actor's name is bound to the corresponding scene to determine its target scene.

[0095] Step 402: Bind each actor's name to the character position parameters in the corresponding target scene to obtain the initial position information of each actor in the corresponding target scene.

[0096] In this step, the initial location information refers to the original location record formed after the actor's name is bound to the character's location parameters in the corresponding target scene.

[0097] In this embodiment, for each actor in a determined target scene, their name is matched with the character position parameters of all keyframes in that scene, and the position parameters are arranged into a position parameter list according to the time order of the keyframes, clarifying the original position of each time node and forming initial position information, laying the foundation for subsequent position optimization.

[0098] Step 403: Based on the first time sequence of the keyframes in the target movie video, and combined with the time interval and position parameter differences of the same actor in adjacent keyframes, calculate the position change data of each actor between adjacent keyframes to update the initial position information of the corresponding actor, and obtain the updated position information of each actor in the corresponding target scene.

[0099] In this step, the first time sequence refers to the order in which keyframes are arranged on the target movie video timeline according to their playback order. Position parameter difference refers to the numerical difference in position parameters for the same actor in adjacent keyframes. Position change data refers to the actor's movement information calculated by combining the time interval between adjacent keyframes and the position parameter difference. Updated position information refers to the precise position record obtained after correcting the initial position information using position change data, reflecting a more coherent positional trajectory of the actor within the target scene.

[0100] In this embodiment, keyframes are sorted by timestamp to obtain a first time sequence. The time interval between adjacent keyframes and the difference in position parameters of the same actor are calculated to obtain position change data. Based on this data, the initial position information is corrected, the position gaps between keyframes are filled, and the updated position information is formed.

[0101] Step 404: Based on the preset script format, and combining the actor's name, updated location information, the transcribed speech text, and the character action recognition results for each target scene, generate the target format script corresponding to each target scene.

[0102] In this step, the preset script format refers to the pre-defined text script structure specification, which includes fixed field types and arrangement rules.

[0103] In this embodiment of the application, step 404 specifically includes the following steps:

[0104] Step 411: Set the first script format.

[0105] In this step, the fixed fields of the first script format include scene number field, scene duration field, actor list field, actor position description field, actor action description field, and dialogue content field. Each field is used to record scene identifier, duration, actors appearing, actor positions, actions, and dialogue information to ensure that the script format is uniform and standardized.

[0106] Step 412: Based on the first correspondence, arrange all the actor names in each target scene to form an actor list.

[0107] In this step, the actor list refers to the sequence of all actor names in the target scene arranged according to preset rules.

[0108] In this embodiment of the application, all actor names in the current scene are filtered out according to the first correspondence relationship, and sorted according to the time order of the actors' first appearance in the scene to form an ordered actor list, ensuring that the order of the list matches the appearance logic of the actors in the scene.

[0109] Step 413: Use the identification information of each target scene as the scene number, use the time span of the keyframes corresponding to each target scene in the target movie video as the scene duration, and fill in the scene number, scene duration, and actor list of each target scene into the first script format to obtain the second format script corresponding to each target scene.

[0110] In this step, the scene number serves as the unique identifier for the scene, and the scene duration is the time span of the corresponding keyframes. The second format script is the initial draft of the script to complete the basic field filling.

[0111] In this embodiment of the application, the unique identifier of the scene is used as the scene number, and the scene duration is obtained by subtracting the earliest keyframe timestamp from the latest keyframe timestamp in the scene. The three are then filled into the corresponding fields of the first script format to form a second format script containing basic scene information.

[0112] Step 414: Convert the updated position information of all actors in the corresponding target scene into position descriptions, and fill the position descriptions into the second format script to obtain the third format script.

[0113] In this step, the location description refers to mapping the updated location information in coordinate form to a natural language expression. The third-format script is the script version supplemented with actor location information.

[0114] In this embodiment of the application, the coordinate values ​​of the updated position information are converted into natural language according to the area division rules of the left / center / right side of the screen. For example, coordinates 100,200 correspond to the left area. The position description of each actor is matched with their name in the second format script and filled into the position description field to form the third format script.

[0115] Step 415: Extract the action records of all actors in the corresponding target scene from the character action recognition results, fill the action records into the third format script, and obtain the fourth format script.

[0116] In this step, motion recording refers to the specific actions performed by the actor within the target scene. The fourth-format script is a version of the script supplemented with actor motion information.

[0117] In this embodiment of the application, actions whose timestamps fall within the current scene interval in the character action recognition results are selected as action records; the action records of each actor are matched with their names in the third format script and filled into the action description field to form a fourth format script containing action information.

[0118] Step 416: Based on the second correspondence, determine the target speaker identifier corresponding to the actor in each target scene, and filter out the text content associated with the target speaker identifier and the target keyframe duration from the speech transcription text, and fill the text content into the fourth format script to obtain the target format script corresponding to each target scene.

[0119] In this step, the target speaker identifier is a speaker tag associated with the actor's name; the text content is the dialogue text that matches the speaker identifier and the scene time. In this embodiment, the target speaker identifier corresponding to the actor is determined based on the second correspondence, the text content in the speech transcription text that matches the speaker identifier and the timestamp within the scene interval is filtered, associated with the corresponding actor's name in the fourth format script, and filled into the dialogue content field to form a complete target format script.

[0120] Step 405: Arrange the target format scripts corresponding to all target scenes according to the third time sequence of each target scene in the target movie video to form a scene list.

[0121] In this step, the third time sequence refers to the order in which the target scenes are arranged on the target movie video timeline according to the playback order.

[0122] In this embodiment, the start timestamp of each target scene is obtained, and all target scenes are sorted according to a third time order. The target format scripts corresponding to each scene are arranged in this order to form an ordered script set, i.e., a scene list, ensuring that the time sequence of the list is consistent with the movie playback logic.

[0123] This application's embodiments ensure accurate and reliable correspondence between actors and scenes through a time-related and actor-matching mechanism, avoiding confusion regarding scene attribution. Updated location information more realistically reflects the actors' movement trajectories within the scene, improving the coherence of location descriptions and achieving a comprehensive presentation of scene content. The chronologically arranged scene list guarantees the logic and completeness of the script's narrative, solving the problems of low efficiency and fragmented information inherent in traditional manual annotation.

[0124] Step 105: Based on the scene, generate a tree diagram, process the tree diagram to obtain a plot summary, and identify the target movie video frames that match the plot summary to form a database.

[0125] In this step, the tree diagram refers to a tree structure with the target film's storyline as the root node, dividing scenes into first-level child nodes based on their temporal relationships, and individual scenes as second-level child nodes, with each second-level child node bound to a description of the target content. The plot summary refers to the core narrative threads of the film obtained by integrating the core content of each scene group in the tree diagram. The database refers to a structured data set formed by integrating the plot summary, plot segments, and corresponding matching video frames, providing support for subsequent script and summary generation.

[0126] In this embodiment of the application, step 105 specifically includes the following steps:

[0127] Step 501: Identify scene environment information and character interaction behavior from the keyframes corresponding to each scene to generate a target content description for each scene. Determine the temporal relationship between different scenes based on the fourth time sequence of each scene in the target movie video.

[0128] In this step, scene environment information refers to environmental details such as objects, lighting, and spatial attributes identified from keyframes. Character interaction behavior refers to the identified dynamic relationships such as character dialogue and action interactions; target content description refers to the core text content of the scene formed by integrating the two. The fourth temporal order refers to the order in which scenes are sorted according to their starting timestamps. Temporal correlation refers to the temporal connection between scenes.

[0129] In this embodiment, computer vision technology is used to identify environmental information and character interaction behaviors in keyframes, and then integrates these to generate a target content description. For example, one generated target content description could be an outdoor street scene where actor A and actor B are arguing. The start timestamps of each scene are obtained to determine the fourth time sequence, and the sequential connection between scenes is judged accordingly, providing a foundation for subsequent tree diagram construction.

[0130] Step 502: Using the target movie storyline as the root node of the tree diagram, divide the multiple scenes into multiple scene groups according to the chronological relationship, and construct a tree diagram with each scene group as a first-level child node under the root node and each scene in each scene group as a second-level child node under the corresponding first-level child node. Each second-level child node is bound to the target content description of the corresponding scene.

[0131] In this step, the target film storyline refers to the summarized core narrative threads of the film. Scene groups refer to sets of scenes divided according to plot development stages; first-level sub-nodes correspond to scene groups, and second-level sub-nodes correspond to individual scenes.

[0132] In this embodiment, the core narrative clues are first summarized based on all target content descriptions as the root node; then, scene groups are divided according to temporal relationships and plot stages as first-level child nodes; and each scene is treated as a second-level child node of the corresponding first-level child node, and its target content description is bound to it to construct a tree diagram.

[0133] Step 503: Integrate the target content descriptions with a repetition rate higher than a preset repetition threshold in all second-level sub-nodes under each first-level sub-node in the tree diagram into a target plot description, integrate the target plot descriptions of all first-level sub-nodes, and generate a plot summary.

[0134] In this step, "repetition rate" refers to the percentage of keyword overlap in the target content description. The preset repetition threshold is the critical value used to determine whether to integrate the content. The target plot description refers to the core plot text obtained by integrating highly repetitive content within the scene group.

[0135] In this embodiment, keywords of the target content description within each scene group are extracted and the repetition rate is calculated; content with a repetition rate higher than a preset threshold is merged and deduplicated to generate the target plot description for each scene group; all target plot descriptions are integrated in the fourth time sequence to form a plot summary reflecting the complete plot development of the movie.

[0136] Step 504: Divide the plot summary into multiple plot segments, match each plot segment with a corresponding target scene, and extract target movie video frames from the target scene to form a set of matching frames for the corresponding plot segment.

[0137] In this step, plot fragments refer to independent plot units broken down according to plot turning points; matching target scenes refer to scenes that have the highest degree of matching with the core information of plot fragments; and matching frame sets refer to the set of video frames selected that are consistent with the plot description.

[0138] In this embodiment of the application, the plot summary is split into plot segments according to the plot turning point. For example, one splitting method is to split the plot summary according to the changes in character relationships and the initiation of events. The plot segments are compared with the target content descriptions of each scene to determine the matching target scenes. Video frames are extracted from the matching target scenes, and frames whose in-frame environment and character behavior are consistent with the plot description are selected to form a set of matching frames for each plot segment.

[0139] Step 505: Construct a database based on the plot summary, each plot segment, and the set of matching frames corresponding to each plot segment.

[0140] In this step, the database is a structured storage medium that contains the association between plot and visual materials.

[0141] In this embodiment, three storage modules are constructed, including plot summary, plot fragments, and matching frame set; the plot summary and plot fragments sorted by time are stored in the corresponding modules respectively; each plot fragment is associated with the matching frame set and then stored, and a numbered index is established to achieve fast query and complete the database construction.

[0142] This application's embodiments can clearly outline the narrative structure of a film, avoiding plot confusion caused by fragmented scenes, and are especially suitable for films with complex narratives; through keyword comparison and frame information matching, it achieves accurate association between plot and visual materials, and the constructed database provides two-way support for subsequent content generation; without relying on the subjective judgment of pre-trained models, it improves the objectivity and accuracy of plot organization, laying a data foundation for the efficient generation of high-quality film scripts and summaries.

[0143] Step 106: Extract the character behavior elements and target plot elements corresponding to the second correspondence from the target format script and the database respectively to generate a story outline, and combine it with the scene list to generate a movie script and movie summary.

[0144] In this step, character behavior elements refer to elements formed by integrating the action states and positional information of each actor in the corresponding scene according to the second correspondence. Target plot elements refer to event information directly related to the actors, selected from the plot development elements in the database. The story outline refers to the narrative framework formed by integrating multiple types of elements according to scene chronology. The film script and film summary are generated based on the story outline, scene list, and database information, respectively presenting complete scene content and core plot lines.

[0145] In this embodiment of the application, step 106 specifically includes the following steps:

[0146] Step 601: Classify the scene number, actor name, updated position information, and character action status corresponding to each scene in the target format script as scene basic elements, and classify the event development information of each plot segment in the database as plot development elements.

[0147] In this step, the scene basic elements refer to the set of core basic information extracted and categorized scene by scene from the target format script. These elements include scene number, actor name, updated location information, and character action status, providing a basic framework for story outline construction. The plot development elements refer to the set formed by summarizing and categorizing the event development information of each plot segment in the database. These elements include descriptions of the event's beginning, development, and outcome, supporting the plot logic of the story outline.

[0148] In this embodiment, scene numbers, actor names, updated location information, and character action states are extracted from the target format script scene by scene, and categorized by scene to form basic scene elements. Simultaneously, plot fragments are retrieved from the database, and descriptions of the beginning, development, and outcome of events in each fragment are extracted, summarized, and categorized to obtain plot development elements. Both serve as core materials for generating the story outline.

[0149] Step 602: Based on the second correspondence, associate the actor's name in the scene basic elements with the target speaker identifier, determine the action state and position information of the actor corresponding to each target speaker identifier in the corresponding scene, integrate the action state and position information into character behavior elements, and select actor-related event information as target plot elements from the plot development elements.

[0150] In this step, "action state" refers to the specific action descriptions of the actors extracted from the target format script, and "position information" refers to the revised spatial position descriptions of the actors. These two elements are integrated to form the character's behavioral elements. "Actor-related event information" refers to the content recorded in the plot development elements concerning specific actor participation; this information is filtered and used as the target plot elements to demonstrate the connection between the actors and the plot.

[0151] In this embodiment, the actor's name in the basic scene elements is associated and matched with the target speaker identifier based on the second correspondence relationship. The action state and updated position information of the actor corresponding to each identifier in the corresponding scene are located and merged into a coherent description to form a character behavior element. Information containing actor participation records is filtered from the plot development elements to obtain the target plot element.

[0152] Step 603: Based on the fourth time sequence of each scene in the target movie video, integrate the basic elements of the scene, the plot development elements, the character behavior elements, and the target plot elements into a story outline.

[0153] In this step, the fourth temporal sequence refers to the order in which scenes are sorted according to their starting timestamps, serving as the benchmark for element integration. The story outline refers to the complete narrative framework formed by integrating basic scene elements, plot development elements, character behavior elements, and target plot elements. This story outline includes the scene temporal sequence, character behavior chains, and core plot nodes.

[0154] In this embodiment, the fourth time sequence is determined by the scene start timestamp, and the four types of elements are integrated based on this. The scene numbers are sorted by time to form a scene number set, and the character behavior elements of the same actor in different scenes are linked together to form a character behavior chain. The key event points that drive the plot are extracted as core plot nodes, and the three are logically integrated into a complete story outline.

[0155] Step 604: Associate the set of scene numbers in the story outline with the scene sequence numbers and target format scripts in the scene list to form the first draft of the movie script.

[0156] In this step, the scene number refers to the sequence number assigned to the scene list according to the fourth chronological order, used to link the story outline with the target format script. The first draft of the film script refers to the preliminary script formed by organizing scene titles, characters, dialogue, and action instructions according to a preset format, covering the core content of all scenes.

[0157] In this embodiment, a matching association is established between the set of scene numbers in the story outline and the scene sequence numbers in the scene list, and the corresponding target format script is associated through the sequence number. Environmental information is extracted from the script to extract scene titles, actor names are extracted as characters, matched voice-transcribed text is selected as character dialogue, and action states are extracted as action instructions. The script is then organized according to a preset format to form a first draft of the film script.

[0158] Step 605: Extract the core events, key character behaviors, and plot turning points of each scene from the initial draft of the film script, combine them with the plot summary in the database to generate a preliminary film summary, and combine them with the initial draft of the film script to generate the film script and film summary.

[0159] In this step, core events refer to the main events in the scenes that drive the plot forward, key character actions refer to the actions of actors that have a significant impact on the plot, and plot turning points refer to the key events that change the direction of the plot. The preliminary film summary refers to the core plot summary formed by integrating the above information with the plot outline in the database, and the final integration yields the film script and film summary.

[0160] In this embodiment, the initial draft of the film script is analyzed scene by scene to extract core events, key character behaviors, and plot twists. A plot summary is retrieved from the database, and the extracted information is compared with the plot summary to remove redundant information and form a preliminary film summary. Finally, the initial draft of the film script and the preliminary film summary are integrated to obtain the final film script and film summary.

[0161] This application's embodiments establish precise associations between characters, actions, and plots, avoiding content deviations caused by a single information source and improving the accuracy of character and scene matching in the script. The story outline ensures narrative chronology, and the preliminary film summary addresses the issue of loose logic in summaries within complex narratives. Integrating multimodal information overcomes the limitations of a single dimension, enhancing content richness and quality. Full-process automation reduces manual intervention, improves generation efficiency, adapts to resource-constrained scenarios, and the generated scripts and summaries provide reliable guidance for production and marketing teams, meeting the core needs of the film industry.

[0162] Figure 3 This application provides a schematic diagram illustrating a specific implementation of a system for automatically generating movie scripts and movie summaries, with reference to... Figure 3 The system may include:

[0163] The acquisition module 21 is used to acquire the target movie video, extract the key frame set, speech-transcribed text and character action recognition results from the target movie video, and extract multiple visual feature vectors from the key frame set.

[0164] Search module 22 is used to determine the scene segmentation scheme with the minimum total bit cost based on the preset scene division principle and the visual feature vector, so as to segment the target movie video into multiple scenes;

[0165] The filtering module 23 is used to extract an initial set of actor information images from video clips including the cast and crew list in the target movie video, filter target actor information images from the initial set of actor information images to construct a distance matrix, and generate a first correspondence between actor names and scenes and a second correspondence between actor names and speakers based on the distance matrix.

[0166] The generation module 24 is used to generate a target format script and a scene list for each target scene based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text, and the character action recognition results;

[0167] The identification module 25 is used to generate a tree diagram based on the scene, process the tree diagram to obtain a plot summary, and identify target movie video frames that match the plot summary to form a database.

[0168] Extraction module 26 is used to extract character behavior elements and target plot elements corresponding to the second correspondence from the target format script and the database, respectively, to generate a story outline, and combine it with the scene list to generate a movie script and movie summary.

[0169] This application provides an automatic movie script and movie summary generation system to implement the aforementioned automatic movie script and movie summary generation method. Therefore, the specific implementation of the automatic movie script and movie summary generation system can be found in the embodiment section of the automatic movie script and movie summary generation method above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0170] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described methods for automatically generating movie scripts and movie summaries.

[0171] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described methods for automatically generating movie scripts and movie summaries.

[0172] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0173] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the method for automatically generating movie scripts and movie summaries.

[0174] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] The foregoing has provided a detailed description of a method and system for automatically generating movie scripts and movie summaries. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for automatically generating film scripts and film summaries, characterized in that, include: The target movie video is acquired, and keyframe sets, speech-transcribed text, and character action recognition results are extracted from the target movie video. Multiple visual feature vectors are also extracted from the keyframe sets. Based on the preset scene division principle and the visual feature vector, the scene segmentation scheme with the minimum total bit cost is determined to segment the target movie video into multiple scenes; Based on the target movie video, a distance matrix is ​​constructed, and based on the distance matrix, a first correspondence between actor names and scenes and a second correspondence between actor names and speakers are generated; Based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text and the character action recognition results, a target format script and scene list are generated for each target scene. Based on the scenario, a tree diagram is generated, the tree diagram is processed to obtain a plot summary, and target movie video frames that match the plot summary are identified to form a database. The scene number, actor name, updated location information, and character action status corresponding to each scene in the target format script are classified as scene basic elements, and the event development information of each plot segment in the database is classified as plot development elements. According to the second correspondence, the actor's name in the basic scene element is associated with the target speaker identifier, the action state and position information of the actor corresponding to each target speaker identifier in the corresponding scene are determined, and the action state and position information are integrated into the character behavior element. The actor-related event information is selected from the plot development element as the target plot element. Based on the fourth chronological order of each scene in the target movie video, the basic elements of the scene, the plot development elements, the character behavior elements, and the target plot elements are integrated into a story outline. The scene number set in the story outline is associated with the scene sequence number and target format script in the scene list to form the first draft of the movie script; Extract the core events, key character behaviors, and plot twists of each scene from the initial draft of the film script. Combine this with the plot summary in the database to generate a preliminary film summary. Then, combine this with the initial draft of the film script to generate the film script and the film summary.

2. The method according to claim 1, characterized in that, Based on the preset scene segmentation principles and the aforementioned visual feature vectors, a scene segmentation scheme with the minimum total bit cost is determined, including: Based on the preset scene division principle, the visual feature vector is divided into multiple vector segment combinations; Calculate the probability density value of all visual feature vectors and the corresponding average feature vector within each vector segment in each vector segment combination, and calculate the total segmentation cost of each vector segment combination based on the probability density value. Calculate the first number of bits required to encode all visual feature vectors within each vector segment in each vector segment combination and the second number of bits required to encode the corresponding average feature vector, and calculate the total bit cost for each vector segment combination; Based on the second time sequence of the target movie video, the segmentation range of the vector segment combination is adjusted to form multiple adjusted vector segment combinations. The total bit cost of each adjusted vector segment combination is calculated, and the vector segment combination or adjusted vector segment combination with the minimum total bit cost is selected as the scene segmentation scheme.

3. The method according to claim 1, characterized in that, Based on the target movie video, a distance matrix is ​​constructed. Based on the distance matrix, a first correspondence between actor names and scenes and a second correspondence between actor names and speakers are generated, including: Multiple first actor information images and second actor information images in different scenes are extracted from the target movie video; Facial feature information is extracted from the first actor information image to calculate the facial feature similarity between different first actor information images, and the first actor information images with facial feature similarity higher than a preset threshold are integrated into the target actor information image; Based on the facial feature information in the target actor information image, calculate the feature difference value between any two target actor information images to construct a distance matrix; Calculate the matching degree between each target actor information image and the second actor information image. Combined with the distance matrix, assign the actor name in the target actor information image with the highest matching degree to the corresponding scene to form a first correspondence between actor name and scene. The speaker's voice features and text identifiers are extracted from the target movie video. Combined with the actor's name in the target actor information image and the distance matrix, a second correspondence between the actor's name and the speaker is determined.

4. The method according to claim 1, characterized in that, Based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text, and the character action recognition results, a target format script and scene list corresponding to each target scene are generated, including: The character position parameters are extracted from the keyframe set. Based on the temporal correlation between the scene and each keyframe in the keyframe set in the target movie video and the first correspondence, the target scene corresponding to each actor's name is determined. Each actor's name is bound to the character's position parameters in the corresponding target scene to obtain the initial position information of each actor in the corresponding target scene; Based on the first time sequence of the keyframes in the target movie video, and combined with the time interval and position parameter differences of the same actor in adjacent keyframes, the position change data of each actor between adjacent keyframes is calculated to update the initial position information of the corresponding actor, and the updated position information of each actor in the corresponding target scene is obtained. Based on the preset script format, and combined with the actor's name, updated location information, the transcribed speech text, and the character action recognition results for each target scene, a target format script corresponding to each target scene is generated. Based on the third time sequence of each target scene in the target movie video, arrange the target format scripts corresponding to all target scenes to form a scene list.

5. The method according to claim 4, characterized in that, Based on a preset script format, and combining the actor's name, updated location information, the transcribed speech text, and the character action recognition results for each target scene, a target format script is generated for each target scene, including: Set the first script format; Based on the first correspondence, arrange all the actor names in each target scene to form an actor list; The identification information of each target scene is used as the scene number, the time span of the keyframes corresponding to each target scene in the target movie video is used as the scene duration, and the scene number, scene duration, and actor list of each target scene are filled into the first script format to obtain the second format script corresponding to each target scene. The updated position information of all actors in the corresponding target scene is converted into position descriptions, and the position descriptions are filled into the second format script to obtain the third format script; Extract all actors' action records in the corresponding target scene from the character action recognition results, fill the action records into the third format script to obtain the fourth format script; Based on the second correspondence, the target speaker identifier corresponding to the actor in each target scene is determined, and the text content associated with the target speaker identifier and the target keyframe duration is filtered out from the speech transcription text. The text content is then filled into the fourth format script to obtain the target format script corresponding to each target scene.

6. The method according to claim 1, characterized in that, Based on the scenario, a tree diagram is generated; the tree diagram is processed to obtain a plot summary; and target movie video frames matching the plot summary are identified to form a database, including: The scene environment information and character interaction behavior are identified from the keyframes corresponding to each scene to generate a target content description for each scene. Based on the fourth time sequence of each scene in the target movie video, the temporal relationship between different scenes is determined. Using the target movie storyline as the root node of the tree diagram, and according to the chronological relationship, multiple scenes are divided into multiple scene groups. Each scene group is a first-level child node under the root node, and each scene within the scene group is a second-level child node under the corresponding first-level child node. The tree diagram is constructed such that each second-level child node is bound to the target content description of the corresponding scene. The target content descriptions with a repetition rate higher than a preset repetition threshold in all second-level sub-nodes under each first-level sub-node in the tree diagram are integrated into a target plot description. The target plot descriptions of all first-level sub-nodes are integrated to generate a plot summary. The plot summary is divided into multiple plot segments, each plot segment is matched with a corresponding target scene, and target movie video frames are extracted from the target scenes to form a set of matching frames for the corresponding plot segment. A database is constructed based on the plot summary, each plot segment, and the set of matching frames corresponding to each plot segment.

7. A system for automatically generating film scripts and film summaries, used in accordance with the method for automatically generating film scripts and film summaries as described in any one of claims 1 to 6, characterized in that, include: The acquisition module is used to acquire the target movie video, extract the key frame set, speech-transcribed text and character action recognition results from the target movie video, and extract multiple visual feature vectors from the key frame set. The search module is used to determine the scene segmentation scheme with the minimum total bit cost based on the preset scene division principle and the visual feature vector, so as to segment the target movie video into multiple scenes. The filtering module is used to extract an initial set of actor information images from video clips including the cast and crew list in the target movie video, filter target actor information images from the initial set of actor information images to construct a distance matrix, and generate a first correspondence between actor names and scenes and a second correspondence between actor names and speakers based on the distance matrix. The generation module is used to generate a target format script and a scene list for each target scene based on the first correspondence and the character position parameters in the target movie video, combined with the first correspondence, the speech-transcribed text and the character action recognition results. The identification module is used to generate a tree diagram based on the scene, process the tree diagram to obtain a plot summary, and identify target movie video frames that match the plot summary to form a database. The extraction module is used to extract character behavior elements and target plot elements corresponding to the second correspondence from the target format script and the database, respectively, to generate a story outline, and combine it with the scene list to generate a movie script and a movie summary.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of an automatic method for generating movie scripts and movie summaries as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables an automatic method for generating a movie script and movie summary as described in any one of claims 1 to 6.

Images