Video script generation method and apparatus, and electronic device

Abstract

The present application relates to the field of computers, and discloses a video script generation method and apparatus, and a device. The video script generation method comprises: on the basis of a material content description of each material file in a material set, generating script generation prompt words, wherein the material set is used for generating a video, the script generation prompt words are used for instructing a large language model to generate storyline description information on the basis of the material content description of each material file, screening and ranking the material files on the basis of the storyline description information, and outputting a ranking result of selected N candidate materials and caption texts corresponding to the N candidate materials, the material set comprises at least two material files, the material file is a picture material file or video material file, and N is an integer greater than 1; inputting the script generation prompt words into the large language model, to output the ranking result of the N candidate materials and the caption texts corresponding to the N candidate materials, wherein one candidate material corresponds to one caption text; and generating a video script on the basis of the ranking result of the N candidate materials and the caption texts corresponding to the N candidate materials.

Description

Video script generation method and device and electronic equipment

[0001] Cross-reference to Related Applications

[0002] The present application claims priority to the Chinese patent application No. 202411046252.8, filed on August 01, 2024, and entitled "Video script generation method and device and electronic equipment", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application belongs to the technical field of artificial intelligence, and specifically relates to a video script generation method, device and electronic equipment. BACKGROUND

[0004] With the rise of digital media, video creation has been widely applied. Video script creation is a key step in video creation. In the prior art, video script creation is mainly manually written by users, which requires a lot of time and effort of users, resulting in low efficiency of video creation. For non-professional users, it is a challenging task to create a video script that matches the content and artistic conception of the video material.

[0005] The video script of the prior art needs to be edited by the user using professional video editing software, and the script is added on each frame of video image through the editing software. The video editing process of the user is tedious and time-consuming, the efficiency of video script generation is low, and the matching degree of the generated video script and the video material is low. SUMMARY

[0006] The purpose of the embodiments of the present application is to provide a video script generation method, device and equipment, which can make the plot development of each script in the video script more attractive, the connection between each script more smooth, and the picture content of each script and the overall theme of the video more consistent, so as to make the overall video script more story-like. Moreover, the video script can be automatically generated, and the generation efficiency of the video script is improved.

[0007] In a first aspect, an embodiment of the present application provides a video script generation method, which comprises: generating a script generation prompt word based on a material content description of each material file in a material set; wherein the material set is used to generate a video, the script generation prompt word is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N selected materials and a script corresponding to the N selected materials; the material set comprises at least two material files, the material file is a picture material file or a video material file, and N is an integer greater than 1; inputting the script generation prompt word into the large language model to output the sorting result of the N selected materials and the script corresponding to the N selected materials; wherein one selected material corresponds to one script; and generating a video script based on the sorting result of the N selected materials and the script corresponding to the N selected materials.

[0008] In a second aspect, an embodiment of the present application provides a video script generation device, which comprises: a script prompt word generation module configured to generate a script generation prompt word based on a material content description of each material file in a material set; wherein the material set is used to generate a video, the script generation prompt word is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N selected materials and a script corresponding to the N selected materials; the material set comprises at least two material files, the material file is a picture material file or a video material file, and N is an integer greater than 1; a sorting and script acquisition module configured to input the script generation prompt word into the large language model to output the sorting result of the N selected materials and the script corresponding to the N selected materials; wherein one selected material corresponds to one script; and a video script generation module configured to generate a video script based on the sorting result of the N selected materials and the script corresponding to the N selected materials.

[0009] In a third aspect, an embodiment of the present application provides an electronic device, which comprises a processor and a memory, the memory stores programs or instructions executable on the processor, and the programs or instructions are executed by the processor to implement the steps of the method according to the first aspect.

[0010] In the embodiments of the present application, by generating a material content description of each material file in a material set used for generating a video, a story line description information is generated for instructing a large language model to generate the story line description information based on the material content description of each material file, the each material file is filtered and sorted based on the story line description information, and a sorting result of N candidate materials obtained by filtering and a script generation prompt word corresponding to the N candidate materials are output. The script generation prompt word is input into the large language model, which can guide the large language model to comprehensively consider the material content description of each material file to generate the story line description information, filter and sort the material files based on the story line description information generated by comprehensively considering the material content description of each material file, and output the sorting result of N candidate materials obtained by filtering and the script corresponding to the N candidate materials. Based on the sorting result of N candidate materials obtained by the above method and the script corresponding to the N candidate materials, the video script can be generated, which can make the plot development of each script in the video script more attractive, the connection between each script more smooth, and each script more consistent with the picture content of each material and the overall theme of the video, so that the overall video script is more story-based. Moreover, the above method can automatically generate the video script, and improve the generation efficiency of the video script. BRIEF DESCRIPTION OF DRAWINGS

[0011] FIG. 1 is a flowchart of a video script generation method according to some embodiments of the present application;

[0012] FIG. 2 is a flowchart of a material set acquisition process according to some embodiments of the present application;

[0013] FIG. 3 is an interface diagram of a material set acquisition example according to some embodiments of the present application;

[0014] FIG. 4 is a flowchart of a material set acquisition process according to some embodiments of the present application;

[0015] FIG. 5 is an interface diagram of a material set acquisition example according to some embodiments of the present application;

[0016] FIG. 6 is a flowchart of a material content description acquisition process according to some embodiments of the present application;

[0017] FIG. 7 is a flowchart of a script creation tag set acquisition process according to some embodiments of the present application;

[0018] FIG. 8 is an example of a script generation prompt word according to some embodiments of the present application;

[0019] FIG. 9 is a flowchart of a video script generation method according to some embodiments of the present application;

[0020] FIG. 10 is an example of a style recommendation prompt word according to some embodiments of the present application;

[0021] FIG. 11 is an example of a script generation prompt word provided by some embodiments of the present application;

[0022] FIG. 12 is a flowchart of a video script generation method provided by some embodiments of the present application;

[0023] FIG. 13 is an example of a script generation prompt word provided by some embodiments of the present application;

[0024] FIG. 14 is an output flowchart of a video script provided by some embodiments of the present application;

[0025] FIG. 15a is an example of a video script provided by some embodiments of the present application;

[0026] FIG. 15b is an interface diagram of a display example of a video script provided by some embodiments of the present application;

[0027] FIG. 16 is a script modification flowchart provided by some embodiments of the present application;

[0028] FIG. 17 is an example of a script modification prompt word provided by some embodiments of the present application;

[0029] FIG. 18 is a module diagram of a video script generation device provided by some embodiments of the present application;

[0030] FIG. 19 is a structural diagram of an electronic device provided by some embodiments of the present application;

[0031] FIG. 20 is a hardware structure diagram of an electronic device provided by some embodiments of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0033] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a class and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in an "or" relationship.

[0034] The video script generation method provided by the embodiments of the present application will be described in detail below in conjunction with the drawings, specific embodiments and application scenarios.

[0035] In the embodiments of the present application, the professional terms involved include:

[0036] Large Language Model (LLM): a machine learning model that uses a large amount of language data to generate natural language text. It is a deep learning-based model, usually using a recurrent neural network (RNN) or a variant of neural network structure (such as transformers). This model can learn the structure, syntax and semantics of language by training a large amount of text data, and then use this learning to generate natural language text, which can be used in chat automatic reply, article abstract extraction, chat robots and other application scenarios.

[0037] Prompt, which refers to the prompt word or prompt text input when the large language model generates natural language text. The prompt word can be regarded as an initial input of the large language model, which can help the large language model to generate text content more accurately. In many application scenarios, users can use the prompt word to guide the model to generate text that meets the needs.

[0038] It should be noted that the video script generation method provided by the embodiments of the present application can be executed by a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, etc. In some embodiments of the present application, the electronic device is taken as an example to execute the video script generation method provided by the embodiments of the present application, and the video script generation method provided by the embodiments of the present application is described.

[0039] The method provided by the embodiments of the present application can be applied to the scene of video creation.

[0040] In one specific application scenario, when a user is creating a video, the user clicks a video generation control of an electronic device. In response to the user's click operation on the video generation control, the electronic device can jump to a photo album interface to enable the user to select multimedia files in the photo album interface. Then, the electronic device can obtain a material set for generating a video based on the multimedia files selected by the user. The material set includes a plurality of material files. Each material file can be a picture or a video. Then, the electronic device can generate a script generation prompt based on a material content description of each material file in the material set. The script generation prompt is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N candidate materials obtained by filtering and a script corresponding to the N candidate materials. Then, the script generation prompt is input into the large language model to output the sorting result of the N candidate materials and the script corresponding to the N candidate materials. Then, a video script is generated based on the sorting result of the N candidate materials and the script corresponding to the N candidate materials. Then, a video is generated based on the video script, and the video is output on a video output interface of the electronic device.

[0041] In another specific application scenario, when a user is creating a video, the user inputs a message including video clip indication information in a chat interface provided by an electronic device. According to the message, the electronic device can find matching multimedia files from a photo album program, obtain a material set for generating a video based on the found multimedia files, and filter and sort each material file in the material set based on a material content description of each material file. Then, a script generation prompt is generated based on the material content description of each material file. The script generation prompt is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N candidate materials obtained by filtering and a script corresponding to the N candidate materials. Then, the script generation prompt is input into the large language model to output the sorting result of the N candidate materials and the script corresponding to the N candidate materials. Then, a video script is generated based on the sorting result of the N candidate materials and the script corresponding to the N candidate materials. Then, a video is generated based on the video script, and a chat message including the video is output on the chat interface.

[0042] FIG. 1 is a flow diagram of a video script generation method according to an embodiment of the present application. As shown in FIG. 1, the video script generation method according to an embodiment of the present application includes the following steps:

[0043] In step 101, a script generation prompt word is generated based on the material content description of each material file in the material set.

[0044] The material set is used to generate a video, and contains at least two material files, which are picture material files or video material files. The material files in the material set can all be picture material files, or all be video material files, or part be picture material files and part be video material files.

[0045] For example, when a user is creating a video, the user can click a video generation control of an electronic device. In response to the user's click operation on the video generation control, the electronic device can jump to a photo album interface to enable the user to select multimedia files in the photo album interface. Then, the electronic device can obtain a material set for generating a video based on the multimedia files selected by the user, and generate a script prompt word based on the material content description of each material file in the material set.

[0046] For another example, when a user is creating a video, the user can input a message including video clip indication information in a chat interface provided by the electronic device. According to the message, the electronic device can find matching multimedia files from a photo album program, obtain a material set for generating a video based on the found multimedia files, and then generate a script generation prompt word based on the material content description of each material file in the material set.

[0047] The material content description is used to describe the picture content of the material file, which can be a text description of the picture content of the material file, such as a picture description text of the material file, or a picture description text of the picture of the material file and a script creation tag of the material file. The picture description text of the material file is used to describe the picture content of the material file, which can be a text description of the picture of the material file, for example, "peach blossoms cover the hillside, and the distant mountains are surrounded by clouds and mist". The script creation tag of the material file is used to identify the summary information of the picture content of the material file, which can be a tag of the overall picture of the material file, for example, "mountain peak".

[0048] The script generation prompt word is used to instruct the large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output the sorting result of N candidate materials obtained by filtering and the script corresponding to the N candidate materials. N is an integer greater than 1.

[0049] The story line description information is a linear arrangement and narration of the story development, progress and ending, which covers a series of events with logical relationship in the story, which collectively drive the story forward until the final ending. For example, "In the spring of May, we came to the picturesque Tibet, where we met the blooming flowers, met the misty mountains, and experienced the nature, harmony and inclusiveness."

[0050] The alternative material is a material file selected by the large language model from the material files in the material set according to the story line description information. For example, N alternative materials are a plurality of material files matched with a plurality of story plots of a story, which are selected from a plurality of material files in the material set according to the plurality of story plots. The ranking result of the alternative material is determined based on the position of the story plot matched by the alternative material in the story.

[0051] The text script is a sentence of explanatory text corresponding to the alternative material, which is used as the subtitle of the alternative material when generating the video based on the alternative material. For example, the alternative material is picture 1, and when generating the video, picture 1 is used as a frame of image in the video, and the text script corresponding to picture 1 is used as the subtitle of the frame of image.

[0052] In specific implementation, the material set can be obtained by a suitable manner.

[0053] Please refer to FIG. 2, which is a schematic diagram of a material set obtaining process provided by some embodiments of the present application. As shown in FIG. 2, before step 101, the method provided by the embodiments of the present application further includes:

[0054] In step 201, a file selection input of a user to a photo album interface is received.

[0055] In some embodiments of the present application, the above-mentioned file selection input is used to input the multimedia file selected in the photo album interface. The file selection input can be a selection operation of the user to the image or video in the photo album interface. For example, the above-mentioned file selection input includes but is not limited to the touch input of the user to the specified image or video in the photo album interface by using a touch device such as a finger or a stylus, or a voice instruction input by the user, or a specific gesture input by the user, or other feasible input. The specific gesture in the embodiments of the present application can be any one of a single-click gesture, a sliding gesture, a dragging gesture, a pressure recognition gesture, a long-press gesture, an area change gesture, a double-press gesture, a double-click gesture, etc. The click input in the embodiments of the present application can be a single-click input, a double-click input or any number of click inputs, etc. It can also be a long-press input or a short-press input. For example, the above-mentioned file selection input can be a click input of the user to the specified image or video in the photo album interface.

[0056] In the case that the electronic device receives a video generation request or a video script generation request input by the user, the electronic device can jump to the album interface to enable the user to perform file selection input in the album interface.

[0057] For example, a video generation control can be provided on the electronic device. In response to a click operation of the video generation control by the user, the electronic device can jump to the album interface of the album program to enable the user to select pictures or videos in the album interface.

[0058] In response to the file selection input, the multimedia files selected by the file selection input are determined as a material set for generating a video.

[0059] The multimedia files include at least one of an image and a video. Based on the set of multimedia files selected by the user, a material set for generating a video can be obtained. For example, as shown in FIG. 3, if the user selects picture 1, picture 2, and picture 3, the set of picture 1, picture 2, and picture 3 can be determined as a material set for generating a video, and each of picture 1, picture 2, and picture 3 is a material file in the material set.

[0060] Referring to FIG. 4, another material set obtaining flowchart is provided by some embodiments of the present application. As shown in FIG. 4, before step 101, the method provided by the embodiments of the present application further includes:

[0061] Step 401: receiving a chat message input by the user in a chat interface.

[0062] The electronic device can provide a chat interface for the user to input a chat message.

[0063] Step 402: in the case that the chat message includes video clip indication information, obtaining a clip requirement keyword in the chat message.

[0064] When the user needs to generate a video, the user can input a chat message including video clip indication information in the chat interface of the electronic device. As shown in FIG. 5, the chat message including video clip indication information input by the user in the chat interface is, for example, “Help me clip the things shot in Beijing last week into a short film”.

[0065] The clip requirement keyword is used to search for multimedia files to obtain a material set for generating a video, which can include a keyword in a time dimension and a keyword in a location dimension. The clip requirement keyword can be obtained by performing semantic analysis on the chat message. For example, for the chat message shown in FIG. 5, semantic analysis can be performed to obtain the keywords “last week” and “Beijing”.

[0066] Step 403, searching for at least one multimedia file matching the clip requirement keyword in the album program.

[0067] Continuing with the example of FIG. 5, the electronic device can search for pictures taken in Beijing in the last week and videos taken in Beijing in the last week in its album program.

[0068] Step 404, determining the at least one multimedia file matching the clip requirement keyword as the material set for generating the video.

[0069] Continuing with the example of FIG. 5, the electronic device can determine the set of pictures taken in Beijing in the last week and videos taken in Beijing in the last week as the material set for generating the video.

[0070] In the above embodiment, the file selection input selected multimedia file is determined as the material set for generating the video, or the multimedia file matching the clip requirement keyword is determined as the material set for generating the video. The multimedia file includes a picture or a video, one picture can be used as one material file in the material set, and one video can be used as one material file in the material set.

[0071] In an implementation, to avoid using a video directly as a material file, which has a too long video length and affects the script generation effect, when the file selection input selected multimedia file is determined as the material set for generating the video, or when the multimedia file matching the clip requirement keyword is determined as the material set for generating the video, for the multimedia file of the video type, the video length of the multimedia file can be detected to determine whether the video length exceeds a predefined material length, and if so, the multimedia file is split based on the predefined material length, and each split multimedia file is used as one material file in the material set.

[0072] For example, assuming that the user selects video 1 in the album interface, in the determination of the material set for generating the video, the video length of video 1 can be detected to determine whether the video length exceeds the predefined material length, assuming that the predefined material length is 1 second and the video length of video 1 is 3 seconds, the video length of video 1 exceeds the predefined material length, and at this time, video 1 can be split into video 11, video 12, and video 13 based on the predefined material length, and each of video 11, video 12, and video 13 is used as one material file in the material set. The video length of each of video 11, video 12, and video 13 is 1 second.

[0073] As described above, the material content description in the embodiment of the present application includes a picture description text of a material file.

[0074] The picture description text of the material file is usually a detailed description of the picture content of the material file, which is used to provide content information of the material file for the large language model, so that the large language model screens the candidate material based on the content information of the material file, sorts the candidate material, and generates the copywriting script corresponding to the candidate material.

[0075] In an implementation manner, to provide more abundant material content information for the large language model and improve the accuracy of the sorting result of the candidate material output by the large language model and the copywriting script corresponding to the candidate material, on the basis of the picture description text, the material content description further includes a copywriting label. That is, the material content description includes the picture description text and the copywriting label. The copywriting label is a label suitable for video copywriting, such as “grassland”, “snow mountain”, “peach blossom”, “mountain peak”, and “cloud sea”, which is used to provide summary information of the picture content of the material file. The picture description text is a text description of the picture content of the material, such as “This is a peach blossom tree in full bloom, with dense flowers and a spring atmosphere”, which is used to provide detailed information of the picture content of the material file. The material content description of each material file contains one copywriting label and one picture description text. By generating a copywriting prompt word based on the material content information including the picture description text and the copywriting label, the large language model can be provided with material content information in two dimensions of summary and detail dimensions, so that the large language model screens more accurate candidate materials based on the multi-dimensional material content information and generates more accurate copywriting scripts.

[0076] In specific implementation, the material content description can be obtained in an appropriate manner.

[0077] Please refer to FIG. 6 for a flowchart of obtaining a material content description provided by some embodiments of the present application. As shown in FIG. 6, after determining the material set used to generate a video, before step 101, the video copywriting method described in the embodiments of the present application further includes:

[0078] Step 601: determining a copywriting label of each material file in the material set based on a copywriting label in the copywriting label set that has the highest similarity with the material file.

[0079] The copywriting label in the copywriting label set is obtained by screening from the candidate label set based on a copywriting suitability score of the candidate label. The copywriting label of each material file can be determined based on a copywriting label in the copywriting label set that has the highest similarity with the material file.

[0080] The material file in the material set is a picture material file or a video material file.

[0081] For a picture material file, a similarity of the picture material file and each copywriting label in the copywriting label set can be calculated, and a copywriting label with the highest similarity to the picture material file is determined as the copywriting label of each picture material file.

[0082] For a video material file, at least two video images of the video material file can be extracted according to a video frame extraction interval, a similarity of a copywriting label and the video material file can be determined based on a similarity of the copywriting label and each video image in the at least two video images, and a copywriting label with the highest similarity to the video material file in the copywriting label set is determined as the copywriting label of each video material file.

[0083] In this way, the similarity of the copywriting label and the video material file can be determined as a sum of similarities of the copywriting label and each video image in the at least two video images of the video material file, and the copywriting label with the highest similarity to the video material file in the copywriting label set is determined as the copywriting label of the video material file.

[0084] In specific implementation, the similarity of the picture material file and each copywriting label can be calculated by a multi-modal model CLIP (Contrastive Language-Image Pre-training, a pre-training model using contrastive learning), or the similarity of each video image and each copywriting label can be calculated by the multi-modal model CLIP.

[0085] The multi-modal model CLIP is a pre-training model using contrastive learning, which can map images and texts to the same feature space and calculate the similarity between images and texts.

[0086] In the calculation of the similarity of the picture material file and the copywriting label based on the multi-modal model CLIP, the picture material file and the copywriting label can be input into the multi-modal model CLIP, the multi-modal model CLIP can extract image features of the picture material file from the picture material file and extract text features of the copywriting label from the copywriting label, calculate the similarity between the image features of the picture material file and the text features of the copywriting label, and obtain and output the similarity of the picture material file and the copywriting label.

[0087] When the similarity between the video image and the script creation label is calculated based on the multi-modal model CLIP, the video image and the script creation label can be input into the multi-modal model CLIP. The multi-modal model CLIP can extract image features of the video image from the video image, extract text features of the script creation label from the script creation label, calculate the similarity between the image features of the video image and the text features of the script creation label, and thus obtain the similarity between the video image and the script creation label and output the similarity.

[0088] For example, assuming that the script creation labels in the script creation label set are “snow mountain”, “peach blossom”, “mountain peak”, “cloud sea”, and “grassland”, for the picture material file 1 in the material set, it can be input into the CLIP model with the script creation labels “snow mountain”, “peach blossom”, “mountain peak”, “cloud sea”, and “grassland” in the script creation label set respectively, to calculate the similarity between the picture material file 1 and the script creation label “snow mountain”, the similarity between the picture material file 1 and the script creation label “peach blossom”, the similarity between the picture material file 1 and the script creation label “mountain peak”, the similarity between the picture material file 1 and the script creation label “cloud sea”, and the similarity between the picture material file 1 and the script creation label “grassland”. If the similarity between the picture material file 1 and the script creation label “grassland” is the highest among these similarities, the script creation label “grassland” can be determined as the script creation label of the picture material file 1.

[0089] For example, assuming that the script creation labels in the script creation label set are “snow mountain”, “peach blossom”, “mountain peak”, “cloud sea”, and “grassland”, for the picture material file 1 in the material set, it can be input into the CLIP model with the script creation labels “snow mountain”, “peach blossom”, “mountain peak”, “cloud sea”, and “grassland” in the script creation label set respectively, to calculate the similarity between the picture material file 1 and the script creation label “snow mountain”, the similarity between the picture material file 1 and the script creation label “peach blossom”, the similarity between the picture material file 1 and the script creation label “mountain peak”, the similarity between the picture material file 1 and the script creation label “cloud sea”, and the similarity between the picture material file 1 and the script creation label “grassland”. If the similarity between the picture material file 1 and the script creation label “grassland” is the highest among these similarities, the script creation label “grassland” can be determined as the script creation label of the picture material file 1.

[0090] The script creation tag set in the embodiments of the present application is a tag system specially established for video script creation. Selecting a script creation tag of a material file from the script creation tag set can provide more useful material content information for the large model in screening and sorting the alternative materials and generating the script, thereby improving the pertinence of the large model output information for video script creation. For example, for the tags "mountain peak" and "black", the mountain peak can represent more summary information of the picture, and the "mountain peak" can be determined as a script creation tag and added to the script creation tag set, while the "black" is not used as a script creation tag.

[0091] FIG. 7 is a schematic diagram of a script creation tag set acquisition process according to some embodiments of the present application. As shown in FIG. 7, the video script generation method provided by some embodiments of the present application further includes the following steps before step 601:

[0092] Step 701: generating an image description tag set based on the image description text set.

[0093] The image description text set is a description set of an existing public image description data set, which contains a plurality of image description texts. The image description texts in the image description set can be segmented to obtain description keywords of each image description text, and then the description keywords of the plurality of image description texts can be clustered to obtain an image description tag set. The image description tags in the image description tag set are the clustered description keywords.

[0094] Step 702: generating a candidate tag set based on the image description tag set and an image classification tag set.

[0095] The image classification tag set is a label set of an existing public image classification data set, which contains a plurality of image classification tags. The image description tag set and the image classification tag set can be merged to obtain the union of the image description tag set and the image classification tag set, and the candidate tag set can be obtained based on the N tags with the highest frequency in the union.

[0096] Step 703: determining the script creation suitability score of each candidate tag in the candidate tag set based on the appearance frequency of each candidate tag in the candidate tag set and the script generation beauty degree of the candidate tag.

[0097] The appearance frequency of the candidate tag can be determined based on the number of times the candidate tag appears in the union of the image description tag set and the image classification tag set. The script generation beauty degree of the candidate tag can be determined by a person.

[0098] In a specific implementation, for each candidate label, the occurrence frequency score of the candidate label can be determined according to the number of times the candidate label appears in the union of the image description label set and the image classification label set, the text generation beauty degree of the candidate label can be determined according to the text generation beauty degree of the candidate label, and the text creation suitability score of the candidate label can be determined based on the occurrence frequency score and the text generation beauty degree score of the candidate label.

[0099] For example, for label 1, the corresponding occurrence frequency score is 6, and the text generation beauty degree score is 8, so the text creation suitability score corresponding to label 1 can be (6+8) / 2, that is, 7 points.

[0100] Step 704, based on the text creation suitability score of each candidate label, all candidate labels are screened to obtain a text creation label set.

[0101] Among them, the text creation suitability score of each candidate label can be compared with the score threshold, and the candidate label with a text creation suitability score greater than the score threshold is determined as a text creation label, so as to obtain a set of text creation labels, that is, a text creation label set.

[0102] By establishing a text creation label set more suitable for video text creation based on the existing image description text set and image classification label set, selecting the text creation label corresponding to the material file from the text creation label set as the information in the material content description provided to the large language model, the large language model can output more appropriate text creation, thereby improving the generation quality of the video text.

[0103] Step 602, input each material file in the material set into the description model to output the picture description text of each material file.

[0104] The material file in the material set is a picture material file or a video material file.

[0105] For a picture material file, it can be directly input into the description model for picture content description to obtain the picture description text of the picture material file.

[0106] For example, for picture material file 1, picture material file 1 can be input into the description model for picture content description to obtain the picture description text of the picture. The picture description text of picture material file 1 is, for example, “This is a peach blossom tree in full bloom, with dense flowers and a spring atmosphere.”

[0107] For a video material file, at least two frames of video images of the video material file can be extracted according to a video frame extraction interval; each frame of video image in the at least two frames of video images is input into a description model to output picture description text of each frame of video image; based on the picture description text of each frame of image, a description summary prompt word is generated, which is used to instruct a large language model to summarize the picture description text of each frame of video image and output summarized description text; the description summary prompt word is input into the large language model to output the summarized description text; and based on the summarized description text, picture description text of the video material file is generated.

[0108] For example, for a video material file 1, three frames of video images, i.e., video image 1, video image 2, and video image 3, can be extracted from the video material file at a frequency of two frames per second, and then the video image 1, the video image 2, and the video image 3 are input into a description model for picture content description to obtain picture description text of the video image 1, picture description text of the video image 2, and picture description text of the video image 3; then, based on the picture description text of the video image 1, the picture description text of the video image 2, and the picture description text of the video image 3, a description summary prompt word is generated, which is input into a large language model to obtain summarized description text; and the summarized description text is determined as the picture description text of the video material file 1.

[0109] For example, the picture description text of the video image 1 is “Chinese traditional architectural style of archway and stone lion sculpture”, the picture description text of the video image 2 is “a Chinese traditional style of painted archway and a stone lion sculpture in front of the guard”, and the picture description text of the video image 3 is “a bright archway with majestic stone lions guarding”. The large language model summarizes the picture description text of the video images to output summarized description text, for example, “a Chinese-style archway with bright colors guarded by majestic stone lions, showing the unique charm of Chinese traditional architecture”. The summarized description text can be determined as the picture description text of the video material file 1.

[0110] The script generation prompt word in the embodiment of the present application is used to instruct the large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N candidate materials obtained by filtering and script texts corresponding to the N candidate materials.

[0111] In specific implementation, the material content description of each material file in the material set can be embedded into a predefined script prompt word template to generate a script generation prompt word. The script generation prompt word is shown as “input” in FIG. 8, for example.

[0112] Step 102, input the copy generation prompt word into the large language model, output the sorting result of N candidate materials and the copy corresponding to the N candidate materials.

[0113] Wherein, one candidate material corresponds to one copy. The large language model can perform corresponding processing under the constraint of the copy generation prompt word according to the indication of the copy generation prompt word, and output the sorting result of N candidate materials and the copy corresponding to the N candidate materials.

[0114] In the example of FIG. 8, when the copy generation prompt word shown as "input" in FIG. 8 is input into the large language model, the output of the large language model is shown as "output" in FIG. 8. Wherein, the label of the material in the input of FIG. 8 is the copy creation label of the material, the description of the material in the input of FIG. 8 is the picture description text of the material, "bathed in the spring of May, we came to the poetic Tibet, here we met the blooming flowers, met the misty mountains, experienced nature, harmony and inclusion" in the output of FIG. 8 is the story line description information, 1, 4, 2, 3 in the output of FIG. 8 are four candidate materials, and the copy corresponding to the materials 1, 4, 2, 3 in the output of FIG. 8 is the copy corresponding to the materials.

[0115] Step 103, generating a video copy based on the sorting result of the N candidate materials and the copy corresponding to the N candidate materials.

[0116] Wherein, the video copy is the overall copy obtained based on the sorting result of the N candidate materials and the copy corresponding to the N candidate materials, which is a story copy. The N candidate materials and the copy corresponding to the N candidate materials can be assembled according to the pre-defined format based on the sorting result of the N candidate materials, so as to obtain the video copy.

[0117] The video copy generated based on the sorting result of the four candidate materials and the copy corresponding to the four candidate materials shown in the output of FIG. 8 is shown in FIG. 15a, for example.

[0118] In the embodiment of the present application, by generating a material content description of each material file in a material set used to generate a video, a story line description information generation instruction is generated for a large language model to generate a story line description information based on the material content description of each material file, the story line description information is used to filter and sort the material files, and a sorting result of N candidate materials obtained by filtering and a script generation prompt word corresponding to the N candidate materials are output. The script generation prompt word is input into the large language model, which can guide the large language model to comprehensively consider the material content description of each material file to generate the story line description information, filter and sort the material files based on the story line description information generated by comprehensively considering the material content description of each material file, and output the sorting result of N candidate materials obtained by filtering and the script corresponding to the N candidate materials. Based on the sorting result of N candidate materials obtained by the above method and the script corresponding to the N candidate materials, the video script can be generated, which can make the plot development of each script in the video script more attractive, the connection between each script more smooth, and each script more consistent with the picture content of each material and the overall theme of the video, so that the overall video script is more story-based. Moreover, the above method can automatically generate a video script and improve the generation efficiency of the video script.

[0119] FIG. 9 is a flowchart of another video script generation method provided by an embodiment of the present application. As shown in FIG. 9, the video script generation method provided by an embodiment of the present application includes the following steps:

[0120] In step 901, user portrait information of a user is obtained.

[0121] The user portrait information includes at least one of the following: gender, age, zodiac, type, occupation, and interest. The user is a user of an electronic device, for example, the owner of the electronic device.

[0122] In step 902, a style recommendation prompt word is generated based on the user portrait information.

[0123] The style recommendation prompt word is used to instruct a large language model to output recommended style information based on the user portrait information, for example, as shown in “input” in FIG. 10.

[0124] In a specific implementation, the user portrait information can be embedded into a pre-defined style recommendation prompt word template to generate the style recommendation prompt word.

[0125] In step 903, the style recommendation prompt word is input into the large language model to obtain recommended style information.

[0126] The large language model can perform the processing indicated by the style recommendation prompt word under the constraint of the style recommendation prompt word.

[0127] Following the example of FIG. 10, when the style recommendation prompt word described as “input” in FIG. 10 is input into the large language model, the output of the large language model is as shown in “output” of FIG. 10.

[0128] The recommended style information includes recommended author information and copywriting style recommendation information. In specific implementation, the recommended author information and the copywriting style recommendation information can be extracted from the output information of the large language model, so as to obtain the recommended style information.

[0129] Following the example of FIG. 10, “copywriting style recommendation: emotional, warm, poetic” and “recommended author: Joanne Harris” can be extracted from the output information shown in “output” of FIG. 10, and the above information is assembled to obtain the recommended style information.

[0130] At step 904, a first copywriting generation prompt word is generated based on the material content description of each material file in the material set and the recommended style information.

[0131] The first copywriting generation prompt word is used to instruct the large language model to generate story line description information based on the material content description of each material file, to filter and sort the material files based on the story line description information, to output the sorting result of N alternative materials, and to output the copywriting corresponding to the N alternative materials based on the story line description information and the recommended style information.

[0132] In some embodiments of the present application, the material content description includes copywriting creation tags and picture description texts. The first copywriting generation prompt word is specifically used to instruct the large language model to generate story line description information based on the copywriting creation tags and picture description texts of each material file, to filter and sort the material files based on the story line description information, to output the sorting result of N alternative materials, and to output the copywriting corresponding to the N alternative materials based on the story line description information and the recommended style information.

[0133] The first copywriting generation prompt word is shown in FIG. 11, for example. As shown in FIG. 11, the first copywriting generation prompt word is basically the same as the copywriting generation prompt word shown in “input” of FIG. 8, except that the first copywriting generation prompt word increases the content of “and, please based on the writing style of Joanne Harris emotional, warm, poetic”, which is generated based on the recommended style information.

[0134] In specific implementation, the material content description of each material file and the recommended style information can be embedded into a predefined copywriting generation prompt word template to generate the copywriting generation prompt word.

[0135] Step 905, input the first script generation prompt word into the large language model, and output the sorting result of N candidate materials and the script text corresponding to the N candidate materials.

[0136] The large language model can perform corresponding processing under the constraint of the second script generation prompt word according to the indication of the second script generation prompt word, and output the sorting result of the N candidate materials and the script text corresponding to the N candidate materials.

[0137] Step 906, generate a video script based on the sorting result of the N candidate materials and the script text corresponding to the N candidate materials.

[0138] By generating a first script generation prompt word for indicating the large language model to generate story line description information based on the material content description of each material file in the material set, filter and sort the each material file based on the story line description information, output the sorting result of N candidate materials, and output the script text corresponding to the N candidate materials based on the story line description information and the recommended style information, the first script generation prompt word is input into the large language model, which can make the large language model generate a user personalized script text, and realize video script creation for thousands of people.

[0139] FIG. 12 is a flow diagram of another video script generation method provided by an embodiment of the present application. As shown in FIG. 12, the video script generation method provided by an embodiment of the present application includes:

[0140] Step 1201, obtain material shooting information of each material file in a material set.

[0141] The material shooting information includes at least one of the following: shooting time, shooting location, and shooting object.

[0142] The shooting time is the time when the material file is shot, the shooting location is the location where the material file is shot, and the shooting object is the main character or person in the image of the material file. The shooting time, the shooting location, and the shooting object change with the change of the material file. The material shooting information such as the shooting time, the shooting location, and the shooting object can be obtained from the album program, for example, from the database of the album program.

[0143] Step 1202, generate shooting background information based on the material shooting information of each material file.

[0144] The background summary prompt word can be generated based on the material shooting information of each material file, and is used to instruct the large language model to summarize the material shooting information of each material file, and output the shooting background information. Then, the information summary prompt word can be input into the large language model to obtain the shooting background information.

[0145] For example, assuming that the shooting time of each material file is “May 1st” and the shooting location is “Tibet”, the shooting background information can be “shot in Tibet on May 1st”.

[0146] In step 1203, a second script generation prompt word is generated based on the material content description of each material file in the material set, the recommended style information, and the shooting background information.

[0147] The second script generation prompt word is used to instruct the large language model to generate story line description information based on the material content description of each material file and the shooting background information, to filter and sort each material file based on the story line description information, to output the sorting result of N alternative materials, and to output the corresponding script text of the N alternative materials based on the story line description information and the recommended style information.

[0148] In some embodiments of the present application, the material content description includes a script creation tag and a picture description text. The second script generation prompt word is specifically used to instruct the large language model to generate story line description information based on the script creation tag, the picture description text of each material file, and the shooting background information, to filter and sort each material file based on the story line description information, to output the sorting result of N alternative materials, and to output the corresponding script text of the N alternative materials based on the story line description information and the recommended style information.

[0149] The second script generation prompt word is, for example, as shown in FIG. 13. As shown in FIG. 13, the second script generation prompt word is basically the same as the first script generation prompt word shown in FIG. 11, except that the second script generation prompt word has an additional content of “shot in Tibet on May 1st”. This part of the content is generated based on the shooting background information.

[0150] In specific implementation, the material content description of each material file, the recommended style information, and the shooting background information can be embedded in a predefined script generation prompt word template to generate the second script generation prompt word.

[0151] In step 1204, the second script generation prompt word is input into the large language model to output the sorting result of N alternative materials and the corresponding script text of the N alternative materials.

[0152] The large language model can execute corresponding processing under the constraint of the second script generation prompt word according to the indication of the second script generation prompt word, and output the sorting result of the N candidate materials and the script corresponding to the N candidate materials.

[0153] In step 1205, the video script is generated based on the sorting result of the N candidate materials and the script corresponding to the N candidate materials.

[0154] By obtaining the shooting background information and generating the script generation prompt word based on the shooting background information, more rich material information can be provided for the large language model, so that the large language model fully considers the material content description and shooting background information of each material file, generates more high-quality story line description information, and further improves the story of the video script.

[0155] As can be seen from the above embodiments, the first script generation prompt word shown in FIG. 11 and the second script generation prompt word shown in FIG. 13 are basically the same as the script generation prompt word shown in FIG. 8 as "input". Compared with the script generation prompt word shown in FIG. 8, the first script generation prompt word increases the content related to the recommended style information, and the second script generation prompt word increases the content related to the shooting background information. In specific implementation, the recommended style information and the shooting background information can be obtained, and then the obtained information and the material content description of each material file are assembled using a predefined script generation prompt word template to generate a corresponding script generation prompt word. For example, in the case of obtaining the recommended style information, the material content description of each material file and the recommended style information can be assembled using a predefined script generation prompt word template to generate the first script generation prompt word. In the case of obtaining the recommended style information and the shooting background information, the material content description of each material file and the recommended style information and the shooting background information can be assembled using a predefined script generation prompt word template to generate the second script generation prompt word.

[0156] Of course, in the case of only obtaining the shooting background information, a pre-defined script generation prompt template can be used to assemble the material content description of each material file and the shooting background information to generate a third script generation prompt, which is used to instruct the large language model to generate story line description information based on the material content description of each material file and the shooting background information, filter and sort each material file based on the story line description information, and output the sorting result of N selected materials and the corresponding script text of the N selected materials. Then, the third script generation prompt can be input into the large language model to output the sorting result of N selected materials and the corresponding script text of the N selected materials. Subsequently, the video script can be generated based on the sorting result of the N selected materials and the corresponding script text of the N selected materials.

[0157] The script generation prompt template is similar to the second script generation prompt shown in FIG. 13, except that the content of the material content description, the recommended style information, and the shooting background information corresponding to each material file in the template is a placeholder. In actual application, when some information is not obtained, the corresponding information item in the template can be made empty to generate the corresponding script generation prompt.

[0158] Further, for the convenience of video creation, in some embodiments of the present application, after generating the video script, a video can be generated based on the video script.

[0159] For example, assuming that the selected material 1, the selected material 2, and the selected material 3 in the video script are all pictures, and their sorting result is 2, 1, and 3, respectively, the selected material 2 can be used as the first frame image of the video, the corresponding script text 2 of the selected material 2 can be used as the caption of the first frame image, the selected material 1 can be used as the second frame image of the video, the corresponding script text 1 of the selected material can be used as the caption of the second frame image, the selected material 3 can be used as the third frame image of the video, and the corresponding script text 3 of the selected material can be used as the caption of the third frame image, thereby generating a video.

[0160] In some embodiments, a video can be directly generated based on the video script.

[0161] For example, in a case where the user clicks a video generation control provided by the electronic device, the electronic device can jump to a photo album interface, make the user select a multimedia file, and then obtain a material set for generating a video according to the multimedia file selected by the user. Subsequently, the electronic device can generate a video script based on the material set, generate a video based on the video script, and then output the video through a video output control. The video output control can include a preview control, which can be clicked by the user to play the video. The video output control can include an export control, which can be clicked to export the video. Of course, the video creation output control can also include both the preview control and the export control, so that the user can preview and export the video.

[0162] For example, in a case where the user inputs a chat message including video clip indication information in a chat interface, the electronic device can find a multimedia file from a photo album program, obtain a material set for generating a video, generate a video script based on the material set, generate a video based on the video script, and then output the chat message including the video in the chat interface.

[0163] In some embodiments, the video can be generated based on the video script in a case where a video generation request input by the user is received, where the video includes the script text, and the display order of the script text in each video image in the video is consistent with the sorting order of the N alternative materials.

[0164] For example, in a case where the user clicks a video generation control provided by the electronic device, the electronic device can jump to a photo album interface, make the user select a multimedia file, and then obtain a material set for generating a video according to the multimedia file selected by the user. Subsequently, the electronic device can generate a video script based on the material set, and then output the video script through a video script output control provided by the electronic device. Subsequently, in a case where the user clicks a video generation control provided by the electronic device, the electronic device can take the images of the N alternative materials in the video script as N video images of a video to be generated based on the video script, sort the N video images according to the sorting result of the N alternative materials, and display the script text corresponding to the N alternative materials on the corresponding N video images, respectively, so as to generate a video, and then output the video through a video output control.

[0165] For another example, when the user inputs a chat message including video clip indication information in the chat interface, the electronic device can find the multimedia file from the album program, obtain a material set for generating a video, then generate a video script based on the material set, output a chat message including the video script in the chat interface, and then, when receiving a chat message including indication information for generating a video based on the video script input by the user, the electronic device can sort N frames of video images of N alternative materials in the video script as N frames of video images of the video to be generated based on the video script, and sort the N frames of video images according to the sorting result of the N alternative materials, and display the text script corresponding to the N alternative materials on the corresponding N frames of video images respectively, so as to generate a video, and then output a chat message including the video in the chat interface.

[0166] In the embodiments of the present application, the video can be generated based on the video script, which can reduce the burden of the user's video creation. In addition, the video script generated by the embodiments of the present application has stronger story, and the video generated based on the video script can enhance the expressiveness of the video content, enhance the information transmission and emotional expression ability of the video, and improve the attractiveness of the video.

[0167] In the above embodiments, the video script is an intermediate result in the process of generating a video according to the user's video generation request, which can be output to the user or not. In some cases, the video script can be generated and output directly according to the user's video script generation request. In some embodiments of the present application, generating the script generation prompt word based on the material content description of each material file in the material set can include: in the case of receiving the video script generation request input by the user, generating the script generation prompt word based on the material content description of each material file in the material set.

[0168] For example, the video script generation request control can be provided on the electronic device, and in response to the user's click operation on the video script generation request control, the electronic device can generate the script generation prompt word based on the material content description of each material file in the material set, input the script generation prompt word into the large language model, output the sorting result of the N alternative materials and the text script corresponding to the N alternative materials, generate the video script based on the sorting result of the N alternative materials and the text script corresponding to the N alternative materials, and then output the video script.

[0169] By generating and outputting the video script according to the user's video script generation request, the user can further create a video based on the video script, and the user experience can be further improved. For example, the user can add special effect information to some alternative materials in the video script, and then generate a video based on the video script with added special effect information.

[0170] In some embodiments of the present application, in the case of outputting a video script, in order to improve the quality of the output video script, after the video script is generated, the quality score of the video script can be determined, and when the quality score of the video script is greater than a quality score threshold, the video script is outputted again.

[0171] FIG. 14 is a schematic diagram of a video script output process according to some embodiments of the present application. As shown in FIG. 14, in some embodiments of the present application, after the video script is generated, further comprising:

[0172] Step 1401, determining the quality score of the video script based on the dimension information of the scoring dimension of the script text corresponding to the N candidate materials.

[0173] The scoring dimension includes at least one of the following: the number of words of the script text, the number of times of continuous repeated descriptions in the script text, and the number of times of error characters in the script text.

[0174] For example, the number of script texts with a word count greater than a first word count threshold in the script texts of the N candidate materials can be counted, denoted as a; the number of script texts with a word count less than a second word count threshold can be counted, denoted as b; the number of times of format errors such as <tag, description text> in the script texts can be counted, denoted as c; the number of times of format errors such as <four-word description, four-word description> in the script texts can be counted, denoted as d; the number of times of repeated fields in the script texts can be counted, denoted as e; the number of times of format errors such as “place” in the script texts can be counted, denoted as f. Then, the quality score S of the video script is calculated based on the formula S = 100-a*10-b*10-M, where M = 50 when any one of b, c, d, e, f is greater than N*0.4, otherwise M = 0.

[0175] Step 1402, displaying the N script text identifiers in the case where the quality score of the video script is greater than the quality score threshold.

[0176] The script text identifier is used to indicate a candidate material and the script text corresponding to the candidate material, which can be a thumbnail of the candidate material image with the script text added.

[0177] Continuing with the above example, assuming that the quality score threshold is 60, when the quality score S of the video script is greater than 60, the thumbnail of the image of the N candidate materials can be displayed according to the sorting result of the N candidate materials, and the script text corresponding to the candidate material can be displayed at a specified position of each candidate material image thumbnail.

[0178] For example, assuming that the quality score of the video script generated based on the N candidate materials corresponding to the captions shown in the output of FIG. 8 is greater than the quality score threshold, the video script can be displayed as shown in FIG. 15b. As shown in FIG. 15b, the candidate materials in the output of FIG. 8 are picture 1, picture 2, picture 3, and picture 4, and the ranking result is: 1, 4, 2, 3. The electronic device can display the thumbnails of picture 1, picture 4, picture 2, and picture 3 according to the ranking result, and display the captions corresponding to the thumbnails of picture 1, picture 4, picture 2, and picture 3, thereby displaying the video script.

[0179] Step 1403, in the case where the quality score of the video script is less than or equal to the quality score threshold, re-generate the video script.

[0180] In the re-generation of the video script, steps 101-103 can be re-executed, the script generation prompt words can be re-generated based on the material content description of each material file in the material set, the script generation prompt words can be re-input into the large language model, the large language model can be instructed to re-generate the story line description information based on the material content description of each material file, the N candidate materials can be re-output based on the story line description information, and the N candidate materials corresponding to the captions can be re-output based on the ranking result of the N candidate materials and the captions corresponding to the N candidate materials. Then, the video script can be generated based on the re-output ranking result of the N candidate materials and the captions corresponding to the N candidate materials. After that, steps 1401-1403 can be re-executed until the video script with a quality score greater than the quality score threshold is output.

[0181] By determining the quality score of the video script, the video script is output again when the quality score of the video script is greater than the quality score threshold, which can improve the quality of the output video script and facilitate the user to create a high-quality video based on the high-quality video script.

[0182] FIG. 16 is a schematic diagram of a modification process of a caption script according to some embodiments of the present application. As shown in FIG. 16, in order to provide a video script that meets the user's needs and improve the user experience, after displaying the N caption script identifiers, the process further includes:

[0183] Step 1601, receiving a selection input of a target caption script identifier from the N caption script identifiers.

[0184] The selection input is used to select a caption script that the user is not satisfied with.

[0185] Following the example of FIG. 15b, the electronic device can set a check box below the candidate material thumbnail displaying the caption script of each candidate material. The user can check the check box of the candidate material thumbnail whose caption script is not satisfactory. According to the selection operation of the user on the check box corresponding to each candidate material thumbnail, the electronic device can determine the caption script in the candidate material thumbnail corresponding to the checked check box as the target caption script.

[0186] In step 1602, a caption modification prompt word is generated based on the target caption script indicated by the target caption script identification and the target material corresponding to the target caption script.

[0187] The caption modification prompt word is used to instruct the large language model to regenerate the caption script corresponding to the target material. For example, assuming that the target caption script is caption script 2, and the caption script 2 corresponds to candidate material 2, the candidate material 2 is determined as the target material.

[0188] Following the example of FIG. 15b, in FIG. 15b, the user selects the caption script corresponding to material 2 and material 3. The caption modification prompt word generated based on material 2 and material 3 and the corresponding caption script is shown as “input” in FIG. 17.

[0189] In step 1603, the caption modification prompt word is input into the large language model, and the regenerated caption script corresponding to the target material is output.

[0190] The large language model can perform corresponding processing according to the instruction of the caption modification prompt word, regenerate the caption script corresponding to the target material, and output.

[0191] Following the example shown in FIG. 17, when the caption modification prompt word shown in FIG. 17 is input into the large language model, the output of the large language model is shown as “output” in FIG. 17.

[0192] By allowing the user to select the unsatisfactory caption script and regenerate the unsatisfactory caption script, the user can be provided with a satisfactory caption script, and the caption script can be more in line with the user's needs, thereby improving the user experience.

[0193] The video caption generation method provided in the embodiment of the present application can be executed by a video caption generation device. In the embodiment of the present application, the video caption generation device is taken as an example to illustrate the video caption generation device provided in the embodiment of the present application.

[0194] FIG. 18 is a module schematic diagram of a video caption generation device provided in an embodiment of the present application. As shown in FIG. 18, the video caption generation device 1800 provided in the embodiment of the present application comprises:

[0195] The script prompt word generation module 1801 is configured to generate a script generation prompt word based on the material content description of each material file in the material set, wherein the material set is used to generate a video, the script generation prompt word is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N candidate materials obtained by filtering and a script corresponding to the N candidate materials, the material set includes at least two material files, the material file is a picture material file or a video material file, and N is an integer greater than 1.

[0196] The sorting and script obtaining module 1802 is configured to input the script generation prompt word into a large language model and output a sorting result of N candidate materials and a script corresponding to the N candidate materials, wherein one candidate material corresponds to one script.

[0197] The video script generation module 1803 is configured to generate a video script based on the sorting result of the N candidate materials and the script corresponding to the N candidate materials.

[0198] In some embodiments of the present application, the device 1800 further includes:

[0199] The obtaining module is configured to obtain user portrait information of a user, wherein the user portrait information includes at least one of the following: gender, age, zodiac, type, occupation, and interest.

[0200] The style prompt word generation module is configured to generate a style recommendation prompt word based on the user portrait information, wherein the style recommendation prompt word is used to instruct a large language model to output recommended style information based on the user portrait information.

[0201] The recommended style obtaining module is configured to input the style recommendation prompt word into a large language model and output recommended style information.

[0202] The script prompt word generation module is specifically configured to generate a first script generation prompt word based on the material content description of each material file in the material set and the recommended style information, wherein the first script generation prompt word is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, to output a sorting result of N candidate materials, and to output a script corresponding to the N candidate materials based on the story line description information and the recommended style information.

[0203] The sorting material and script obtaining module is specifically configured to: input the first script generation prompt word into a large language model, and output a sorting result of N candidate materials and script texts corresponding to the N candidate materials.

[0204] In some embodiments of the present application, the material content description includes a script creation label and a picture description text.

[0205] The obtaining module is further configured to obtain material shooting information of each material file in the material set, the material shooting information including at least one of the following: shooting time, shooting location, and shooting object.

[0206] The device further includes a shooting background generation module configured to generate shooting background information based on the material shooting information of each material file.

[0207] The script prompt word generation module is specifically configured to: generate a second script generation prompt word based on the material content description of each material file in the material set, the recommended style information, and the shooting background information, the second script generation prompt word being used to instruct the large language model to generate story line description information based on the material content description of each material file and the shooting background information, to filter and sort each material file based on the story line description information, to output a sorting result of N candidate materials, and to output script texts corresponding to the N candidate materials based on the story line description information and the recommended style information.

[0208] The sorting and script obtaining module is specifically configured to:

[0209] input the second script generation prompt word into the large language model, and output a sorting result of N candidate materials and video scripts corresponding to the N candidate materials.

[0210] In some embodiments of the present application, the device 1800 further includes:

[0211] A label determination module is configured to determine a script creation label of each material file in the material set based on a script creation label in a script creation label set that has the highest similarity with the material file; the script creation label in the script creation label set is obtained by filtering from a candidate label set based on a script creation suitability score of the candidate label.

[0212] A description text obtaining module is configured to input each material file in the material set into a description model, and output a picture description text of each material file.

[0213] In some embodiments of the present application, the material file includes a video material file; and the device 1800 further includes:

[0214] The frame extraction module is configured to extract at least two video images of each video material file according to a video frame extraction interval.

[0215] The tag determination module is specifically configured to:

[0216] determine the similarity between the copywriting tag and the video material file based on the similarity between the copywriting tag and each of the at least two video images.

[0217] determine the copywriting tag of each video material file based on the copywriting tag with the highest similarity to the video material file in the copywriting tag set.

[0218] The description text acquisition module is specifically configured to:

[0219] input each of the at least two video images into a description model to output a picture description text of each video image.

[0220] generate a description summary prompt word based on the picture description text of each image, the description summary prompt word being used to instruct a large language model to summarize the picture description text of each video image and output a summary description text.

[0221] input the description summary prompt word into the large language model to output the summary description text.

[0222] generate a picture description text of the video material file based on the summary description text.

[0223] In some embodiments of the present application, the apparatus 1800 further includes:

[0224] The description tag set generation module is configured to generate an image description tag set based on the image description text set.

[0225] The candidate tag set generation module is configured to generate a candidate tag set based on the image description tag set and an image classification tag set.

[0226] The tag score determination module is configured to determine a copywriting suitability score of each candidate tag in the candidate tag set based on the occurrence frequency of each candidate tag in the candidate tag set and the copywriting beauty degree.

[0227] The copywriting tag set generation module is configured to filter all candidate tags based on the copywriting suitability score of each candidate tag to obtain a copywriting tag set.

[0228] In some embodiments of the present application, the apparatus 1800 further includes:

[0229] The material selection receiving module is configured to receive a file selection input of a user on the album interface.

[0230] The first material set determining module is configured to determine, in response to the file selection input, multimedia files selected by the file selection input as a material set for generating a video.

[0231] The multimedia files include at least one of an image and a video.

[0232] In some embodiments of the present application, the apparatus 1800 further includes:

[0233] The chat information receiving module is configured to receive a chat message input by a user on a chat interface.

[0234] The keyword obtaining module is configured to, in a case where the chat message includes video clip indication information, obtain a clip requirement keyword in the chat message.

[0235] The searching module is configured to search, in an album program, at least one multimedia file matching the clip requirement keyword.

[0236] The second material set determining module is configured to determine, as a material set for generating a video, at least one multimedia file matching the clip requirement keyword.

[0237] In some embodiments of the present application, the apparatus 1800 further includes:

[0238] The video generating module is configured to, in a case where a video generation request input by a user is received, generate a video based on the video script.

[0239] In some embodiments of the present application, the apparatus 1800 further includes:

[0240] The quality score determining module is configured to determine a quality score of the video script based on dimension information of a scoring dimension of the script corresponding to the N alternative materials, the scoring dimension including at least one of a number of words in the script, a number of times of continuous repeated descriptions in the script, and a number of times of error characters in the script.

[0241] The display module is configured to, in a case where the quality score of the video script is greater than a quality score threshold, display N script identifiers, the script identifier being used to indicate an alternative material and a script corresponding to the alternative material.

[0242] The apparatus is further configured to, in a case where the quality score of the video script is less than or equal to the quality score threshold, regenerate a video script.

[0243] In some embodiments of the present application, the apparatus 1800 further comprises:

[0244] The script selection receiving module is configured to receive a selection input of a target script template identifier in the N script template identifiers by the user; the selection input is used to select a script template that the user is not satisfied with;

[0245] The script modification prompt word generating module is configured to generate a script modification prompt word based on the target script template and the target material corresponding to the target script template, the script modification prompt word being used to instruct the large language model to regenerate the script template corresponding to the target material;

[0246] The script modification prompt word generating module is configured to generate a script modification prompt word based on the target script template and the target material corresponding to the target script template, the script modification prompt word being used to instruct the large language model to regenerate the script template corresponding to the target material;

[0247] In some embodiments of the present application, the script prompt word generating module 1801 is further configured to:

[0248] In the case of receiving a video script generation request input by the user, generate a script generation prompt word based on the material content description of each material file in the material set.

[0249] In the embodiments of the present application, by obtaining the material content description of each material in the material set used by the user to generate a video, instructing the large language model to generate a story line based on the material content description corresponding to each material, filtering and sorting the materials based on the story line, generating the script template corresponding to each material, and further generating a video script, the video script can be generated based on the story line generated by comprehensively considering the material content description of each material. Compared with the video script generated based on the prior art, the connection of the script template of each material in the video script generated based on the apparatus provided in the embodiments of the present application is smoother, the script template of each material is more consistent with the content of each material and the overall theme of the video, and the overall video script is more story-like. Therefore, based on the apparatus provided in the embodiments of the present application, the expressiveness of the video content can be improved, the information transmission and emotional expression ability of the video can be enhanced, and the burden of video script creation can be reduced. Moreover, the apparatus provided in the embodiments of the present application can automatically generate a video script, thereby improving the generation efficiency of the video script.

[0250] The video script generation apparatus in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the like. The electronic device can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like. The embodiments of the present application are not limited in this regard.

[0251] The video script generation apparatus in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an ios operating system, or other possible operating systems. The embodiments of the present application are not limited in this regard.

[0252] The video script generation apparatus provided in the embodiments of the present application can implement each process implemented by the method embodiments of FIGS. 1 to 17. To avoid repetition, the details are not described here.

[0253] Optionally, as shown in FIG. 19, the embodiments of the present application further provide an electronic device 1900, which includes a processor 1901 and a memory 1902. The memory 1902 stores programs or instructions executable on the processor 1901. When the programs or instructions are executed by the processor 1901, the following functions are implemented:

[0254] generating a script generation prompt word based on the material content description of each material file in the material set, wherein the material set is used to generate a video, the script generation prompt word is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N candidate materials obtained by filtering and a script corresponding to the N candidate materials, the material set includes at least two material files, the material file is a picture material file or a video material file, and N is an integer greater than 1;

[0255] input the script generation prompt word into the large language model, and output a sorting result of N candidate materials and a script text corresponding to the N candidate materials;

[0256] generate a video script based on the sorting result of the N candidate materials and the script text corresponding to the N candidate materials.

[0257] In some embodiments of the present application, the processor 1901 is specifically configured to obtain user portrait information of a user, the user portrait information including at least one of the following: gender, age, zodiac, type, occupation, and interest; generate a style recommendation prompt word based on the user portrait information, the style recommendation prompt word being used to instruct the large language model to output recommended style information based on the user portrait information; and input the style recommendation prompt word into the large language model to output the recommended style information.

[0258] generate a first script generation prompt word based on the material content description of each material file in the material set and the recommended style information, the first script generation prompt word being used to instruct the large language model to generate story line description information based on the material content description of each material file, filter and sort the material file based on the story line description information, output a sorting result of N candidate materials, and output a script text corresponding to the N candidate materials based on the story line description information and the recommended style information;

[0259] input the first script generation prompt word into the large language model, and output a sorting result of N candidate materials and a script text corresponding to the N candidate materials.

[0260] In some embodiments of the present application, the material content description includes a script creation label and a picture description text; and the processor 1901 is specifically configured to:

[0261] obtain material shooting information of each material file in the material set, the material shooting information including at least one of the following: shooting time, shooting location, and shooting object;

[0262] generate shooting background information based on the material shooting information of each material file;

[0263] generate a second script generation prompt word based on the material content description of each material file in the material set, the recommended style information, and the shooting background information, the second script generation prompt word being used to instruct a large language model to generate story line description information based on the material content description of each material file and the shooting background information, filter and sort the each material file based on the story line description information, output a sorting result of N alternative materials, and output a script corresponding to the N alternative materials based on the story line description information and the recommended style information;

[0264] input the second script generation prompt word into a large language model, and output a sorting result of N alternative materials and a video script corresponding to the N alternative materials.

[0265] In some embodiments of the present application, the processor 1901 is specifically configured to:

[0266] determine a script creation tag of each material file in the material set based on a script creation tag in the script creation tag set that is most similar to the material file; the script creation tag in the script creation tag set is obtained by filtering from a candidate tag set based on a script creation suitability score of the candidate tag;

[0267] input each material file in the material set into a description model, and output a picture description text of each material file.

[0268] In some embodiments of the present application, the processor 1901 is specifically configured to:

[0269] extract at least two frames of video images of each video material file according to a video frame extraction interval;

[0270] determine a similarity between a script creation tag and the video material file based on a similarity between the script creation tag and each frame of video image in the at least two frames of video images;

[0271] determine a script creation tag of each video material file based on a script creation tag in the script creation tag set that is most similar to the video material file;

[0272] input each frame of video image in the at least two frames of video images into a description model, and output a picture description text of each frame of video image;

[0273] generate a description summary prompt word based on the picture description text of each frame of image, the description summary prompt word being used to instruct a large language model to summarize the picture description text of each frame of video image, and output a summary description text;

[0274] input the description summary prompt word into a large language model, and output a summary description text;

[0275] generate the picture description text of the video material file based on the summary description text.

[0276] In some embodiments of the present application, the processor 1901 is specifically configured to:

[0277] generate an image description label set based on the image description text set;

[0278] generate a candidate label set based on the image description label set and an image classification label set;

[0279] determine a script creation suitability score of each candidate label in the candidate label set based on the occurrence frequency of each candidate label in the candidate label set and the elegance degree of the script;

[0280] screen all candidate labels based on the script creation suitability score of each candidate label to obtain a script creation label set.

[0281] In some embodiments of the present application, the processor 1901 is specifically configured to, before generating the script generation prompt word based on the material content description of each material file in the material set,

[0282] receive a file selection input of a user on a photo album interface;

[0283] In response to the file selection input, determine the multimedia files selected by the file selection input as a material set for generating a video;

[0284] The multimedia file includes at least one of the following: an image, a video.

[0285] In some embodiments of the present application, the processor 1901 is specifically configured to, before generating the script generation prompt word based on the material content description of each material file in the material set,

[0286] receive a chat message input by a user on a chat interface;

[0287] In the case where the chat message includes video clip indication information, obtain a clip requirement keyword in the chat message;

[0288] find at least one multimedia file matching the clip requirement keyword in a photo album program;

[0289] determine the at least one multimedia file matching the clip requirement keyword as a material set for generating a video.

[0290] In some embodiments of the present application, the processor 1901 is specifically configured to: after generating the video script based on the ranking results of the N candidate materials and the script templates corresponding to the N candidate materials,

[0291] In the case of receiving a video generation request input by the user, generate a video based on the video script.

[0292] In some embodiments of the present application, the processor 1901 is specifically configured to: after generating the video script based on the ranking results of the N candidate materials and the script templates corresponding to the N candidate materials,

[0293] Based on the dimension information of the scoring dimension of the script template corresponding to the N candidate materials, determine the quality score of the video script, the scoring dimension includes at least one of the following: the number of words of the script template, the number of times of continuous repeated description in the script template, and the number of times of error characters in the script template;

[0294] In the case where the quality score of the video script is greater than the quality score threshold, display the N candidate materials and the script templates corresponding to the N candidate materials;

[0295] In the case where the quality score of the video script is less than or equal to the quality score threshold, regenerate the video script.

[0296] In some embodiments of the present application, the processor 1901 is specifically configured to: after displaying the N script template identifiers,

[0297] Receive a selection input of a target script template in the script templates corresponding to the N candidate materials by the user; the selection input is used to select a script template that the user is not satisfied with;

[0298] Based on the target script template and the target material corresponding to the target script template, generate a script modification prompt word, the script modification prompt word is used to instruct a large language model to regenerate a script template corresponding to the target material; the target material is a candidate material corresponding to the target script template;

[0299] Input the script modification prompt word into the large language model, and output the regenerated script template corresponding to the target material.

[0300] In some embodiments of the present application, the processor 1901 is specifically configured to:

[0301] In the case of receiving a video script generation request input by the user, generate a script generation prompt word based on the material content description of each material file in the material set.

[0302] In the embodiment of the present application, by obtaining the material content description of each material in the material set used by the user to generate the video, instructing the large language model to generate a story line based on the material content description corresponding to each material, filtering and sorting each material based on the story line, generating the text matching script corresponding to each material, and further generating the video script, the video script can be generated based on the story line generated by comprehensively considering the material content description of each material. Compared with the video script generated based on the prior art, the connection of the text matching script of each material in the video script generated based on the device provided in the embodiment of the present application is smoother, the text matching script of each material is more consistent with the content of each material and the overall theme of the video, and the overall video script is more story-based. Therefore, based on the device provided in the embodiment of the present application, the expressiveness of the video content can be improved, the information transmission and emotional expression ability of the video can be enhanced, and the burden of video script creation can be reduced. Moreover, the device provided in the embodiment of the present application can automatically generate the video script, thereby improving the generation efficiency of the video script.

[0303] It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0304] FIG. 20 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application.

[0305] The electronic device 2000 includes, but is not limited to, a radio frequency unit 2001, a network module 2002, an audio output unit 2003, an input unit 2004, a sensor 2005, a display unit 2006, a user input unit 2007, an interface unit 2008, a memory 2009, and a processor 2010, etc.

[0306] Those skilled in the art can understand that the electronic device 2000 can further include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 2010 through a power management system, so as to realize the functions of power management, such as charging, discharging, and power consumption management, through the power management system. The electronic device structure shown in FIG. 20 does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the illustrated components, or combine certain components, or different component arrangements, which are not described here again.

[0307] It should be understood that in the embodiments of the present application, the input unit 2004 can include a graphics processor (GPU) 20041 and a microphone 20042. The graphics processor 20041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 2006 can include a display panel 20061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 2007 includes at least one of a touch panel 20071 and other input devices 20072. The touch panel 20071 is also referred to as a touch screen. The touch panel 20071 can include two parts of a touch detection device and a touch controller. The other input devices 20072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, a joystick, and the like, which will not be described here.

[0308] The memory 2009 can be used to store software programs and various data. The memory 2009 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 2009 can include a volatile memory or a non-volatile memory, or the memory 2009 can include both a volatile memory and a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 2009 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0309] The processor 2010 can include one or more processing units; optionally, the processor 2010 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 2010.

[0310] The embodiments of the present application also provide a readable storage medium, and the readable storage medium stores programs or instructions, which are executed by a processor to realize each process of the above-mentioned video script generation method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0311] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0312] It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that includes the element. In addition, it should be noted that the scope of the methods and apparatus of the present embodiments are not limited by the order of the steps or the sequences for performing the steps, as some steps can occur in different order or concurrently with each other. For example, described methods can occur in an order other than that described, and / or various illustrative steps can occur in different orders or concurrently with each other. Additionally, features described with respect to certain examples can be combined in other examples.

[0313] From the above description of the embodiments, it can be clear to those skilled in the art that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network device, etc.) execute the methods described in various embodiments of the present application.

[0314] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative, not restrictive, and those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A video script generation method, the method comprising: generating a script generation prompt word based on a material content description of each material file in a material set, wherein the material set is used to generate a video, the script generation prompt word is used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, and to output a sorting result of N candidate materials obtained by filtering and a text matching script corresponding to the N candidate materials, the material set includes at least two material files, the material file is a picture material file or a video material file, and N is an integer greater than 1; inputting the script generation prompt word into a large language model to output a sorting result of N candidate materials and a text matching script corresponding to the N candidate materials, wherein one candidate material corresponds to one text matching script; generating a video script based on the sorting result of the N candidate materials and the text matching script corresponding to the N candidate materials.

2. The method of claim 1, wherein, The method further comprises: obtaining user portrait information of a user, the user portrait information including at least one of the following: gender, age, zodiac, type, occupation, and interest; generating a style recommendation prompt word based on the user portrait information, the style recommendation prompt word being used to instruct a large language model to output recommended style information based on the user portrait information; inputting the style recommendation prompt word into a large language model to output recommended style information; The generating a script generation prompt word based on a material content description of each material file in a material set comprises: generating a first script generation prompt word based on the material content description of each material file in the material set and the recommended style information, the first script generation prompt word being used to instruct a large language model to generate story line description information based on the material content description of each material file, to filter and sort each material file based on the story line description information, to output a sorting result of N candidate materials, and to output a text matching script corresponding to the N candidate materials based on the story line description information and the recommended style information; The inputting the script generation prompt word into a large language model to output a sorting result of N candidate materials and a text matching script corresponding to the N candidate materials comprises: inputting the first script generation prompt word into a large language model to output a sorting result of N candidate materials and a text matching script corresponding to the N candidate materials.

3. The method of claim 2, wherein, The material content description includes a script creation tag and a picture description text; the method further comprises: obtaining material shooting information of each material file in the material set, the material shooting information including at least one of the following: shooting time, shooting location, and shooting object; generating shooting background information based on the material shooting information of each material file; The generating a script generation prompt word based on a material content description of each material file in a material set comprises: generate a second script generation prompt word based on the material content description of each material file in the material set, the recommended style information, and the shooting background information, the second script generation prompt word being used to instruct a large language model to generate story line description information based on the material content description of each material file and the shooting background information, filter and sort the each material file based on the story line description information, output a sorting result of N alternative materials, and output a script corresponding to the N alternative materials based on the story line description information and the recommended style information; the inputting the script generation prompt word into the large language model and the outputting the sorting result of N alternative materials and the video script corresponding to the N alternative materials include: the inputting the second script generation prompt word into the large language model and the outputting the sorting result of N alternative materials and the video script corresponding to the N alternative materials.

4. The method of claim 3, wherein, The method further includes: determining a script creation tag of each material file in the material set based on a script creation tag in a script creation tag set that is most similar to the material file; the script creation tag in the script creation tag set being selected from a candidate tag set based on a script creation suitability score of each candidate tag in the candidate tag set; inputting each material file in the material set into a description model to output a picture description text of each material file.

5. The method of claim 4, wherein, The material file includes a video material file; the method further includes: extracting at least two frames of video images of each video material file according to a video frame extraction interval; the determining the script creation tag of each material file in the material set based on a script creation tag in a script creation tag set that is most similar to the material file includes: determining a similarity between the script creation tag and the video material file based on a similarity between the script creation tag and each frame of video image in the at least two frames of video images; determining the script creation tag of each video material file based on a script creation tag in a script creation tag set that is most similar to the video material file; the inputting each material file in the material set into a description model to output a picture description text of each material file includes: inputting each frame of video image in the at least two frames of video images into the description model to output a picture description text of each frame of video image; generating a description summary prompt word based on the picture description text of each frame of image, the description summary prompt word being used to instruct a large language model to summarize the picture description text of each frame of video image and output a summary description text; inputting the description summary prompt word into the large language model to output the summary description text; generating the picture description text of the video material file based on the summary description text.

6. The method of claim 4, wherein, The method further includes: generating an image description tag set based on an image description text set; generating a candidate tag set based on the image description tag set and an image classification tag set; determining a script creation suitability score of each candidate tag in the candidate tag set based on a frequency of occurrence of each candidate tag in the candidate tag set and a script generation beauty degree; Filter all candidate labels based on the script creation suitability score of each candidate label to obtain a script creation label set.

7. The method of claim 1, wherein, Before generating the script generation prompt word based on the material content description of each material file in the material set, the method further includes: receiving a file selection input of a user on the album interface; in response to the file selection input, determining the multimedia files selected by the file selection input as a material set for generating a video; wherein the multimedia files include at least one of the following: images, videos.

8. The method of claim 1, wherein, Before generating the script generation prompt word based on the material content description of each material file in the material set, the method further includes: receiving a chat message input by a user on a chat interface; in a case where the chat message includes video clip indication information, obtaining clip requirement keywords in the chat message; finding at least one multimedia file matching the clip requirement keywords in the album program; determining the at least one multimedia file matching the clip requirement keywords as a material set for generating a video.

9. The method of claim 1, 7, or 8, wherein, After generating the video script based on the ranking result of the N candidate materials and the script text corresponding to the N candidate materials, the method further includes: in a case where a video generation request input by a user is received, generating a video based on the video script; wherein the video includes the script text, and the display order of the script text in each frame of video image in the video is consistent with the ranking order of the N candidate materials.

10. The method of claim 1, wherein, After generating the video script based on the ranking result of the N candidate materials and the script text corresponding to the N candidate materials, the method further includes: determining a quality score of the video script based on dimension information of a scoring dimension of the script text corresponding to the N candidate materials, the scoring dimension including at least one of the following: the number of words in the script text, the number of times of continuous repeated description appearing in the script text, and the number of times of error characters appearing in the script text; in a case where the quality score of the video script is greater than a quality score threshold, displaying N script text identifiers, the script text identifier being used to indicate a candidate material and the script text corresponding to the candidate material; in a case where the quality score of the video script is less than or equal to the quality score threshold, regenerating a video script.

11. The method of claim 10, wherein, After displaying the N script text identifiers, the method further includes: receiving a selection input of a target script text identifier in the N script text identifiers by a user, the selection input being used to select a script text that a user is not satisfied with; generating a script modification prompt word based on a target script text indicated by the target script text identifier and a target material corresponding to the target script text, the script modification prompt word being used to instruct a large language model to regenerate the script text corresponding to the target material; inputting the script modification prompt word into the large language model to output the regenerated script text corresponding to the target material.

12. The method of claim 1, 7, or 8, wherein, The method of generating the script generation prompt word based on the material content description of each material file in the material set includes: In a case where a video script generation request input by a user is received, script generation prompt words are generated based on material content descriptions of each material file in a material set.

13. A video script generation apparatus, comprising: a script prompt word generation module configured to generate script generation prompt words based on material content descriptions of each material file in a material set, wherein the material set is used to generate a video, the script generation prompt words are used to instruct a large language model to generate story line description information based on the material content descriptions of the each material file, to screen and sort the each material file based on the story line description information, and to output sorting results of N candidate materials obtained by screening and script text corresponding to the N candidate materials, the material set includes at least two material files, the material files are picture material files or video material files, and N is an integer greater than 1; a sorting and script obtaining module configured to input the script generation prompt words into the large language model, and output the sorting results of the N candidate materials and the script text corresponding to the N candidate materials, wherein one candidate material corresponds to one script text; a video script generation module configured to generate a video script based on the sorting results of the N candidate materials and the script text corresponding to the N candidate materials.

14. The apparatus of claim 13, wherein, The apparatus further comprises: an obtaining module configured to obtain user portrait information of a user, wherein the user portrait information includes at least one of the following: gender, age, zodiac, type, occupation, and interest; a style prompt word generation module configured to generate style recommendation prompt words based on the user portrait information, wherein the style recommendation prompt words are used to instruct the large language model to output recommended style information based on the user portrait information; a recommended style obtaining module configured to input the style recommendation prompt words into the large language model, and output the recommended style information; the script prompt word generation module is specifically configured to generate first script generation prompt words based on the material content descriptions of each material file in the material set and the recommended style information, wherein the first script generation prompt words are used to instruct the large language model to generate story line description information based on the material content descriptions of the each material file, to screen and sort the each material file based on the story line description information, to output sorting results of N candidate materials, and to output script text corresponding to the N candidate materials based on the story line description information and the recommended style information; the sorting material and script obtaining module is specifically configured to input the first script generation prompt words into the large language model, and output the sorting results of the N candidate materials and the script text corresponding to the N candidate materials.

15. The apparatus of claim 14, wherein, The material content descriptions include script creation tags and picture description texts. The obtaining module is further configured to obtain material shooting information of each material file in the material set, wherein the material shooting information includes at least one of the following: shooting time, shooting location, and shooting object. The apparatus further comprises a shooting background generation module configured to generate shooting background information based on the material shooting information of the each material file. The script prompt word generation module is specifically configured to: based on the material content description of each material file in the material set, the recommended style information, and the shooting background information, generate second script generation prompt words, the second script generation prompt words being used to instruct a large language model to generate story line description information based on the material content description of each material file and the shooting background information, filter and sort the each material file based on the story line description information, output a sorting result of N candidate materials, and output a script corresponding to the N candidate materials based on the story line description information and the recommended style information; The sorting and script obtaining module is specifically configured to: input the second script generation prompt words into the large language model, and output the sorting result of the N candidate materials and the video script corresponding to the N candidate materials.

16. The apparatus of claim 14, wherein, The device further comprises: A label determination module is configured to determine a script creation label of each material file in the material set based on a script creation label in a script creation label set that has the highest similarity with the material file; the script creation label in the script creation label set is obtained by filtering candidate labels based on a script creation suitability score of the candidate labels; A description text obtaining module is configured to input each material file in the material set into a description model, and output a picture description text of each material file.

17. The apparatus of claim 16, wherein, The material file comprises a video material file; the device further comprises: An extracting module is configured to extract at least two video images of each video material file according to a video frame extraction interval; The label determination module is specifically configured to: determine a similarity between a script creation label and the video material file based on a similarity between the script creation label and each video image in the at least two video images; determine a script creation label of each video material file based on a script creation label in the script creation label set that has the highest similarity with the video material file; The description text obtaining module is specifically configured to: input each video image in the at least two video images into the description model, and output a picture description text of each video image; generate a description summary prompt word based on the picture description text of each image, the description summary prompt word being used to instruct the large language model to summarize the picture description text of each video image, and output a summary description text; input the description summary prompt word into the large language model, and output the summary description text; generate a picture description text of the video material file based on the summary description text.

18. The apparatus of claim 16, wherein, The device further comprises: A description label set generation module is configured to generate an image description label set based on an image description text set; A candidate label set generation module is configured to generate a candidate label set based on the image description label set and an image classification label set; A label score determination module is configured to determine a script creation suitability score of each candidate label in the candidate label set based on a frequency of occurrence of each candidate label in the candidate label set and a script generation beauty degree. The script creation tag set generation module is configured to filter all candidate tags based on the script creation suitability score of each candidate tag to obtain a script creation tag set.

19. The apparatus of claim 13, wherein, The device further includes: The material selection receiving module is configured to receive a file selection input of a user on the album interface. The first material set determination module is configured to determine, in response to the file selection input, multimedia files selected by the file selection input as a material set for generating a video. The multimedia files include at least one of the following: images, videos.

20. The apparatus of claim 13, wherein, The device further includes: The chat information receiving module is configured to receive a chat message input by a user on a chat interface. The keyword obtaining module is configured to, in a case where the chat message includes video clip indication information, obtain a clip requirement keyword in the chat message. The searching module is configured to search, in an album program, at least one multimedia file matching the clip requirement keyword. The second material set determination module is configured to determine the at least one multimedia file matching the clip requirement keyword as a material set for generating a video.

21. The apparatus of claim 13, 19, or 20, wherein, The device further includes: The video generation module is configured to, in a case where a video generation request input by a user is received, generate a video based on the video script. The video includes the script, and a display order of the script in each frame of video image in the video is consistent with an order of the N candidate materials.

22. The apparatus of claim 13, wherein, The device further includes: The quality score determination module is configured to determine a quality score of the video script based on dimension information of a scoring dimension of the script corresponding to the N candidate materials, the scoring dimension including at least one of the following: a word count of the script, a number of times of continuous repeated descriptions in the script, and a number of times of error characters in the script. The display module is configured to, in a case where the quality score of the video script is greater than a quality score threshold, display N script identifiers, the script identifiers being used to indicate one candidate material and a script corresponding to the candidate material. The device is further configured to, in a case where the quality score of the video script is less than or equal to the quality score threshold, regenerate a video script.

23. The apparatus of claim 22, wherein, The device further includes: The script selection receiving module is configured to receive a selection input of a target script identifier in the N script identifiers by a user, the selection input being used to select a script that is not satisfactory to the user. The script modification prompt word generation module is configured to generate a script modification prompt word based on a target script indicated by the target script identifier and a target material corresponding to the target script, the script modification prompt word being used to instruct a large language model to regenerate a script corresponding to the target material. The modified script obtaining module is configured to input the script modification prompt word into the large language model to output a regenerated script corresponding to the target material.

24. The apparatus of claim 13, 19, or 20, wherein, The script prompt word generation module is further configured to: In a case where a video script generation request input by a user is received, generate a script generation prompt word based on a material content description of each material file in a material set.

25. An electronic device comprising a processor and a memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implement the steps of the video script generation method of any one of claims 1-12.

Images