Video script generation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a video script generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining user input data, the user input data comprises video demand parameters, and the video demand parameters at least represent the video content features of a to-be-generated video; user input data is processed through a pre-trained large language model, first script data is generated, the first script data comprises at least two segments of picture description texts for a to-be-generated video, and the picture description texts are used for describing image content under a video sub-mirror in the video; therefore, the user can complete the shooting of the original video according to the picture description text, and the problems that the video works are high in shooting cost and low in quality due to the fact that the user designs the video split contents of the video to be shot by himself / herself according to the historical shooting experience of himself / herself are solved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of video processing, and particularly relate to a video script generation method and device, electronic equipment and storage medium. BACKGROUND

[0002] Currently, more and more users begin to create videos by using some video production applications (APPs). Before creating a video, the user needs to shoot an original video, and then the user edits the original video by using a video production application to complete the video creation. When the user shoots the original video, the user needs to complete a series of shooting according to a video storyboard.

[0003] However, in the prior art, the video storyboard needs to be designed manually, that is, the user designs the video storyboard content of the video to be shot according to his own historical shooting experience, resulting in high video production cost and low video quality. SUMMARY

[0004] Embodiments of the present disclosure provide a video script generation method and device, electronic equipment and storage medium to overcome the problem of high video production cost and low video quality.

[0005] In a first aspect, embodiments of the present disclosure provide a video script generation method, comprising:

[0006] obtaining user input data, wherein the user input data includes video requirement parameters, and the video requirement parameters at least represent video content features of a video to be generated; processing the user input data by using a pre-trained large language model to generate first script data, wherein the first script data includes at least two pieces of picture description text for the video to be generated, and the picture description text is used to describe image content under a video storyboard of the video.

[0007] In a second aspect, embodiments of the present disclosure provide a video script generation device, comprising:

[0008] an obtaining unit, configured to obtain user input data, wherein the user input data includes video requirement parameters, and the video requirement parameters at least represent video content features of a video to be generated; a processing unit, configured to process the user input data by using a pre-trained large language model to generate first script data, wherein the first script data includes at least two pieces of picture description text for the video to be generated, and the picture description text is used to describe image content under a video storyboard of the video.

[0009] In a third aspect, embodiments of the present disclosure provide an electronic device, comprising a processor and a memory.

[0010] The memory stores computer-executable instructions;

[0011] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video script generation method as described in the first aspect and various possible designs of the first aspect.

[0012] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, which stores computer-executable instructions, and when a processor executes the computer-executable instructions, the video script generation method as described in the first aspect and various possible designs of the first aspect is implemented.

[0013] In a fifth aspect, the embodiments of the present disclosure provide a computer program product, which includes a computer program, and when a processor executes the computer program, the video script generation method as described in the first aspect and various possible designs of the first aspect is implemented.

[0014] The video script generation method, device, electronic equipment and storage medium provided by the embodiments can obtain user input data, the user input data includes video requirement parameters, and the video requirement parameters at least represent video content features of a to-be-generated video; the user input data is processed by a pre-trained large language model to generate first script data, and the first script data includes at least two picture description texts for the to-be-generated video, and the picture description texts are used to describe image content under a video shot of the video. Since the pre-trained large language model is trained based on a large amount of video script data of a large number of users, and then the video content features of the to-be-generated video corresponding to the user input data are processed by the pre-trained large language model based on the user input data, the first script data guiding the user to shoot the original video can be generated, wherein the first script data includes at least two picture description texts for the to-be-generated video (original video), and the picture description texts correspond to image content under a video shot of the video, so that the user can complete the shooting of the original video according to the picture description texts, that is, the problem of high video production cost and low quality caused by the user designing the video shot content of the to-be-generated video according to the historical shooting experience is solved. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0016] Figure 1 This is an application scenario diagram of the video script generation method provided in the embodiments of this disclosure;

[0017] Figure 2 A flowchart illustrating the video script generation method provided in this embodiment. Figure 1 ;

[0018] Figure 3 This is a schematic diagram illustrating a process for generating first script data, provided in an embodiment of the present disclosure.

[0019] Figure 4 Flowchart of the video script generation method provided in this embodiment of the disclosure Figure 2 ;

[0020] Figure 5 for Figure 4 A flowchart illustrating the specific implementation of step S202 in the illustrated embodiment;

[0021] Figure 6 A schematic diagram illustrating a process for generating second script data provided in an embodiment of this disclosure;

[0022] Figure 7 for Figure 5 A flowchart illustrating the specific implementation of step S2021 in the illustrated embodiment;

[0023] Figure 8 A schematic diagram illustrating the process of generating initial scene description text corresponding to source video using a multimodal processing model provided in this embodiment of the disclosure;

[0024] Figure 9 This is a schematic diagram illustrating a process for generating first script data for a video to be generated, provided in an embodiment of the present disclosure.

[0025] Figure 10 This is a structural block diagram of the video script generation apparatus provided in the embodiments of this disclosure;

[0026] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0027] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0029] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0030] The application scenarios of the embodiments of this disclosure are explained below:

[0031] The video script generation method provided in this disclosure can be applied to applications (APPs) with video production functions, such as camera applications, video editing applications, and short video applications. More specifically, it can be applied to application scenarios where original videos are shot based on the generated video script. The execution subject of this embodiment can be a terminal device running the aforementioned application with video production functions, a server deploying the server corresponding to the aforementioned application, or other electronic devices that perform similar functions.

[0032] In some embodiments, the terminal device or server can implement the video script generation method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be program-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be local applications, i.e., programs that need to be installed in the operating system to run; or they can be mini-programs embedded in any APP, i.e., programs that run based on a browser environment. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin, and the specific implementation can be configured as needed. Further, in some embodiments, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud storage, cloud communication, cloud databases, cloud computing, cloud functions, network services, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The cloud services can be interactive processing services that can be invoked by the terminal device.

[0033] Figure 1 This is an application scenario diagram of the video script generation method provided in this disclosure embodiment, with reference to... Figure 1 As shown in the diagram, taking the server as the execution entity as an example, the terminal device runs an application with video production capabilities (hereinafter referred to as the application). After loading user input data through the application, the terminal device sends the user input data to the server for processing by triggering corresponding functional components, such as the "Generate Script" component shown in the diagram. The server uses the method provided in this embodiment to call a pre-trained large language model to process the user input data, generate corresponding first script data, and then send it back to the terminal device for display. This allows the user to determine the video storyboard based on the first script data, thereby completing the shooting of the original video. Of course, it is understood that in another possible implementation, the execution entity of the method provided in this embodiment can also be the terminal device itself. That is, the above processing is entirely executed by the terminal device. The models and algorithms used in the execution process can be entirely deployed on the local terminal device, or partially deployed on the local terminal device and partially deployed on external devices and called by the terminal device. The specific implementation process is similar and will not be repeated here.

[0034] In existing technologies, the aforementioned methods for determining video storyboards are typically based on manual design. This means users need to design the video storyboard content themselves, drawing on their past shooting experience, to complete the initial video recording. Specifically, for example, if a user wants to shoot a video introducing a power adapter with fast charging capabilities, they need to design the video storyboard content based on their past shooting experience. For instance, if the user designs a shot of using the power adapter to quickly charge a phone to full capacity, then "quickly charging to full capacity" would be a close-up video storyboard. The user then uses this close-up video storyboard to complete the video introducing the fast-charging power adapter. However, designing the video storyboard based on personal experience is highly subjective, leading to high production costs and low quality.

[0035] This disclosure provides a video script generation method to solve the above-mentioned problems.

[0036] refer to Figure 2 , Figure 2 A flowchart illustrating the video script generation method provided in this embodiment. Figure 1 The method described in this embodiment can be applied to servers, terminal devices, or other electronic devices. This video script generation method includes:

[0037] Step S101: Obtain user input data, which includes video requirement parameters. The video requirement parameters at least characterize the video content features of the video to be generated.

[0038] Step S102: Process the user input data through a pre-trained large language model to generate first script data. The first script data includes at least two segments of scene description text for the video to be generated. The scene description text is used to describe the image content of a video shot in the targeted video.

[0039] refer to Figure 1The illustrated application scenario diagram illustrates the video script generation method provided in this embodiment, using a server as the execution entity. For example, the server deploys a server-side application with video production capabilities, while the terminal device runs a client-side application of that application. Communication between the server and the terminal device is based on server-client communication, thereby obtaining user input data sent by the terminal device. This user input data corresponds to the video requirement parameters for the video the user wants to shoot. These video requirement parameters at least characterize the video content features of the video to be generated. Specifically, for example, if the user needs to shoot a video introducing a power adapter with fast charging capabilities, for example, the user input data would be "generate a video script introducing a power adapter with fast charging capabilities," then the video requirement parameters would be "power adapter with fast charging capabilities," and the video content features would be "fast charging" and "power adapter." Furthermore, after the server receives user input data, it processes the user input data by calling a pre-trained large language model to generate the first script data. The first script data includes at least two scene description texts for the video to be generated. The scene description texts describe the image content of a video shot in the video. Specifically, for example, the pre-trained large language model generates two scene description texts for the video to be generated based on the user input data "generate a video script introducing a power adapter with fast charging function". The first scene description text is "A man wakes up in the morning and finds that his mobile phone battery is low and he needs to go to the airport to catch a flight in 20 minutes". The second scene description text is "This man uses a power adapter with fast charging function to charge his mobile phone. When the phone starts charging and is fully charged, the man finishes eating a sandwich, and exactly 10 minutes have passed". Then, the server sends the first script data generated by the pre-trained large language model back to the terminal device and displays it on the terminal device, so that the user can determine the video shot based on the first script data and complete the shooting of the original video.

[0040] In one possible implementation, the user input data also includes video script; correspondingly, the first script data includes at least two script clauses that constitute the video script, with each script clause corresponding to a screen description text, and the screen description text is generated based on the corresponding script clause.

[0041] Figure 3 This is a schematic diagram illustrating a process for generating first script data according to an embodiment of the present disclosure, such as... Figure 3As shown, the video shooting theme corresponding to the user input data is "Generate a video script introducing a power adapter with fast charging function". The video script in the user input data is "A man was getting ready to catch a plane and found that his phone battery was low. Then he used a power adapter with fast charging function to charge his phone, which took 10 minutes to fully charge it". Then, the pre-trained large language model splits the video script into at least two script clauses based on the video shooting theme and the video script. For example, if it is split into two script clauses, the first script clause is "A man was getting ready to catch a plane and found that his phone battery was low", and the second script clause is "Then he used a power adapter with fast charging function to charge his phone, which took 10 minutes to fully charge it". "To full charge"; then, the pre-trained large language model can generate two scene description texts for the video to be generated based on the video shooting theme and these two copy sentences, that is, generate the first script data. In this case, the copy sentences correspond one-to-one with the scene description text. The first scene description text is "A man woke up in the morning and found that his phone battery was low. He also needed to go to the airport to catch a flight in 20 minutes." The first scene description text is generated based on the first copy sentence. The second scene description text is "The man used a power adapter with fast charging function to charge his phone. When the phone started charging and was fully charged, the man finished eating a sandwich. Exactly 10 minutes had passed." The second scene description text is generated based on the second copy sentence.

[0042] Furthermore, after step S102, the following may also be included:

[0043] Step S103: Based on the first script data, generate a shooting guidance video for the video to be generated, wherein the shooting guidance video includes at least two shooting guidance video frames that correspond one-to-one with the scene description text for the video to be generated.

[0044] Optionally, the shooting instruction video frame may display corresponding scene description text and / or text sentences. The scene description text and / or text sentences displayed within the shooting instruction video frame are used to remind the user of the main content to be shot for that shooting instruction video frame.

[0045] For example, after the pre-trained large language model generates the first script data, the server further calls the pre-trained large language model to generate a shooting guidance video for the video to be generated based on the first script data, so that the user can intuitively shoot the original video according to the shooting guidance video. The shooting guidance video includes at least two shooting guidance video frames, and each shooting guidance video frame corresponds one-to-one with the screen description text for the video to be generated. Specifically, for example, if the first scene description text is "A man wakes up in the morning and finds his phone battery low, and he needs to leave for the airport in 20 minutes to catch a flight," then the first shooting guidance video frame generated by the pre-trained large language model will display a scene including "A man stretching in bed, and a smart speaker in the room announces the man's schedule and daily reminders: 'Good morning, sir. The current time is 7:00 AM. You need to leave for the airport in 20 minutes. Also, your phone battery is low and needs to be charged as soon as possible to avoid delaying your trip.' The brightness of the bedroom gradually changes from dark to bright." Further, optionally, the first shooting guidance video frame may also include the first scene description text. For example, the first scene description text can be located in the lower area of ​​the scene displayed in the first shooting guidance video frame, so that the user can quickly and accurately determine the main content to be shot based on the scene description text (and / or, sentence breaks).

[0046] In this embodiment, user input data is acquired, including video requirement parameters, which at least characterize the video content features of the video to be generated. The user input data is then processed by a pre-trained large language model to generate first script data. This first script data includes at least two segments of descriptive text for the video to be generated, each describing the image content within a specific video segment. Since the pre-trained large language model is trained on a large amount of video script data from numerous users, by processing the video content features of the video to be generated based on the acquired user input data, first script data guiding the user to shoot the original video can be generated. This first script data includes at least two segments of descriptive text for the original video, each corresponding to an image content within a specific video segment. This allows the user to shoot the original video based on the descriptive text, thus solving the problem of high production costs and low quality caused by users designing their own video segments based on their past shooting experience.

[0047] refer to Figure 4 , Figure 4 Flowchart of the video script generation method provided in this embodiment of the disclosure Figure 2 This embodiment is inFigure 2 Based on the illustrated embodiment, step S102 is further refined, and the video script generation method includes:

[0048] Step S201: Obtain source video.

[0049] Step S202: Process the source video using a multimodal processing model to obtain second script data, which includes at least two segments of descriptive text for the source video.

[0050] For example, a multimodal processing model is a machine learning model that can process and fuse data from multiple different types of data sources (such as text, images, audio, video, etc.). After the server receives the source video, it processes the source video by calling the multimodal processing model. Specifically, the multimodal processing model processes and parses the text, images, and audio in the source video to generate a visual description text for the source video, which is the second script data. The second script data includes at least two visual description texts for describing the source video.

[0051] In one possible implementation, Figure 5 for Figure 4 The flowchart of the specific implementation of step S202 in the illustrated embodiment is as follows: Figure 5 As shown, the specific implementation of step S202 includes:

[0052] Step S2021: Process the source video using a multimodal processing model to obtain initial image description text for the source video.

[0053] Step S2022: Separate the personalized description information in the initial image description text to obtain the image description text for the source video.

[0054] Step S2023: Generate second script data based on the scene description text for the source video.

[0055] For example, Figure 6 This is a schematic diagram illustrating a process for generating second script data according to an embodiment of the present disclosure, such as... Figure 6As shown, for a 2-minute video clip, which consists of 4 video segments, the multimodal processing model processes the video clip to obtain initial screen description text. This initial screen description text includes the same number of segment screen description texts as the number of video segments. Specifically, processing these 4 video segments yields the initial screen description text, which includes 4 segment screen description texts, and these 4 segment screen description texts correspond one-to-one with the 4 video segments. That is, video segment `video_footage_1` corresponds to segment screen description text `text_snippets_1`, and video segment `video_footage_1` corresponds to segment screen description text `text_snippets_1`. Footage_2 corresponds to the snippet description text text_snippets_2, video_footage_3 corresponds to the snippet description text text_snippets_3, and video_footage_4 corresponds to the snippet description text text_snippets_4. Furthermore, if the core theme of the video footage is introducing a power adapter with fast charging capabilities, then the power adapter and related electronic devices are the core subjects of the video footage. Consequently, if the snippet description text includes information about the power adapter... The description text for the power adapter and related electronic devices is extracted. If the description text for a video clip does not contain description text for the power adapter and related electronic devices, it is deleted. This process separates the personalized description information from the initial video clip description text. All the extracted description texts are then the video clip description texts for the source video. These description texts are then summarized and processed to obtain the second script data. For example, the video clip description texts text_snippets_1, text_snippets_3, and text_snippets_4 all contain description text for the power adapter and related electronic devices. The description text (personalized description information) of the electronic device is then separated from at least three of the above description texts. Since the fragment screen description text text_snippets_2 does not contain description text for the power adapter and related electronic devices, the fragment screen description text text_snippets_2 is deleted. All the description texts separated above are the screen description texts for the source video. The screen description texts include description texts text_descr_1, text_descr_3, and text_descr_4. Then, the screen description texts for the source video are processed and integrated to generate the second script data.

[0056] Furthermore, exemplarily, the multimodal processing model includes an image feature extraction module, a text feature extraction module, and a fusion inference module. Figure 7 forFigure 5 The flowchart of the specific implementation method of step S2021 in the embodiment shown is as follows: Figure 7 As shown, the specific implementation of step S2021 includes:

[0057] Step S2021A: Extract the text from the source video using the text feature extraction module. The text is used to represent the text content and / or dialogue content that appears in the source video.

[0058] Step S2021B: Extract video content features from the source video using the image feature extraction module. These video content features are used to characterize the video content information of the source video.

[0059] Step S2021C: Convert the video content information into corresponding content description text.

[0060] Step S2021D: Merge the text and content description text into text features, and process the text features and video content features through the fusion reasoning module to obtain the initial screen description text for the source video.

[0061] For example, Figure 8 This is a schematic diagram illustrating the process of generating initial scene description text corresponding to source video using a multimodal processing model provided in this disclosure embodiment. The following is in conjunction with... Figure 8 The process of generating the initial screen description text described above will be introduced, such as... Figure 8As shown, by extracting text features through the text feature extraction module, the text content and / or dialogue content appearing in the source video can be obtained, that is, the script text in the source video can be obtained. Specifically, for example, the text feature extraction module includes a speech recognition model and / or an optical character recognition model. Furthermore, if the source video displays text subtitles, that is, text content is displayed, the text content can be extracted through the optical character recognition model to obtain the script text in the source video. Further, if the text content displayed in the source video has different colors based on different characters, for example, the text content corresponding to character person_1 is displayed in red, the text content corresponding to character person_2 is displayed in blue, and the text content corresponding to the narration is displayed in black, then the text content can be extracted through the optical character recognition model and classified according to the color corresponding to the text content to obtain the script text in the source video classified according to different characters. Furthermore, the dialogue content in the source video can also be extracted through the speech recognition model to obtain the script text in the source video. The speech recognition model can not distinguish the voice of the dialogue content. Audio feature information (such as timbre, pitch, volume, etc.) is used to extract dialogue content from source videos to obtain the text text. This text text does not have classification features, or the speech recognition model can classify and extract dialogue content based on audio feature information to obtain text text with classification features. It can be understood that, based on the video features of the source video, the text feature extraction module can extract the text text based on a speech recognition model and / or an optical character recognition model. For example, if the source video is silent but has subtitles, the text text can be extracted using an optical character recognition model. If the source video has sound but no subtitles, the text is extracted using a speech recognition model. If the source video has both sound and subtitles, either model can be used to extract the text, or both a speech recognition model and an optical character recognition model can be used simultaneously. When both models are used for text extraction, the accuracy of text extraction and / or classification can be further improved, providing accurate intermediate process data (text) for the subsequent generation of initial screen description text and second script data. Optionally, the text can include a timestamp.

[0062] Furthermore, such as Figure 8As shown, by extracting the video content information through the image feature extraction module, the video content features of the video can be obtained. Specifically, for example, the image feature extraction module can extract the changes in the video content. For instance, if the video shows an electronic device charging from a low battery level to a full charge, the image feature extraction module can generate corresponding video content information (video content features) based on the battery change process of the electronic device shown in the video. That is, the video content information describes the battery change process of the electronic device at different times. Furthermore, the multimodal processing model also includes a video content feature parsing module. This module converts video content information (video content features) and outputs corresponding content description text, which describes the video content features (change features) of the source video in the form of text. For example, if the video content information describes the battery level change of an electronic device at different times, the video content feature parsing module converts the video content information and outputs the content description text as "At time_1, the electronic device's battery level is 15%, and it starts charging when connected to the power adapter. At time_2, the electronic device is charged to 80%, and at time_3, the electronic device is charged to 100%."

[0063] Furthermore, such as Figure 8As shown, the multimodal processing model also includes a text alignment module. This module aligns the text and content description text according to their timestamps, then fuses them into text features. Specifically, for example, at timestamp time_1, the text is "This man took out a power adapter with fast charging capability to charge his electronic device. Can this electronic device be fully charged in 10 minutes?", and the corresponding content description text is "At timestamp time_1, the electronic device has 15% battery and is connected to the power adapter to start charging." The text alignment module then aligns and fuses the text and content description text based on the same timestamp to generate the corresponding text features. Further, the fusion inference module processes the text features and video content features, aligning and fusion the fused text features with the corresponding video content features and performing logical inference to obtain the initial screen description text of the video material. Specifically, for example, at timestamp time_1, the video content feature is "Using a power adapter to charge a low-battery electronic device." The fused text features are: "At time_1, the electronic device has 15% battery. A man takes out a power adapter with fast charging capability to charge the electronic device. Can the electronic device be fully charged in 10 minutes?" Then, the fusion inference module processes the text features and video content features to generate an initial scene description text at time_1. The initial scene description text describes: "A man uses a power adapter with fast charging capability to charge an electronic device with 15% battery and asks whether it can be fully charged in 10 minutes." The effect is that the "fast charging function" of the power adapter, which is not reflected in the video content features, is supplemented by the text features. The "connection action of the electronic device to the power adapter and the charging status after connection," which is not reflected in the text features, is supplemented by the video content features. In other words, through multi-source, multi-modal information complementarity, a detailed and rich initial scene description text is generated, providing accurate and rich intermediate process data (initial scene description text).

[0064] Further, in one possible implementation, the video content features include a first video content feature and a second video content feature, wherein the first video content feature represents the segment content of at least one video segment constituting the source video; and the second video content feature represents the image content of at least one video frame in the source video. Accordingly, in this case, the specific implementation of step S2022B includes:

[0065] Step S2022B-1: Based on the preset storyboard information, the source video is segmented to obtain at least one video storyboard segment, and the video storyboard segment is feature extracted by the image feature extraction module to obtain the first video content feature. The storyboard information represents the time period corresponding to at least one video storyboard.

[0066] For example, regarding the first video content feature, since the source video includes at least one video clip, the image feature extraction module extracts features from each video clip obtained from segmentation of the source video according to preset clip information, thereby obtaining the first video content feature corresponding to each video clip. The clip information corresponds to the time period of each video clip. Specifically, for example, the source video consists of video clips_1, video clips_2, and video clips_3, with the time period of video clips_1 being T_1 and the time period of video clips_2 being T_2. If the time period of video clip_3 is T_3, then time periods T_1, T_2, and T_3 together constitute the clip information. Based on the clip information, the source video is segmented to obtain video clips_1, video clips_2, and video clips_3. Then, the image feature extraction module extracts features from video clips_1, clips_2, and clips_3 respectively to obtain the first video content features corresponding to each video clip. Furthermore, the first video content features of each video clip are summarized to obtain the first video content features of the source video.

[0067] Step S2022B-2: Based on a preset time interval, extract frames from the source video to obtain at least one source video frame, and extract features from the source video frame using an image feature extraction module to obtain the second video content features.

[0068] For example, regarding the second video content features, based on the video length of the source video, the source video can be divided into several equal parts to obtain several source video frames. That is, frames are extracted from the source video according to a preset time interval to obtain at least one source video frame. Then, the image feature extraction module extracts features from each source video frame to obtain the corresponding second video content features. Specifically, for example, a 20-second source video can be extracted by sampling frames per second to obtain 20 source video frames. The image feature extraction module then extracts features from each of these 20 source video frames to obtain the corresponding second video content features. Furthermore, the second video content features of these 20 source video frames are summarized to obtain the second video content features of the 20-second source video. It can be understood that for the 20-second source video, if frames are extracted from the source video at 500 milliseconds, 40 source video frames can be obtained. The subsequent feature extraction process is similar to the above process and will not be repeated here.

[0069] In this embodiment, based on the time interval corresponding to the preset storyboard information, coarse-grained first video content features are extracted from the source video. Based on the preset time interval, fine-grained second video content features are extracted from the source video. Then, the first video content features and the second video content features are fused to obtain video content features, which improves the accuracy of extracting video content information from the source video. The preset time interval corresponding to the second video content features is shorter than the time interval corresponding to the storyboard information of the first video content features.

[0070] Furthermore, in one possible implementation, the specific implementation of obtaining the initial scene description text for the source video by processing text features and video content features through the fusion inference module in step S2021D includes:

[0071] Step S2021D-1: Process text features and video content features through the fusion reasoning module to obtain at least one chapter identifier and chapter screen description text corresponding to each chapter identifier; wherein, the chapter screen description text includes at least one initial screen description text, which is used to describe the image content under at least one video shot in the video chapter indicated by the corresponding chapter identifier.

[0072] For example, based on the differences in the objects described by text features and video content features, the fusion inference module performs coarse-grained division of text features and video content features, that is, it divides them according to chapters, obtaining at least one corresponding chapter identifier, and obtaining the chapter screen description text corresponding to each chapter identifier. Further, the chapter screen description text includes at least one initial screen description text. Specifically, for example, if the source video is a product promotion video for a power adapter, this source video can be divided into application scenario video clips and product introduction video clips. Then, the fusion inference module divides the text features and video content features into two chapters according to the content emphasized by the text features and video content features, that is, it obtains two chapter identifiers. The first chapter identifier is "Application Scenario", and the corresponding chapter screen description text is "Electronic devices charge from low battery to full battery by connecting to this power adapter." The second chapter is labeled "Product Introduction," and the corresponding chapter description text is "The design concept of this power adapter and the types of electronic devices it is intended for." Further, for the text features and video content features corresponding to the first chapter labeled "Application Scenarios," the fusion reasoning module performs further processing to obtain at least one initial screen description text. For example, for the "Application Scenarios" chapter, three initial screen description texts are obtained: the first initial screen description text mainly describes "a man needs to fully charge a low-battery electronic device in a short time," the second initial screen description text mainly describes "the charging process of the electronic device connected to the power adapter," and the third initial screen description text mainly describes "the electronic device is fully charged within 10 minutes." The process by which the fusion reasoning module generates the initial screen description text is similar to the process in the aforementioned embodiment and will not be repeated here.

[0073] In this embodiment, the fusion reasoning module first divides the text features and video content features into chapters according to the content they emphasize, and then further processes the text features and video content features of each chapter to generate the corresponding initial screen description text. This realizes the hierarchical and structured nature of the initial screen description text, which facilitates the multimodal processing model to process the initial screen description text and accurately generate the second script data.

[0074] Furthermore, in one possible implementation, this embodiment also includes:

[0075] Step S2020A: Obtain user requirement information, which is used to characterize the content requirements of the generated second script data.

[0076] Step S2020B: Based on the user's requirements, generate corresponding model prompts, which contain chapter information representing the chapter division rules.

[0077] For example, the user requirement information reflects the user's requirements for shooting the video, that is, the user's content requirement for the multimodal processing model to generate the second script data. Therefore, the server analyzes and processes the acquired user requirement information to generate corresponding model prompts, so that the multimodal processing model can generate the corresponding second script data based on the model prompts. Specifically, for example, if the model prompts contain chapter information representing the chapter division rules, the multimodal processing model divides the source video into chapters based on the model prompts, and then generates the corresponding second script data. That is, it can be understood that for the same source video, it will be divided into different chapter structures based on different model prompts, and thus, based on different chapter structures, the generated second script data will be different, and the focus of each second script data will also be different.

[0078] Accordingly, in one possible implementation, step S2021D-1 is specifically implemented as follows:

[0079] By processing text features, video content features, and model prompts through the fusion reasoning module, at least one chapter identifier is obtained, along with the chapter scene description text corresponding to each chapter identifier.

[0080] For example, if the source video is a promotional video for a power adapter, this source video can be divided into application scenario video clips and product introduction video clips. If the model prompt is "a shot describing the usage scenario of a charger in detail", the fusion inference module processes the text features, video content features, and model prompt to obtain four chapter identifiers and corresponding chapter description texts. Three chapter identifiers correspond to the application scenario video clips of the source video to describe the application scenario of the power adapter in detail, and the remaining chapter identifier corresponds to the product introduction video clip of the source video. If the model prompt is "a shot describing the product introduction of a charger in detail", the fusion inference module also generates four chapter identifiers, three of which correspond to the product introduction video clips of the source video, and the remaining chapter identifier corresponds to the application scenario video clips of the source video.

[0081] In this embodiment, based on the user's actual shooting needs, the source video is processed accordingly. For example, the source video is divided into chapters according to the user's needs, thereby generating personalized second script data that meets the user's needs, thus improving the user experience.

[0082] Step S203: Obtain user input data, which includes video requirement parameters. The video requirement parameters at least characterize the video content features of the video to be generated.

[0083] Step S204: Process the user input data and the second script data using a pre-trained large language model to generate the first script data for the video to be generated.

[0084] For example, a pre-trained large language model can generate first script data by matching corresponding second script data from a database based on user input data and performing adaptive processing. Specifically, for example, user input data_0 is "generate a video script introducing a power adapter with fast charging function", second script data_1 is script data for filming a user using electronic devices, second script data_2 is script data for filming the power adapter manufacturing process, and second script data_3 is script data for filming a mobile phone charging. The pre-trained large language model determines that second script data_3 is the matching script data based on user input data_0, and then generates first script data by adaptively processing second script data_3. The source videos corresponding to the aforementioned second script data_1, data_2, and data_3 can be video data automatically crawled from the database by the server, video data determined based on user usage habits (likes, favorites, browsing, and forwarding, including at least one of the above usage behaviors), or reference video data uploaded by the user when inputting user input data_0.

[0085] Figure 9 This is a schematic diagram illustrating a process for generating first script data for a video to be generated, as provided in an embodiment of this disclosure. The following is in conjunction with... Figure 9 To provide a more detailed explanation of the above process, such as... Figure 9As shown, taking the scenario where "the user simultaneously uploads reference video data when inputting user input data_0" as an example, firstly, the user inputs user input data_0 and source video (i.e., reference video data) through the terminal device. Then, the terminal device sends the input user input data_0 and source video to the server. The server processes the source video by calling a multimodal processing model, obtaining second script data data_1, data_2, and data_3. The server then calls a pre-trained large language model to match the user input data_0 with the second script data data_1, data_2, and data_3 to determine the matching script data. The matching script data is the second script data data_3. Finally, the pre-trained large language model, based on the user input data_0 and the second script data matching data_0, performs a multimodal processing. This data, data_3, generates the first script data. The process by which the server processes the source videos using a multimodal processing model to obtain the second script data, data_1, data_2, and data_3, includes: For example, if a user uploads three source videos (V_1, V_2, and V_3), the multimodal processing model processes each of these three videos to obtain corresponding initial image description texts, text_1, text_2, and text_3. Then, it separates the personalized description information from these initial image description texts to obtain image description texts td_1 for source video V_1, td_2 for source video V_2, and td_3 for source video V_3. Based on these image description texts, the multimodal processing model generates the corresponding second script data, data_1, data_2, and data_3.

[0086] In this embodiment, the implementation of step S203 is the same as that in this disclosure. Figure 2 The implementation of step S101 in the illustrated embodiment is the same, and will not be described in detail here.

[0087] Corresponding to the video script generation method in the above embodiments, Figure 10 This is a structural block diagram of a video script generation apparatus provided in an embodiment of this disclosure. The method described in the above embodiments can be executed by this video script generation apparatus, which can be implemented by software and / or hardware, and can be integrated into an electronic device with certain data processing capabilities. The electronic device may include, but is not limited to, mobile terminals with big data processing capabilities, as well as fixed terminals with big data processing capabilities such as desktop computers and supercomputers.

[0088] For ease of explanation, only the parts relevant to embodiments of this disclosure are shown. (Refer to...) Figure 10 The video script generation device 3 includes:

[0089] The acquisition unit 31 is used to acquire user input data, which includes video requirement parameters. The video requirement parameters at least characterize the video content features of the video to be generated.

[0090] The processing unit 32 is used to process user input data through a pre-trained large language model to generate first script data. The first script data includes at least two segments of image description text for the video to be generated. The image description text is used to describe the image content of a video scene in the targeted video.

[0091] According to one or more embodiments of this disclosure, the user input data further includes video script; the first script data includes at least two script clauses constituting the video script, the script clauses correspond one-to-one with the on-screen description text, and the on-screen description text is generated based on the corresponding script clauses.

[0092] According to one or more embodiments of this disclosure, the video script generation device 3 is further configured to: acquire source video; process the source video through a multimodal processing model to obtain second script data, the second script data including at least two segments of descriptive text for the source video; when the processing unit 32 processes user input data through a pre-trained large language model to generate first script data for the video to be generated, it is specifically configured to: process user input data and second script data through a pre-trained large language model to generate first script data for the video to be generated.

[0093] According to one or more embodiments of this disclosure, when the video script generation device 3 processes the source video through a multimodal processing model to obtain second script data, it is specifically used to: process the source video through a multimodal processing model to obtain initial screen description text for the source video; separate the personalized description information in the initial screen description text to obtain screen description text for the source video; and generate second script data based on the screen description text for the source video.

[0094] According to one or more embodiments of this disclosure, the multimodal processing model includes an image feature extraction module, a text feature extraction module, and a fusion inference module. When the video script generation device 3 processes the source video using the multimodal processing model to obtain initial scene description text for the source video, it specifically performs the following steps: Extracting text from the source video using the text feature extraction module, the text being used to characterize the text content and / or dialogue appearing in the source video; extracting video content features from the source video using the image feature extraction module, the video content features being used to characterize the video content information of the source video; converting the video content information into corresponding content description text; fusing the text and content description text into text features, and processing the text features and video content features through the fusion inference module to obtain the initial scene description text for the source video.

[0095] According to one or more embodiments of this disclosure, the video content features include a first video content feature and a second video content feature, wherein the first video content feature represents the segment content of at least one video segment constituting the source video; the second video content feature represents the image content of at least one video frame in the source video; when the video script generation device 3 extracts the video content features of the source video through the image feature extraction module, it is specifically used to: segment the source video based on preset storyboard information to obtain at least one video storyboard segment, and extract features from the video storyboard segment through the image feature extraction module to obtain the first video content feature, wherein the storyboard information represents the time period corresponding to at least one video storyboard segment; extract frames from the source video based on a preset time interval to obtain at least one source video frame, and extract features from the source video frame through the image feature extraction module to obtain the second video content feature.

[0096] According to one or more embodiments of this disclosure, when the video script generation device 3 processes text features and video content features through the fusion inference module to obtain initial screen description text for the source video, it is specifically used to: process text features and video content features through the fusion inference module to obtain at least one chapter identifier and chapter screen description text corresponding to each chapter identifier; wherein, the chapter screen description text includes at least one initial screen description text, and the chapter screen description text is used to describe the image content under at least one video shot in the video chapter indicated by the corresponding chapter identifier.

[0097] According to one or more embodiments of this disclosure, the video script generation device 3 is further configured to: acquire user demand information, which is used to characterize the content demand of the generated second script data; generate corresponding model prompt words based on the user demand information, which contain chapter information characterizing the chapter division rules; when the video script generation device 3 processes text features and video content features through the fusion inference module to obtain at least one chapter identifier and chapter screen description text corresponding to each chapter identifier, it is specifically configured to: process text features, video content features and model prompt words through the fusion inference module to obtain at least one chapter identifier and chapter screen description text corresponding to each chapter identifier.

[0098] According to one or more embodiments of this disclosure, the video script generation device 3 is further configured to: generate a shooting guidance video for a video to be generated based on first script data, wherein the shooting guidance video includes at least two shooting guidance video frames that correspond one-to-one with the scene description text for the video to be generated, and the corresponding scene description text is displayed in the shooting guidance video frames.

[0099] The acquisition unit 31 and the processing unit 32 are connected. The video script generation device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0100] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 11 As shown, the electronic device 4 includes:

[0101] Processor 41, and memory 42 communicatively connected to processor 41;

[0102] Memory 42 stores instructions executed by the computer;

[0103] The processor 41 executes computer execution instructions stored in the memory 42 to achieve, for example, Figures 2-9 The video script generation method in the illustrated embodiment.

[0104] Optionally, the processor 41 and the memory 42 are connected via a bus 43.

[0105] For relevant instructions, please refer to the corresponding text. Figures 2-9 The relevant descriptions and effects of the steps in the corresponding embodiments are understood, and will not be elaborated on here.

[0106] This disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement this disclosure. Figures 2-9The video script generation method provided in any of the corresponding embodiments.

[0107] This disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements this disclosure. Figures 2-9 The video script generation method provided in any of the corresponding embodiments.

[0108] To implement the above embodiments, this disclosure also provides an electronic device.

[0109] refer to Figure 12 , Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. It illustrates a structural schematic diagram of an electronic device 900 suitable for implementing the embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers, portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 12 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0110] like Figure 12 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0111] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 12 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0112] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0113] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0114] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0115] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0116] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0118] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units or modules do not necessarily limit the specific unit itself.

[0119] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0120] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] In a first aspect, according to one or more embodiments of this disclosure, a video script generation method is provided, comprising:

[0122] Obtain user input data, which includes video requirement parameters, and the video requirement parameters at least characterize the video content features of the video to be generated; process the user input data through a pre-trained large language model to generate first script data, which includes at least two segments of scene description text for the video to be generated, and the scene description text is used to describe the image content of a video shot in the targeted video.

[0123] According to one or more embodiments of this disclosure, the user input data further includes video script; the first script data includes at least two script clauses constituting the video script, the script clauses corresponding one-to-one with the screen description text, and the screen description text is generated based on the corresponding script clauses.

[0124] According to one or more embodiments of this disclosure, the method further includes: acquiring source video; processing the source video using a multimodal processing model to obtain second script data, the second script data including at least two segments of descriptive text for the source video; and processing the user input data using a pre-trained large language model to generate first script data, including: processing the user input data and the second script data using a pre-trained large language model to generate the first script data for the video to be generated.

[0125] According to one or more embodiments of this disclosure, the step of processing the source video through a multimodal processing model to obtain second script data includes: processing the source video through a multimodal processing model to obtain initial scene description text for the source video; separating personalized description information in the initial scene description text to obtain scene description text for the source video; and generating the second script data based on the scene description text for the source video.

[0126] According to one or more embodiments of this disclosure, the multimodal processing model includes an image feature extraction module, a text feature extraction module, and a fusion inference module. The step of processing the source video using the multimodal processing model to obtain initial scene description text for the source video includes: extracting text from the source video using the text feature extraction module, the text being used to characterize the text content and / or dialogue appearing in the source video; extracting video content features from the source video using the image feature extraction module, the video content features being used to characterize the video content information of the source video; converting the video content information into corresponding content description text; fusing the text and the content description text into text features, and processing the text features and the video content features using the fusion inference module to obtain the initial scene description text for the source video.

[0127] According to one or more embodiments of this disclosure, the video content features include a first video content feature and a second video content feature, wherein the first video content feature represents the segment content of at least one video segment constituting the source video; the second video content feature represents the image content of at least one video frame in the source video; the step of extracting the video content features of the source video through the image feature extraction module includes: segmenting the source video based on preset storyboard information to obtain at least one video storyboard segment, and extracting features from the video storyboard segment through the image feature extraction module to obtain the first video content feature, wherein the storyboard information represents the time period corresponding to at least one video storyboard; and extracting frames from the source video based on a preset time interval to obtain at least one source video frame, and extracting features from the source video frame through the image feature extraction module to obtain the second video content feature.

[0128] According to one or more embodiments of this disclosure, the step of processing the text features and the video content features through the fusion inference module to obtain the initial scene description text for the source video includes: processing the text features and the video content features through the fusion inference module to obtain at least one chapter identifier, and chapter scene description text corresponding to each chapter identifier; wherein, the chapter scene description text includes at least one segment of the initial scene description text, and the chapter scene description text is used to describe the image content under at least one video shot in the video chapter indicated by the corresponding chapter identifier.

[0129] According to one or more embodiments of this disclosure, the method further includes: obtaining user demand information, the user demand information being used to characterize the content demand of the generated second script data; generating corresponding model prompt words based on the user demand information, the model prompt words containing chapter information characterizing chapter division rules; and processing the text features and the video content features through the fusion inference module to obtain at least one chapter identifier and chapter screen description text corresponding to each of the chapter identifiers, comprising: processing the text features, the video content features, and the model prompt words through the fusion inference module to obtain at least one chapter identifier and chapter screen description text corresponding to each of the chapter identifiers.

[0130] According to one or more embodiments of this disclosure, the method further includes: generating a shooting guidance video for the video to be generated based on the first script data, wherein the shooting guidance video includes at least two shooting guidance video frames that correspond one-to-one with the scene description text for the video to be generated, and the shooting guidance video frames display the corresponding scene description text.

[0131] Secondly, according to one or more embodiments of this disclosure, a video script generation apparatus is provided, comprising:

[0132] An acquisition unit is used to acquire user input data, the user input data including video requirement parameters, the video requirement parameters at least characterizing the video content features of the video to be generated;

[0133] The processing unit is configured to process the user input data through a pre-trained large language model to generate first script data. The first script data includes at least two segments of scene description text for the video to be generated, wherein the scene description text is used to describe the image content of a video scene in the targeted video.

[0134] According to one or more embodiments of this disclosure, the user input data further includes video script; the first script data includes at least two script clauses constituting the video script, the script clauses corresponding one-to-one with the screen description text, and the screen description text is generated based on the corresponding script clauses.

[0135] According to one or more embodiments of this disclosure, the video script generation device is further configured to: acquire source video; process the source video through a multimodal processing model to obtain second script data, wherein the second script data includes at least two segments of descriptive text for the source video; and when the processing unit processes the user input data through a pre-trained large language model to generate first script data, it is specifically configured to: process the user input data and the second script data through a pre-trained large language model to generate the first script data for the video to be generated.

[0136] According to one or more embodiments of this disclosure, when the video script generation device processes the source video through a multimodal processing model to obtain second script data, it is specifically used to: process the source video through a multimodal processing model to obtain initial scene description text for the source video; separate personalized description information in the initial scene description text to obtain scene description text for the source video; and generate the second script data based on the scene description text for the source video.

[0137] According to one or more embodiments of this disclosure, the multimodal processing model includes an image feature extraction module, a text feature extraction module, and a fusion inference module. When the video script generation device processes the source video using the multimodal processing model to obtain initial scene description text for the source video, it specifically performs the following steps: Extracting text from the source video using the text feature extraction module, the text being used to characterize the text content and / or dialogue appearing in the source video; extracting video content features from the source video using the image feature extraction module, the video content features being used to characterize the video content information of the source video; converting the video content information into corresponding content description text; fusing the text and the content description text into text features, and processing the text features and the video content features using the fusion inference module to obtain the initial scene description text for the source video.

[0138] According to one or more embodiments of this disclosure, the video content features include a first video content feature and a second video content feature, wherein the first video content feature represents the segment content of at least one video segment constituting the source video; the second video content feature represents the image content of at least one video frame in the source video; when the video script generation device extracts the video content features of the source video through the image feature extraction module, it is specifically used to: segment the source video based on preset storyboard information to obtain at least one video storyboard segment, and extract features from the video storyboard segment through the image feature extraction module to obtain the first video content feature, wherein the storyboard information represents the time period corresponding to at least one video storyboard segment; extract frames from the source video based on a preset time interval to obtain at least one source video frame, and extract features from the source video frame through the image feature extraction module to obtain the second video content feature.

[0139] According to one or more embodiments of this disclosure, when the video script generation device processes the text features and the video content features through the fusion inference module to obtain initial scene description text for the source video, it is specifically used to: process the text features and the video content features through the fusion inference module to obtain at least one chapter identifier, and chapter scene description text corresponding to each chapter identifier; wherein, the chapter scene description text includes at least one segment of the initial scene description text, and the chapter scene description text is used to describe the image content under at least one video shot in the video chapter indicated by the corresponding chapter identifier.

[0140] According to one or more embodiments of this disclosure, the video script generation device is further configured to: acquire user demand information, the user demand information being used to characterize the content demand of the generated second script data; generate corresponding model prompt words based on the user demand information, the model prompt words containing chapter information characterizing chapter division rules; when the video script generation device processes the text features and the video content features through the fusion inference module to obtain at least one chapter identifier and chapter screen description text corresponding to each of the chapter identifiers, it is specifically configured to: process the text features, the video content features, and the model prompt words through the fusion inference module to obtain at least one chapter identifier and chapter screen description text corresponding to each of the chapter identifiers.

[0141] According to one or more embodiments of this disclosure, the video script generation device is further configured to: generate a shooting guidance video for the video to be generated based on the first script data, wherein the shooting guidance video includes at least two shooting guidance video frames that correspond one-to-one with the scene description text for the video to be generated, and the shooting guidance video frames display the corresponding scene description text.

[0142] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0143] The memory stores computer-executed instructions;

[0144] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the video script generation method as described in the first aspect and various possible designs of the first aspect.

[0145] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the video script generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0146] Fifthly, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the video script generation method as described in the first aspect and various possible designs of the first aspect.

[0147] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0148] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0149] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method of generating a video script, characterized by, The method comprises: obtaining user input data, wherein the user input data comprises video requirement parameters representing at least video content features of a to-be-generated video; processing the user input data through a pre-trained large language model to generate first script data, wherein the first script data comprises at least two pieces of picture description text for the to-be-generated video, and the picture description text is used to describe image content under a video split shot of the to-be-generated video.

2. The method of claim 1, wherein, The user input data further comprises a video script; The first script data comprises at least two script sentences constituting the video script, and the script sentences correspond to the picture description text one by one, and the picture description text is generated based on the corresponding script sentence.

3. The method of claim 1, wherein, The method further comprises: obtaining a material video; processing the material video through a multi-modal processing model to obtain second script data, wherein the second script data comprises at least two pieces of picture description text for the material video; processing the user input data through a pre-trained large language model to generate first script data, comprising: processing the user input data and the second script data through a pre-trained large language model to generate first script data of the to-be-generated video.

4. The method of claim 3, wherein, The processing of the material video through the multi-modal processing model to obtain the second script data comprises: processing the material video through the multi-modal processing model to obtain initial picture description text for the material video; separating personalized description information in the initial picture description text to obtain picture description text for the material video; generating the second script data according to the picture description text for the material video.

5. The method of claim 4, wherein, The multi-modal processing model comprises an image feature extraction module, a text feature extraction module, and a fusion reasoning module, and the processing of the material video through the multi-modal processing model to obtain the initial picture description text for the material video comprises: extracting script text in the material video through the text feature extraction module, wherein the script text is used to represent text content and / or dialogue content appearing in the material video; extracting video content features of the material video through the image feature extraction module, wherein the video content features are used to represent video content information of the material video; converting the video content information into corresponding content description text; fusing the script text and the content description text into text features, and processing the text features and the video content features through the fusion reasoning module to obtain the initial picture description text for the material video.

6. The method of claim 5, wherein, The video content features comprise first video content features and second video content features, wherein the first video content features represent segment content of at least one video segment constituting the material video, and the second video content features represent image content of at least one video frame in the material video; the extraction of the video content features of the material video through the image feature extraction module comprises: segment the material video based on preset split shot information to obtain at least one video split shot segment, and perform feature extraction on the video split shot segment by using the image feature extraction module to obtain the first video content feature, wherein the split shot information represents a time period corresponding to at least one video split shot; extract frames from the material video based on a preset time interval to obtain at least one material video frame, and perform feature extraction on the material video frame by using the image feature extraction module to obtain the second video content feature.

7. The method of claim 5, wherein, The processing of the text feature and the video content feature by the fusion reasoning module to obtain the initial picture description text for the material video includes: processing the text feature and the video content feature by the fusion reasoning module to obtain at least one chapter identifier and chapter picture description text corresponding to each chapter identifier; The chapter picture description text includes at least one initial picture description text, and the chapter picture description text is used to describe the image content under at least one video split shot in a video chapter indicated by the corresponding chapter identifier.

8. The method of claim 7, wherein, The method further includes: obtaining user demand information, wherein the user demand information is used to represent the content demand of the generated second script data; generating corresponding model prompt words according to the user demand information, wherein the model prompt words contain chapter information representing chapter division rules; The processing of the text feature and the video content feature by the fusion reasoning module to obtain at least one chapter identifier and chapter picture description text corresponding to each chapter identifier includes: processing the text feature, the video content feature, and the model prompt word by the fusion reasoning module to obtain at least one chapter identifier and chapter picture description text corresponding to each chapter identifier.

9. The method of claim 1, wherein, The method further includes: generating a shooting guide video of the to-be-generated video according to the first script data, wherein the shooting guide video includes at least two shooting guide video frames corresponding one-to-one to the picture description text for the to-be-generated video, and the corresponding picture description text is displayed in the shooting guide video frame.

10. A video script generation apparatus characterized by comprising: includes: an acquisition unit configured to acquire user input data, wherein the user input data includes video demand parameters, and the video demand parameters represent at least video content features of a to-be-generated video; a processing unit configured to process the user input data by using a pre-trained large language model to generate first script data, wherein the first script data includes at least two picture description texts for the to-be-generated video, and the picture description text is used to describe image content under one video split shot in the to-be-generated video.

11. An electronic device, comprising: includes: a processor and a memory; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the processor executes the video script generation method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the video script generation method in any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the video script generation method in any one of claims 1 to 9.

Images