A video data storyboard method and system for intelligent video generation

By extracting copy and storyboards in video data, calculating matching indexes and fine-tuning the large language model, the problem of low efficiency of video data storyboards in the existing technology is solved, intelligent video content generation is realized, and work efficiency and quality are improved.

CN118660115BActive Publication Date: 2024-11-08HAOSHEN TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411107812.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-08
Estimated Expiration
2044-08-13

AI Technical Summary

Technical Problem

In film and television production and animation production, the existing video data storyboard method has high exquisiteness but low work efficiency, which cannot meet the increasing demand, and lacks automated processing methods.

Method used

By obtaining structured video data, extracting copywriting and storyboards, calculating matching indexes, generating training data sets, and fine-tuning based on large language models, optimizing their storyboard generation capabilities to achieve intelligent generation of copywriting and storyboard sequences.

Benefits of technology

It improves the efficiency and quality of video copywriting and storyboard splitting, realizes intelligent video content generation, and meets efficient needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118660115B_ABST
    Figure CN118660115B_ABST
Patent Text Reader

Abstract

The present invention discloses a video data storyboard method and system for intelligent video generation, which relates to the technical field of video generation. The method comprises: obtaining structured video data; extracting text and storyboards in the video data, calculating the matching index of the text and storyboards, and creating a mapping relationship between the text and storyboards according to the matching index to generate a training data set; fine-tuning a large language model for the first time based on the training data set and the prediction result scoring formula to enable it to have the ability to generate storyboards; fine-tuning the large language model for the second time to optimize the model's storyboard generation ability; using the fine-tuned large language model to intelligently generate text and storyboard sequences according to input keywords, and storing the generated text and storyboard sequences in a database. By learning the writing method of the text and the organization method of the storyboards, the text and storyboard sequences that meet the requirements of the video content can be intelligently generated, effectively improving the efficiency and quality of video text writing and storyboard splitting.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video generation technology, and in particular to a video data storyboard method and system for intelligent video generation. Background Art

[0002] Video data storyboarding methods are widely used in film and television production, animation production, advertising creativity, surveillance and security and other fields. In film and television production, storyboarding methods can help directors better plan lens language and enhance the artistic effect of the film; in animation production, storyboarding methods can realize the rapid construction and adjustment of animation scenes. With the increase in the demand for storyboards, although the exquisiteness of manual production is guaranteed, the work efficiency obviously cannot keep up with the increasing demand. If artificial intelligence technology can be introduced to automatically produce storyboards, work efficiency can be greatly improved, thereby meeting the increasing demand. Summary of the invention

[0003] The present invention provides a video data storyboard method for generating smart videos, comprising:

[0004] Step 1. Obtain structured video data;

[0005] Step 2: Extract the text and storyboards from the video data, calculate the matching index between the text and the storyboards, and create a mapping relationship between the text and the storyboards based on the matching index to generate a training data set;

[0006] Step 3: Fine-tune the large language model for the first time based on the training data set and the prediction result scoring formula to enable it to generate storyboards;

[0007] Step 4: Perform a second fine-tuning on the large language model to optimize the model’s storyboard generation capability.

[0008] Step 5. Use the fine-tuned large language model to intelligently generate text and storyboard sequences based on the input keywords, and store the generated text and storyboard sequences in the database.

[0009] The video data storyboard method for intelligent video generation as described above, wherein the text and storyboard in the video data are extracted, the matching index between the text and the storyboard is calculated, and a mapping relationship between the text and the storyboard is created according to the matching index to generate a training data set, specifically comprising the following sub-steps:

[0010] Converting video data into a continuous storyboard sequence;

[0011] Extracting text sequences from audio of video data;

[0012] Calculate the matching index between each storyboard in the storyboard sequence and each copy in the copy sequence, and establish a mapping relationship between the storyboards and the copy that meet the matching index requirements;

[0013] The storyboards and texts with mapping relationships are stored as training data sets.

[0014] The video data storyboard method for intelligent video generation as described above, wherein the matching index between each storyboard in the storyboard sequence and each text in the text sequence is calculated, specifically comprises the following sub-steps:

[0015] Performing image recognition on video frames in the storyboard sequence to obtain storyboard description information;

[0016] Use a large language model to extract the semantic features of the storyboard description information in the storyboard sequence and the semantic features of each copy sentence in the copy sequence;

[0017] Create a matching index calculation function, input the semantic features of the storyboard sequence and the text sequence into the matching index calculation function, and return the calculation result;

[0018] Add a filter to the return of the matching index calculation function to filter out storyboards and texts whose matching index does not meet the requirements.

[0019] The video data storyboard method for intelligent video generation as described above, wherein the large language model is first fine-tuned based on the training data set and the prediction result scoring formula, specifically comprising the following sub-steps:

[0020] Use the text and storyboards with mapping relationships in the training dataset to construct input-output pairs;

[0021] Incremental training of large language models based on input-output pairs;

[0022] During the training process, the model parameters are continuously adjusted until the score of the model prediction results no longer increases.

[0023] The video data storyboard method for intelligent video generation as described above, wherein the large language model is fine-tuned for the second time to optimize the storyboard generation capability of the model, specifically comprises the following sub-steps:

[0024] Create a storyboard optimization training set based on the storyboard semantic features and their corresponding storyboard sequences;

[0025] Create the loss function for the second fine-tuning;

[0026] The loss function is used to calculate the loss value of the large language model on the storyboard optimization training set, and the adjustment formula of the model parameters is used to tune the parameters of the large language model.

[0027] The present invention also provides a video data storyboard system for intelligent video generation, comprising: a storage module, a text storyboard mapping module, a large language model fine-tuning module, and a text storyboard generation module;

[0028] Storage module, used to store structured video data, as well as generated text and storyboard sequences;

[0029] The text-storyboard mapping module is used to extract the text and storyboards in the video data, calculate the matching index between the text and the storyboard, and create a mapping relationship between the text and the storyboard based on the matching index to generate a training data set;

[0030] Large language model fine-tuning module, used to fine-tune the large language model so that it has the ability to generate copywriting and storyboards according to user requirements;

[0031] The copywriting and storyboard generation module is used to use the fine-tuned large language model to intelligently generate copywriting and storyboard sequences based on input keywords.

[0032] A video data storyboard system for intelligent video generation as described above, wherein the text storyboard mapping module includes: a storyboard sequence extraction submodule, a text sequence extraction submodule, a mapping relationship establishment submodule, and a training data set generation submodule;

[0033] A storyboard sequence extraction submodule is used to extract a continuous storyboard sequence from video data;

[0034] A text sequence extraction submodule, used to extract text sequences from the audio of video data;

[0035] A mapping relationship establishment submodule is used to calculate the matching index between each storyboard in the storyboard sequence and each copy in the copy sequence, and to establish a mapping relationship between the storyboards and the copy that meet the matching index requirements;

[0036] The training data set generation submodule is used to store the storyboards and texts with a mapping relationship as a training data set.

[0037] A video data storyboard system for intelligent video generation as described above, wherein the large language model fine-tuning module includes: a first fine-tuning submodule and a second fine-tuning submodule;

[0038] The first fine-tuning submodule is used to perform the first fine-tuning on the large language model based on the training data set and the prediction result scoring formula, so that it has the ability to generate storyboards;

[0039] The second fine-tuning submodule is used to optimize the storyboard generation capability of the large language model.

[0040] The beneficial effects achieved by the present invention are as follows: by learning the writing method of copywriting and the organization method of storyboards from videos on the Internet, it is possible to intelligently generate copywriting and storyboard sequences that meet the requirements of video content, effectively improving the efficiency and quality of video copywriting and storyboard splitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0042] Figure 1 This is a flow chart of a video data storyboard method for intelligent video generation provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0043] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention. Embodiment 1

[0044] like Figure 1 As shown, the first embodiment of the present invention provides a video data storyboard method for intelligent video generation, including:

[0045] Step S10: Obtaining structured video data;

[0046] The video data can be obtained through an Internet platform or a ready-made video material library, and then the obtained video data is stored in the form of field attributes.

[0047] Step S20: extracting the text and storyboards in the video data, calculating the matching index between the text and the storyboards, and creating a mapping relationship between the text and the storyboards according to the matching index to generate a training data set;

[0048] Natural language processing technology and computer vision technology are used to parse the text and storyboards in the video, and a mapping relationship between them is established to form a training data set. Specifically:

[0049] Step S21: converting the video data into a continuous storyboard sequence;

[0050] The video data is divided into continuous shorter video clips, and several key frames are extracted from each video clip as a storyboard to form a continuous storyboard sequence.

[0051] Step S22: extracting a text sequence from the audio of the video data;

[0052] Convert the audio of the video data into text, and use natural language processing technology to segment the entire text into individual copy sentences to form a copy sequence.

[0053] Step S23: calculating the matching index between each storyboard in the storyboard sequence and each text in the text sequence, and establishing a mapping relationship between the storyboards and texts that meet the matching index requirements;

[0054] Calculate the matching index between each storyboard in the storyboard sequence and each text in the text sequence, including:

[0055] Step S231: performing image recognition on the video frames in the storyboard sequence to obtain storyboard description information;

[0056] Use computer vision technology to identify objects, scenes, emotions, actions and other information in video frames, and use this information as descriptive information for storyboards.

[0057] Step S232: using the large language model to extract semantic features of the storyboard description information in the storyboard sequence and semantic features of each copy sentence in the copy sequence;

[0058] Step S233: creating a matching index calculation function, inputting the semantic features of the storyboard sequence and the text sequence into the matching index calculation function, and returning the calculation result;

[0059] The matching index calculation function is expressed as: ,in is the storyboard semantic feature set extracted from the i-th storyboard sequence, is the set of semantic features extracted from the jth copy sequence, are the weights of semantic matching dimension, emotional matching dimension and theme matching dimension respectively, m and n are the lengths of storyboard semantic feature set and copywriting semantic feature set respectively, express and The minimum value in the difference sequence of express and The maximum value in the difference sequence of Adjustable parameters, express The kth eigenvalue in The absolute difference of the kth eigenvalue in , represents the emotion score of the i-th storyboard sequence, represents the sentiment score of the jth copywriting sequence, represents the theme feature value of the i-th storyboard sequence, Represents the topic feature value of the j-th copy sequence.

[0060] Step S234: Add a filter at the return of the matching index calculation function to filter out the storyboards and texts whose matching indexes do not meet the standards;

[0061] The matching condition is a range of matching index values, which is used to control the relevance of the training data set. A matching index that is too high or too low will affect the training effect of the subsequent model. It needs to be adjusted according to the requirements for the rigor of the model.

[0062] Step S24: storing the storyboards and texts having a mapping relationship as a training data set.

[0063] Step S30: fine-tuning the large language model for the first time based on the training data set and the prediction result scoring formula to enable it to have the ability to generate storyboards;

[0064] The large language model originally has the ability to generate text. It can generate a copy by inputting keywords, but it does not have the ability to generate storyboards based on this copy. Therefore, it is necessary to use the training data set to fine-tune the model. Specifically:

[0065] Step S31: construct input-output pairs using the texts and storyboards with mapping relationships in the training data set;

[0066] The semantic features of the text and storyboards with mapping relationships are used as input and output pairs of the large language model. Each training batch contains several input and output pairs.

[0067] Step S32: incrementally training the large language model based on the input-output pairs;

[0068] Step S33: continuously adjusting the model parameters during the training process until the score of the model prediction result no longer increases;

[0069] The adjustment formula of the model parameters is expressed as: in, are model parameters, are the adjusted model parameters, is the learning rate, is the adjustment gradient of the model parameters, S is the score of the prediction result; the score of the prediction result is determined by the gap between the real-time output of the large language model based on an input value and the expected output. The expected output is the output value of the input-output pair of the input value. Then the score calculation formula of the prediction result is expressed as: ,in represents the real-time output of the large language model for the h-th output value, represents the expected output of the hth input value, h ranges from 1 to H, and H represents the size of the training batch. During the adjustment of the model parameters, the change curve of the predicted result score is observed. If the score no longer increases or there is no obvious upward trend, the training is stopped to avoid overfitting of the model.

[0070] Step S40: performing a second fine-tuning on the large language model to optimize the model's storyboard generation capability;

[0071] After the first fine-tuning, the large language model has adapted to generate storyboard semantic features based on the text semantic features. The second fine-tuning is to enable the model to generate a storyboard sequence containing a set of key video frames based on the storyboard semantic features. Specifically:

[0072] Step S41: creating a storyboard optimization training set according to the storyboard semantic features and their corresponding storyboard sequences;

[0073] The storyboard semantic feature describes a group of key video frames, and each group of key video frames is a storyboard sequence extracted from the video data. Therefore, the storyboard semantic feature and the storyboard sequence are in a one-to-one correspondence. Labels are added to the storyboard sequences, and then the storyboard semantic features are used as input and the corresponding storyboard sequence labels are used as output to create a storyboard optimization training set.

[0074] Step S42: creating a loss function for the second fine-tuning;

[0075] The loss function of the second fine-tuning is expressed as: ,in For an input value The storyboard labels predicted by the model, For an input value The actual corresponding storyboard label, is the subscript of the training sample, ranging from 1 to G, where G is the size of the training batch during the second fine-tuning. is the loss value.

[0076] Step S43: using the loss function to calculate the loss value of the large language model on the storyboard optimization training set, and using the adjustment formula of the model parameters to tune the parameters of the large language model;

[0077] The second fine-tuning is to adjust the model parameters based on the model loss value, rather than the predicted result score. You need to replace S in the model parameter adjustment formula with It is expressed as: ,in, are model parameters, are the adjusted model parameters, is the learning rate, is the adjustment gradient of the model parameters.

[0078] Observe the changing curve of the loss function during the training process. If the loss value no longer decreases or there is no obvious downward trend, stop training.

[0079] Step S50: using the fine-tuned large language model to intelligently generate text and storyboard sequences according to the input keywords, and storing the generated text and storyboard sequences in a database;

[0080] Create a human-computer interaction page to input key information for storyboard generation into the model and return the output results of the model. The generated text and its corresponding storyboard sequence are stored in the database for subsequent retrieval and management. The output text and storyboard sequence can be formatted for use in video editing and generation. Embodiment 2

[0081] Embodiment 2 of the present invention provides a video data storyboard system for intelligent video generation, including: a storage module, a text storyboard mapping module, a large language model fine-tuning module, and a text storyboard generation module;

[0082] (1) Storage module, used to store structured video data and generated copy storyboards;

[0083] The video data can be obtained through an Internet platform or a ready-made video material library, and then the obtained video data is stored in the form of field attributes.

[0084] (2) A text-storyboard mapping module, which is used to extract text and storyboards from video data, calculate the matching index between the text and storyboards, and create a mapping relationship between the text and storyboards based on the matching index to generate a training data set, including: a storyboard sequence extraction submodule, a text sequence extraction submodule, a mapping relationship establishment submodule, and a training data set generation submodule;

[0085] 1. A storyboard sequence extraction submodule, used to extract a continuous storyboard sequence from video data;

[0086] The video data is divided into continuous shorter video clips, and several key frames are extracted from each video clip as a storyboard to form a continuous storyboard sequence.

[0087] 2. A text sequence extraction submodule, used to extract text sequences from the audio of video data;

[0088] Convert the audio of the video data into text, and use natural language processing technology to segment the entire text into individual copy sentences to form a copy sequence.

[0089] 3. A mapping relationship establishment submodule is used to calculate the matching index between each storyboard in the storyboard sequence and each copy in the copy sequence, and to establish a mapping relationship between the storyboards and copies that meet the matching index requirements;

[0090] Calculate the matching index between each storyboard in the storyboard sequence and each text in the text sequence, including:

[0091] I. Perform image recognition on the video frames in the storyboard sequence to obtain storyboard description information;

[0092] Use computer vision technology to identify objects, scenes, emotions, actions and other information in video frames, and use this information as descriptive information for storyboards.

[0093] II. Use the large language model to extract the semantic features of the storyboard description information in the storyboard sequence and the semantic features of each copy sentence in the copy sequence;

[0094] III. Create a matching index calculation function, input the semantic features of the storyboard sequence and the text sequence into the matching index calculation function, and return the calculation result;

[0095] The matching index calculation function is expressed as: ,in is the storyboard semantic feature set extracted from the i-th storyboard sequence, is the set of semantic features extracted from the jth copy sequence, are the weights of semantic matching dimension, emotional matching dimension and theme matching dimension respectively, m and n are the lengths of storyboard semantic feature set and copywriting semantic feature set respectively, express and The minimum value in the difference sequence of express and The maximum value in the difference sequence of Adjustable parameters, express The kth eigenvalue in The absolute difference of the kth eigenvalue in , represents the emotion score of the i-th storyboard sequence, represents the sentiment score of the jth copywriting sequence, represents the theme feature value of the i-th storyboard sequence, Represents the topic feature value of the j-th copy sequence.

[0096] IV. Add a filter to the return of the matching index calculation function to filter out the storyboards and texts that do not meet the matching index requirements;

[0097] The matching condition is a range of matching index values, which is used to control the relevance of the training data set. A matching index that is too high or too low will affect the training effect of the subsequent model. It needs to be adjusted according to the requirements for the rigor of the model.

[0098] 4. A training data set generation submodule is used to store storyboards and texts with mapping relationships as training data sets.

[0099] (3) A large language model fine-tuning module, which is used to fine-tune the large language model so that it can generate text and storyboards according to the needs of users, including: a first fine-tuning sub-module and a second fine-tuning sub-module;

[0100] 1. The first fine-tuning submodule is used to perform the first fine-tuning on the large language model based on the training data set and the prediction result scoring formula, so that it has the ability to generate storyboards;

[0101] The large language model originally has the ability to generate text. It can generate a copy by inputting keywords, but it does not have the ability to generate storyboards based on this copy. Therefore, it is necessary to use the training data set to fine-tune the model. Specifically:

[0102] I. Use the text and storyboards with mapping relationships in the training dataset to construct input-output pairs;

[0103] The semantic features of the text and storyboards with mapping relationships are used as input and output pairs of the large language model. Each training batch contains several input and output pairs.

[0104] II. Incremental training of large language models based on input-output pairs;

[0105] III. Continuously adjust the model parameters during the training process until the score of the model prediction results no longer increases;

[0106] The adjustment formula of the model parameters is expressed as: in, are model parameters, are the adjusted model parameters, is the learning rate, is the adjustment gradient of the model parameters, S is the score of the prediction result; the score of the prediction result is determined by the gap between the real-time output of the large language model based on an input value and the expected output. The expected output is the output value of the input-output pair of the input value. Then the score calculation formula of the prediction result is expressed as: ,in represents the real-time output of the large language model for the h-th output value, represents the expected output of the hth input value, h ranges from 1 to H, and H represents the size of the training batch. During the adjustment of the model parameters, the change curve of the predicted result score is observed. If the score no longer increases or there is no obvious upward trend, the training is stopped to avoid overfitting of the model.

[0107] 2. The second fine-tuning submodule is used to optimize the storyboard generation capability of the large language model;

[0108] After the first fine-tuning, the large language model has adapted to generate storyboard semantic features based on the text semantic features. The second fine-tuning is to enable the model to generate a storyboard sequence containing a set of key video frames based on the storyboard semantic features. Specifically:

[0109] I. Create a storyboard optimization training set based on the storyboard semantic features and their corresponding storyboard sequences;

[0110] The storyboard semantic feature describes a group of key video frames, and each group of key video frames is a storyboard sequence extracted from the video data. Therefore, the storyboard semantic feature and the storyboard sequence are in a one-to-one correspondence. Labels are added to the storyboard sequences, and then the storyboard semantic features are used as input and the corresponding storyboard sequence labels are used as output to create a storyboard optimization training set.

[0111] II. Create the loss function for the second fine-tuning;

[0112] The loss function of the second fine-tuning is expressed as: ,in For an input value The storyboard labels predicted by the model, For an input value The actual corresponding storyboard label, g is the subscript of the training sample, ranging from 1 to G, G is the size of the training batch during the second fine-tuning, is the loss value.

[0113] III. Use the loss function to calculate the loss value of the large language model on the storyboard optimization training set, and use the model parameter adjustment formula to tune the parameters of the large language model;

[0114] The second fine-tuning is to adjust the model parameters based on the model loss value, rather than the predicted result score. You need to replace S in the model parameter adjustment formula with It is expressed as: ,in, are model parameters, are the adjusted model parameters, is the learning rate, is the adjustment gradient of the model parameters.

[0115] Observe the changing curve of the loss function during the training process. If the loss value no longer decreases or there is no obvious downward trend, stop training.

[0116] (4) A copywriting and storyboard generation module, which uses a fine-tuned large language model to intelligently generate copywriting and storyboard sequences based on input keywords;

[0117] Create a human-computer interaction page to input key information for storyboard generation into the model and return the output results of the model. The generated text and its corresponding storyboard sequence are stored in the database for subsequent retrieval and management. The output text and storyboard sequence can be formatted for use in video editing and generation.

[0118] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A video data storyboard method for intelligent video generation, characterized in that: The method comprises: Step 1. Obtain structured video data; Step 2: Extract the text and storyboards from the video data, calculate the matching index between the text and the storyboards, and create a mapping relationship between the text and the storyboards based on the matching index to generate a training data set. This includes the following sub-steps: Converting video data into a continuous storyboard sequence; Extracting text sequences from audio of video data; Calculate the matching index between each storyboard in the storyboard sequence and each copy in the copy sequence, and establish a mapping relationship between the storyboards and the copy that meet the matching index requirements; The storyboards and texts with mapping relationships are stored as training data sets; Calculate the matching index between each storyboard in the storyboard sequence and each text in the text sequence, which is specifically divided into the following sub-steps: Performing image recognition on video frames in the storyboard sequence to obtain storyboard description information; Use a large language model to extract the semantic features of the storyboard description information in the storyboard sequence and the semantic features of each copy sentence in the copy sequence; Create a matching index calculation function, input the semantic features of the storyboard sequence and the text sequence into the matching index calculation function, and return the calculation result; The matching index calculation function is expressed as: ,in is the storyboard semantic feature set extracted from the i-th storyboard sequence, is the set of semantic features extracted from the jth copy sequence, , , are the weights of semantic matching dimension, emotional matching dimension and theme matching dimension respectively, m and n are the lengths of storyboard semantic feature set and copywriting semantic feature set respectively, express and The minimum value in the difference sequence of express and The maximum value in the difference sequence of Adjustable parameters, express The kth eigenvalue in The absolute difference of the kth eigenvalue in , represents the emotion score of the i-th storyboard sequence, represents the sentiment score of the jth copywriting sequence, represents the theme feature value of the i-th storyboard sequence, Represents the topic feature value of the jth copywriting sequence; Add a filter to the return of the matching index calculation function to filter out the storyboards and texts that do not meet the matching index requirements; Step 3: Fine-tune the large language model for the first time based on the training data set and the prediction result scoring formula to enable it to generate storyboards; Step 4: Perform a second fine-tuning on the large language model to optimize the model’s storyboard generation capability. Step 5. Use the fine-tuned large language model to intelligently generate text and storyboard sequences based on the input keywords, and store the generated text and storyboard sequences in the database.

2. The video data storyboard method for intelligent video generation according to claim 1, characterized in that: The first fine-tuning of the large language model is performed based on the training dataset and the prediction result scoring formula, which includes the following sub-steps: Use the text and storyboards with mapping relationships in the training dataset to construct input-output pairs; Incremental training of large language models based on input-output pairs; During the training process, the model parameters are continuously adjusted until the score of the model prediction results no longer increases.

3. The video data storyboard method for intelligent video generation according to claim 1, characterized in that: The large language model is fine-tuned for the second time to optimize the model's storyboard generation capability, which includes the following sub-steps: Create a storyboard optimization training set based on the storyboard semantic features and their corresponding storyboard sequences; Create the loss function for the second fine-tuning; The loss function is used to calculate the loss value of the large language model on the storyboard optimization training set, and the adjustment formula of the model parameters is used to tune the parameters of the large language model.

4. A video data storyboard system for intelligent video generation, characterized in that: The system comprises: a storage module, a text storyboard mapping module, a large language model fine-tuning module, and a text storyboard generation module; Storage module, used to store structured video data, as well as generated text and storyboard sequences; The text-storyboard mapping module is used to extract the text and storyboards in the video data, calculate the matching index between the text and the storyboard, and create a mapping relationship between the text and the storyboard based on the matching index to generate a training data set; including: A storyboard sequence extraction submodule is used to extract a continuous storyboard sequence from video data; A text sequence extraction submodule, used to extract text sequences from the audio of video data; A mapping relationship establishment submodule is used to calculate the matching index between each storyboard in the storyboard sequence and each copy in the copy sequence, and to establish a mapping relationship between the storyboards and the copy that meet the matching index requirements; A training data set generation submodule is used to store storyboards and texts with a mapping relationship as a training data set; Calculate the matching index between each storyboard in the storyboard sequence and each text in the text sequence, which is specifically divided into the following sub-steps: Performing image recognition on video frames in the storyboard sequence to obtain storyboard description information; Use a large language model to extract the semantic features of the storyboard description information in the storyboard sequence and the semantic features of each copy sentence in the copy sequence; Create a matching index calculation function, input the semantic features of the storyboard sequence and the text sequence into the matching index calculation function, and return the calculation result; The matching index calculation function is expressed as: ,in is the storyboard semantic feature set extracted from the i-th storyboard sequence, is the set of semantic features extracted from the jth copy sequence, , , are the weights of semantic matching dimension, emotional matching dimension and theme matching dimension respectively, m and n are the lengths of storyboard semantic feature set and copy semantic feature set respectively, express and The minimum value in the difference sequence of express and The maximum value in the difference sequence of Adjustable parameters, express The kth eigenvalue in The absolute difference of the kth eigenvalue in , represents the emotion score of the i-th storyboard sequence, represents the sentiment score of the jth copy sequence, represents the theme feature value of the i-th storyboard sequence, Represents the topic feature value of the jth copywriting sequence; Add a filter to the return of the matching index calculation function to filter out the storyboards and texts that do not meet the matching index requirements; Large language model fine-tuning module, used to fine-tune the large language model so that it has the ability to generate copywriting and storyboards according to user requirements; The copywriting and storyboard generation module is used to use the fine-tuned large language model to intelligently generate copywriting and storyboard sequences based on input keywords.

5. The video data storyboard system for intelligent video generation according to claim 4, characterized in that: The large language model fine-tuning module includes: a first fine-tuning submodule and a second fine-tuning submodule; The first fine-tuning submodule is used to perform the first fine-tuning on the large language model based on the training data set and the prediction result scoring formula, so that it has the ability to generate storyboards; The second fine-tuning submodule is used to optimize the storyboard generation capability of the large language model.