An AIGC video generation scheme intelligent adaptation method and device and medium

Abstract

The application discloses an AIGC video generation scheme intelligent adaptation method and device and medium, and the method comprises the following steps: S01, a script sample dataset is acquired and parsed into multiple storyboards, the text description, visual parameters and scheme parameters of each script sample data are taken as input data, and a corresponding comprehensive score is taken as a label to construct a sample dataset; S02, a deep learning model is trained using the sample dataset, and a scheme evaluation model is obtained after the training is completed; S03, a to-be-adapted script is acquired and parsed into multiple storyboards, the text description and visual parameters of each storyboard and the scheme parameters of multiple video generation schemes required are extracted, and are input into the scheme evaluation model respectively, the comprehensive scores of the video generation schemes are output by the scheme evaluation model and are sorted, and the optimal video generation scheme adapted is determined. The application can improve the adaptation efficiency and precision, and make the storyboards and the whole film have continuity.

Description

Technical Field

[0001] This invention relates to the field of AIGC (Artificial Intelligence Generated Content) video generation technology, and in particular to an intelligent adaptation method, device and medium for AIGC video generation schemes. Background Technology

[0002] AIGC video generation technology utilizes artificial intelligence algorithms such as deep learning models to extract video features and then intelligently generate video content that meets specific requirements. This significantly shortens creation time and reduces production costs. The intelligent adaptation of the AIGC video generation solution automatically adapts and optimizes the video generation scheme based on different application scenarios and user needs, efficiently generating high-quality video content that meets the corresponding requirements.

[0003] A video contains multiple scenes. In existing technologies, intelligent adaptation for AIGC video generation solutions is usually based on manual intervention. This involves first manually breaking down the script into multiple scenes, labeling the plot, actions, and key elements of each scene, then manually evaluating the importance of each scene, such as marking scenes with core plots as highly important, and then simply evaluating the video clarity (such as resolution) and semantic matching (such as the consistency between the generated content and the text description) of each scene under different solutions to determine the adaptation solution.

[0004] However, the importance of different scenes varies. Traditional AIGC video generation technology relies on manual evaluation of the importance of each scene, which not only has a high subjective error rate, low adaptation efficiency and accuracy, making it difficult to adapt to large-scale generation needs, but also the evaluation results are greatly affected by experience and subjective judgment, resulting in a high error rate. In addition, it cannot quantify the visual and semantic coherence between scenes, resulting in poor overall video smoothness. Furthermore, since it only considers the quality of a single scene (such as clarity and semantic matching), it lacks the evaluation of the visual and semantic consistency between scenes, which leads to abrupt transitions between scenes and a broken video style. Summary of the Invention

[0005] The technical problem to be solved by this invention is: In view of the technical problems existing in the prior art, this invention provides an AIGC video generation scheme intelligent adaptation method, device and medium that is simple to implement, has high adaptation efficiency and accuracy, and strong continuity between scenes and the whole film. It can improve the efficiency and accuracy of intelligent adaptation of AIGC video generation scheme, while maintaining the continuity of the overall video style.

[0006] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0007] A smart adaptation method for AIGC video generation scheme, comprising the following steps:

[0008] Step S01. Sample Dataset Generation: Obtain the script sample dataset and parse each script data into multiple scenes. Extract the text description, visual parameters, and scheme parameters of different video generation schemes for each scene. Generate video feature vectors for each scene under different video generation schemes. The scheme parameters include algorithm type, resolution, and style coefficient. Calculate the global consistency value of each video generation scheme and the single-scene score of each scene under different visual generation schemes based on the video feature vectors of each scene under different video generation schemes. Weight the single-scene scores of each scene obtained under each visual generation scheme with the global consistency value to obtain the comprehensive score of each visual generation scheme. Construct the sample dataset using the text description, visual parameters, and scheme parameters corresponding to each script sample data as input data and the corresponding comprehensive score as labels.

[0009] Step S02. Model Training: Construct a deep network model, train the constructed deep learning model using the sample dataset, and obtain the scheme evaluation model after training is completed;

[0010] Step S03. Real-time solution adaptation: Obtain the script to be adapted and parse it into multiple scenes. Extract the text description and visual parameters of each scene, as well as the solution parameters of the required video generation solutions. Input the text description and visual parameters of each scene into the solution evaluation model, corresponding to the solution parameters of each video generation solution. The solution evaluation model outputs the comprehensive score of each video generation solution, and sorts the video generation solutions according to the comprehensive score to determine the optimal adapted video generation solution.

[0011] Further, in step S01, the single-shot score of each scene under different visual generation schemes is calculated based on the semantic alignment value between the image and the text description of each scene under different visual generation schemes, the consistency value between scenes, and the motion coherence value between adjacent frames within a scene. The comprehensive weight corresponding to each scene is calculated based on the semantic keyness and visual complexity of each scene. The semantic keyness is calculated based on the plot keywords in the text description of each scene, and the visual complexity is calculated based on the visual parameters of each scene. The weight of the single-shot score of each scene in the process of weighting to obtain the comprehensive score is the comprehensive weight.

[0012] Furthermore, the expression for calculating the single-shot score of each scene under different visual generation schemes is as follows:

[0013]

[0014]

[0015]

[0016]

[0017] in, Indicates the first The first video generation scheme The score for each individual shot in a storyboard. Indicates the first The first video generation scheme A value indicating the semantic alignment between the generated storyboard image and the storyboard text description. Indicates the first The first video generation scheme Motion coherence values ​​between adjacent frames within a single shot. It is the first Storyboard text description for each scene A text encoder that converts text into feature vectors. It is the first Total number of frames per shot It is the first Frame to the Frame in The optical flow vector at coordinates, Describing the L2 norm, For the first The first scene in the... Inter-scene consistency values ​​under a given video generation scheme , The first The, the +1 panel in the first Video feature vectors under various video generation schemes Represents the cosine similarity function. , , These are the weight parameters.

[0018] Furthermore, the calculation of the comprehensive weight corresponding to each scene based on its semantic keyness and visual complexity includes:

[0019] The BERT (Bidirectional Encoder Representations from Transformers) model was used to extract plot keywords from the scene descriptions in the target script, and the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm was used to calculate the semantic keyness of each scene description.

[0020]

[0021] in, For the first The semantic criticality of each scene. For the first A collection of key plot words for each storyboard. This is a collection of keywords for the entire script. For word frequency, Inverse document frequency, For a single keyword, For the target script, , Each represents a keyword In the Storyboard text Inverse document frequency and keywords In the target script Inverse document frequency;

[0022] Computational visual complexity :

[0023]

[0024]

[0025] in, For the first The visual complexity of each scene For the first Number of characters per storyboard For the first Number of scene changes per shot For the first The number of special effects keywords per scene Indicates the first The overall index of visual complexity of each scene. , ~ They represent the 1st to the 2nd. M The overall index of visual complexity of each scene. M Indicates the number of scenes;

[0026] Based on semantic keyness Visual complexity Calculate the overall weight of each scene. :

[0027]

[0028] in, Indicates the first The overall weight of each scene.

[0029] Further, in step S01, the step of generating video feature vectors for each scene under different video generation schemes includes:

[0030] Multiple key frames were sampled from each scene under different video generation schemes, and high-dimensional features were extracted from the image encoder in the CLIP (Contrastive Language-Image Pretraining) model to obtain the visual features of each key frame in each scene under different video generation schemes.

[0031] The attention weights of each keyframe in each scene under different video generation schemes are calculated based on the semantic relevance values ​​of keyframes and scene text in each scene and the motion salience values ​​of keyframes within the scene.

[0032] The visual features of each keyframe in each shot under different video generation schemes are aggregated with the corresponding attention weights to obtain the video feature vectors of each shot under different video generation schemes.

[0033] Furthermore, the formula for calculating the attention weight is:

[0034]

[0035]

[0036]

[0037] in, Indicates the first The first video generation scheme The first storyboard Attention weights for keyframes Indicates the first The first video generation scheme Frame keyframe and the first Storyboard text for each scene semantic relevance value, For the CLIP model text encoder, Indicates the first The first video generation scheme Frame keyframe at the Motion significance value within each segment Indicates the first Frame keyframe to the Optical flow vector field of keyframe, Indicates the preset coefficient. <1, Represents the cosine similarity function;

[0038] The expressions for calculating the video feature vectors of each scene under different video generation schemes are as follows:

[0039]

[0040] in, Indicates the first The first video generation scheme Video feature vectors of each scene, Indicates the first The first video generation scheme In the first scene Visual features of keyframes Indicates the number of keyframes.

[0041] Further, in step S01, the expression for calculating the global consistency value of each video generation scheme based on the video feature vectors of each scene under different video generation schemes is as follows:

[0042]

[0043] in, For the first The global consistency value of each video generation scheme. For the first The first scene in the... Video feature vectors under various video generation schemes For the first The first scene in the... Video feature vectors under various video generation schemes Represents the cosine similarity function;

[0044] The formula for calculating the overall score of the plan is as follows:

[0045]

[0046] in, For the first The overall score of each video generation scheme. Indicates the first The first video generation scheme The score for each individual shot in a storyboard. For the first The overall weight of each scene.

[0047] Furthermore, the text encoder using the CLIP model is used in deep network models. Textual description of the storyboard Encode the text to obtain text feature vectors. and an image encoder using the CLIP model. For visual parameters Encoding yields visual feature vectors A multimodal cross-attention mechanism is used to fuse text feature vectors. and visual feature vectors Obtain the fused feature vector :

[0048]

[0049]

[0050] in, Text feature vector Dimensions.

[0051] A computer device includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to perform the method described above.

[0052] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0053] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention dynamically calculates the comprehensive weight of each scene based on its semantic criticality and visual complexity during the sample dataset generation process. This adaptively adjusts the weight of each scene, eliminating human intervention bias. Furthermore, it calculates the single-scene score of each scene based on the semantic alignment between scenes, the consistency between scenes, and the running coherence of adjacent frames within a scene. This allows for the comprehensive evaluation of single scenes by considering semantic alignment, consistency, and motion coherence. Simultaneously, by combining the scores of each scene with global consistency, a comprehensive evaluation of the video generation scheme is performed to generate a sample dataset. This effectively quantifies and evaluates the coherence between scenes and the entire video by combining the consistency (local coherence) and global consistency (overall style uniformity) of the video generation scheme, improving the overall smoothness of the video. Combined with a deep learning model, it achieves intelligent adaptation of the AIGC video generation scheme, not only improving the evaluation accuracy of scene and scheme adaptation, but also enhancing the coherence of scenes and the entire video in the AIGC video generation scheme, ensuring a consistent style in the generated video, and effectively solving the problem of style discontinuity in long AIGC videos. Attached Figure Description

[0054] Figure 1 This is a schematic diagram illustrating the implementation process of the intelligent adaptation method for the AIGC video generation scheme in this embodiment. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0056] like Figure 1 As shown, the steps of the intelligent adaptation method for the AIGC video generation scheme in this embodiment include:

[0057] Step S01. Sample Dataset Generation: Obtain the script sample dataset and parse each script data into multiple scenes. Extract the text description, visual parameters, and scheme parameters of different video generation schemes for each scene. Generate video feature vectors for each scene under different video generation schemes. Scheme parameters include algorithm type, resolution, and style coefficient. Calculate the global consistency value of each video generation scheme and the single-scene score of each scene under different visual generation schemes based on the video feature vectors of each scene under different video generation schemes. Weight the single-scene scores of each scene obtained under each visual generation scheme with the global consistency value to obtain the comprehensive score of each visual generation scheme. Construct the sample dataset using the text description, visual parameters, and scheme parameters corresponding to each script sample data as input data and the corresponding comprehensive score as labels.

[0058] In this embodiment, a script sample dataset is first collected. For each script sample data in the dataset, the text in the script can be parsed using Natural Language Processing (NLP) technology to identify scene transitions, character dialogues, action descriptions, etc., breaking it down into multiple independent storyboards. Then, the text descriptions and visual parameters of each storyboard are extracted. The text descriptions are textual explanations of the storyboard content, and the visual parameters are extracted by parsing the number of characters, scene changes, and special effects keywords in the storyboard. Simultaneously, algorithm types, resolution / frame rate, and style coefficients for various video generation schemes are extracted. For example, the algorithm type can be a diffusion model, one-hot encoding, generative adversarial networks, variational autoencoders, image-based factorization methods, etc., which can be configured according to actual needs. The style coefficient is a parameter describing the video's style characteristics, such as realistic style, cartoon style, oil painting style, ink painting style, science fiction style, etc.

[0059] In this embodiment, the storyboard text description is the text that describes the content of the storyboard, such as the protagonist wearing a blue coat walking in the forest; the visual parameters are visually related parameters extracted from the text, including the number of characters, the number of scene changes, special effects keywords, etc.

[0060] Specifically, the script Defined as by It consists of several storyboards, each storyboard... It can be represented as a pair:

[0061] (1)

[0062] in, This is a description of the storyboard text; These are the visual parameters extracted from the text.

[0063] In specific application examples, the storyboard text description extracted by the BERT model can be used. The 768-dimensional embedding is used to obtain visual parameters by parsing the number of characters, scene changes, and special effects keywords in the storyboard. .

[0064] In this embodiment, the steps for generating video feature vectors for each scene under different video generation schemes include:

[0065] Step S111. Keyframe Selection: Select keyframes from different video generation schemes. Each of the following storyboards Medium-sampled multi-frame keyframes , Indicates the number of keyframes;

[0066] Step S112. Single-frame encoding: Using the image encoder in the CLIP model. High-dimensional features were extracted to obtain the visual features of each keyframe in each shot under different video generation schemes. ;

[0067] Step S113. Attention weight calculation: Calculate the attention weight of each key frame in each segment under different video generation schemes based on the semantic relevance value between the key frame and the segment text and the motion salience value of the key frame in the segment.

[0068] Step S114. Aggregate the visual features of each keyframe in each shot under different video generation schemes with the corresponding attention weights to obtain the video feature vector of each shot under different video generation schemes.

[0069] As an optional implementation, the formula for calculating attention weights can be expressed as:

[0070] (2)

[0071] (3)

[0072] (4)

[0073] in, Indicates the first The first video generation scheme The first storyboard Attention weights for keyframes Indicates the first The first video generation scheme Frame keyframe and the first Storyboard text for each scene semantic relevance value, For the CLIP model text encoder, Indicates the first The first video generation scheme Frame keyframe at the Motion significance value within each segment Indicates the first Frame keyframe to the Optical flow vector field of keyframe, Represents the cosine similarity function. Indicates the preset coefficient. <1, for example, static scenes are acceptable In dynamic scenarios, it can be adjusted to 0.5.

[0074] As an optional implementation method, the calculation expression for the video feature vector of each scene under different video generation schemes is as follows:

[0075] (5)

[0076] in, Indicates the first The first video generation scheme Video feature vectors of each scene, Indicates the first The first video generation scheme In the first scene Visual features of keyframes.

[0077] In this embodiment, in order to quantify the importance of different scenes and adaptively determine the corresponding weights in the scheme evaluation process, the comprehensive weight of each scene is calculated based on the semantic keyness and visual complexity of each scene. The semantic keyness is calculated based on the plot keywords in the text description of each scene, and the visual complexity is calculated based on the visual parameters of each scene. The weight of the single scene score of each scene in the process of weighting to obtain the comprehensive score is the comprehensive weight.

[0078] As an optional implementation method, the overall weight of each scene can be calculated according to the following steps:

[0079] Step S121. Extract the text descriptions of each scene in the target script using the BERT model. The plot keywords were used to calculate the semantic keyness of each scene's text description using the TF-IDF algorithm:

[0080] (6)

[0081] in, For the first The semantic criticality of each scene. For the first A collection of key plot words for each storyboard. This is a collection of keywords for the entire script. For word frequency, For a single keyword (i.e., a set) and (elements in) The target script (i.e., the complete script containing all storyboards). , Each represents a keyword In the Storyboard text Inverse document frequency and keywords In the target script Inverse document frequency;

[0082] Step S122. Calculate the corresponding visual complexity based on the number of characters, scene changes, and special effects keywords in each storyboard. :

[0083] (7)

[0084] (8)

[0085] in, For the first The visual complexity of each scene For the first Number of characters per storyboard For the first Number of scene changes per shot For the first The number of special effects keywords per scene , M Indicates the number of scenes. For set The maximum value in;

[0086] Step S123. Based on semantic keyness Visual complexity Calculate the overall weight of each scene. :

[0087] (9)

[0088] in, Indicates the first The overall weight of each scene.

[0089] This embodiment is based on the semantic keyness of each scene. Visual complexity By determining the overall weight of each scene, we can adaptively determine the appropriate weight for different scenes. Then, by using this overall weight to weight each scene, we can obtain a score for the overall plan by combining all the scenes.

[0090] As an optional implementation, the single-shot score of each scene under different visual generation schemes can be calculated according to the following steps:

[0091] Step S131. Based on the similarity between the video feature vectors of adjacent shots under different video generation schemes, calculate the local consistency value of each shot under different video generation schemes. For example, it can be calculated using the following formula:

[0092] (10)

[0093] in, For the first The first scene in the... Inter-scene consistency values ​​under a given video generation scheme For the first The first scene in the... Video feature vectors under various video generation schemes This represents the cosine similarity function. For example, for a dimension of... vector and cosine similarity function Defined as:

[0094] (11)

[0095] Step S132. Based on the similarity between the video feature vectors of each scene under different video generation schemes and the feature vectors obtained by converting the scene text descriptions, calculate the semantic alignment value between the scene images and the scene text descriptions:

[0096] (12)

[0097] in, Indicates the first The first video generation scheme A value indicating the semantic alignment between the generated storyboard image and the storyboard text description. It is the first Storyboard text description for each scene A text encoder that converts text into feature vectors;

[0098] Step S133. Calculate the motion coherence value between adjacent frames within a scene based on the optical flow vectors of each scene:

[0099] (13)

[0100] in, Indicates the first The first video generation scheme Motion coherence values ​​between adjacent frames within a single shot. It is the first Total number of frames per shot It is the first Frame to the Frame in The optical flow vector at coordinates, This represents the L2 norm.

[0101] Step S134. Overall Inter-Scene Consistency Value The semantic alignment value between storyboard images and storyboard text descriptions Motion coherence value between adjacent frames within a storyboard Calculate the single-shot score for each scene under each visual generation scheme:

[0102] (14)

[0103] in, Indicates the first The first video generation scheme The score for each individual shot in a storyboard. , , For the weight parameters, satisfying The value can be adjusted according to the actual needs of the scenario or experimental data (for example, it can be taken as...). , , ; Indicates the first The first video generation scheme The score for each scene in a storyboard.

[0104] The above formula (15) can be used to comprehensively evaluate the consistency between each segment, the semantic alignment between the segment image and the segment text description, and the motion coherence between adjacent frames within the segment, so that each segment in the final generated video scheme can maintain consistency between segments, semantic alignment and operation coherence.

[0105] This embodiment integrates the scores of each scene under various video generation schemes and the global consistency value of each video generation scheme to construct an evaluation model for the overall scheme score. This model calculates the overall scheme score for different video generation schemes, accurately assessing the adaptability of different schemes while ensuring consistency throughout the entire film. As an optional implementation, the calculation expression for the overall scheme score can be expressed as:

[0106] (15)

[0107] in, For the first The overall score of each video generation scheme. For the first The weight of each scene, M For the number of scenes, For the first The first video generation scheme The score for each individual shot in a storyboard. For the first The global consistency value of each video generation scheme.

[0108] As an optional implementation, the global consistency value of each video generation scheme can be calculated based on the video feature vectors of each scene under different input video generation schemes. The calculation expression can be expressed as:

[0109] (16)

[0110] in, For the first The global consistency value of each video generation scheme. For the first The first scene in the... Video feature vectors under various video generation schemes For the first The first scene in the... Video feature vectors under various video generation schemes Represents the cosine similarity function, the first... The first scene is the one that is currently in the [number]th frame. Each storyboard has a different storyboard.

[0111] By using the above formula (15) to calculate the comprehensive score of the scheme, we can consider the consistency between different scenes, the degree of semantic alignment, and the motion coherence of each scene under different schemes from a local perspective, and at the same time consider the global consistency of the overall scheme from a global perspective. We can evaluate each scheme by combining local and global consistency, so that we can prioritize the scheme that can ensure consistency between scenes, semantic alignment, motion coherence, and the consistency and coherence of the whole film. Using this method to generate a sample dataset can effectively improve the accuracy of scene and scheme adaptation.

[0112] Step S02. Model Training: Construct a deep network model, train the constructed deep learning model using the sample dataset, and obtain the scheme evaluation model after training is completed.

[0113] In this embodiment, a deep network model is first constructed. The parameters input to the model's input layer include BERT text embeddings (...). Visual complexity vector The system includes scheme parameters such as algorithm type, resolution / frame rate, etc., and optimizes the deep network architecture for the temporal and multimodal characteristics of AIGC videos.

[0114] After the model is built, it is trained. The objective function during training is:

[0115] (17)

[0116] in, Let N be the objective function; N is the number of training samples. For the scheme... And sample n, The global score given by the model; This is for testing the overall score. After the model is trained, the scheme evaluation model is obtained. Using this scheme evaluation model, the comprehensive score of different video generation schemes can be automatically calculated for the target script, and the schemes can be sorted according to the comprehensive score.

[0117] In this embodiment, the deep network model is optimized, the CLIP model is used for feature encoding, and the text encoder of the CLIP model is used. Description of storyboard text Encode the text to obtain text feature vectors. Simultaneously using the CLIP model image encoder For visual parameters Encoding is performed to simulate the image feature extraction process, resulting in visual feature vectors. This places textual and visual features in the same dimensional space.

[0118] Furthermore, fused features can be obtained by fusing textual and visual features based on multimodal cross-attention. :

[0119] (18)

[0120] (19)

[0121] in, The weights of the text feature vectors. It is a text feature vector The dimension (i.e., the size of the hidden layer dimension of the text features) is used to scale the dot product results of text and visual features to avoid the dot product value being too large due to the feature dimension being too high, which would affect the stability of Softmax.

[0122] Step S03. Real-time solution adaptation: Obtain the script to be adapted and parse it into multiple scenes. Extract the text description, visual parameters, and solution parameters of multiple video generation solutions required for each scene. Input them into the solution evaluation model, and the solution evaluation model outputs the optimal video generation solution.

[0123] In a specific application embodiment, for the target script, the script is first parsed into M scenes, and the text description of each scene is extracted. and visual parameters (Number of characters, etc.) Use the BERT model to extract text descriptions. The 768-dimensional embedding is used to analyze visual parameters through structured parsing. And extract the following parameters: algorithm type (3D one-hot encoding) + resolution / frame rate (3D normalization) + style coefficients (6D), using the CLIP model for the text encoder. Textual description of the storyboard Encode the text to obtain text feature vectors. and an image encoder using the CLIP model. For visual parameters Encoding yields visual feature vectors A multimodal cross-attention mechanism is used to fuse text feature vectors. and visual feature vectors Obtain the fused feature vector The input is fed into a pre-trained scheme evaluation model, which then outputs the optimal video generation scheme.

[0124] This invention evaluates each scene by comprehensively considering semantic alignment, motion coherence, and consistency among scenes. It dynamically adjusts the weights of each scene based on its semantic criticality and visual complexity, employing a temporal attention mechanism to achieve deep fusion of multimodal features rather than static splicing. Furthermore, it comprehensively evaluates the video generation scheme by combining the scores of each scene with global consistency, generating a sample dataset. Combined with a deep network model, it achieves intelligent adaptation of the AIGC video generation scheme. This invention quantifies the coherence of scenes and the entire film by combining local consistency indicators of scenes with global consistency indicators, effectively solving the problem of style discontinuity in long AIGC videos.

[0125] Furthermore, lightweight inference branches (reducing the number of parameters by 60%) can be constructed to support dynamic adjustments to the scheme during video generation (e.g., optimizing the parameters of scene 4 when generating scene 3); and incremental learning mechanisms can be set up, such as using the K-means clustering algorithm to cluster new script types and using a small number of samples to update the model (reducing the update time from 8 hours to 1.5 hours) to adapt to new scenarios (such as virtual human live streaming).

[0126] To verify the effectiveness of this invention, a dataset of 300 scripts (covering animation, realism, and documentary types) containing 5200 storyboards (each storyboard averaging 243 frames) was selected. Experiments were conducted using the method described above, and compared with the manual adaptation method. The results are shown in Table 1. The overall score accuracy measures the consistency between the model's predicted comprehensive score and the manually labeled true comprehensive score; a higher value indicates that the predicted score is closer to the true level. Ranking accuracy includes Top-1 accuracy and the Spearman coefficient. Top-1 accuracy refers to the percentage of samples where the model's recommended optimal solution matches the manually labeled optimal solution. The Spearman coefficient reflects the overall correlation between model ranking and manual ranking (range [-1, 1], with a stronger correlation closer to 1). Adaptation time is the total time (in seconds) from script parsing completion to outputting the optimal solution. The solution of this invention specifically includes storyboard feature extraction and model inference. The adaptation time for the manual adaptation method is the time required to generate the video plus the time required for manual review by general practitioners.

[0127] Table 1: Comparison Results

[0128]

[0129] As shown in Table 1, in terms of overall score accuracy, the proposed solution, through optimizations such as dynamic calculation of scene weights and multimodal fusion, achieves a significantly higher accuracy than manual adaptation, indicating a substantial improvement in evaluation precision. Regarding ranking accuracy, manual adaptation is significantly influenced by subjective factors, while the proposed solution, through a ranking logic that integrates single-scene scores and global consistency, significantly improves both the optimal solution matching rate and the overall ranking relevance. In terms of adaptation time, the proposed solution, through automated reasoning using a deep learning model, drastically reduces the time consumption compared to manual adaptation, resulting in a significant efficiency improvement that can meet the needs of large-scale script processing.

[0130] This embodiment further provides a computer device, including a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to perform the method as described above.

[0131] It is understood that the method described in this embodiment can be executed by a single device, such as a computer or server, or it can be applied to a distributed scenario where multiple devices cooperate to complete the task. In a distributed scenario, one of the multiple devices may execute only one or more steps of the method described in this embodiment, and the multiple devices interact to complete the method. The processor can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the method described in this embodiment. The memory can be implemented using read-only memory (ROM), random access memory (RAM), static storage devices, and dynamic storage devices. The memory can store the operating system and other applications. When the method described in this embodiment is implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0132] This embodiment further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0133] Those skilled in the art will understand that the above embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce implementations of the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for intelligent adaptation of AIGC video generation scheme, characterized in that the steps include: include: Step S01. Sample Dataset Generation: Obtain the script sample dataset and parse each script data into multiple scenes. Extract the text description, visual parameters, and scheme parameters of different video generation schemes for each scene. Generate video feature vectors for each scene under different video generation schemes. The scheme parameters include algorithm type, resolution, and style coefficient. Calculate the global consistency value of each video generation scheme and the single-scene score of each scene under different visual generation schemes based on the video feature vectors of each scene under different video generation schemes. Weight the single-scene scores of each scene obtained under each visual generation scheme with the global consistency value to obtain the comprehensive score of each visual generation scheme. Construct the sample dataset using the text description, visual parameters, and scheme parameters corresponding to each script sample data as input data and the corresponding comprehensive score as labels. Step S02. Model Training: Construct a deep network model, train the constructed deep learning model using the sample dataset, and obtain the scheme evaluation model after training is completed; Step S03. Real-time solution adaptation: Obtain the script to be adapted and parse it into multiple scenes. Extract the text description and visual parameters of each scene, as well as the solution parameters of the required video generation solutions. Input the text description and visual parameters of each scene into the solution evaluation model, and output the comprehensive score of each video generation solution. Sort the video generation solutions according to the comprehensive score and determine the optimal video generation solution to be adapted. In step S01, the expression for calculating the global consistency value of each video generation scheme based on the video feature vectors of each scene under different video generation schemes is as follows: in, For the first The global consistency value of each video generation scheme. For the first The first scene in the... Video feature vectors under various video generation schemes For the first The first scene in the... Video feature vectors under various video generation schemes Represents the cosine similarity function. M Indicates the number of scenes; The formula for calculating the overall score of the plan is as follows: in, For the first The overall score of each video generation scheme. Indicates the first The first video generation scheme The score for each individual shot in a storyboard. For the first The overall weight of each scene.

2. The intelligent adaptation method for AIGC video generation scheme according to claim 1, characterized in that, In step S01, the single-shot scores of each scene under different visual generation schemes are calculated based on the semantic alignment between the image and the text description of each scene, the consistency between scenes, and the motion coherence between adjacent frames within a scene. The comprehensive weight of each scene is calculated based on the semantic keyness and visual complexity of each scene. The semantic keyness is calculated based on the plot keywords in the text description of each scene, and the visual complexity is calculated based on the visual parameters of each scene. The weight of the single-shot scores of each scene in the process of weighting to obtain the comprehensive score is the comprehensive weight.

3. The intelligent adaptation method for AIGC video generation scheme according to claim 2, characterized in that, The expression for calculating the single-shot score of each scene under different visual generation schemes is as follows: in, Indicates the first The first video generation scheme The score for each individual shot in a storyboard. Indicates the first The first video generation scheme A value indicating the semantic alignment between the generated storyboard image and the storyboard text description. Indicates the first The first video generation scheme Motion coherence values ​​between adjacent frames within a single shot. It is the first Storyboard text description for each scene A text encoder that converts text into feature vectors. It is the first Total number of frames per shot It is the first Frame to the Frame in The optical flow vector at coordinates, Describing the L2 norm, For the first The first scene in the... Inter-scene consistency values ​​under a given video generation scheme , The first The, the +1 panel in the first Video feature vectors under various video generation schemes Represents the cosine similarity function. , , These are the weight parameters.

4. The intelligent adaptation method for AIGC video generation scheme according to claim 2, characterized in that, The calculation of the comprehensive weight corresponding to each scene based on the semantic keyness and visual complexity includes: The BERT model was used to extract plot keywords from the scene descriptions in the target script, and the TF-IDF algorithm was used to calculate the semantic keyness of each scene description. : in, For the first The semantic criticality of each scene. For the first A collection of key plot words for each storyboard. This is a collection of keywords for the entire script. For word frequency, Inverse document frequency, For a single keyword, For the target script, , Each represents a keyword In the Storyboard text Inverse document frequency and keywords In the target script Inverse document frequency; The visual complexity is calculated based on the number of characters, scene changes, and special effects keywords in each storyboard. : in, For the first The visual complexity of each scene For the first Number of characters per storyboard For the first Number of scene changes per shot For the first The number of special effects keywords per scene Indicates the first The overall index of visual complexity of each scene. , ~ They represent the 1st to the 2nd. M The overall index of visual complexity of each scene. M Indicates the number of scenes; Based on semantic keyness Visual complexity Calculate the overall weight of each scene. : in, Indicates the first The overall weight of each scene.

5. The intelligent adaptation method for AIGC video generation scheme according to claim 1, characterized in that, In step S01, the step of generating video feature vectors for each scene under different video generation schemes includes: Multiple key frames were sampled from each scene under different video generation schemes, and high-dimensional features were extracted using the image encoder in the CLIP model to obtain the visual features of each key frame in each scene under different video generation schemes. The attention weights of each keyframe in each scene under different video generation schemes are calculated based on the semantic relevance values ​​of keyframes and scene text in each scene and the motion salience values ​​of keyframes within the scene. The visual features of each keyframe in each shot under different video generation schemes are aggregated with the corresponding attention weights to obtain the video feature vectors of each shot under different video generation schemes.

6. The intelligent adaptation method for AIGC video generation scheme according to claim 5, characterized in that, The formula for calculating the attention weight is: in, Indicates the first The first video generation scheme The first storyboard Attention weights for keyframes Indicates the first The first video generation scheme Frame keyframe and the first Storyboard text for each scene semantic relevance value, For the CLIP model text encoder, Indicates the first The first video generation scheme Frame keyframe at the Motion significance value within each segment Indicates the first Frame keyframe to the Optical flow vector field of keyframe, Indicates the preset coefficient. <1, Represents the cosine similarity function; The expressions for calculating the video feature vectors of each scene under different video generation schemes are as follows: in, Indicates the first The first video generation scheme Video feature vectors of each scene, Indicates the first The first video generation scheme In the first scene Visual features of keyframes Indicates the number of keyframes.

7. The intelligent adaptation method for AIGC video generation scheme according to any one of claims 1 to 6, characterized in that, The deep network model uses a CLIP model text encoder. The text description of the storyboard is encoded to obtain a text feature vector. and an image encoder using the CLIP model. Visual parameters are encoded to obtain visual feature vectors. A multimodal cross-attention mechanism is used to fuse text feature vectors. and visual feature vectors Obtain the fused feature vector : in, Text feature vector Dimensions.

8. A computer device comprising a processor and a memory, the memory being used to store a computer program, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images