Video generation method and system of self-adaptive model and related equipment

Abstract

The invention provides a video generation method and system of a self-adaptive model and related equipment. The method comprises the following steps: constructing a multi-model aggregation video generation platform; performing model characteristic labeling of a preset table on each model in the multi-model aggregation video generation platform; receiving contents and demands input by a user; based on the content and the demand input by the user and in combination with the model characteristic label, performing model matching of adaptive modes including a user preference priority mode, a resource priority mode, a quality priority mode, a cost priority mode and a fault-tolerant priority mode, and obtaining a recommended video generation model; and processing the content input by the user based on the recommended video generation model to obtain a target video. According to the technical scheme, the user only needs to input the content and the demand, the multi-model aggregation video generation platform can adaptively recommend the appropriate model based on the content and the demand input by the user, and the efficiency and the convenience of video production of the user are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video production, and in particular to a video generation method based on adaptive model, a system and related equipment. BACKGROUND

[0002] At present, there are many video generation models in the market, each model has its characteristics and advantages, and it is difficult for creators to quickly determine which model is most suitable for their needs. For example, when selecting a model for advertising production, the performance of the model in terms of picture quality, style matching degree and other aspects needs to be considered, which makes model selection a complex task, and the prior art lacks a video generation method based on adaptive model.

[0003] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0004] The present application provides a video generation method based on adaptive model, a system and related equipment, and the main purpose of the present application is to solve the technical problems mentioned in the background of the prior art.

[0005] The first aspect of the present application provides a video generation method based on adaptive model, comprising: constructing a multi-model aggregation video generation platform; annotating the model characteristics of each model in the multi-model aggregation video generation platform with a preset table; receiving user input content and requirements; based on the content and requirements input by the user and combined with the model characteristic annotation, performing model matching in adaptive mode to obtain a recommended video generation model; processing the content input by the user based on the recommended video generation model to obtain a target video.

[0006] In an optional implementation of the first aspect of the present application, the model characteristic annotation of each model in the multi-model aggregation video generation platform with a preset table comprises: annotating model performance parameters, the model performance parameters including computing resource requirements, processing speed, concurrency capability, maximum load capacity and response time; annotating model functional characteristics, the model functional characteristics including supported video formats, resolution ranges, frame rate ranges, special effect capabilities and audio / video processing capabilities; annotating model quality indicators, the model quality indicators including video sharpness, color restoration, compression ratio, encoding efficiency and decoding performance; annotating model compatibility information, the model compatibility information including interoperability with other models, data format compatibility, interface protocol support; annotating model maintenance information, the model maintenance information including update frequency, version control, bug fixing record, and performance optimization history.

[0007] In an optional implementation of the first aspect of the present application, the model matching in the adaptive mode based on the content and the requirement input by the user and in combination with the model characteristic annotation to obtain a recommended video generation model comprises: obtaining a model type of video generation based on the content input by the user; obtaining a target video generation model set based on the model type; obtaining a target model matching mode based on the requirement input by the user, the target model matching mode being one or more of the adaptive modes; obtaining a recommended video generation model from the target video generation model set based on the target model matching mode and in combination with the model characteristic annotation.

[0008] In an optional implementation of the first aspect of the present application, the obtaining of a recommended video generation model from the target video generation model set based on the target model matching mode and in combination with the model characteristic annotation comprises: if the target model matching mode is a user preference priority mode, obtaining historical video generation cases of each model in the target video generation model set; comparing the content input by the user with historical input content of the historical video generation cases of each model in terms of similarity; performing a half-screen filtering on the target video generation model set based on the similarity comparison result to obtain a candidate video generation model set; performing a scoring and weighted operation on generation ability, technical ability, and consistency ability of each model in the candidate video generation model set based on the model characteristic annotation to obtain a model with the highest score from the candidate video generation model set as a recommended video generation model.

[0009] In an optional implementation of the first aspect of the present application, the obtaining of a recommended video generation model from the target video generation model set based on the target model matching mode and in combination with the model characteristic annotation further comprises: if the target model matching mode is a resource priority mode, obtaining a current load state of the system; analyzing current available resources based on the current load state of the system; comparing the load state and the available resources with a characteristic annotation related to resource occupation of each model in the target video generation model set, the characteristic annotation related to resource occupation including computing resource demand, memory demand, and storage demand; The optimization algorithm is used to filter the constraint conditions of the target video generation model and to score the adaptability; The model with the best adaptability score is selected from the target video generation model set as the recommended video generation model.

[0010] In an optional implementation of the first aspect of the present application, the recommended video generation model is obtained from the target video generation model set based on the target model matching mode and in combination with the model characteristic annotations, and the method further comprises: If the target model matching mode is a quality priority mode, the model generation quality related characteristic annotations of each model are collected from the target video generation model set, and the model generation quality related characteristic annotations include definition, color restoration, sound quality and content coherence; According to the requirements of the current generation task, different weights are given to each of the model generation quality related characteristic annotations, and a preset quality evaluation algorithm is applied to score each model in the target video generation model set; According to the generation quality score of each model, the model with the highest score is selected from the target video generation model set as the recommended video generation model.

[0011] In an optional implementation of the first aspect of the present application, the recommended video generation model is obtained from the target video generation model set based on the target model matching mode and in combination with the model characteristic annotations, and the method further comprises: If the target model matching mode is a cost priority mode, the cost related characteristic annotations of each model are obtained from the target video generation model set; According to the cost budget and target of the user, a corresponding scoring standard is set to guide the model selection; Based on the set scoring standard, the cost score of each model in the target video generation model set is calculated; According to the cost score, the model with the lowest cost is selected from the target video generation model set as the recommended video generation model.

[0012] The second aspect of the present application provides a self-adaptive model video generation system, which comprises: A platform construction module is configured to construct a multi-model aggregated video generation platform; A characteristic annotation module is configured to perform model characteristic annotation on each model in the multi-model aggregated video generation platform according to a preset table; An input receiving module is configured to receive the content and requirements input by the user; A model recommendation module is configured to perform adaptive mode model matching based on the content and requirements input by the user and in combination with the model characteristic annotations, and to obtain a recommended video generation model; The video generation module is configured to process the content input by the user based on the recommended video generation model to obtain a target video.

[0013] The third aspect of the present application provides a video generation device of an adaptive model, comprising a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected by a circuit; The at least one processor invokes the instructions in the memory to enable the video generation device of the adaptive model to perform the video generation method of the adaptive model according to any one of the first aspect of the present application.

[0014] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the video generation method of the adaptive model according to any one of the first aspect of the present application.

[0015] Beneficial effects: The present application provides a video generation method, system and related device of an adaptive model, which comprises constructing a multi-model aggregation video generation platform, performing model characteristic annotation on a preset table for each model in the multi-model aggregation video generation platform, receiving content and requirements input by a user, performing model matching in an adaptive mode based on the content and the requirements input by the user and in combination with the model characteristic annotation, obtaining a recommended video generation model, and processing the content input by the user based on the recommended video generation model to obtain a target video. The present application has the beneficial effects that a user only needs to input content and requirements, and the multi-model aggregation video generation platform can adaptively recommend a suitable model based on the content and the requirements input by the user, thereby improving the efficiency and convenience of video production by the user. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 An embodiment schematic diagram of a video generation method of an adaptive model according to the present application; Figure 2 An embodiment schematic diagram of a video generation system of an adaptive model according to the present application; Figure 3 An embodiment schematic diagram of a video generation device of an adaptive model according to the present application. DETAILED DESCRIPTION

[0017] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-mentioned drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so-termed "first", "second", "third", "fourth" and the like, if any, in the description and in the claims of the present application is simply intended to differentiate between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used can be interchanged, where appropriate, so that the embodiments described herein can be carried out in sequences other than those illustrated or described herein. Furthermore, the term "comprising" or "having" and any of its variants, is intended to cover both the inclusive and the exclusive case, e.g., a process, method, system, product, or apparatus that comprises a list of steps or units, without necessarily being limited to those clearly listed, but can include other steps or units not clearly listed or inherent to such process, method, product, or apparatus.

[0018] For the purpose of facilitating understanding, the specific flow of the embodiments of the present application is described below, please refer to Figure 1 The first aspect of the present application provides a video generation method of an adaptive model, comprising: S100, a multi-model aggregation video generation platform is constructed; in the present application, the multi-model aggregation video generation platform is integrated with a video generation model (LitAI) independently researched by the present application and a video generation model publicly applied on the market. The video generation model (LitAI) independently researched by the present application adopts a multi-modal diffusion architecture and has a text-image-video cross-modal alignment capability. Input type: supporting text description (200 tokens) + reference image (1024x1024) + style guide vector. Hierarchical feature extraction is performed: text encoding, image encoding, and modal alignment. According to the multi-modal diffusion architecture, feature fusion and decoding are performed, which include dynamic rendering decoding, such as input: a small dog is running on the grass. First, extract the key elements of the text: a small dog; light rendering: on the grass; physical simulation: running; generate the frame number in combination with multi-modal diffusion.

[0019] S200, model characteristic labeling of preset table is carried out to each model in the multi-model aggregation video generation platform; in the application, the purpose of model characteristic labeling is to adapt the model based on user demand subsequently, recommend suitable video generation model to the user, in an optional embodiment of the application, the step can comprise: model performance parameters are labeled, the model performance parameters include computing resource demand, processing speed, concurrent ability, maximum load and response time; model function characteristics are labeled, the model function characteristics include supported video format, resolution range, frame rate range, special effect ability and audio and video processing ability; model quality indexes are labeled, the model quality indexes include video definition, color restoration degree, compression ratio, coding efficiency and decoding performance; model compatibility information is labeled, the model compatibility information includes interoperability with other models, data format compatibility, interface protocol support; model maintenance information is labeled, the model maintenance information includes update frequency, version control, error repair record and performance optimization history.

[0020] S300, receiving user input content and demand; in the application, the user input content can be one or more of text, picture and video, that is, the application supports multiple input modes, including but not limited to image-based video generation, text-based video generation, video-based video generation and other specific purpose generation models (such as style conversion, special effect generation, etc.). The user input demand can be the user's expectation of generating a video directly, or it can be a direct description of the required model matching mode. If the user input demand is the user's expectation of generating a video, the user's expectation of generating a video needs to be further analyzed in multiple granularity semantics, and the model matching mode required by the user is inferred.

[0021] S400, model matching of adaptive mode is carried out based on the content and the demand input by the user and combined with the model characteristic labeling, and a recommended video generation model is obtained, the adaptive mode includes user preference priority mode, resource priority mode, quality priority mode, cost priority mode and fault tolerance priority mode. In an optional embodiment of the application, the content and the demand input by the user can be segmented and disassembled, and the segmented content and the demand are paired based on content description scene, so that each disassembled content can use different adaptive mode matching models, and finally the videos generated by different models are spliced to obtain the final generated video.

[0022] In an optional embodiment of step S400 of the present application, the model matching in the adaptive mode based on the content and the requirement input by the user and in combination with the model characteristic label to obtain the recommended video generation model can further include: obtaining a video generation model type based on the content input by the user; obtaining a target video generation model set based on the model type; obtaining a target model matching mode based on the requirement input by the user, the target model matching mode being one or more of the adaptive modes; and obtaining the recommended video generation model from the target video generation model set based on the target model matching mode and in combination with the model characteristic label. In this embodiment, for example, the content input by the user is a picture, the model type for this video generation can be a picture-to-video model, and the content input by the user is music, the model type for this video generation can be a music-to-video model. In the present application, the scenario for obtaining the target model matching mode based on the requirement can be, for example, that the user's expectation description is that the user hopes to complete the video generation within a certain time, and the generated video needs to meet at least 2k resolution and more than 1 hour in length, with a hairstyle local redrawn character special effect. In the present application, the platform first performs semantic analysis on the user's expectation description to obtain keywords related to the mode, and then uses the keywords related to the mode to obtain the corresponding target model matching mode. In an optional embodiment of step S400 of the present application, the model matching in the adaptive mode based on the content and the requirement input by the user and in combination with the model characteristic label to obtain the recommended video generation model can further include: Scenario 1: If the target model matching mode is the user preference priority mode, the historical video generation cases of each model in the target video generation model set are obtained; the content input by the user is compared with the historical input content of the historical video generation cases of each model in terms of similarity; the target video generation model set is half-screened based on the similarity comparison result to obtain a candidate video generation model set; and the scoring and weighted operation of the generation ability, technical ability and consistency ability of each model in the candidate video generation model set are performed based on the model characteristic label to obtain the model with the highest score from the candidate video generation model set as the recommended video generation model.

[0023] Specifically, when the model recommendation in the user preference priority mode is performed, first, the system retrieves the historical video generation case library of each model from the target video generation model set. The case library contains the historical input content and the corresponding output video data recorded when each model performs a task in the past. For example, for a text-to-video model A, the case library storage entry can include: {input text: "riding scene at the foot of the snow-capped mountain", output video: video_A001.mp4}. Then, the content description text input by the user (such as "riding scene at the foot of the snow-capped mountain") is compared with the historical input content of each model in terms of similarity.Figure 1 The "Urban Nighttime Time-Lapse" shown is compared with the historical input content in each model historical case for similarity comparison. In a preferred embodiment: the text is vectorized using natural language processing techniques, such as extracting a 768-dimensional semantic feature vector through the BERT model, calculating the cosine similarity of the user input vector and each historical case vector, taking the highest historical case similarity of each model as the matching score of the model, based on the similarity comparison result, the system performs dynamic half-screening: arrange the target model set in descending order of model similarity score, and select the top 50% of models to form the candidate video generation model set (Note: when the number of models is odd, it can be rounded up, for example, 7 models are retained in the top 4), for each model in the pre-recommended model set, the system performs multi-dimensional capability evaluation according to its pre-stored model characteristic annotations: generation capability: based on the objective indicators of the model output video (such as resolution, support frame rate stability, rendering quality score); technical capability: according to the model architecture characteristics (such as whether to support light and shadow physics simulation, particle system complexity); consistency capability: evaluate the input-output alignment of historical cases (such as the semantic matching degree of user instructions and generated content), use weighted operation to calculate the comprehensive score of the model, comprehensive score = α × generation capability + β × technical capability + γ × consistency capability, where the weight coefficients satisfy α + β + γ = 1, in the user preference priority mode, a higher consistency weight is set in priority (for example, α = 0.3, β = 0.2, γ = 0.5), finally, the system selects the model with the highest comprehensive score as the recommended video generation model output.

[0024] Scenario 2: If the target model matching mode is the resource priority mode, obtain the current load state of the system; analyze the available resources based on the current load state of the system; compare the load state and the available resources with the resource occupation related characteristic annotations of each model in the target video generation model set, the resource occupation related characteristic annotations include computing resource demand, memory demand and storage demand; use optimization algorithm to filter the target video generation model with constraints and adaptive scoring; select the model with the best adaptive score from the target video generation model set as the recommended video generation model.

[0025] Specifically, when the user selects the resource priority mode (such as Figure 3The system collects the following load parameters in real time through the built-in resource monitoring module: real-time utilization of CPU / GPU, free memory capacity, storage I / O read / write rate, and network bandwidth available amount. Exemplary: for example, the current monitoring value is {CPU: 65%, GPU: 78%, free memory: 12.3 GB, storage bandwidth: 320 MB / s}. The resource analysis engine converts the load state into an available resource vector: V_available=[Compute, Memory, Storage] where: Compute=(1-max(CPU%, GPU%)) × total computing core number × frequency coefficient; Memory=free memory × memory bandwidth attenuation factor; Storage = min(storage I / O rate, network bandwidth) × concurrency coefficient. Model characteristic labeling comparison, in the system's preset video generation model library, each model is associated with structured characteristic labeling: models that pass the compliance check enter the multi-objective optimization engine and perform the following processing flow: constraint filtering: exclude models that do not meet the hard latency requirement (such as generation time <5s), adaptive scoring: use a weighted scoring algorithm Score=α×(1-Computing resource ratio)+β×(1-Memory ratio) +γ×QoE_history score (where α+β+γ=1, QoE_history score is based on user feedback data), optimal model recommendation, sort the candidate models in descending order of score, and select the Top1 model as the recommended result.

[0026] Scenario 3: If the target model matching mode is the quality priority mode, collect the quality-related characteristic labels of each model from the target video generation model set, and the quality-related characteristic labels include definition, color restoration, sound quality, and content coherence; according to the requirements of the current generation task, give different weights to each of the quality-related characteristic labels, and apply the preset quality evaluation algorithm to score each model in the target video generation model set; according to the generation quality score of each model, select the recommended video generation model with the highest score from the target video generation model set.

[0027] Specifically, in the present application, sharpness is used to evaluate the sharpness and detail performance of the video picture, whether there are blurred noise points or artifacts. Color restoration degree is used to evaluate whether the color of the video is accurate, natural and matched with the original intention or reference material. Sound quality is used to evaluate the clarity and fidelity of the accompanying sound in the video (if any), whether there are distortion, noise or sound picture synchronization problems. Content coherence is used to evaluate the logical fluency of the video in time sequence, the naturalness of scene / object movement and the coherence of the overall narrative, avoiding abrupt changes, logical breaks or content inconsistencies. The weight configuration module of the system assigns different importance weights to the collected generation quality characteristic labels according to the specific demand information of the current generated task. The weight assignment principle: the task demand may come from the user's explicit specification (for example, the user emphasizes that "the color must be accurate") or the system automatically analyzes according to the task context (for example, generating artistic creation video may pay more attention to color and creative coherence, and generating teaching demonstration video may pay more attention to clarity and content coherence). The weight value is usually a numerical value (for example, 0-1 range, total is 1; or 1-10 level). The model evaluation module of the system uses the preset quality evaluation algorithm, combines the collected specific characteristic label values of each model and the determined dynamic weight, and performs comprehensive quality scoring on each candidate model in the target video generation model set. The evaluation process: For each candidate model M_i, its characteristic label vector (C_i, F_i, A_i, H_i) is input into the evaluation algorithm, where C_i represents the clarity score, F_i represents the color restoration score, A_i represents the sound quality score, and H_i represents the content coherence score. The core of the algorithm is to calculate a weighted comprehensive score: QS_i=w_c*C_i+w_f*F_i+w_a*A_i+w_h * H_i, where w_c, w_f, w_a, w_h are the weights of clarity, color restoration, sound quality and content coherence respectively, and satisfy w_c+w_f+w_a+w_h=1. The generation quality score (QS_i) of each candidate model is screened. From the target video generation model set, the candidate model with the highest generation quality score is selected.

[0028] Scenario 4: If the target model matching mode is the cost priority mode, the cost-related characteristic labels of each model are obtained from the target video generation model set; according to the user's cost budget and target, the corresponding scoring standard is set to guide model selection; based on the set scoring standard, each model in the target video generation model set is scored for cost; according to the cost score, the model with the lowest cost is selected from the target video generation model set as the recommended video generation model.

[0029] Specifically, after the system identifies the current matching mode as cost-first, it extracts cost-related feature annotations for each model from the pre-built target video generation model set. These cost feature annotations include, but are not limited to, the following dimensions: single inference computation cost, hardware resource requirements, service-oriented deployment cost, and license fee structure. The system obtains the above structured annotation data by accessing the model metadata database, forming a cost feature vector C={c1,c2,...,c...} n}, where n is the total number of models in the model set. Based on the user-submitted cost budget constraint value B (e.g., the highest acceptable cost per generation) and cost optimization objective O (e.g., minimizing the cost of generating video per unit duration), the system automatically constructs a scoring function: When O is a hard budget constraint, define a Boolean scoring criterion:

[0030] Where w k For each cost dimension, the normalized weighting coefficients are used. When O is the elastic optimization objective, a linear scoring function is constructed: α is a cost sensitivity adjustment factor, and its value is negatively correlated with B.

[0031] Multi-dimensional cost scoring. The extracted cost feature vectors are input into the scoring function, which performs the following: for each model M in the model set... i Conduct a quantitative cost assessment and generate a standardized cost score S. i For heterogeneous cost parameters (such as currency units and computing resource units), a pre-set unified unit conversion module is called (for example, converting GPU time consumption to equivalent currency based on the real-time quote of the cloud computing platform). When there are time-sensitive cost parameters (such as spot instance discount rate), the external cloud service API is dynamically accessed to obtain real-time data.

[0032] The system executes a filtering strategy based on the scoring results: when using Boolean scoring criteria, it selects candidates from those meeting the Score... cost From the models with a score of 1, any model is randomly selected as the recommendation output. When using a continuous scoring function, the model with the maximum score is selected. cost (M i Model M of the value opt .

[0033] S500: Based on the recommended video generation model, process the content input by the user to obtain the target video.

[0034] In summary, the adaptive model video generation method of this invention can bring the following technical effects.

[0035] 1. Diverse selection: Users can access multiple different video generation models without being limited to a single tool or platform. This allows for selection of the most suitable model based on specific needs such as video style, length, effects, etc.

[0036] 2. Automated recommendations: The platform can intelligently recommend the most suitable video generation model based on user input content and needs, reducing selection and trial-and-error costs for users. This improves work efficiency, especially for users who are not familiar with technology.

[0037] 3. Time and effort saving: The aggregation platform integrates multiple video generation models, allowing users to operate on a single platform and avoiding the need to switch between multiple different platforms, improving the efficiency and convenience of creation.

[0038] 4. Optimized results: Automatic selection of the most suitable model ensures that the generated video achieves the best results. The platform can learn user preferences and creation habits through algorithms, improving the accuracy and effectiveness of recommended models.

[0039] 5. Cost-effectiveness: Compared to purchasing multiple model licenses or tools individually, the platform usually offers subscription or one-time purchase options, saving users a significant amount of cost, especially for small and medium-sized enterprises or individual creators.

[0040] Reference Figure 2 , the second aspect of the present application provides a self-adaptive model video generation system, the self-adaptive model video generation system comprises: A platform construction module 10 is used to construct a multi-model aggregation video generation platform; A characteristic labeling module 20 is used to label the model characteristics of each model in the multi-model aggregation video generation platform through a preset table; An input receiving module 30 is used to receive user input content and requirements; A model recommendation module 40 is used to match models in an adaptive mode based on user input content and requirements and combined with model characteristic labeling, obtain recommended video generation models, and the adaptive mode includes user preference priority mode, resource priority mode, quality priority mode, cost priority mode, and fault tolerance priority mode; A video generation module 50 is used to process user input content based on the recommended video generation model to obtain a target video.

[0041] In an optional embodiment of the second aspect of the present application, the characteristic labeling module comprises: A performance parameter labeling unit is used to label model performance parameters, including computing resource demand, processing speed, concurrency capability, maximum load capacity, and response time. a function characteristic labeling unit configured to label model function characteristics, the model function characteristics including supported video formats, resolution ranges, frame rate ranges, special effect capabilities, and audio / video processing capabilities; a quality index labeling unit configured to label model quality indexes, the model quality indexes including video clarity, color restoration, compression ratios, encoding efficiency, and decoding performance; a compatibility information labeling unit configured to label model compatibility information, the model compatibility information including interoperability with other models, data format compatibility, and interface protocol support; a maintenance information labeling unit configured to label model maintenance information, the model maintenance information including update frequencies, version control, error repair records, and performance optimization histories.

[0042] In an optional implementation of the second aspect of the present application, the model recommendation module comprises: a model type acquisition unit configured to obtain a model type of video generation based on the content input by the user; a model type screening unit configured to obtain a target video generation model set based on the model type; a matching mode acquisition unit configured to obtain a target model matching mode based on the requirement input by the user, the target model matching mode being one or more of the adaptive modes; a model recommendation unit configured to obtain a recommended video generation model from the target video generation model set based on the target model matching mode and in combination with the model characteristic labeling.

[0043] In an optional implementation of the second aspect of the present application, the model recommendation unit comprises: a user preference recommendation subunit configured to, if the target model matching mode is a user preference priority mode, obtain historical video generation cases of each model in the target video generation model set; perform similarity comparison between the content input by the user and historical input content of the historical video generation cases of each model; perform half-screen filtering on the target video generation model set based on a similarity comparison result to obtain a candidate video generation model set; and perform scoring and weighted operation on generation capability, technical capability, and consistency capability of each model in the candidate video generation model set based on the model characteristic labeling to obtain, from the candidate video generation model set, a model with the highest score as the recommended video generation model.

[0044] In an optional implementation of the second aspect of the present application, the model recommendation unit further comprises: The resource priority recommendation subunit is configured to: if the target model matching mode is a resource priority mode, acquire a current load state of the system; analyze available resources at present based on the current load state of the system; compare the load state and the available resources with characteristic annotations related to resource occupation of each model in the target video generation model set, wherein the characteristic annotations related to resource occupation include computational resource demand, memory demand and storage demand; filter the target video generation model by using an optimization algorithm under a constraint condition and score the target video generation model adaptively; and select a model with the best adaptive score from the target video generation model set as a recommended video generation model.

[0045] In an optional implementation of the second aspect of the present application, the model recommendation unit further comprises: The quality priority recommendation subunit is configured to: if the target model matching mode is a quality priority mode, collect characteristic annotations related to generation quality of each model from the target video generation model set, wherein the characteristic annotations related to generation quality include definition, color restoration, sound quality and content coherence; give different weights to each of the characteristic annotations related to generation quality according to requirements of a current generation task, and score each model in the target video generation model set by using a preset quality evaluation algorithm; and select a recommended video generation model with the highest score from the target video generation model set according to generation quality scores of each model.

[0046] In an optional implementation of the second aspect of the present application, the model recommendation unit further comprises: The cost priority recommendation subunit is configured to: if the target model matching mode is a cost priority mode, acquire characteristic annotations related to cost of each model from the target video generation model set; set a corresponding scoring standard to guide model selection according to a cost budget and a target of a user; score each model in the target video generation model set based on the set scoring standard; and select a model with the lowest cost from the target video generation model set as a recommended video generation model according to the cost score.

[0047] Figure 3is a structural schematic diagram of a video generation device of an adaptive model provided by an embodiment of the present application. The video generation device of the adaptive model can have a large difference due to different configurations or performances, and can include one or more processors 60 (central processing units, CPU) (for example, one or more processors) and a memory 70, one or more storage media 80 (for example, one or more mass storage devices) for storing application programs or data. The memory and the storage medium can be temporary storage or persistent storage. The programs stored in the storage medium can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the video generation device of the adaptive model. Further, the processor can be configured to communicate with the storage medium and execute the series of instruction operations in the storage medium on the video generation device of the adaptive model.

[0048] The video generation device of the adaptive model can further include one or more power supplies 90, one or more wired or wireless network interfaces 100, one or more input / output interfaces 110, and / or one or more operating systems, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that Figure 3 The structure of the video generation device of the adaptive model shown does not constitute a limitation on the video generation device of the adaptive model, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0049] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium, and the computer readable storage medium stores instructions, which, when executed on a computer, cause the computer to perform the steps of the video generation system of the adaptive model.

[0050] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system or system, unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0051] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a self-adaptive model video generation device, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0052] The above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for video generation of an adaptive model, characterized in that, The method comprises the following steps: constructing a multi-model aggregation video generation platform; annotating the model characteristics of each model in the multi-model aggregation video generation platform with a preset table; receiving user input content and requirements; based on the content and requirements input by the user and combined with the model characteristic annotation, performing model matching in an adaptive mode to obtain a recommended video generation model, the adaptive mode including user preference priority mode, resource priority mode, quality priority mode, cost priority mode, and fault tolerance priority mode; processing the content input by the user based on the recommended video generation model to obtain a target video.

2. The method of claim 1, wherein, The model characteristic annotation of each model in the multi-model aggregation video generation platform with a preset table comprises: annotating model performance parameters, including computing resource requirements, processing speed, concurrency capability, maximum load capacity, and response time; annotating model functional characteristics, including supported video formats, resolution ranges, frame rate ranges, special effect capabilities, and audio and video processing capabilities; annotating model quality indicators, including video clarity, color restoration, compression ratio, encoding efficiency, and decoding performance; annotating model compatibility information, including interoperability with other models, data format compatibility, and interface protocol support; annotating model maintenance information, including update frequency, version control, error repair records, and performance optimization history. 3.The method of claim 1, wherein, The model matching in an adaptive mode based on the content and requirements input by the user and combined with the model characteristic annotation to obtain a recommended video generation model comprises: obtaining a model type of video generation based on the content input by the user; obtaining a target video generation model set based on the model type; obtaining a target model matching mode based on the requirements input by the user, the target model matching mode being one or more of the adaptive modes; obtaining a recommended video generation model from the target video generation model set based on the target model matching mode and combined with the model characteristic annotation.

4. The method of claim 3, wherein, The obtaining of a recommended video generation model from the target video generation model set based on the target model matching mode and combined with the model characteristic annotation comprises: if the target model matching mode is the user preference priority mode, obtaining historical video generation cases of each model in the target video generation model set; comparing the content input by the user with historical input content of the historical video generation cases of each model in terms of similarity; based on the similarity comparison result, performing half-screening on the target video generation model set to obtain a candidate video generation model set; based on the model characteristic annotation, scoring and weighting the generation capability, technical capability, and consistency capability of each model in the candidate video generation model set to obtain the model with the highest score from the candidate video generation model set as the recommended video generation model.

5. The method of claim 3, wherein, The obtaining of a recommended video generation model from the target video generation model set based on the target model matching mode and combined with the model characteristic annotation further comprises: if the target model matching mode is a resource priority mode, obtaining a current load state of a system; analyzing current available resources based on the current load state of the system; comparing the load state and the available resources with model resource occupation related characteristic annotations of each model in the target video generation model set, the model resource occupation related characteristic annotations including computational resource demand, memory demand, and storage demand; using an optimization algorithm to perform constraint filtering and adaptive scoring on the target video generation model; selecting a model with the best adaptive score from the target video generation model set as a recommended video generation model.

6. The method of claim 3, wherein, The obtaining of the recommended video generation model from the target video generation model set based on the target model matching mode and in combination with the model characteristic annotations further includes: if the target model matching mode is a quality priority mode, collecting model generation quality related characteristic annotations of each model in the target video generation model set, the model generation quality related characteristic annotations including definition, color restoration, sound quality, and content coherence; assigning different weights to each of the model generation quality related characteristic annotations according to requirements of a current generation task, and applying a preset quality evaluation algorithm to score each model in the target video generation model set; selecting a recommended video generation model with the highest score from the target video generation model set according to the obtained generation quality scores of each model.

7. The method of claim 3, wherein, The obtaining of the recommended video generation model from the target video generation model set based on the target model matching mode and in combination with the model characteristic annotations further includes: if the target model matching mode is a cost priority mode, obtaining cost related characteristic annotations of each model in the target video generation model set; setting a corresponding scoring standard to guide model selection according to a user's cost budget and target; performing cost scoring on each model in the target video generation model set based on the set scoring standard; selecting a recommended video generation model with the lowest cost from the target video generation model set according to the cost scores.

8. A video generation system of an adaptive model, characterized by, The adaptive model video generation system includes: a platform construction module configured to construct a multi-model aggregated video generation platform; a characteristic annotation module configured to perform model characteristic annotation on each model in the multi-model aggregated video generation platform using a preset table; an input receiving module configured to receive content and requirements input by a user; a model recommendation module configured to perform adaptive mode model matching based on the content and the requirements input by the user and in combination with the model characteristic annotations, and obtain a recommended video generation model; a video generation module configured to process the content input by the user based on the recommended video generation model, and obtain a target video.

9. A video generation device of an adaptive model, characterized by, The adaptive model video generation device includes a memory and at least one processor, the memory has instructions stored therein, and the memory and the at least one processor are interconnected by a circuit. The at least one processor invokes the instructions in the memory to cause a video generation device of the adaptive model to perform the adaptive model video generation method of any of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program, which when executed by a processor, implements the adaptive model video generation method of any of claims 1-7.

Images