An artificial intelligence-based short video creation method, device, and medium

Abstract

The application discloses a short video creation method and device based on artificial intelligence and a medium. The method comprises the following steps: acquiring and analyzing multi-dimensional creation intention information of a user to generate a creation parameter list; calling a multi-modal semantic mapping library according to the creation parameter list and a preset multi-modal dynamic cooperation mechanism to generate preliminary creation content; in a multi-person cooperation mode, synchronizing the preliminary creation content to each cooperation terminal, receiving creative modification instructions and fusing conflictive creativity through an intention matching degree algorithm to generate cooperative creation content; according to a selected target creation scene and a target playing terminal, calling a preset scene and terminal adaptation rule library to perform resolution adjustment, format conversion and interactive adaptation on the cooperative creation content to generate adapted video content; and performing whole-process quality evaluation through a preset quality evaluation node to generate a final short video product. The application realizes end-to-end intelligent creation from intention understanding to product output.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a short video creation method, device and medium based on artificial intelligence. Background Technology

[0002] With the deep integration of mobile internet and AIGC technology, short videos have become the core carrier of information dissemination and content consumption. The demand for creation has rapidly spread from professional creators to the general public, and short video creation technology based on artificial intelligence has emerged accordingly.

[0003] However, existing AI-based short video creation technologies, particularly intelligent material and script generation solutions, largely rely on fixed templates or single trending data, lacking in-depth analysis of users' potential creative intentions and personalized preferences. This results in highly homogenized generated content, making it difficult to meet diverse expressive needs. Secondly, multimodal technologies are mostly linear superpositions of text, visual, and audio modalities, without establishing a unified semantic association model, often leading to issues such as a disconnect between visuals and text, and conflicts between special effects and styles. Collaborative solutions only reach the level of material sharing or segment splicing, lacking mechanisms for real-time creative fusion and rapid iteration among multiple users. This results in repeated communication during multi-person collaborations, slow response to temporary needs, and low creative efficiency.

[0004] These shortcomings mean that existing technologies cannot meet the creative demands of ordinary users for low-threshold, highly personalized, and highly interactive content, nor can they adapt to the production needs of professional teams for efficient collaboration, becoming the main bottleneck for the further development of short video intelligent creation technology. Summary of the Invention

[0005] To address the technical problems existing in the background art, embodiments of this application provide a short video creation method, device, and medium based on artificial intelligence. The method includes: acquiring and parsing multi-dimensional creative intent information of users to generate a list of creative parameters; generating preliminary creative content by calling a preset multi-modal semantic mapping library according to the list of creative parameters and a preset multi-modal dynamic collaboration mechanism; the preliminary creative content includes at least text scripts, visual materials, matching audio, and special effects; in a multi-person collaboration mode, synchronizing the preliminary creative content to each collaboration terminal, receiving creative modification instructions from each collaboration terminal, and merging conflicting creative ideas in the creative modification instructions through an intent matching degree algorithm to generate collaborative creative content; adjusting the resolution, converting the format, and adapting the interaction of the collaborative creative content according to the selected target creation scene and target playback terminal by calling a preset scene-based parameter template and terminal adaptation rule library to generate adapted video content; and evaluating the quality of the multi-dimensional creative intent information, the preliminary creative content, and the adapted video content through a preset quality evaluation node to generate the final short video product.

[0006] In one example, based on the creation parameter list and a preset multimodal dynamic collaboration mechanism, a preset multimodal semantic mapping library is invoked to generate preliminary creative content. Specifically, this includes: invoking the multimodal semantic mapping library; generating a structured text script including shot descriptions, narration text, and subtitle positions using a generative large model based on the theme keywords and style tendencies in the creation parameter list; analyzing visual elements in a preset material library using an image recognition model based on the shot descriptions in the structured text script, selecting visual materials matching the shot descriptions, and matching filters and shot transition rhythms based on style parameters in the creation parameter list; identifying the emotional category of the narration text using an emotion analysis model based on the emotional tone of the narration text in the structured text script, matching background music in a preset audio library, and generating narration audio using a speech synthesis model; adding special effects to the shots based on the shot types in the visual materials and the creation parameter list; and performing multimodal fusion of the generated text script, visual materials, background music, narration audio, and special effects to output the preliminary creative content.

[0007] In one example, in a multi-person collaboration mode, the initial creative content is synchronized to each collaborative terminal, and creative modification instructions from each collaborative terminal are received. A matching algorithm is used to merge conflicting creative ideas within these instructions to generate collaborative content. Specifically, this includes: when a multi-person collaboration mode is detected to be activated, the initial creative content is sent to a lightweight collaboration cloud space and synchronized to each collaborative terminal; creative modification instructions submitted by each collaborative terminal are received; these instructions include at least one of material replacement, parameter adjustment, and shot addition / removal; when multiple collaborative terminals submit conflicting creative modification instructions for the same creative element, the matching degree between each instruction and the creative intent is calculated; the matching degree calculation is based on three indicators: thematic fit, style consistency, and element completeness of the initial creative content, obtained through a weighted summation; the instruction with the highest matching degree is used as the basic framework, and elements from other instructions that do not conflict with the basic framework are identified and extracted for fusion to generate an integrated creative solution; based on the integrated creative solution, the creative parameter list is updated, and a multimodal dynamic collaboration mechanism is triggered to regenerate the corresponding creative content, outputting the collaborative content.

[0008] In one example, a quality assessment is performed on the multi-dimensional creative intent information, the preliminary creative content, and the adapted video content through preset quality assessment nodes to generate the final short video product. Specifically, this includes setting at least three quality assessment nodes throughout the short video creation process; these nodes are a material quality pre-assessment node, a multi-modal collaborative assessment node, and a final product comprehensive assessment node. At the material quality pre-assessment node, the multi-dimensional creative intent information is subjected to clarity detection, watermark removal verification, and copyright compliance comparison. If the clarity score of any material is lower than a preset threshold, or if it contains a watermark or poses a copyright risk, an optimization prompt is issued to the user. At the multi-modal collaborative assessment node… The evaluation node assesses the initial creative content for semantic consistency between visuals and text, rhythmic matching between audio and visuals, and suitability of special effects and style. If the score for any evaluation item is lower than the corresponding preset threshold, the corresponding creative elements are adjusted and regeneration is triggered. In the final product comprehensive evaluation node, the adapted video content is comprehensively evaluated for content completeness, visual comfort, and audience suitability. If the comprehensive score is lower than the preset passing threshold, relevant parameters are adjusted according to preset optimization suggestions, and the generation steps are re-executed until the comprehensive score reaches the passing threshold. When the evaluation results of all quality evaluation nodes meet the preset requirements, the current video content is output as the final short video product.

[0009] In one example, the process involves acquiring and parsing multi-dimensional creative intent information from users to generate a list of creative parameters. This includes: receiving text instructions, reference materials, and scene tags input by the user; extracting thematic keywords, style preferences, duration requirements, and core audience elements using a pre-trained natural language understanding model to identify explicit demand elements; accessing a user creative behavior database to analyze the user's historical creative style characteristics, audience feedback data of historical works, and scene-related demands corresponding to the current scene tag; and mining the user's implicit creative intent using a collaborative filtering algorithm. Finally, the explicit demand elements and the implicit creative intent are fused to generate a list of creative parameters.

[0010] In one example, based on the selected target creation scenario and target playback terminal, a preset scenario-based parameter template library and terminal adaptation rule library are invoked to adjust the resolution, convert the format, and adapt the interaction of the collaboratively created content, generating adapted video content. Specifically, this includes: receiving the user-selected target creation scenario and target playback terminal; the target creation scenario includes at least one of commercial promotion, knowledge popularization, and personal recording; the target playback terminal includes at least one of mobile phone, tablet, smart TV, and mini-program; based on the target creation scenario, the preset scenario-based parameter template library is invoked to match the corresponding scenario parameter template; the scenario parameter template includes at least one of... The video content is designed to optimize various aspects, including: product presentation duration, camera rhythm, narration speed, subtitle font size, and filter style. Based on the target playback terminal type, a pre-defined terminal adaptation rule library is invoked to match the corresponding terminal adaptation rule. This rule includes at least resolution standards, encoding format requirements, bitrate range, and interactive operation methods. According to the matched scene parameter template, the corresponding creative elements in the collaborative content are adjusted. Based on the matched terminal adaptation rules, the adjusted collaborative content undergoes resolution scaling, format conversion, and interactive method adaptation. The video content, after parameter adjustment and terminal adaptation, is then output as adapted video content.

[0011] In one example, the method also includes storing the final short video output, the corresponding list of creative parameters, the creative conflict matching record, and the quality assessment result in the user database as historical data to provide a basis for mining the implicit creative intentions of subsequent creations.

[0012] In one example, the method also includes: when in single-player mode, performing subsequent resolution adjustments, format conversions, and interactive adaptations on the initially created content.

[0013] On the other hand, embodiments of this application provide an artificial intelligence-based short video creation device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-mentioned artificial intelligence-based short video creation methods.

[0014] On the other hand, embodiments of this application provide a non-volatile computer storage medium for short video creation based on artificial intelligence, which stores computer-executable instructions that can execute any of the above-mentioned short video creation methods based on artificial intelligence.

[0015] Compared with existing short video intelligent creation technologies, the technical solution of this invention achieves significant breakthroughs in personalized expression, multimodal collaboration, collaboration efficiency, generalization adaptation, and quality assurance. Specific beneficial effects are as follows: By employing a multi-dimensional modeling mechanism combining explicit needs and implicit intentions, this technology overcomes the limitations of existing technologies that rely solely on fixed templates or single data. It achieves precise binding between creative needs and content elements, significantly improving the accuracy of creative intent matching and highlighting personalized expression. The constructed multi-modal semantic association model and dynamic linkage mechanism solve the problems of linear superposition and weak correlation between modalities in existing technologies. Text scripts, visuals, and audio effects can be intelligently linked and adjusted, improving the semantic consistency and rhythmic coordination of short videos and avoiding situations where "visuals and text are disconnected, or effects and style conflict." The real-time collaboration module supports real-time synchronization and intelligent fusion of multi-person creative ideas, eliminating the need for repeated manual communication and greatly optimizing collaboration efficiency and iteration flexibility. Through a scenario-based parameter configuration library and terminal adaptation rule library, it breaks the limitations of existing technologies that are "single-scenario / terminal-specific." The adaptation process requires no manual secondary editing, improving adaptation efficiency and significantly reducing the operational costs of cross-scenario and cross-platform publishing. The fully embedded quality assessment mechanism achieves a real-time closed loop of "creation-assessment-optimization," avoiding the problems of post-event evaluation and difficulty in traceability inherent in existing technologies. Quality control throughout the entire process, from material selection to final product output, improves the overall compliance rate of short videos in terms of content integrity, visual comfort, and audience suitability. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, some embodiments of this application will be described in detail below with reference to the accompanying drawings, in which: Figure 1 A flowchart illustrating an artificial intelligence-based short video creation method provided in this application embodiment; Figure 2 A module architecture and data flow diagram of an artificial intelligence-based short video creation method provided in this application embodiment; Figure 3 A diagram illustrating the execution steps of an artificial intelligence-based short video creation method provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of an artificial intelligence-based short video creation device provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Some embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0019] Figure 1 This is a flowchart illustrating an artificial intelligence-based short video creation method provided in this application embodiment. This method can be applied to different business domains. Certain input parameters or intermediate results in this process allow for manual intervention and adjustment to help improve accuracy.

[0020] The analysis method involved in the embodiments of this application can be implemented by a terminal device or a server, and this application does not impose any special limitations on it. For ease of understanding and description, the following embodiments are all described in detail using a server as an example.

[0021] Based on this Figure 1 The process may include the following steps: S101: Obtain and parse the user's multi-dimensional creative intent information to generate a list of creative parameters.

[0022] In some embodiments of this application, this step achieves accurate capture and structured transformation of users' explicit needs and potential creative intentions through a multi-dimensional intent modeling module.

[0023] First, it's important to note that before acquiring multi-dimensional creative intent information from users, system deployment is required. The server-side utilizes a GPU cluster (including 8 NVIDIA A100 graphics cards) to support parallel computing of large models and real-time collaboration among multiple users. The client supports Windows / macOS operating systems, Android / iOS mobile devices, and smart TV terminals, and is compatible with mainstream browsers such as Chrome and Safari. Furthermore, it comes pre-configured with 12 core scene parameter template libraries, a multimodal semantic association library (containing over 1 million labeled data points), a commercial copyright-free material library (over 5 million images and over 100,000 audio files), and a terminal adaptation rule library (covering over 20 mainstream terminal models).

[0024] Furthermore, the system receives explicit requirements input by users through the interactive interface. These explicit requirements include at least one of text instructions, reference materials, and scene tags. For example, in a commercial promotion scenario, Product Operations A inputs the text instruction "Create a short promotional video for a smart jump rope, 60 seconds long, with a vibrant style, highlighting the core selling points of 'accurate counting' and 'quiet and non-slip'," while uploading three real-life product photos and selecting the "Commercial Promotion - Fitness Equipment" scene tag. The system parses the above input using a pre-trained natural language understanding model. This model is based on an improvement of BERT, and its improvements include: incorporating a short video creation dictionary during pre-training, containing over 500 professional terms such as "camera rhythm," "transition style," and "copywriting tone." Additionally, a "requirement element classification header" is added to the output layer, enabling end-to-end extraction of six categories of elements, including theme keywords, style preferences, duration requirements, and core audience, which are then identified as explicit requirement elements.

[0025] Simultaneously, the system accesses the user's creative behavior database to analyze the user's historical creative style characteristics (such as frequently used filters, camera rhythm, and copywriting tone), audience feedback data of past works (such as keywords for likes and comments on past works), and scene-related needs corresponding to the current scene tag (such as adding product display shots by default in commercial promotion scenarios). Through a collaborative filtering algorithm, the system uncovers the user's implicit creative intentions. The collaborative filtering algorithm's feature dimensions include 12 user creative style features, 8 scene-related features, and 10 audience preference features. Similarity calculation uses weighted cosine similarity, with weights allocated as follows: style features 0.4, scene features 0.3, and audience features 0.3. For example, in the aforementioned commercial promotion scenario, the system analysis reveals that the user frequently uses fast-paced transitions and high-saturation filters in their past creations, and that audience feedback data focuses on product usage effects and data visualization, thus uncovering potential intentions such as "adding before-and-after data comparison shots of user usage, and close-ups of product details."

[0026] Finally, the system integrates explicit requirements with implicit creative intentions, quantifying them into a structured list of creative parameters. This list includes at least content structure parameters (such as the number of shots and the percentage of segment duration), style parameters (such as filter type, transition rhythm, and text tone), and core element parameters (such as product display duration and the percentage of people appearing on screen).

[0027] For example, in a commercial promotion implementation, the generated list of creative parameters specifically includes: in terms of content structure, the number of shots is 8-10, with close-up shots of the product accounting for 30%, shots of usage scenarios accounting for 40%, shots of data display accounting for 25%, and shots of appeal at the end accounting for 5%; in terms of style parameters, the filter uses vibrant orange (saturation +20%), the transition rhythm is 0.5-1 second / shot, the copywriting tone is passionate and infectious, and the font size of the subtitles is 28-point boldface; in terms of core elements, the product display time is no less than 18 seconds, the data visualization effect uses dynamic number jumping, and the background music matches electronic music with a rhythm of 130 BPM.

[0028] The generated parameter list allows users to manually fine-tune it. For example, if Product Operations A adjusts the "data display shot percentage" from the default value to 25%, the system will update the parameter list accordingly. Through the three-level mechanism of "explicit requirement analysis + implicit intent mining + dynamic parameter generation" mentioned above, this step accurately quantifies the user's vague creative intent into executable structured parameters, laying the foundation for subsequent multimodal collaborative generation.

[0029] S102: Based on the list of creation parameters and the preset multimodal dynamic collaboration mechanism, call the preset multimodal semantic mapping library to generate preliminary creation content; the preliminary creation content includes at least text scripts, visual materials, matching audio and special effects.

[0030] In some embodiments of this application, this step achieves intelligent linkage and adaptive generation of creative elements through a multimodal dynamic collaboration module.

[0031] First, the system calls a pre-built multimodal semantic mapping library. This library pre-trains a text-visual-audio association model using more than 1 million labeled data points, establishing semantic mapping relationships such as "energetic style corresponds to high-saturation filters, fast-paced transitions, and exciting background music".

[0032] Based on this mapping library, a structured text script is generated using a generative large-scale model according to the theme keywords and style preferences in the creation parameter list. This generative large-scale model is an improvement on GPT and incorporates short video script creation rules, including limitations on the number of shots, segment length ratios, and narration length specifications. The generated text script includes at least shot descriptions, narration text, and subtitle positions. For example, in a commercial promotion embodiment, the system-generated structured script includes shot descriptions such as: "Wide view: Fitness enthusiast enters holding a smart jump rope," "Close-up: Anti-slip texture on the jump rope handle," and "Dynamic data: Screen displays the number of jumps," with narration text and subtitle positions generated simultaneously.

[0033] Next, based on the shot description in the generated text script, the visual elements in the preset material library are analyzed by an image recognition model, and visual materials that match the shot description are selected. At the same time, the filter and shot switching rhythm are automatically matched according to the style parameters in the creation parameter list.

[0034] It should be noted that this image recognition model is an improvement upon YOLOv9. The improvements include an optimized neck feature extraction layer, the addition of product-specific detection anchor boxes, and optimization of the detection head for the three core elements commonly found in short videos: products, people, and scenes. This results in an accuracy rate exceeding 95%. For example, in a commercial promotion implementation, the system filters out scene materials such as gyms and home workouts, applies a "vibrant orange" filter, and combines the shots at a pace of 0.8 seconds per shot.

[0035] Meanwhile, based on the emotional tone of the narration in the text script, a sentiment analysis model is used to identify the emotional category of the narration. This sentiment analysis model uses a TextCNN network structure to classify the emotional tone of the text into four categories: passionate, warm, professional, and humorous.

[0036] After identifying the emotional tone, the system matches suitable background music from a preset audio library; for example, music with a tempo of at least 120 BPM corresponds to an upbeat tone. Simultaneously, a voice-over audio synchronized with the visual rhythm is generated using a speech synthesis model. This speech synthesis model is an improvement upon VITS2, optimizing the logic for matching speech rate and tone, supporting speech rate adjustment from 120 to 180 words per minute, and achieving precise alignment between lip movements and subtitles. For example, in a commercial promotion implementation, the system matches electronic background music at 130 BPM and generates voice-over audio at a speech rate of 150 words per minute, ensuring audio synchronization with the visual rhythm.

[0037] In addition, the system dynamically adds appropriate special effects to specific shots based on the shot type and creative parameter list in the visual materials, with the effect parameters and visual style linked in real time. For example, a highlight border effect is added to a product close-up shot, and a dynamic number jumping effect is added to a data display shot, with the timing of the effect triggering synchronized with the background music beat.

[0038] It should be noted that the core of this step lies in constructing and applying a multimodal dynamic collaboration mechanism. This mechanism is specifically manifested as a 4×4 modal association weight matrix, used to quantify the collaborative relationships between four modalities: text, visual, audio, and special effects. When any modal element is adjusted, the system automatically calculates the adjustment magnitude of the associated modalities based on the weight matrix, thereby achieving dynamic multimodal linkage.

[0039] For example, in the above process, when the emotional tone of the narration in the text script changes, the system will adjust the style and rhythm of the background music accordingly; when the camera rhythm of the visual material is adjusted, the system will adjust the speech rate of the narration audio and the trigger frequency of special effects accordingly.

[0040] It should also be noted that, by default, the association weights between text scripts and other modalities in this application are 0.8 for visual, 0.75 for audio, and 0.6 for special effects; the association weights between visual materials and other modalities are 0.8 for text, 0.7 for audio, and 0.75 for special effects; the association weights between audio and other modalities are 0.75 for text, 0.7 for visual, and 0.65 for special effects; and the association weights between special effects and other modalities are 0.6 for text, 0.75 for visual, and 0.65 for audio.

[0041] Through the aforementioned technical means, the system deeply integrates the generated text scripts, visual materials, background music, narration audio, and special effects, outputting preliminary creative content with a unified format and coordinated elements, providing high-quality input for subsequent multi-person collaboration and quality optimization.

[0042] S103: In multi-person collaboration mode, the initial creative content is synchronized to each collaborative terminal, creative modification instructions from each collaborative terminal are received, and conflicting creative ideas in the creative modification instructions are merged through intent matching algorithm to generate collaborative creative content.

[0043] In some embodiments of this application, this step achieves real-time synchronization, intelligent fusion, and rapid response of multiple people's ideas through a real-time collaboration and iteration module.

[0044] First, the system monitors the creation mode in real time. When a multi-user collaboration mode is detected, the preliminary creation content generated in step S102 and the corresponding list of creation parameters are pushed to a lightweight collaboration cloud space, which then synchronizes the content to each collaborative terminal in real time. This cloud space is designed to ensure real-time synchronization of operation logs, with a synchronization latency controlled within 100ms. Based on 100 sets of real-world test data for multi-user collaboration, the latency is no more than 80ms in 95% of scenarios, meeting the requirements for real-time interaction. For example, in a commercial promotion embodiment, product operator A, copywriter B, and video editor C access the same creation task through their account permissions, viewing the preliminary script and materials in real time. The measured synchronization latency across all terminals is 80ms.

[0045] Subsequently, the system receives creative modification instructions submitted by each collaborative terminal in real time. These instructions include at least one of the following: material replacement, parameter adjustment, and addition / removal of shots. For example, in a commercial promotion embodiment, B modified the narration, changing "Silent bearing design, no complaints from downstairs neighbors~" to "Silent non-slip handle, no disturbance to neighbors during night runs, safer exercise!"; C proposed "Adding a shot comparing the heart rate of fitness enthusiasts before and after use." When the system detects that multiple collaborative terminals have submitted conflicting creative modification instructions for the same creative element, it triggers the creative conflict resolution mechanism.

[0046] Specifically, the system calculates the matching degree between each modification scheme and the core creative intent. This matching degree calculation is based on three categories of indicators: thematic fit with the initial creative content, stylistic consistency, and element completeness. These are obtained through a weighted summation method, with the weights of the three indicators being 0.4, 0.3, and 0.3, respectively. For example, in a commercial promotion implementation, if A and B propose different modification suggestions for the filter style, the system calculates the matching degree between the two schemes and the core intent of "vibrant and highlighting product selling points." The scheme with the higher matching degree is used as the basic framework.

[0047] Furthermore, reasonable elements from other modified schemes that do not conflict with the basic framework are identified and extracted for fusion, generating an integrated creative scheme, and the source of each creative idea is marked for user confirmation. For example, in the personal record embodiment, couple E and F respectively proposed "increasing the brightness of the sunrise shot of Erhai Lake" and "extending the duration of the stroll in the ancient city". The system calculated that the matching degree of the two creative ideas with the core intention "warm and healing travel record" was 85%, and a weighted fusion strategy was used to retain both creative ideas, achieving conflict-free integration.

[0048] Furthermore, this step also supports a rapid response mechanism for temporary requests: when a user's temporary modification instruction is received, such as "change the style to warm and healing," the system quickly updates the creation parameters and triggers a "parameter linkage update" mechanism. It does not require regenerating all content; only style-related elements (filters, music, transitions) are locally and adaptively adjusted, with a response time controlled within 3 seconds. For example, in a commercial promotion embodiment, A requests "add a technological feel to the style." The system quickly updates the parameters, adding technological particle effects to the product close-up shot, while simultaneously increasing the background music tempo from 130 BPM to 135 BPM, with a measured response time of 2.3 seconds.

[0049] It should also be noted that when single-player mode is detected, the initially created content will undergo subsequent resolution adjustments, format conversions, and interactive adaptations.

[0050] Through the above-mentioned mechanism of "creative synchronization - intelligent integration - rapid iteration", this step effectively solves the problems of creative conflict and response delay in multi-person collaboration. The generated collaborative content not only integrates the creativity of multiple parties, but also maintains a high degree of consistency with the core creative intention.

[0051] S104: Based on the selected target creation scene and target playback terminal, call the preset scene-based parameter template library and terminal adaptation rule library to adjust the resolution, convert the format and adapt the interaction of the collaborative creation content, and generate adapted video content.

[0052] In some embodiments of this application, this step achieves generalized adaptation and seamless output of created content through a cross-scene and cross-terminal adaptation module.

[0053] First, the system receives the user's selected target creation scenario and target playback terminal. The target creation scenario includes at least one of the following: commercial promotion, knowledge dissemination, or personal recording; the target playback terminal includes at least one of the following: mobile phone, tablet, smart TV, or mini-program. For example, in the commercial promotion embodiment, the user selects a mobile phone as the target playback terminal; in the knowledge dissemination embodiment, the user selects a tablet as the target playback terminal; and in the personal recording embodiment, the user selects a smart TV as the target playback terminal.

[0054] Furthermore, based on the selected target creation scenario, the system invokes a pre-defined scenario-based parameter template library to match the corresponding scenario parameter template. This library pre-defines parameter templates for 12 core creation scenarios, including commercial promotion, knowledge popularization, and personal recording, each of which supports personalized adjustments according to user needs. For example, in the knowledge popularization embodiment, the system matches the "Knowledge Popularization - Astronomy and Geography" scenario template, which specifies parameters such as a shot rhythm of 1.5-2 seconds per shot, a narration speed of 130 words per minute, a subtitle font size of 32, and an animation effect ratio of 40%.

[0055] At the same time, based on the selected target playback terminal type, the system calls a preset terminal adaptation rule library to match the corresponding terminal adaptation rules. This rule library stores adaptation rules for mainstream terminals, including resolution standards, encoding format requirements, bitrate ranges, and interactive operation methods.

[0056] After matching the scene parameter template with the terminal adaptation rules, the system first adjusts the parameters of the corresponding creative elements in the collaborative content according to the scene parameter template, such as adjusting the camera rhythm, narration speed, and subtitle font size. Then, according to the terminal adaptation rules, it performs resolution scaling, format conversion, and interaction method adaptation on the content after parameter adjustment. Specifically, when adapting a horizontal video to a vertical video, the system automatically identifies and retains core elements such as people and products in the video frame using an AI cropping algorithm based on an attention mechanism. The algorithm's core element retention accuracy is no less than 92%, ensuring that the adapted image is not distorted or loses key information. For example, when adapting popular science content to mobile devices, the AI ​​cropping algorithm automatically identifies and retains core popular science elements such as planets and orbits. The completed video achieved a 38% completion rate on mobile devices, higher than the industry average of 25%.

[0057] After completing all adaptation processing, the system outputs adapted video content that meets the requirements of the target scene and terminal. This content is perfectly matched with the target playback environment in terms of resolution, format, and interaction method, without requiring secondary editing by the user.

[0058] S105: Through preset quality assessment nodes, the multi-dimensional creative intent information, the preliminary creative content and the adapted video content are assessed for quality so as to generate the final short video product.

[0059] In some embodiments of this application, this step enables dynamic quality monitoring and iterative optimization of the entire creation process.

[0060] Specifically, the system sets up at least three key quality assessment nodes in the entire short video creation process: material quality pre-assessment node, multimodal collaborative assessment node, and finished product comprehensive assessment node, forming a real-time closed loop of "creation-assessment-optimization".

[0061] At the material quality pre-assessment stage, the system performs clarity detection, watermark removal verification, and copyright compliance comparison on the original materials involved in the multi-dimensional creative intent information.

[0062] The clarity detection is based on an improved BRISQUE algorithm, which optimizes noise robustness. The evaluation score ranges from 0 to 100, with a threshold set at no less than 70 points. This threshold is based on 5000 sets of test data; footage with a score of 70 or higher achieves a clarity satisfaction rate of at least 85% on short video platforms. If any footage's clarity score falls below the preset threshold, or if a watermark is detected, or if there is a copyright risk, the system automatically replaces it with alternative footage or issues an optimization prompt to the user. For example, in a commercial promotion example, a product photo with a clarity score of 85 points, no watermark, and copyright compliance successfully passes the pre-evaluation.

[0063] At the multimodal collaborative evaluation node, the system conducts a multi-dimensional collaborative evaluation of the preliminary creative content generated in step S102, including the evaluation of semantic consistency between visuals and text, the evaluation of rhythm matching between audio and visuals, and the evaluation of the suitability of special effects and style.

[0064] Semantic consistency evaluation is based on a fusion algorithm of TF-IDF and edit distance, requiring a consistency score of no less than 80 points. Rhythm matching evaluation analyzes the correlation between audio beats and shot switching frequencies, requiring a matching score of no less than 75 points. Special effects adaptation evaluation is based on a scene-based style scoring model, requiring an adaptation score of no less than 80 points. If the score of any evaluation item is lower than the corresponding preset threshold, the system automatically adjusts the corresponding creative elements and triggers regeneration. For example, in a commercial promotion embodiment, the multimodal collaborative evaluation found that the matching score between data effects and background music beats was 72 points, which is lower than the 75-point threshold. The system automatically adjusted the effect trigger time (e.g., a 0.1-second delay), and after re-evaluation, the matching score improved to 81 points.

[0065] At the final product evaluation stage, the system performs a comprehensive quality score on the adapted video content. The scoring dimensions include content completeness (30 points), visual comfort (30 points), and audience suitability (40 points). A total score of no less than 85 points is considered passing.

[0066] It should be noted that this scoring system has been validated through testing data from 1000 finished products, and its correlation with user satisfaction is no less than 0.88. If the overall score falls below the preset passing threshold, the system automatically adjusts relevant parameters and re-executes the generation process based on preset optimization suggestions until the overall score reaches the passing threshold. For example, in a commercial promotion embodiment, if the overall score of the finished product is 88 points, the system suggests "increasing the font size of the data display subtitles." After the user selects one-click optimization, the system adjusts the subtitle font size from 28 points to 30 points. In another example, in a knowledge popularization embodiment, if the overall score of the finished product is 91 points, the system suggests "adding an animation demonstration of the asteroid belt." After the user accepts the optimization, the system automatically adds the relevant footage, raising the score to 93 points.

[0067] When the evaluation results of all quality assessment nodes meet the preset requirements, the system outputs the current video content as the final short video product. For example, in a commercial promotion embodiment, the system ultimately outputs a 60-second promotional short video for a smart jump rope, which product operator A can directly upload to the Douyin platform for publication.

[0068] Through the aforementioned quality assessment and dynamic optimization mechanism embedded throughout the entire process, this step ensures that every step from material selection to finished product output is in a state of quality control, and the overall compliance rate of finished products is increased to over 90%, effectively solving the problems of post-event evaluation and difficulty in traceability in existing technologies.

[0069] It should also be noted that this application will also store the final output short video, the corresponding list of creative parameters, the creative conflict matching record and the quality assessment result in the user database, so as to provide a basis for the mining of implicit creative intentions in subsequent creations as historical data, which increases the reusability of creative data and continuously optimizes the long-term user experience.

[0070] It should be noted that, although the embodiments in this application are based on... Figure 1 Steps S101 to S105 will be described sequentially, but this does not mean that steps S101 and S105 must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S101 to S105 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S101 to S105 can be appropriately adjusted according to actual needs.

[0071] Figure 1 This application utilizes a multi-dimensional modeling mechanism combining explicit needs and implicit intentions to overcome the limitations of existing technologies that rely solely on fixed templates or single data. This achieves precise binding between creative needs and content elements, significantly improving the accuracy of creative intention matching and highlighting personalized expression. The constructed multi-modal semantic association model and dynamic linkage mechanism solve the problems of linear superposition and weak correlation between modalities in existing technologies. Text scripts, visual images, and audio effects can be intelligently linked and adjusted, improving the semantic consistency and rhythmic coordination of short videos and avoiding situations where "images and text are disconnected, or effects and style conflict." The real-time collaboration module supports real-time synchronization and intelligent fusion of multi-person creative ideas, eliminating the need for repeated manual communication and greatly optimizing collaboration efficiency and iteration flexibility. Through a scenario-based parameter configuration library and terminal adaptation rule library, the limitations of existing technologies that are "single-scenario / terminal-specific" are overcome. The adaptation process requires no manual secondary editing, improving adaptation efficiency and significantly reducing the operational costs of cross-scenario and cross-platform publishing. The fully embedded quality assessment mechanism achieves a real-time closed loop of "creation-assessment-optimization," avoiding the problems of post-event evaluation and difficulty in traceability in existing technologies. Quality control throughout the entire process, from material selection to final product output, improves the overall compliance rate of short videos in terms of content integrity, visual comfort, and audience suitability.

[0072] Figure 2 The present application provides a module architecture and data flow diagram for an artificial intelligence-based short video creation method.

[0073] exist Figure 2 The document clearly illustrates the connection logic and data transmission paths of the five core modules, with annotations for each component as follows: Multi-dimensional Intent Modeling Module: The core input processing unit receives explicit user needs (text instructions / reference materials / scene tags) and outputs structured creation parameters (content structure, style, and core elements). Multi-modal Dynamic Collaboration Module: The core content generation unit receives creation parameters and outputs preliminary created content (script + visual materials + audio + special effects). Real-time Collaboration and Iteration Module: A collaborative interaction unit that connects bidirectionally to the multi-modal dynamic collaboration module, supporting multi-user creative synchronization, fusion, and temporary demand response (synchronization latency ≤100ms). Cross-Scenario and Cross-Terminal Adaptation Module: An adaptation processing unit that receives preliminary created content and outputs content adapted to the target scene and terminal (supporting 12 scene types + 4...). The system comprises the following modules: Terminal-like interface; Real-time quality closed-loop optimization module: Quality control unit, bidirectionally connected to the multimodal dynamic collaboration module and the cross-scenario cross-terminal adaptation module, outputting quality assessment results and optimization instructions (three-node assessment mechanism); User interaction interface: User operation entry point, connecting the multi-dimensional intent modeling module and the real-time collaboration and iteration module, supporting requirement input, parameter adjustment, and creative submission; Creation database: Data storage unit, connecting all core modules, storing user behavior data, scene parameter templates, multimodal association libraries, collaboration logs, etc.; Finished product output interface: Result output unit, connecting to the cross-scenario cross-terminal adaptation module, outputting the final short video product to the target terminal (formats supported: MP4 / AVI / FLV).

[0074] Figure 3 This diagram illustrates the execution steps of an artificial intelligence-based short video creation method provided in this application embodiment.

[0075] exist Figure 3 The document covers the entire process from demand input to finished product output, with annotations for each step as follows: User input and intent modeling: Users input explicit needs, and the system generates a structured list of creation parameters through "explicit parsing + implicit mining," supporting manual fine-tuning; Multimodal element collaborative generation: Based on creation parameters, scripts are generated, materials, audio, and special effects are matched, and preliminary creation content is output; Real-time collaboration and creative fusion (optional): In multi-person collaboration scenarios, preliminary creation content is synchronized, and multiple people's creative ideas are merged to output collaborative content; Single-person creation scenarios are skipped directly; Cross-scene and cross-terminal adaptation: Scene parameter templates and terminal adaptation rules are called to adjust the resolution, convert the format, and adapt the interaction of the creation content; Quality closed-loop optimization: Material pre-evaluation, multimodal collaborative evaluation, and final product comprehensive evaluation are executed in sequence (a comprehensive score ≥85 is considered qualified). If it passes, it proceeds to step 6; if it fails, the relevant elements are adjusted in reverse, and steps 2-5 are executed again; Final product output and storage: The final short video product is output to the target terminal, and the creation process data is stored in the creation database.

[0076] Figure 4A schematic diagram of the structure of an artificial intelligence-based short video creation device provided in this application embodiment includes: At least one processor; and, A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, which, when executed by at least one processor, enable the at least one processor to perform any of the above-mentioned methods of short video creation.

[0077] Some embodiments of this application provide a non-volatile computer storage medium for short video creation based on artificial intelligence, which stores computer-executable instructions that can execute any of the above-mentioned short video creation methods based on artificial intelligence.

[0078] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0079] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0085] Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0086] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0088] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the technical principles of this application should fall within the protection scope of this application.

Claims

1. A short video creation method based on artificial intelligence, characterized in that, The method includes: Acquire and analyze the user's multi-dimensional creative intent information to generate a list of creative parameters; Based on the list of creation parameters and the preset multimodal dynamic collaboration mechanism, a preset multimodal semantic mapping library is invoked to generate preliminary creation content; the preliminary creation content includes at least text scripts, visual materials, matching audio, and special effects; In multi-person collaboration mode, the initial creative content is synchronized to each collaborative terminal, and creative modification instructions from each collaborative terminal are received. Conflicting creative ideas in the creative modification instructions are merged through an intent matching algorithm to generate collaborative creative content. Based on the selected target creation scenario and target playback terminal, the preset scenario-based parameter template library and terminal adaptation rule library are invoked to adjust the resolution, convert the format and adapt the interaction of the collaborative creation content, and generate adapted video content. The multi-dimensional creative intent information, the preliminary creative content, and the adapted video content are evaluated using at least three preset quality assessment nodes to generate the final short video product.

2. The method according to claim 1, characterized in that, The step of generating preliminary creative content by invoking a preset multimodal semantic mapping library based on the list of creative parameters and a preset multimodal dynamic collaboration mechanism specifically includes: The multimodal semantic mapping library is invoked, and a structured text script including shot descriptions, narration text, and subtitle positions is generated through a preset generative large model based on the theme keywords and style tendencies in the creation parameter list. Based on the shot description in the structured text script, visual elements in the preset material library are analyzed by an image recognition model, and visual materials that match the shot description are selected. At the same time, filters and shot switching rhythms are matched according to the style parameters in the creative parameter list. Based on the emotional tone of the narration text in the structured text script, the emotional category of the narration text is identified through an emotional analysis model, background music is matched in a preset audio library, and narration audio is generated through a speech synthesis model. Add special effects to the shots based on the shot types in the visual materials and the list of creative parameters; According to the preset multimodal dynamic collaboration mechanism, the generated text script, visual materials, background music, narration audio and special effects are multimodally fused to output preliminary creative content; the multimodal dynamic collaboration mechanism is a modal association weight matrix, which is used to quantify the collaborative relationship between the modalities of text script, visual materials, background music, narration audio and special effects. When any modal element is adjusted, the adjustment range of the associated modality is calculated based on the weight.

3. The method according to claim 1, characterized in that, In the multi-person collaboration mode, the initial creative content is synchronized to each collaborative terminal, creative modification instructions are received from each collaborative terminal, and conflicting creative ideas in the creative modification instructions are merged through an intent matching algorithm to generate collaborative creative content. Specifically, this includes: When the multi-person collaboration mode is detected to be activated, the initial creation content is sent to the lightweight collaboration cloud space and synchronized to each collaboration terminal. Receive creative modification instructions submitted by each collaborative terminal; the creative modification instructions include at least one of material replacement, parameter adjustment, and shot addition / deletion; When multiple collaborative terminals submit conflicting creative modification instructions for the same creative element, the matching degree between each creative modification instruction and the creative intent is calculated. The matching degree calculation is based on three types of indicators: thematic fit with the initial creative content, style consistency, and element completeness, and is obtained through a weighted summation method. The creative modification instruction with the highest matching degree is used as the basic framework. Elements that do not conflict with the basic framework in other creative modification instructions are identified and extracted and integrated to generate an integrated creative solution. Based on the integrated creative scheme, the list of creative parameters is updated and a multimodal dynamic collaboration mechanism is triggered to regenerate the corresponding creative content and output the collaborative creative content.

4. The method according to claim 1, characterized in that, The process of evaluating the multi-dimensional creative intent information, the preliminary creative content, and the adapted video content through preset quality assessment nodes to generate the final short video product specifically includes: At least three quality assessment nodes are set up in the entire short video creation process; the quality assessment nodes are the material quality pre-assessment node, the multimodal collaborative assessment node, and the finished product comprehensive assessment node. At the material quality pre-assessment node, the original materials involved in the multi-dimensional creative intent information are subjected to clarity detection, watermark-free verification, and copyright compliance comparison. If the clarity score of any material is lower than the preset threshold or has a watermark or copyright risk, an optimization prompt is issued to the user. At the multimodal collaborative evaluation node, the initial creative content is evaluated for semantic consistency between visuals and text, rhythm matching between audio and visuals, and suitability between special effects and style. If the score of any evaluation item is lower than the corresponding preset threshold, the corresponding creative elements are adjusted and regeneration is triggered. At the finished product comprehensive evaluation node, the adapted video content is given a comprehensive quality score in terms of content completeness, visual comfort and audience suitability. If the comprehensive score is lower than the preset qualified threshold, the relevant parameters are adjusted according to the preset optimization suggestions and the generation steps are re-executed until the comprehensive score reaches the qualified threshold. When the evaluation results of all quality assessment nodes meet the preset requirements, the current video content will be output as the final short video product.

5. The method according to claim 1, characterized in that, The process of acquiring and parsing the user's multi-dimensional creative intent information to generate a list of creative parameters specifically includes: It receives text instructions, reference materials, and scene tags from users, and extracts thematic keywords, style preferences, duration requirements, and core audience elements through a pre-trained natural language understanding model to identify explicit demand elements. By accessing the user's creative behavior database, analyzing the user's historical creative style characteristics, audience feedback data of historical works, and the scene-related needs corresponding to the current scene tags, the implicit creative intentions of users are mined through collaborative filtering algorithms. The explicit requirements are integrated with the implicit creative intent to generate a list of creative parameters.

6. The method according to claim 1, characterized in that, The step involves, based on the selected target creation scenario and target playback terminal, calling a preset scenario-based parameter template library and terminal adaptation rule library to adjust the resolution, convert the format, and adapt the interaction of the collaboratively created content, generating adapted video content. Specifically, this includes: The system receives the user's selected target creation scenario and target playback terminal; the target creation scenario includes at least one of commercial promotion, knowledge popularization, and personal recording, and the target playback terminal includes at least one of mobile phone, tablet, smart TV, and mini-program. Based on the target creation scenario, a preset scenario-based parameter template library is invoked to match the corresponding scenario parameter template; the scenario parameter template includes at least the product display duration percentage, camera rhythm range, narration speed range, subtitle font size range, and filter style preference. Based on the target playback terminal type, a preset terminal adaptation rule library is invoked to match the corresponding terminal adaptation rule; the terminal adaptation rule includes at least resolution standard, encoding format requirements, bit rate range and interactive operation method; Based on the matched scene parameter template, the parameters of the corresponding creative elements in the collaborative creation content are adjusted; Based on the matched terminal adaptation rules, the collaborative creation content with adjusted parameters is subjected to resolution scaling, format conversion, and interaction method adaptation. When adapting a horizontal video to a vertical video, the preset attention-based AI cropping algorithm identifies and preserves elements in the video frame. The video content, after parameter adjustments and terminal adaptation, will be output as adapted video content.

7. The method according to claim 1, characterized in that, The method further includes: In single-player mode, the initial content creation will undergo subsequent resolution adjustments, format conversions, and interactive adaptations.

8. The method according to claim 1, characterized in that, The method further includes: The final short video output, the corresponding list of creative parameters, the creative conflict matching record, and the quality assessment results are stored in the user database as historical data to provide a basis for mining the implicit creative intentions of subsequent creations.

9. A short video creation device based on artificial intelligence, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform an artificial intelligence-based short video creation method as described in any one of claims 1-8.

10. A short video creation storage medium based on artificial intelligence, storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of executing the artificial intelligence-based short video creation method described in any one of claims 1-8.

Images