Short video intelligent creation system and method based on AI image

Abstract

The present invention relates to the technical field of short video intelligent creation, and in particular to a short video intelligent creation system and method based on AI images. Its technical solution includes an image acquisition module, an image preprocessing module, an image feature extraction module, a video basic stream generation module, an AI script generation module, a video rendering module, and a storage output module. The short video intelligent creation system and method based on AI images of the present invention have remarkable results. It opens up the whole process from image acquisition to finished video output. On the one hand, with the help of multi-dimensional acquisition, intelligent preprocessing, automatic stream generation and other links, the creation efficiency is improved several times, allowing creators to quickly output; on the other hand, with precise feature extraction, intelligent script creation, and adaptive rendering, it ensures high video quality, supports personalized customization and multi-language creation, and can also retain metadata to help manage reuse, enabling short video creation in all directions.

Description

Technical Field

[0001] The present invention relates to the technical field of short video intelligent creation, and in particular to a short video intelligent creation system and method based on AI images. Background Art

[0002] Traditional short video creation mainly relies on professional creators, who use professional video editing software such as Adobe Premiere and Final Cut Pro to produce short videos through a series of complex and time-consuming processes such as manually selecting materials, editing and splicing, adding special effects, and writing scripts. This not only requires creators to have superb professional skills, including proficiency in video editing techniques, deep aesthetic literacy, and solid copywriting skills, but also the creation cycle is often long, which makes it difficult to meet the current rapidly changing content needs and the growing creative desire of the general public.

[0003] At the same time, with the in-depth application of artificial intelligence technology in the field of image and video processing, some tools to assist in the creation of short videos have begun to emerge. Early tools mainly focused on simple image filter addition and basic video editing functions. For example, some mobile phone apps can quickly apply preset filters to videos shot by users, or achieve simple video clip cropping and splicing. However, they lack a deep understanding of image content and intelligent creative process guidance, and cannot automatically generate coherent and creative short video content based on image materials.

[0004] In recent years, although some research has been devoted to using AI technology for video creation and has made certain progress in image feature extraction and automatic video editing, there are still many problems that need to be solved. On the one hand, in the image acquisition stage, most existing solutions can only obtain limited types of materials from a single data source, which is difficult to meet the needs of diverse creative themes, and lacks accurate means to efficiently screen massive materials, resulting in creators spending a lot of time on material identification; on the other hand, in the process of generating basic video streams, either they rely too much on fixed templates, making the created videos monotonous and lacking in personality, or they fail to fully consider the complex semantics and visual correlations between images, resulting in poor coherence and logical confusion in the generated video clips, and unable to effectively convey information. In addition, the combination of script creation and video rendering is also unsatisfactory. The script content is often out of touch with the actual image material, and the rendering effect cannot adapt to scripts of different styles, which ultimately affects the overall quality of short videos.

[0005] In summary, this application proposes an AI image-based short video intelligent creation system and method. Summary of the invention

[0006] The purpose of the present invention is to propose an AI image-based short video intelligent creation system and method to address the problems in the background technology that AI video creation lacks accurate means for efficient screening of massive materials and over-reliance on fixed templates in the process of generating basic video streams.

[0007] In the first aspect, the present application provides an AI image-based short video intelligent creation system, comprising: An image acquisition module is used to acquire original image materials, which cover a variety of themes, scenes and shooting styles and are acquired from different image data sources; An image preprocessing module is connected to the image acquisition module and is used to optimize the quality of the collected original image materials, including but not limited to denoising, color correction, and resolution adjustment, so that the image meets the requirements of subsequent processing. Denoising uses an adaptive filtering algorithm based on deep learning to dynamically adjust the filtering parameters according to the noise characteristics of different areas of the image. Color correction is based on a predefined color style template to adjust the image color to adapt to the theme style of the short video creation; An image feature extraction module, connected to the image preprocessing module, uses a pretrained convolutional neural network model to extract visual features of the preprocessed image, wherein the visual features include but are not limited to object shape, texture, color distribution, and scene category. The image feature extraction module can simultaneously extract global features of the image and local features based on the region of interest, and store them separately for subsequent analysis; A video basic stream generation module is connected to the image feature extraction module, and plans a video clip sequence by constructing a directed graph model based on the extracted image features, wherein nodes represent images, and edge weights are determined by feature similarity between images and a preset theme coherence rule. An initial video basic stream is generated based on the directed graph using a shortest path algorithm, and the initial video basic stream includes a sequence of image frames arranged in order and has preliminary visual coherence; The AI ​​script generation module is equipped with a natural language processing model. It automatically generates a script describing the video content based on the image features, theme information and preset creative style requirements corresponding to the basic video stream. The script content covers picture description, camera movement instructions and narration. The generation of narration is based on the results of image semantic understanding and sentiment analysis to adapt to different scene atmospheres. A video rendering module is connected to the video basic stream generation module and the AI ​​script generation module, and renders the video basic stream according to the script generated by the AI ​​script generation module, including adding transition effects, special effect filters, and subtitles. The transition effects are dynamically selected based on the scene differences and emotional atmosphere of adjacent images, and the parameters of the special effect filters are intelligently adjusted according to the overall tone and style of the image to enhance the visual appeal of the short video; The storage output module is used to store the completed short videos and provides multiple format output options to adapt to the playback requirements of different platforms. When storing, the creation metadata of the short video is also recorded, including the original image source, creation time, and the script template information used, to facilitate subsequent tracing and analysis.

[0008] Optionally, the image data source of the image acquisition module includes but is not limited to online photo libraries, user local albums, and real-time camera images. The image acquisition module also has an image screening function. According to preset keywords, image tags, and user-defined screening rules, candidate images related to the short video creation theme are preliminarily screened out from a large amount of original image materials. The screening process combines content-based image retrieval technology with a machine learning classification model to improve screening accuracy and efficiency. The machine learning classification model is continuously trained and optimized based on historical screening data. The machine learning classification model uses a support vector machine (SVM) algorithm. For a given image sample set, , and its corresponding category label is , the classification hyperplane is constructed by solving the following optimization problem: ; Among them, w is the normal vector of the hyperplane, b is the offset, is the slack variable, C is the penalty parameter, To sample The mapping function is mapped to the high-dimensional feature space, and the model parameters w and b obtained through training are used to classify and screen new images.

[0009] Optionally, the image preprocessing module adopts super-resolution reconstruction technology when adjusting the resolution. For low-resolution images, the high-frequency detail information of the image is predicted based on the generative adversarial network (GAN) model to increase the image resolution to a preset high-definition standard. In the processing process, the visual effect and semantic content authenticity of the reconstructed image are ensured through the perceptual loss function to avoid image blur or content distortion. The generator G and the discriminator D in the generative adversarial network are trained adversarially as follows: The goal of the generator G is to minimize the following loss function: ; The goal of the discriminator is to maximize the following loss function: ; Where z is a random noise vector, is the noise distribution, is the true image distribution, is the perceptual loss function used to measure the generated image The difference in perception from the real high-resolution image y, is the weight parameter that balances the perceptual loss and the adversarial loss, and is optimized alternately and , enabling the generator G to generate high-quality super-resolution images.

[0010] Optionally, when constructing a directed graph model, the video basic stream generation module introduces time dimension information, and for image materials with time series correlation, including continuous photos and video screenshots, optimizes the node connection relationship according to the chronological order of shooting and the dynamic change characteristics of the picture, so that the generated video basic stream can show a natural and smooth time evolution effect, which conforms to human visual perception habits.

[0011] Optionally, the AI ​​script generation module supports multilingual script creation, has built-in language models and grammar rule libraries for multiple languages, and automatically generates voice-over text and screen descriptions in the corresponding language based on the target language specified by the user. During the translation process, it combines the semantics of the image content to accurately translate professional vocabulary and cultural metaphors.

[0012] In a second aspect, the present application provides a short video intelligent creation method based on AI image, comprising the following steps: Original image acquisition steps: collect original image materials from multiple data sources, annotate the source information of the collected materials, and store them according to image subject, shooting time, and resolution attributes. The data sources include public network resources, user personal device storage, and real-time shooting acquisition methods; Image preprocessing steps: The collected original images are processed in sequence for denoising, color correction and resolution adaptation. Denoising uses a blind denoising algorithm based on a deep neural network to automatically identify and remove different types of noise. Color correction refers to the short video target style color profile and adjusts the image color channel value through a mapping function. Resolution adaptation is optimized using an instance-based super-resolution algorithm or downsampling technology based on the target platform playback requirements and the importance of the image content. Image feature extraction and analysis steps: Use a multi-layer convolutional neural network to extract multi-scale features from the preprocessed image, quantify the features into feature vectors, calculate the similarity matrix between images based on the feature vectors, and combine the image semantic labels with scene classification information to explore the potential combination correlation of the images, providing a basis for subsequent video stream construction; Video basic stream construction steps: Based on image features and associated information, a weighted directed graph is constructed, where nodes are images and edge weights comprehensively consider image visual similarity, semantic relevance, and preset narrative logic rules. A heuristic search algorithm is used, using the A* algorithm to search for the optimal path on the directed graph, determine the arrangement order of video frames, and generate a video basic stream with preliminary logical coherence; AI script creation steps: Input the image sequence features of the basic video stream, the preset short video theme and style requirements into the natural language generation model. The model combines the pre-trained knowledge graph and sentiment analysis module to generate script content including detailed picture description, lens movement guidance, and emotional adaptation narration. The script creation process is optimized with real-time feedback, and the model parameters are dynamically adjusted according to the matching degree between the generated text and the image content. Video rendering and output steps: Based on the created script, render and synthesize the basic video stream, add a variety of transition effects, including gradient dissolution and rotation switching, intelligently match the filter effects according to the image style and emotional atmosphere, the filter effects include retro filters and dreamy filters, add subtitles synchronized with the picture, and finally output high-quality short video works in the format specified by the user, and store the metadata of the creation process for subsequent review and improvement.

[0013] Optionally, the original image acquisition step also includes real-time image acquisition optimization. When acquiring images from the camera in real time, the camera parameters, including exposure time, sensitivity, and white balance, are dynamically adjusted according to changes in ambient light. The image enhancement algorithm is used to optimize the quality of the acquired picture in real time to ensure that the acquired real-time image material is clear and usable. At the same time, the target detection model is combined to automatically identify key elements and human subjects in the picture to provide targeted materials for subsequent creation.

[0014] Optionally, in the image feature extraction and analysis step, an attention mechanism is introduced to enable the convolutional neural network to focus on the key areas and significant objects of the image when extracting features, thereby improving the effectiveness and pertinence of feature extraction. For images with complex scenes, the feature extraction range is gradually refined through multiple iterations of the attention module, the core visual information of the image is accurately captured, and the visual guidance of subsequent video creation is enhanced. The attention mechanism uses a scaled dot product attention algorithm. For the input query matrix Q, key matrix K and value matrix V, the attention distribution A is calculated as follows: ; in, is the dimension of the key vector, the output attention weight value for: ; In this way, the neural network focuses on key areas to extract features.

[0015] Optionally, after the video basic stream construction step and before the video rendering step, a user interaction adjustment step is further provided to adapt to the personalized narrative needs of different users, specifically including: The system also includes a visual interactive interface, which displays a sequence of thumbnail images of the currently generated video elementary stream and a schematic diagram of the corresponding node connection relationship. The user can rearrange the order of the video clips by directly dragging the image frame thumbnails, or select a specific image node from the local file library or the system recommended material library and insert it into a specified position; When the user performs the above operations, the system obtains the user operation information in real time and updates the directed graph model and edge weights according to the following rules: Assume that the user inserts the image node as, and the set of directed graph nodes in the original video basic stream is , the edge set is ,in Represents a slave node To Node For the edge of the inserted node , calculate its visual similarity with each node in the set N , cosine similarity function based on feature vectors, semantic relevance , using the entity association path length in the knowledge graph to measure relevance, according to the formula Calculate inserted node With the original node The initial weight of the edge ,in , The weight adjustment coefficient is preset, and the directed graph is reconstructed according to the new node and edge information. The same heuristic search algorithm as in the video basic stream construction step is adopted, and the A* algorithm is used to search for the optimal path on the updated directed graph and re-plan the video basic stream to ensure that the overall visual and narrative coherence is maintained while satisfying the user's creativity.

[0016] Optionally, a video quality assessment feedback link is added to the video rendering and output, and a pre-trained image quality evaluation model and a video smoothness detection model are used to perform a quantitative evaluation on the rendered short video. The transition special effect parameters, filter strength, and subtitle display effect are automatically adjusted according to the evaluation results. If video freezes or blurred images are found, the process goes back to the previous steps for targeted optimization until a short video work that meets the preset quality standards is generated.

[0017] Compared with the prior art, the present invention has at least one of the following beneficial technical effects: 1. The image acquisition of the present invention is diverse and convenient, covering a variety of sources such as the Internet, local photo albums, and real-time shooting. It collects materials in one stop, and the creator does not need to search everywhere to start creation quickly. Through deep learning algorithms, noise can be efficiently removed, color can be accurately corrected, and resolution can be flexibly adapted, so that the image can meet the standard instantly, eliminating the tedious manual optimization. By using image features to build directed graph models and intelligent algorithms, the logic of materials can be quickly sorted out, and a coherent video framework can be built in a short time, which greatly compresses the creation time and greatly improves the creation efficiency.

[0018] 2. Through the convolutional neural network, we can mine global and local features, capture all-round visual and semantic information of objects and scenes, and inject rich connotations into subsequent creations. Through AI scripts, we can generate fitting pictures. The natural language processing model combines image semantics and emotions to produce scripts that complement the video. The narration, description, and camera guidance are very detailed to avoid the disconnection between the picture and the text. Through the transition effects, we can switch flexibly according to the atmosphere of adjacent image scenes, intelligently adjust the filter parameters according to the image style, and accurately synchronize the subtitles with the picture, which can comprehensively improve the video's viewing and professionalism, and significantly improve the quality of creation.

[0019] 3. Users can customize screening rules as needed to accurately pick out images that fit the theme, making their creations unique from the source; the visual interactive interface supports free dragging and insertion of image nodes, and real-time updates of video streams, fully demonstrating user creativity and meeting diverse narrative styles; multi-language creation is barrier-free, supports global mainstream languages, accurately translates image semantics, facilitates cross-cultural communication, expands creative boundaries, and deeply meets personalized needs.

[0020] 4. Through the complete record of creative metadata, the source, time, and script information are clear at a glance, which is convenient for creators to reflect and continuously improve their skills. It is convenient for team collaboration and sharing, and members can optimize collaboratively based on metadata; it is conducive to secondary development, based on existing materials and processes, and quickly produce new videos according to different needs, reducing costs, improving efficiency, and effectively assisting creation management and reuse.

[0021] 5. The AI ​​image-based short video intelligent creation system and method of the present invention has achieved remarkable results. It connects the entire process from image acquisition to finished video output. On the one hand, it uses multi-dimensional acquisition, intelligent preprocessing, automatic stream generation and other links to increase the creation efficiency several times, allowing creators to produce quickly; on the other hand, it relies on precise feature extraction, intelligent script creation, and adaptive rendering to ensure high video quality, support personalized customization and multi-language creation, and retain metadata to assist management and reuse, enabling short video creation in all aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a block diagram of the AI ​​image-based short video intelligent creation system. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0024] Example 1 like Figure 1 As shown, the present invention proposes an AI image-based short video intelligent creation system, including an image acquisition module, an image preprocessing module, an image feature extraction module, a video basic stream generation module, an AI script generation module, a video rendering module, and a storage output module. Each module is described in detail below.

[0025] Among them, the image acquisition module is used to collect original image materials, which cover a variety of themes, scenes and shooting styles, and are collected from different image data sources; the image data sources of the image acquisition module include but are not limited to online photo libraries, user local albums, and real-time camera shooting images. The image acquisition module also has an image screening function. According to preset keywords, image tags and user-defined screening rules, candidate images related to the short video creation theme are preliminarily screened out from massive original image materials. The screening process combines content-based image retrieval technology with machine learning classification models to improve screening accuracy and efficiency. The machine learning classification model is continuously trained and optimized based on historical screening data. Among them, the machine learning classification model adopts the support vector machine (SVM) algorithm. For a given image sample set , and its corresponding category label is , the classification hyperplane is constructed by solving the following optimization problem: ; Among them, w is the normal vector of the hyperplane, b is the offset, is the slack variable, C is the penalty parameter, To sample The mapping function mapped to the high-dimensional feature space, the model parameters w and b obtained through training are used to classify and screen new images. Diversified image acquisition: The image acquisition module can collect materials across multiple data sources, whether it is the massive online gallery, the precious memories accumulated in the user's personal album, or the current moment captured by the real-time camera, all can be used as the cornerstone of creation. This allows creators to search for materials without going through multiple platforms, and gather the required images in one stop, saving a lot of preparatory time and providing a guarantee for the rapid start of creation.

[0026] In this embodiment, the image preprocessing module is connected to the image acquisition module, and is used to optimize the quality of the collected original image material, including but not limited to denoising, color correction, and resolution adjustment, so that the image meets the requirements of subsequent processing, wherein denoising adopts an adaptive filtering algorithm based on deep learning, and dynamically adjusts the filtering parameters according to the noise characteristics of different areas of the image, and color correction adjusts the image color to adapt to the theme style of short video creation according to a predefined color style template; when adjusting the resolution, the image preprocessing module adopts super-resolution reconstruction technology, and for low-resolution images, the high-frequency detail information of the image is predicted based on the generative adversarial network (GAN) model, and the image resolution is increased to a preset high-definition standard, and the visual effect and semantic content authenticity of the reconstructed image are ensured through the perceptual loss function during the processing process, to avoid image blur or content distortion, wherein the generator G and the discriminator D in the generative adversarial network perform the following adversarial training: The goal of the generator G is to minimize the following loss function: ; The goal of the discriminator is to maximize the following loss function: ; Where z is a random noise vector, is the noise distribution, is the true image distribution, is the perceptual loss function used to measure the generated image The difference in perception from the real high-resolution image y, is the weight parameter that balances the perceptual loss and the adversarial loss, and is optimized alternately and , so that the generator G can generate high-quality super-resolution images. By using advanced deep learning algorithms, such as adaptive filtering denoising, super-resolution reconstruction and other technologies, the image quality is automatically optimized. The denoising process accurately identifies the noise characteristics of different regions and dynamically adjusts the filtering parameters to avoid the loss of image details caused by one-size-fits-all processing; super-resolution reconstruction is based on the generative adversarial network and cleverly integrates the perceptual loss function to improve image clarity while ensuring semantic authenticity, giving low-quality images a new lease of life and laying a solid foundation for subsequent high-quality creation. Creators do not need to manually adjust image defects one by one, and the creation process is smoother.

[0027] In addition, the image feature extraction module is connected to the image preprocessing module, and uses the pre-trained convolutional neural network model to extract the visual features of the pre-processed image. The visual features include but are not limited to object shape, texture, color distribution, and scene category. The image feature extraction module can simultaneously extract the global features of the image and the local features based on the region of interest, and store them separately for subsequent analysis; the pre-trained convolutional neural network model deeply explores the visual features of the image, taking into account both global and local features. It not only accurately captures intuitive elements such as object shape, texture, and color distribution, but also focuses on the region of interest and extracts deep semantic information, such as the emotional atmosphere and cultural metaphors behind the scene category, providing rich and accurate image interpretation for subsequent creation links, so that the video content is deeply integrated in visual presentation and connotation communication.

[0028] It is worth noting that the video basic stream generation module is connected to the image feature extraction module. Based on the extracted image features, a directed graph model is constructed to plan the video clip sequence, in which the nodes represent images, and the weights of the edges are determined by the feature similarity between images and the preset theme coherence rules. Based on this directed graph, the shortest path algorithm is used to generate the initial video basic stream, which contains a sequence of image frames arranged in order and has preliminary visual coherence. When constructing the directed graph model, the video basic stream generation module introduces time dimension information. For image materials with time series correlation, including continuous photos and video screenshots, the node connection relationship is optimized according to the order of shooting time and the dynamic change characteristics of the picture, so that the generated video basic stream can show a natural and smooth time evolution effect, which is in line with human visual perception habits. A directed graph model is constructed based on image feature extraction, and the shortest path algorithm is used to plan the video clip sequence. By relying on the rules of feature similarity and theme coherence between images, the internal logic of massive image materials can be quickly sorted out, and disordered pictures can be converted into a preliminary coherent video framework. It is like having an intelligent editing assistant, which can quickly arrange image frames according to visual and narrative logic, greatly shortening the construction time from material to prototype video, so creators can devote more energy to creative refinement.

[0029] Furthermore, the AI ​​script generation module is equipped with a natural language processing model. According to the image features, theme information and preset creative style requirements corresponding to the basic stream of the video, the script describing the video content is automatically generated. The script content covers picture description, lens movement instructions, and narration copywriting. The generation of narration copywriting combines the image semantic understanding and sentiment analysis results to adapt to different scene atmospheres; the AI ​​script generation module supports multi-language script creation, with built-in language models and grammar rule libraries in multiple languages. According to the target language specified by the user, the narration copywriting and picture description in the corresponding language are automatically generated. In the translation process, the semantics of the image content are combined to accurately translate professional vocabulary and cultural metaphors. Combining the natural language processing model with image understanding capabilities, the generated script and the video screen complement each other. The narration copywriting is closely linked to the image semantics and emotional tone, the picture description is meticulous, and the lens movement instructions are professional and reasonable. The soul is injected into the video from the text level, avoiding the embarrassment of the "two skins" of the script and the picture in traditional creation, ensuring the overall style of the video is unified, the narrative is smooth and natural, and the artistic appeal of the work is effectively enhanced.

[0030] In this embodiment, the video rendering module is connected to the video basic stream generation module and the AI ​​script generation module, and renders the video basic stream according to the script generated by the AI ​​script generation module, including adding transition effects, special effects filters, and subtitles. The transition effects are dynamically selected based on the scene differences and emotional atmosphere of adjacent images, and the parameters of the special effects filters are intelligently adjusted according to the overall tone and style of the image to enhance the visual appeal of the short video.

[0031] The storage output module is used to store the completed short videos and provides multiple output formats to meet the playback requirements of different platforms. When storing, the creation metadata of the short videos is also recorded, including the original image source, creation time, and the script template information used, to facilitate subsequent tracing and analysis. According to the script and image style, the transition effects, special effect filters, and subtitle addition are all intelligently adapted. The transition is flexibly selected based on the adjacent image scenes and emotional differences to achieve a smooth transition; the special effect filter parameters are intelligently adjusted according to the image tone to enhance the visual impact; the subtitles are synchronized and coordinated with the picture, showing the quality in the details, and the video texture is sculpted in all directions, making the final work comparable to the masterpieces carefully created by a professional team.

[0032] Example 2 This embodiment provides a short video intelligent creation method based on AI image, including the following steps: Original image acquisition steps: collect original image materials from multiple data sources, annotate the source information of the collected materials, and store them according to the image theme, shooting time, and resolution attributes. The data sources include public network resources, user personal device storage, and real-time shooting acquisition methods; it also includes real-time image acquisition optimization. When collecting images from the camera in real time, dynamically adjust the camera parameters according to the changes in ambient light, including exposure time, sensitivity, and white balance. Use image enhancement algorithms to optimize the quality of the collected images in real time to ensure that the collected real-time image materials are clear and usable. At the same time, combine the target detection model to automatically identify the key elements and human subjects in the picture, providing targeted materials for subsequent creation; Image preprocessing steps: The collected original images are processed in sequence for denoising, color correction and resolution adaptation. Denoising uses a blind denoising algorithm based on a deep neural network to automatically identify and remove different types of noise. Color correction refers to the short video target style color profile and adjusts the image color channel value through a mapping function. Resolution adaptation is optimized using an instance-based super-resolution algorithm or downsampling technology based on the target platform playback requirements and the importance of the image content. Image feature extraction and analysis steps: Use a multi-layer convolutional neural network to extract multi-scale features from the preprocessed image, quantify the features into feature vectors, calculate the similarity matrix between images based on the feature vectors, and combine the image semantic labels and scene classification information to explore the potential combination of images. Provide a basis for subsequent video stream construction, introduce the attention mechanism, so that the convolutional neural network focuses on the key areas and significant objects of the image when extracting features, and improve the effectiveness and pertinence of feature extraction. For images with complex scenes, through multiple iterations of the attention module, the feature extraction range is gradually refined, the core visual information of the image is accurately captured, and the visual guidance of subsequent video creation is enhanced. Among them, the attention mechanism uses the Scaled Dot-Product Attention algorithm. For the input query matrix Q, key matrix K and value matrix V, the attention distribution A is calculated as follows: ; in, is the dimension of the key vector, the output attention weight value for: ; In this way, the neural network can focus on key areas to extract features; Video basic stream construction steps: Based on image features and associated information, a weighted directed graph is constructed, where nodes are images and edge weights comprehensively consider image visual similarity, semantic relevance, and preset narrative logic rules. A heuristic search algorithm is used, using the A* algorithm to search for the optimal path on the directed graph, determine the arrangement order of video frames, and generate a video basic stream with preliminary logical coherence; AI script creation steps: Input the image sequence features of the basic video stream, the preset short video theme and style requirements into the natural language generation model. The model combines the pre-trained knowledge graph and sentiment analysis module to generate script content including detailed picture description, lens movement guidance, and emotional adaptation narration. The script creation process is optimized with real-time feedback, and the model parameters are dynamically adjusted according to the matching degree between the generated text and the image content. Video rendering and output steps: Based on the created script, render and synthesize the basic video stream, add a variety of transition effects, including gradient dissolution and rotation switching, intelligently match the filter effects according to the image style and emotional atmosphere, the filter effects include retro filters and dreamy filters, add subtitles synchronized with the picture, and finally output high-quality short video works in the format specified by the user, and store the metadata of the creation process for subsequent review and improvement.

[0033] Example 3 Based on Embodiment 2, this embodiment further includes a user interaction adjustment step after the video basic stream construction step and before the video rendering step, so as to adapt to the personalized narrative needs of different users, specifically including: The system also includes a visual interactive interface, which displays a sequence of thumbnail images of the currently generated video elementary stream and a schematic diagram of the corresponding node connection relationship. The user can rearrange the order of the video clips by directly dragging the image frame thumbnails, or select a specific image node from the local file library or the system recommended material library and insert it into a specified position; When the user performs the above operations, the system obtains the user operation information in real time and updates the directed graph model and edge weights according to the following rules: Assume that the user inserts the image node as, and the set of directed graph nodes in the original video basic stream is , the edge set is ,in Represents a slave node To Node For the edge of the inserted node , calculate its visual similarity with each node in the set N , cosine similarity function based on feature vectors, semantic relevance , using the entity association path length in the knowledge graph to measure relevance, according to the formula Calculate inserted node With the original node The initial weight of the edge ,in , The weight adjustment coefficient is preset, and the directed graph is reconstructed according to the new node and edge information. The same heuristic search algorithm as in the video basic stream construction step is adopted, and the A* algorithm is used to search for the optimal path on the updated directed graph and re-plan the video basic stream to ensure that the overall visual and narrative coherence is maintained while satisfying the user's creativity.

[0034] Example 4 Based on Example 3, this embodiment adds a video quality assessment feedback link in the video rendering and output, uses a pre-trained image quality evaluation model and a video smoothness detection model to perform a quantitative evaluation on the rendered short video, and automatically adjusts the transition special effect parameters, filter strength, and subtitle display effect according to the evaluation results. If video freezes or blurred images are found, the method goes back to the previous steps for targeted optimization until a short video work that meets the preset quality standards is generated.

[0035] The above specific embodiments are only several optional embodiments of the present invention. Based on the technical solutions of the present invention and the relevant inspirations of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A short video intelligent creation system based on AI image, characterized in that: include: An image acquisition module is used to acquire original image materials, which cover a variety of themes, scenes and shooting styles and are acquired from different image data sources; An image preprocessing module, connected to the image acquisition module, for performing quality optimization processing on the acquired original image material; An image feature extraction module, connected to the image preprocessing module, extracts visual features of the preprocessed image using a pretrained convolutional neural network model; A video basic stream generation module is connected to the image feature extraction module and plans a video clip sequence by constructing a directed graph model based on the extracted image features; The AI ​​script generation module is equipped with a natural language processing model. It automatically generates a script describing the video content based on the image features, subject information, and preset creative style requirements corresponding to the basic video stream. The script content includes picture description, camera movement instructions, and narration copy. A video rendering module, connected to the video basic stream generation module and the AI ​​script generation module, and rendering the video basic stream according to the script generated by the AI ​​script generation module; The storage output module is used to store the completed short videos and provides multiple format output options to adapt to the playback requirements of different platforms. When storing, the creation metadata of the short video is also recorded, including the original image source, creation time, and the script template information used.

2. The AI ​​image-based short video intelligent creation system according to claim 1, characterized in that: The image data sources of the image acquisition module include but are not limited to online photo libraries, user local albums, and real-time camera images. The image acquisition module also has an image screening function. According to preset keywords, image tags, and user-defined screening rules, candidate images related to the theme of short video creation are preliminarily screened out from a large amount of original image materials. The screening process combines content-based image retrieval technology with a machine learning classification model. The machine learning classification model is continuously trained and optimized based on historical screening data. Among them, the machine learning classification model adopts a support vector machine (SVM) algorithm. For a given image sample set , and its corresponding category label is , the classification hyperplane is constructed by solving the following optimization problem: ; Among them, w is the normal vector of the hyperplane, b is the offset, is the slack variable, C is the penalty parameter, To sample The mapping function is mapped to the high-dimensional feature space, and the model parameters w and b obtained through training are used to classify and screen new images.

3. The AI ​​image-based short video intelligent creation system according to claim 1, characterized in that: The image preprocessing module adopts super-resolution reconstruction technology when adjusting the resolution. For low-resolution images, the high-frequency detail information of the image is predicted based on the generative adversarial network model, and the image resolution is increased to the preset high-definition standard. In the processing process, the visual effect and semantic content authenticity of the reconstructed image are ensured through the perceptual loss function. The generator G and the discriminator D in the generative adversarial network are trained as follows: the goal of the generator G is to minimize the following loss function: ； The goal of the discriminator is to maximize the following loss function: ； Where z is a random noise vector, is the noise distribution, is the true image distribution, is the perceptual loss function used to measure the generated image The difference in perception from the real high-resolution image y, is the weight parameter that balances the perceptual loss and the adversarial loss, and is optimized alternately and , enabling the generator G to generate high-quality super-resolution images.

4. The AI ​​image-based short video intelligent creation system according to claim 1, characterized in that: The video basic stream generation module introduces time dimension information when constructing a directed graph model. For image materials with time series correlation, including continuous photos and video screenshots, the node connection relationship is optimized according to the chronological order of shooting and the dynamic change characteristics of the picture.

5. The AI ​​image-based short video intelligent creation system according to claim 1, characterized in that: The AI ​​script generation module supports multilingual script creation, and has built-in language models and grammar rule libraries for multiple languages. It automatically generates voice-over text and screen descriptions in the corresponding language based on the target language specified by the user, and translates professional vocabulary and cultural metaphors in combination with the semantics of the image content during the translation process.

6. A short video intelligent creation method based on AI image, characterized in that: The following steps are involved: Original image acquisition steps: collect original image materials from multiple data sources, annotate the source information of the collected materials, and store them according to image subject, shooting time, and resolution attributes. The data sources include public network resources, user personal device storage, and real-time shooting acquisition methods; Image preprocessing steps: The collected original images are processed in sequence for denoising, color correction and resolution adaptation. Denoising uses a blind denoising algorithm based on a deep neural network to automatically identify and remove different types of noise. Color correction refers to the short video target style color profile and adjusts the image color channel value through a mapping function. Resolution adaptation is optimized using an instance-based super-resolution algorithm or downsampling technology based on the target platform playback requirements and the importance of the image content. Image feature extraction and analysis steps: Use a multi-layer convolutional neural network to extract multi-scale features from the preprocessed image, quantify the features into feature vectors, calculate the similarity matrix between images based on the feature vectors, and combine the image semantic labels and scene classification information to construct the image combination correlation; Video basic stream construction steps: Based on image features and associated information, a weighted directed graph is constructed, in which nodes are images, and edge weights comprehensively consider image visual similarity, semantic relevance, and preset narrative logic rules. A heuristic search algorithm is used, and the A* algorithm is used to search for the optimal path, determine the arrangement order of video frames, and generate a video basic stream; AI script creation steps: Input the image sequence features of the basic video stream, the preset short video theme and style requirements into the natural language generation model. The model combines the pre-trained knowledge graph and sentiment analysis module to generate script content including detailed picture description, lens movement guidance, and emotional adaptation narration. The script creation process is optimized with real-time feedback, and the model parameters are dynamically adjusted according to the matching degree between the generated text and the image content. Video rendering and output steps: Based on the created script, render and synthesize the basic video stream, add a variety of transition effects, including gradient dissolution and rotation switching, and intelligently match the filter effects according to the image style and emotional atmosphere.

7. The AI ​​image-based short video intelligent creation method according to claim 6, characterized in that: The raw image acquisition step also includes real-time image acquisition optimization. When acquiring images from the camera in real time, the camera parameters are dynamically adjusted according to changes in ambient light, including exposure time, sensitivity, and white balance, and the image enhancement algorithm is used to optimize the quality of the acquired images in real time. Combined with the target detection model, it can automatically identify the key elements and human subjects in the picture.

8. The AI ​​image-based short video intelligent creation method according to claim 6, characterized in that: In the image feature extraction and analysis step, an attention mechanism is introduced to enable the convolutional neural network to focus on the key areas and salient objects of the image when extracting features. For images with complex scenes, the feature extraction range is gradually refined through multiple iterations of the attention module. The attention mechanism uses a scaled dot product attention algorithm. For the input query matrix Q, key matrix K and value matrix V, the attention distribution A is calculated as follows: ； in, is the dimension of the key vector, the output attention weight value for: ; In this way, the neural network focuses on key areas to extract features.

9. The AI ​​image-based short video intelligent creation method according to claim 6, characterized in that: After the video basic stream construction step and before the video rendering step, a user interaction adjustment step is further provided, which specifically includes: The system also includes a visual interactive interface, which displays a sequence of thumbnail images of the currently generated video elementary stream and a schematic diagram of the corresponding node connection relationship. The order of the video clips can be rearranged by directly dragging the image frame thumbnails, or a specific image node can be selected from the local file library or the system recommended material library and inserted into the specified position; Get user operation information in real time and update the directed graph model and edge weights: Assume that the user inserts an image node and the set of directed graph nodes in the original video basic stream is , the edge set is ,in Represents a slave node To Node For the edge of the inserted node , calculate its visual similarity with each node in the set N , cosine similarity function based on feature vectors, semantic relevance , using the entity association path length in the knowledge graph to measure relevance, according to the formula Calculate inserted node With the original node The initial weight of the edge ,in , For the preset weight adjustment coefficient, the directed graph is reconstructed according to the new node and edge information, and the same heuristic search algorithm as in the video basic stream construction step is adopted. The A* algorithm is used to search for the optimal path on the updated directed graph and re-plan the video basic stream.

10. The AI ​​image-based short video intelligent creation method according to claim 6, characterized in that: A video quality assessment feedback link is added to the video rendering and output, and a pre-trained image quality evaluation model and a video smoothness detection model are used to perform a quantitative evaluation on the rendered short video. The transition special effect parameters, filter strength, and subtitle display effect are automatically adjusted according to the evaluation results. If video freezes or blurred images are found, the process goes back to the previous steps for targeted optimization until a short video work that meets the preset quality standards is generated.

Images