An AI-based method, device, medium, and product for generating short videos of marketing products

Abstract

The present invention discloses an AI-based method, device, medium, and product for generating short videos of marketing products, relating to the field of short video generation. The method includes: constructing a short video industry knowledge base; obtaining question information input by a user; searching for answers from the short video industry knowledge base based on the question information; obtaining a video selected by the user from the answers as a reference short video, and obtaining product information uploaded by the user; the product information includes product selling points and product materials; retrieving reference text from the short video industry knowledge base based on the product selling points, and using the reference text as supplementary knowledge information; generating a script text based on the reference short video and the supplementary knowledge information, and generating screen information based on the script text and the product materials; and editing the screen information to obtain a short video of the product information. The present invention can improve the output efficiency of running volume materials.

Description

Technical Field

[0001] The present invention relates to the field of short video generation, and in particular to an AI-based method, device, medium and product for generating short videos of marketing products. Background Art

[0002] With the rise of digital media, short videos have become a crucial channel for brand marketing and personal promotion. Driven especially by social media platforms like Douyin, Xiaohongshu, and Tiktok, the potential and influence of short video marketing are becoming increasingly significant.

[0003] Short video marketing generally consists of two major stages: 1) creative development; 2) creative delivery strategy. Current media advertising platforms, based on the iterations of OCPX (Optimized Cost Per Action) bidding technology, have enabled delivery strategies to become automated and managed. Therefore, the development of creatives plays a crucial role in determining the ROI (Return on Investment) of marketing campaigns.

[0004] However, the production of marketing creative materials faces the problem of difficulty in producing large quantities of materials, which costs brand clients a lot of money. The main problems are: (1) Complex process and low production efficiency: First, the content strategy direction needs to be determined, and then the director produces a specific execution brief and script. The brief is a creative brief used to guide the shooting and editing staff to shoot video materials and edit content, and then match the screen according to the script and add special effects. (2) High demand and lack of inspiration: The life cycle of short videos on the platform is about one week. In order to meet the needs of the release, a large amount of creative materials are needed. When producing hundreds or thousands of creative ideas, the director will run out of ideas and words. (3) The quality of the produced materials depends on personal experience and there is no systematic data to support and assist. Summary of the Invention

[0005] The purpose of the present invention is to provide an AI-based method, device, medium and product for generating short videos of marketing products, which can improve the output efficiency of running volume materials.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] An AI-based method for generating short videos of marketing products, the method comprising:

[0008] Constructing a short video industry knowledge base; wherein the short videos in the short video industry knowledge base are structured data with labels;

[0009] Get the question information entered by the user;

[0010] Searching for answers from the short video industry knowledge base based on the question information; the answers include multiple corresponding short videos and content information of each short video;

[0011] Obtaining a video selected by the user from the answers as a reference short video, and obtaining product information uploaded by the user; the product information includes product selling points and product materials;

[0012] Retrieving reference text from the short video industry knowledge base based on the product selling point, using the reference text as supplementary knowledge information, generating a script text based on the reference short video and the supplementary knowledge information, and generating screen information based on the script text and the product material;

[0013] The screen information is edited to obtain a short video of the product information.

[0014] Optionally, the specific process of building a short video industry knowledge base includes:

[0015] Collect a preset number of short video samples;

[0016] Preprocessing each of the short video samples to obtain a plurality of preprocessed short video samples; the preprocessing operations include storyboarding and frame extraction, text error correction, and background music separation;

[0017] Performing video analysis on each of the pre-processed short video samples to obtain structured data with labels; wherein the video analysis includes overall analysis, storyline analysis, and text and picture label analysis.

[0018] Optionally, preprocessing each of the short video samples to obtain a plurality of preprocessed short video samples specifically includes:

[0019] Applying the PySceneDetect algorithm to each of the short video samples to divide the video into independent shot segments according to shot transitions, and extracting representative frames of the shot segments to obtain representative frame shots;

[0020] Using ffmpeg to separate the speech from each short video sample, and using the ASR algorithm to convert each speech into initial text;

[0021] Performing OCR recognition on each of the short video samples, and correcting the initial text according to the recognition results to obtain a corrected text;

[0022] The speech sound and BGM of each short video sample are separated to obtain the separated BGM background sound, and the representative frame shot, the error-corrected text and the separated BGM background sound are used as preprocessed short video samples.

[0023] Optionally, video labeling is performed on each of the pre-processed short video samples to obtain structured data with labels, specifically including:

[0024] The trained Saga multimodal model is applied to perform video analysis on each of the preprocessed short video samples to obtain structured data with labels.

[0025] Optionally, based on the question information, RAG technology is applied to search for multiple corresponding short videos from the short video industry knowledge base and determine the answer to the question information.

[0026] Optionally, based on the question information, RAG technology is applied to search for answers from the short video industry knowledge base, specifically including:

[0027] Each corresponding short video is divided into small blocks of preset length by chunking;

[0028] Apply the BGE model to convert the small blocks into embedding vector form and store them in the knowledge vector database of the ES vector engine;

[0029] Convert the question information into an embedding vector form, and identify the corresponding meta information and index type from the knowledge vector database to obtain the TopN search results;

[0030] Applying the reranker model to the TopN search results to obtain TopK search results;

[0031] Apply LLM to the TopK search results to obtain the answer.

[0032] Optionally, based on the product selling point, a reference copy is retrieved from the short video industry knowledge base, and the reference copy is used as knowledge supplementary information; a script copy is generated based on the reference short video and the knowledge supplementary information; and screen information is generated based on the script copy and the product material, specifically including:

[0033] Retrieving reference copy from the short video industry knowledge base based on the product selling points;

[0034] Performing style analysis on the reference short video to obtain a style analysis result;

[0035] Generate a script copy based on the reference copy and the style analysis result;

[0036] Preprocessing the commodity material to obtain preprocessed material;

[0037] Retrieving and sorting corresponding materials from the pre-processed materials according to the frames of the reference short video to obtain sorted materials;

[0038] Screen information is generated according to the script text and the sorted materials.

[0039] A computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned AI-based methods for generating short videos of marketing products.

[0040] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned AI-based methods for generating short videos of marketing products.

[0041] A computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned AI-based methods for generating short videos of marketing products.

[0042] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0043] This invention discloses an AI-based method, device, medium, and product for generating short videos for marketing products. By utilizing automated data collection tools, the efficiency and accuracy of short video data collection are significantly improved, enabling video creators to quickly identify and acquire viral videos within their industry. This reduces the time spent searching for and analyzing viral videos, allowing them to focus more energy and resources on creativity and content production, and thus increasing the viral density of short marketing videos. The data analysis and industry trend insight capabilities of the invention provide marketers with in-depth and comprehensive industry analysis reports, helping them understand market dynamics and consumer behavior. Customized conversational bots collect user needs and generate tailored industry trend analysis reports, providing robust data support for marketing decisions. These insights not only help marketers develop more precise marketing strategies but also improve the ROI (return on investment) of marketing campaigns, optimize the allocation of marketing resources, and assist marketers in gaining insight into industry trends and supporting decision-making. The use of AI technology for in-depth analysis and structured decomposition of video content ensures high accuracy and comprehensiveness, providing a solid foundation for video reproduction. AI-assisted video reproduction technology not only improves the quality and fidelity of reproduced videos but also accelerates the reproduction process, enabling video creators to quickly respond to market trends and promptly launch marketing content that meets user preferences. Therefore, this invention utilizes AI technology to enhance the accuracy and quality of video disassembly and reproduction. In summary, this invention will significantly promote the innovation and development of short video marketing tools, providing video creators and marketers with a more efficient and intelligent work platform, thereby enhancing the competitiveness and market responsiveness of the entire industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram of the flow of the AI-based method for generating short videos of marketing products provided in Example 1 of the present invention;

[0046] Figure 2 A schematic diagram of the structure of a short video generation system that applies the AI-based marketing product short video generation method provided in Example 1 of the present invention;

[0047] Figure 3 A schematic diagram of the structure of a video understanding module provided in Example 1 of the present invention;

[0048] Figure 4 A schematic diagram of the Saga multimodal model structure provided in Example 1 of the present invention;

[0049] Figure 5 A schematic diagram of the content strategy module structure provided in Example 1 of the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of the hot-selling replica module provided in Example 1 of the present invention;

[0051] Figure 7 This is a diagram of the internal structure of a computer device. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] The purpose of the present invention is to provide an AI-based method, device, medium and product for generating short videos of marketing products, aiming to improve the output efficiency of running volume materials.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1

[0056] The present invention provides an AI-based method for generating short videos of marketing products, such as Figure 2 As shown, the overall process can be roughly divided into the following steps: (1) The collected industry hot-selling videos are converted from unstructured data to structured data with rich tags through the video understanding module and stored in the industry knowledge base. (2) When using it, the user can first ask a series of questions to the content strategy module to understand the hot content direction of the industry; the system will search for relevant videos in the knowledge base, use LLM to summarize them, and return answers and video links. (3) The user selects a video as a reference hot-selling product and uploads product selling points and materials to enter the replication stage. In the replication stage, the system will search for excellent copywriting expressions in the knowledge base based on the product selling points as a knowledge supplement to the LLM, and imitate the copywriting in the style of the reference hot-selling product; then automatically match the material images related to the copywriting. (4) Finally, the user exports the brief / cutting project for post-production fine editing.

[0057] Specifically, if Figures 1 to 6As shown, the AI-based marketing product short video generation method in this embodiment includes:

[0058] Step S1: Construct a short video industry knowledge base; wherein the short videos in the short video industry knowledge base are structured data with labels.

[0059] S1 specifically includes:

[0060] Step S11: Collect a preset number of short video samples.

[0061] Step S12: pre-processing each of the short video samples to obtain a plurality of pre-processed short video samples; the pre-processing operations include storyboarding and frame extraction, text error correction, and separation of BGM background sound.

[0062] S12 specifically includes:

[0063] Step S121: applying the PySceneDetect (automated video scene analysis and cutting tool) algorithm to each of the short video samples to divide the video into independent shot segments according to shot transitions, and extracting representative frames of the shot segments to obtain representative frame shots.

[0064] Step S122: using ffmpeg (Fast forward moving picture expert group) to separate the speech from each short video sample, and using the ASR algorithm to convert each speech into an initial text.

[0065] Step S123: performing OCR (Optical Character Recognition) on each of the short video samples, and correcting the initial text according to the recognition result to obtain a corrected text.

[0066] Step S124: Separate the speech sound and BGM (Background music) of each short video sample to obtain separated BGM background sound, and use the representative frame shot, the error-corrected text and the separated BGM background sound as preprocessed short video samples.

[0067] Step S13: performing video analysis on each of the pre-processed short video samples to obtain structured data with labels; wherein the video analysis includes overall analysis, storyline analysis, and text and picture label analysis.

[0068] The pre-processed short video samples are analyzed using the trained Saga multimodal model to generate labeled structured data. Saga is a series of multimodal large language models trained by Ultron.

[0069] In practical applications, such as Figure 3 and Figure 4 As shown, the present invention collects popular videos of industries and converts them from unstructured data into structured data with rich tags through the video understanding module, which is then stored in the industry knowledge base. The data processing flow of the video understanding module is divided into two stages: the first stage is video preprocessing, and the second stage is video labeling.

[0070] like Figure 3 As shown in Figure 2, the video preprocessing process includes:

[0071] 1) Use the PySceneDetect algorithm to split the video into independent shot segments according to shot transitions, and extract representative frames from the segments to prepare for image labeling.

[0072] 2) Use FFmpeg to separate the speech and utilize an ASR (Automatic Speech Recognition) algorithm to convert the spoken content into text, preparing for text tagging. However, because short video marketing often contains a lot of specialized vocabulary and is often accompanied by background music and other distractions, typical ASR algorithms have a low word count. To improve text recognition accuracy, the system also performs OCR on selected frames and then performs error correction on the ASR output.

[0073] 3) Separate the spoken voice and BGM to prepare for timbre recognition and BGM style labeling.

[0074] The video labeling process includes:

[0075] The system uses the Saga multimodal model, such as Figure 4 As shown, a comprehensive multi-perspective understanding is conducted from the overall -> storyboard segments -> images & text. The overall part is analyzed from the dimensions of video structure, audience, psychology, etc. The storyboards will have storyline semantic labels, content summary of each storyboard segment, creative techniques, etc. The images under the storyboards will describe the content of the images, the characters / scenes / props that appear, the shooting techniques of the images, etc. Each spoken copy will extract knowledge through NER (Named Entity Recognition), such as: the pain points described, product ingredients, usage effects, etc. Figure 4In the context of CLS, the full name is Classification, which means classification algorithm in Chinese. CLS Tagging, as a term, generally refers to using a deep learning classification algorithm to classify and label sequences. Caption Tags is an algorithm that uses deep learning technology to extract tags from image descriptions (captions). VGGish is an audio classification model developed by Google.

[0076] Saga draws on the principles of BLIP2 (Bootstrapping Language-Image Pre-training with Frozen Image Encoders 2.0), an innovative visual-language pre-training model. Through continuous training with marketing corpora, it develops a visual-language multimodal model for marketing. It introduces a lightweight querying transformer, Q-Former (a lightweight Transformer architecture), as a bridge between the frozen image encoder and the large language model (LLM). Q-Former is designed to achieve effective semantic alignment between visual and language models while addressing the high training costs.

[0077] The Q-Former structure consists of two Transformer submodules: the image transformer and the text transformer. These two submodules share the same self-attention (self-attention mechanism in natural language processing) layer, but the image transformer has an additional cross-attention (Cross-Attention Mechanism) layer to interact with the frozen image encoder. The image transformer is responsible for extracting the most relevant visual features of the text, while the text transformer acts as both a text encoder and decoder to process and generate text data.

[0078] The training strategy of Q-Former is divided into two stages:

[0079] 1) Visual-Language Representation Learning: Q-Former learns how to extract the most relevant visual representations from a frozen image encoder. The system transforms industry semantic labels and entities of interest in marketing into image-text contrastive learning (ITC) and image-text matching (ITM) tasks, and transforms image language into image-text generation (ITG) tasks, effectively aligning visual and text features.

[0080] 2) Vision-to-language generative learning: In the second stage, the output of Q-Former is connected to the frozen LLM for generative learning, and pre-trained with language model loss so that the visual representation learned by Q-Former can be interpreted by LLM.

[0081] Step S2: Obtain question information input by the user.

[0082] Step S3: According to the question information, search for answers from the short video industry knowledge base; the answers include multiple corresponding short videos and content information of each short video.

[0083] Specifically, according to the question information, RAG (Retrieval-Augmented Generation) technology is applied to search for multiple corresponding short videos and content information of each short video from the short video industry knowledge base.

[0084] Furthermore, S3 specifically includes:

[0085] Step S31: Each corresponding short video is divided into small blocks of a preset length using the chunk function. The chunk function can evenly divide the input tensor along a specified dimension into a specific number of tensor blocks and return a tuple whose elements are tensor blocks.

[0086] Step S32: Apply the BGE (BAAI General Embedding, an open source general vector model developed by the Beijing Academy of Artificial Intelligence) model to convert the small block into an embedding vector form and store it in the knowledge vector database of the ES (ElasticSearch, a distributed RESTful search and analysis engine) vector engine.

[0087] Step S33: Convert the question information into an embedding vector format, and identify the corresponding meta information and index type from the knowledge vector database to obtain the TopN search results. The meta information contains the original information of various content features of the video (such as video size, source, etc.).

[0088] Step S34: applying the reranker (reranking the candidate document set initially retrieved) model to the TopN retrieval results to obtain TopK (selecting K results with the highest confidence scores for evaluation) retrieval results.

[0089] Step S35: Apply LLM (Large Language Model) to the TopK search results to obtain the answer.

[0090] In actual application, when using it, users can first ask a series of questions to the content strategy module to understand the popular content direction of the industry; the system will retrieve relevant videos in the knowledge base, use LLM to summarize them, and return answers and video links.

[0091] Specifically, if Figure 5 As shown, the present invention generally uses RAG technology to go through three major stages from accepting user questions to returning the generated answer: (1) indexing and retrieval stage; (2) enhancement stage; and (3) generation stage. The answer contains links to related videos and a summary of the content. Based on the content of the answer, a related video can be selected as a reference video for subsequent popular reproduction. The summarized content (e.g., the people, scenes, pain points, etc.) can also be used as input for script imitation, so that the generated copy can better hit the pain points of the people in the scene and improve conversion.

[0092] Among them, (1) indexing & retrieval stage specifically includes:

[0093] During the indexing phase, the structured data obtained during the video understanding phase is first split into 512-bit chunks. The BGE model then converts the chunks into embedding vectors and stores them in the ES vector engine. During the search phase, user queries are also converted into embedding vectors, and the top N results are then returned through the search.

[0094] Chunking is crucial to avoid incomplete speech caused by truncated sentences due to the fixed length of 512. This invention reorganizes structured data into overall, shot-by-shot, and shot-by-text categories, and stores industry, brand, and category as meta fields.

[0095] During the retrieval phase, the user query (the search information entered by the user) is first understood, and metadata and index types are identified. Recall is performed through a hybrid search method (ANN (Artificial Neural Network) vector similarity and BM25 (Best Matching 25, an algorithm used for information retrieval and text mining) keyword matching).

[0096] (2) The enhancement phase specifically includes:

[0097] Although hybrid retrieval methods can improve the coverage of relevant doc (Document) recall, there is a problem of incomparable scores between different retrieval methods, and the results must be integrated and standardized.

[0098] Reranker model: for each<query, doc> Unified scoring is performed to make the scores standardized and comparable. In addition, by concatenating the query and doc together and inputting them into the transformer model, Reranker allows for more complete interaction between the two when calculating attention (a mechanism for information interaction in deep learning models), resulting in more accurate scoring.

[0099] MRR algorithm: A deduplication algorithm used to increase the diversity of results. While maintaining query relevance, it reduces the redundancy of results to optimize document reranking and text summary generation. The core idea is to select the item that is least similar to the current candidate list in each iteration to increase the diversity of the list. The MRR (Mean Reciprocal Rank, an algorithm that evaluates the performance of a search algorithm) algorithm is a submodule of the reranker. The reranker generally recalculates the scores of the docs under the query before sorting them. MRR takes duplication control into consideration when sorting. Suppose there are K docs, and the recalculated ranking is (doc1, doc2, doc3, ..., docK), but the content of doc2 is very relevant to doc1. After adjustment based on MRR, it will be adjusted to the i-th position (no longer ranked second).

[0100] (3) The generation phase specifically includes:

[0101] Different approaches are used to generate answers for different types of user queries. 1) Textual answers: LLM uses the results of the retrieval and enhancement phases, combined with the user's original query, as context input. LLM's summarization capabilities are then leveraged to generate textual output. 2) Chart-based answers: When a user's question involves tag distribution or competitive comparison, LLM's reasoning capabilities are leveraged, invoking appropriate statistical tools and generating parameters. These are then combined with other visualization components (such as echarts) for display. ECharts is an open-source JavaScript-based visualization charting library.

[0102] Step S4: obtaining a video selected by the user from the plurality of corresponding short videos as a reference short video, and obtaining product information uploaded by the user; the product information includes product selling points and product materials.

[0103] Step S5: Retrieve reference text from the short video industry knowledge base based on the product selling point, use the reference text as knowledge supplement information, generate script text based on the reference short video and the knowledge supplement information, and generate picture information based on the script text and the product material.

[0104] S5 specifically includes:

[0105] Step S51: Retrieve reference copy from the short video industry knowledge base based on the product selling point.

[0106] Step S52: performing style analysis on the reference short video to obtain a style analysis result.

[0107] Step S53: Generate a script copy based on the reference copy and the style analysis result.

[0108] Step S54: pre-processing the commodity material to obtain pre-processed material.

[0109] Step S55: Retrieve and sort corresponding materials from the pre-processed materials according to the frames of the reference short video to obtain sorted materials.

[0110] Step S56: generating screen information according to the script text and the sorted materials.

[0111] In practice, users select a video as a reference for a popular product and upload the product's selling points and materials, entering the replication phase. During this phase, the system searches the knowledge base for excellent copywriting based on the product's selling points as a supplement to the LLM knowledge base, and then rewrites the copy in the style of the referenced popular product. It then automatically matches relevant material images to the copy.

[0112] Specifically, if Figure 6 As shown, the hot replica is mainly a tool to improve the efficiency of making hot replica videos, mainly including:

[0113] (1) One-click reproduction of the reference video, one-click generation of imitation script and video material matching.

[0114] 1) Copywriting: First, the style of the reference video is analyzed, such as the overall storyline structure, key sentence patterns, rhetoric, and word choice. Next, for the input product selling points, the knowledge base is searched to obtain richer expressions of the selling points. Then, using LLM, combined with the style analysis of the reference copy and the selling points and expressions of the target product, the copywriting is planned and executed. Finally, through reflection, the consistency between the style and facts is tested, and whether there is any violation of the Advertising Law, etc., and corrections are made where necessary.

[0115] 2) Material matching: After acquiring the video material, the present invention will automatically understand the material content and segment it, and then intelligently match the best material segment based on the imitation oral copy, saving the user the time of watching the material, remembering the material, and finding the material. In the process of material matching, the target product material will be pre-processed. The pre-processing process is as follows: Figure 3As shown, material preprocessing is also video preprocessing.

[0116] According to the popular video, that is, the screen and subtitles of the target video, relevant content is retrieved from the material index of the material preprocessing result, and the retrieved results are sorted to obtain the sorting result.

[0117] This resulted in the imitation copy and corresponding pictures.

[0118] (2) After the replication is completed, you can export the Jianying project file, package the imitation script and the matched video materials into it, and open it directly in Jianying to facilitate subsequent fine editing.

[0119] Step S6: Edit the image information to obtain a short video of the product information.

[0120] In actual applications, users export brief / cutting projects for later fine-tuning.

[0121] In existing technologies, video understanding can be achieved through Google Gemini (Google's artificial intelligence model) and OpenAI GPT-4V (an innovative feature of OpenAI's advanced model GPT-4) to understand images, but there are two problems: 1) data security; 2) not all tags can be implemented by writing prompts (text instructions used in large language models to prompt the output of model results); and strategic insights can be built through the open source RAG (Retrieval-Augmented Generation) project to build services, but because there is no optimization for specialized video structured data and query retrieval intent, the accuracy of the answers may not be enough; in addition, hot-selling replicas can be built through products such as COZE (an intelligent entity building platform launched by ByteDance) to build copywriting tools, but the scale of the selling point knowledge base makes automatic matching of materials impossible.

[0122] The video understanding of this invention trains the Saga multimodal model to understand the video from a coarse to fine granularity, from the video as a whole -> storyboard -> images & text; and analyzes and understands the video at different levels of abstraction, from entities -> semantic tags -> psychological tags -> creative tags. Strategic insight, based on the RAG technical approach, optimizes index construction, query understanding, search enhancement, and LLM Tools invocation for strategic insight, improving the accuracy of answers. The hit replica function uses AI to automatically imitate text and match materials, achieving rough editing of hit replicas and greatly improving efficiency. The RAG+SFT (Supervised Fine-Tun-ing) model improves the consistency of imitation, and optimizes factual consistency, advertising law, and other issues through post-processing. The Saga model is also used to achieve multimodal and cross-modal material matching.

[0123] Therefore, the AI-based marketing product short video generation method provided by the present invention has the following technical effects:

[0124] 1. Improve the automation and efficiency of short video marketing.

[0125] 2. Improve the accuracy and quality of video disassembly and reproduction through AI technology.

[0126] 3. Provide marketers with insights into industry trends and assist in decision-making.

[0127] Example 2

[0128] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the AI-based method for generating short videos of marketing products in Example 1.

[0129] Example 3

[0130] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the AI-based marketing product short video generation method in Example 1.

[0131] Example 4

[0132] A computer program product includes a computer program, which, when executed by a processor, implements the AI-based marketing product short video generation method in Example 1.

[0133] Example 5

[0134] A computer device, which may be a database, may have an internal structure as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the AI-based marketing product short video generation method in Example 1 is implemented.

[0135] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0136] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, and the like.

[0137] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for generating short videos of marketing products based on AI, characterized in that: The method comprises: Constructing a short video industry knowledge base; wherein the short videos in the short video industry knowledge base are structured data with labels; Get the question information entered by the user; According to the question information, RAG technology is applied to search for answers from the short video industry knowledge base; the answers include multiple corresponding short videos and content information of each short video; Obtaining a video selected by the user from the answers as a reference short video, and obtaining product information uploaded by the user; the product information includes product selling points and product materials; Retrieving reference text from the short video industry knowledge base based on the product selling point, using the reference text as supplementary knowledge information, generating a script text based on the reference short video and the supplementary knowledge information, and generating screen information based on the script text and the product material; Editing the image information to obtain a short video of the product information; Based on the question information, RAG technology is used to search for answers from the short video industry knowledge base, specifically including: Each corresponding short video is divided into small blocks of preset length by chunking; Apply the BGE model to convert the small blocks into embedding vector form and store them in the knowledge vector database of the ES vector engine; Convert the question information into an embedding vector form, and identify the corresponding meta information and index type from the knowledge vector database to obtain the TopN search results; Applying a reranker model to the TopN search results to obtain a TopK search result; Apply LLM to the TopK search results to obtain the answer.

2. The AI-based marketing product short video generation method according to claim 1 is characterized in that: The specific process of building a short video industry knowledge base includes: Collect a preset number of short video samples; Preprocessing each of the short video samples to obtain a plurality of preprocessed short video samples; the preprocessing operations include storyboarding and frame extraction, text error correction, and background music separation; Performing video analysis on each of the pre-processed short video samples to obtain structured data with labels; wherein the video analysis includes overall analysis, storyline analysis, and text and picture label analysis.

3. The AI-based marketing product short video generation method according to claim 2, characterized in that: Preprocessing each of the short video samples to obtain a plurality of preprocessed short video samples specifically includes: Applying the PySceneDetect algorithm to each of the short video samples to divide the video into independent shot segments according to shot transitions, and extracting representative frames of the shot segments to obtain representative frame shots; Using ffmpeg to separate the speech from each short video sample, and using the ASR algorithm to convert each speech into initial text; Performing OCR recognition on each of the short video samples, and correcting the initial text according to the recognition results to obtain a corrected text; The speech sound and BGM of each short video sample are separated to obtain the separated BGM background sound, and the representative frame shot, the error-corrected text and the separated BGM background sound are used as preprocessed short video samples.

4. The AI-based marketing product short video generation method according to claim 1, characterized in that: Retrieving reference text from the short video industry knowledge base based on the product selling point, using the reference text as supplementary knowledge information, generating a script text based on the reference short video and the supplementary knowledge information, and generating screen information based on the script text and the product material, specifically including: Retrieving reference copy from the short video industry knowledge base based on the product selling points; Performing style analysis on the reference short video to obtain a style analysis result; Generate a script copy based on the reference copy and the style analysis result; Preprocessing the commodity material to obtain preprocessed material; Retrieving and sorting corresponding materials from the pre-processed materials according to the frames of the reference short video to obtain sorted materials; Screen information is generated according to the script text and the sorted materials.

5. A computer device comprising: A memory and a processor with a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the AI-based method for generating short videos of marketing products as described in any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the AI-based method for generating short videos of marketing products as described in any one of claims 1 to 4.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the AI-based method for generating short videos of marketing products as described in any one of claims 1 to 4.

Images