Method and device for automatically generating multi-split mouth broadcast video fusing product selection results

Abstract

The invention provides a method and a device for automatically generating a multi-split mouth broadcast video fusing product selection results. The method comprises the following steps of: selecting products according to sales volume and commission, or drawing lines by a person and historical behaviors of a platform person; commodity information of the selected commodities is extracted; preprocessing the commodity information by using an LLM large model and a video generation large model to obtain a split video; creating and polishing the copywriting of each split video; and matching proper sound in a preset tone library according to the copywriting, performing digital human lip-shaped driving, generating a short video title, and synthesizing a short video with goods. According to the method, the commodity content is fused in the video content instead of purely playing without real objects, so that more diversified video representation forms are provided; and the problem of incomplete scene coverage caused by the fact that video materials are generated in advance is also avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of spoken word video generation, and in particular to a method and device for automatically generating a multi-storyboard spoken word video integrating product selection results. Background Art

[0002] The current solutions for generating task images with large models commonly used in the market are not realistic enough, and their sense of appeal and attraction do not always meet the requirements of use; specifically, the generated character images may not be fine enough in detail processing, and it is difficult to meet the high-value standards generally recognized by the public, and the clothing and makeup effects may also lack diversity and realism. At the same time, the angles of video / photo shooting and character movements are single. Most of the generated videos use fixed shooting angles, lack changes, and it is difficult to achieve real lens movement effects, such as push-pull, shaking, etc.; and the character makeup and clothing are single, and the emotional expression, action, background, and lighting effects are single. The realism and dynamic changes of the generated background are usually fake and the degree of integration is not high due to the background obtained by the method of cutting out the image to change the background. There are also copyright and privacy issues. This is because the frequent use of public videos on the Internet may involve risks of portrait rights and privacy rights; the works generated by changing the expressions and actions of characters in the video may have originality problems due to excessive similarity. Finally, this solution is difficult to implement technically and has high costs when used in practice.

[0003] Specifically, existing speech-by-video generation solutions mostly use manual shooting, which is costly; they use online video materials for editing without obtaining copyright; and most videos are generated using large models, using the portrait + prompt method to generate speech-by-video short video materials, which has unreliable results, false details, high difficulty, high failure rate, and uncontrollable quality. They also use body-driven / expression-driven solutions with fixed movements, fixed makeup, and fixed clothing, and single materials and lack of diversity.

[0004] In view of this, this application is filed. Summary of the invention

[0005] The present invention provides a method and device for automatically generating a multi-storyboard spoken-word video integrating product selection results, which can at least partially improve the above-mentioned problem.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for automatically generating a multi-storyboard spoken video integrating product selection results, comprising:

[0008] Select products based on sales volume and commission, or based on influencer lines and platform influencer historical behavior;

[0009] Extracting product information of the selected product, wherein the product information includes product selling points, product categories, and copywriting;

[0010] The product information is preprocessed using the LLM big model and the video generation big model to obtain a storyboard video, wherein the LLM big model generates a video text for each storyboard, and the video generation big model generates the storyboard video;

[0011] Create and polish the copy for each of the storyboard videos;

[0012] According to the text, the appropriate voice is matched in the preset tone library, the digital human lip shape is driven, the short video title is generated, and the short video with product promotion is synthesized.

[0013] The present invention also provides a device for automatically generating a multi-storyboard oral video integrating product selection results, which comprises:

[0014] Product selection unit, used to select products based on sales volume and commission, or based on influencer lines and historical behaviors of influencers on the platform;

[0015] A product information extraction unit, used to extract product information of the selected product, wherein the product information includes product selling points, product categories, and copywriting;

[0016] A video storyboard generation unit, configured to pre-process the product information using the LLM big model and the video generation big model to obtain a storyboard video, wherein the LLM big model generates a video text for each storyboard, and the video generation big model generates the storyboard video;

[0017] A copywriting and polishing unit, used to create and polish the copy of each of the storyboard videos;

[0018] The product-selling short video synthesis unit is used to match the appropriate voice in the preset tone library according to the text, drive the digital human lip shape, generate the short video title, and synthesize the product-selling short video.

[0019] In summary, the method for automatically generating multi-storyboard spoken-word videos that integrates product selection results provides a solution for generating video materials based on product selection results, avoiding the problem of incomplete scene coverage caused by generating video materials first. In addition, the video content integrates product content, rather than pure spoken-word without physical objects, providing more diverse video presentation forms. At the same time, you can choose to generate multiple storyboards, which improves the diversity of materials and solves the problem of insufficient material length. In short, the method for automatically generating multi-storyboard spoken-word videos that integrates product selection results provides a new solution for generating spoken-word videos, which increases the diversity of video generation results; increases video length; multiple storyboards; and integrates the visual content of products. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of a method for automatically generating a multi-storyboard spoken video integrating product selection results provided by the first embodiment of the present invention;

[0021] Figure 2 It is a module diagram of a device for automatically generating a multi-storyboard spoken-word video that integrates product selection results, provided in a second embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] refer to Figure 1 As shown, the first embodiment of the present invention discloses a method for automatically generating a multi-storyboard oral broadcast video integrating product selection results, which can be executed by a multi-storyboard oral broadcast video automatic generation device integrating product selection results (hereinafter referred to as a generation device), and in particular, executed by one or more processors in the generation device to implement the following method:

[0024] S101, select products based on sales volume and commission, or based on influencer lines and platform influencer historical behavior;

[0025] In this embodiment, two methods of product selection are provided; one is automatic product selection: selecting products with high sales and high commissions, for example, using the product selection platform to sort by commission and select the top k products. The other is product selection recommendation: recommending products based on the profile of the influencer and the historical behavior of the influencer on the platform.

[0026] S102, extracting product information of the selected product, wherein the product information includes product selling points, product categories, and copywriting;

[0027] Specifically, step S102 includes: extracting the product selling point t1 of the selected product and obtaining the product category label;

[0028] Retrieve short video materials of the same product with sales volume higher than the preset value, and extract video text t2.

[0029] In this embodiment, the selling point t1 of the product is extracted to obtain the product category label; short video materials with high sales of the same product are retrieved, and the copy t2 is extracted.

[0030] S103, using the LLM big model and the video generation big model to pre-process the product information to obtain a storyboard video, wherein the LLM big model generates a video text for each storyboard, and the video generation big model generates the storyboard video;

[0031] Specifically, step S103 includes: generating character feature requirements according to the product information;

[0032] Using the LLM big model to generate video storyboard requirements based on the character feature requirements, wherein the video storyboard requirements include the number of storyboards, the character in each storyboard, and the product performance;

[0033] Call the LLM big model to generate the video copy for each storyboard according to the product content and the video storyboard requirements;

[0034] Get the starting picture of the video storyboard;

[0035] A storyboard video is generated according to the copy content of each storyboard, wherein a video generation model is used to generate a storyboard video using a starting image and storyboard copy that have been integrated into the product image.

[0036] Preferably, obtaining the starting picture of the video storyboard is specifically as follows:

[0037] According to the video storyboard requirements, each storyboard generates n starting pictures, where n is an integer;

[0038] Using the multimodal big model to select the starting picture, wherein, for the purpose of authenticity and in combination with the multimodal big model's own aesthetic evaluation ability, a photo that best meets the requirements is selected as the starting picture for each storyboard;

[0039] Incorporate product images into your starter image.

[0040] Preferably, the product image is integrated into the starting image, specifically:

[0041] Use the preset integration model to insert the specified objects into the starting image, specifically:

[0042] Establish a virtual 3D scene including virtual objects and proxy planes;

[0043] Use a physically based renderer to simulate the interaction between ambient lighting and inserted virtual objects, and how this affects the background scene;

[0044] Render virtual objects using a physically based path tracing algorithm to generate a foreground image that is consistent with the scene lighting;

[0045] Optimize rendered images using a custom-tuned diffusion model;

[0046] Introducing the fractional distillation loss LDS based on the diffusion model to guide the optimization in the rendering process by calculating the difference between the personalized model output and the non-personalized model output;

[0047] Use a spherical Gaussian illumination model to represent scene illumination, initialize dual ambient illumination maps, regularize ambient illumination fusion, and differentiate tone mapping curves to ensure that virtual objects appear most realistic in images or videos.

[0048] Handle virtual object insertion in dynamic scenes;

[0049] Insert virtual objects into scenes shot from different perspectives to complete the integration of product images.

[0050] Preferably, the product image is integrated into the starting image, specifically:

[0051] After the character image is generated, the product image is pasted on the desktop of the picture. Specifically:

[0052] Cut out the specified product image and cut out the product;

[0053] Requirements for generating character images, including the position of the characters in the images;

[0054] Reduce the cut-out product to the ratio of the actual size and the real person;

[0055] Use the segmentation model to find the desktop or counter, get the coordinates, draw the smallest rectangle that can completely contain the desktop or counter, and calculate the coordinates of the center point of the rectangle;

[0056] Place the cut-out product in the center of the rectangle to complete the integration of the product image.

[0057] In this embodiment, video storyboards are generated based on product information. Taking the snack and beverage scene as an example: character feature requirements are generated; the LLM large model generates video storyboard requirements, which include the number of storyboards, the character in each storyboard, and the performance of the product: such as placing on the table, holding in the palm of the hand, placing on the ground, etc.

[0058] The LLM big model generates the video copy for each storyboard based on the product content and storyboard requirements; the video storyboard starting picture is obtained. The specific steps are:

[0059] The first step is to generate n starting pictures for each storyboard according to the storyboard requirements, for example, n is 10.

[0060] The second step is to use the multimodal large model to select the starting picture: for the purpose of authenticity, combined with the aesthetic evaluation ability of the large model itself, select the best photo for each storyboard as the starting picture. Among them, the judgment standard of the multimodal large model is the basic ability to use the multimodal model, such as gpt4o, qwen2-vl, etc. The authenticity index refers to: judging whether each generated picture is the same as the original person (prompt), and filtering out inconsistent ones; judging whether the generated product is consistent with the original product (prompt), and filtering out inconsistent ones. Aesthetic evaluation refers to whether the character's limbs are normal and complete; whether the character's face is clear and straight; whether the expression is appropriate (prompt). According to these dimensions, the comprehensive score is taken, and the picture with the highest score is taken as the starting picture.

[0061] The third step is to integrate the product image. This embodiment adopts two solutions for integration.

[0062] Solution 1: Use the model to insert specified items (commodities) into images or videos. The model reference paper is: https: / / arxiv.org / pdf / 2408.09702.

[0063] First, a virtual scene is constructed. To perform 3D scene modeling, DiPIR creates a virtual 3D scene based on the input image, including virtual objects and proxy geometry in the scene (such as ground planes, etc.). These geometries are used to capture lighting effects such as shadows and reflections. Users can manually specify the position of objects, or automatically determine the position of objects by detecting the ground in the scene or using depth data. Starting from an input picture, first build a virtual 3D scene containing virtual objects and proxy planes. This virtual scene is designed to simulate lighting, shadows, and reflections in real scenes.

[0064] Secondly, perform physical rendering. Use a physically based renderer to simulate the interaction between ambient lighting and inserted virtual objects, as well as the effect of this interaction on the background scene (such as shadows). The purpose of this step is to generate a rendering effect with physical reality so that virtual objects can be truly integrated into the image. And perform foreground rendering, use a physically based path tracing algorithm to render virtual objects, and generate a foreground image that is consistent with the scene lighting. This includes processing the interaction between lighting and object materials, such as reflection, refraction, etc. Among them, the shadow ratio is calculated as follows: DiPIR calculates the shadow cast by the virtual object in the scene, and calculates the light intensity ratio of the shadow area by comparing the scene brightness before and after the object is inserted. This ratio is used to adjust the shadow effect of the background image to make it consistent with the insertion effect of the virtual object.

[0065] Again, the diffusion model is guided. Using a personalized diffusion model, the rendered image is passed to a personalized diffusion model. This diffusion model is responsible for further optimizing the image so that the virtual object blends more naturally with the background scene. The pre-trained diffusion model is personalized to make it more suitable for specific input scenes. The powerful image generation capability of the diffusion model is used to guide the optimization of lighting and tone mapping parameters. In this process, the diffusion model uses the gradient of the adapted Score Distillation formula to feedback optimization information to help adjust the ambient lighting map and tone mapping curve. At the same time, the score distillation loss (SDS) is used: DiPIR introduces a score distillation loss based on the diffusion model, called LDS (LoRA Distillation Sampling), which provides feedback signals through a personalized diffusion model related to the scene to optimize the realism of virtual object insertion. This loss function guides the optimization in the rendering process by calculating the difference between the personalized model output and the non-personalized model output.

[0066] From this, lighting and tone mapping optimization are performed. A spherical Gaussian illumination model is used, and the scene illumination is represented by multiple spherical Gaussian (SG) functions, which are optimized to simulate the ambient lighting in the scene. The direction and intensity of the illumination are represented and calculated by this set of optimization parameters to ensure that virtual objects can match the lighting conditions of the scene. Dual ambient illumination map initialization: In the early stages of optimization, DiPIR handles the lighting consistency problem by initializing two independent ambient illumination maps (one for foreground objects and one for casting shadows). During the training process, the two illumination maps are gradually merged into a unified ambient illumination map to achieve higher lighting accuracy. Regularization of ambient illumination fusion is also performed. By using regularization terms, DiPIR ensures the brightness and tone consistency of the illumination, while suppressing unnecessary ambient illumination to generate sharper shadows and more realistic lighting effects. And use differentiable tone mapping curves: In order to match the tone mapping of the input image (usually determined by the camera sensor), DiPIR uses an optimizable tone mapping curve to adjust the inserted virtual objects and the shadows they cast. These curves are optimized to ensure that the color and brightness of the virtual objects are consistent with the background scene. Throughout the iterative optimization process, by adjusting the ambient lighting and tone mapping curves, the lighting and tone parameters that can perfectly blend with the background scene are finally restored. These parameters ensure that virtual objects present the most realistic effects in pictures or videos.

[0067] In addition, dynamic scene processing is also required, and DiPIR can also handle the insertion of virtual objects in dynamic scenes. For example, the background image can be animated, or the position of the virtual object can be moved to create a dynamic scene effect.

[0068] Finally, multi-view expansion is performed. This method also supports inserting virtual objects into scenes shot from different viewpoints, ensuring that the objects maintain consistent lighting and fusion effects under all viewpoints.

[0069] Solution 2: Product cutout: After the character image is generated, paste the product image on the desktop of the picture. The specific steps include:

[0070] First, cut out the specified product image to remove the product;

[0071] Secondly, there are many requirements for the generation of character images, including the position of the character in the picture: left (with a display stand or table on the right), right (with a display stand or table on the left), sitting (with a table in front of the person), the character occupies less than 1 / 3 of the screen, leaving more space for the display stand or table, and the display stand or table is used to place goods.

[0072] Again, the cut-out products are scaled down according to the actual size and the proportion of the real person;

[0073] From this, use the segmentation model to find the desktop or counter, get the coordinates, draw the smallest rectangle that can completely fit the desktop or counter, and calculate the coordinates of the center point of the rectangle;

[0074] Finally, place the removed product in the center of the rectangle (the position can be adjusted manually).

[0075] Then, based on the copy content of each storyboard, a storyboard video is generated. Using the starting image (already integrated into the product image) + storyboard copy, a video is generated using a large model (such as using Kling).

[0076] S104, creating and polishing the text of each storyboard video;

[0077] Specifically, step S104 includes: receiving the copy of each storyboard, calling the LLM big model to rewrite the copy content;

[0078] Adjust the length of each storyboard to ensure that the length of each storyboard is basically consistent with the length of the copy.

[0079] In this embodiment, the copywriting is polished. The copywriting of each storyboard is spliced ​​together, and Llm is asked to rewrite the copywriting content to ensure that the expression style of the entire copywriting is consistent; and the duration of each storyboard is added to ensure that it is basically consistent with the duration of the copywriting (assuming a spoken scene, 4-5 words are read per second).

[0080] S105, according to the text, a suitable voice is matched in the preset tone library, the digital human lip shape is driven, a short video title is generated, and a short video with a product is synthesized.

[0081] Specifically, step S105 includes: matching appropriate voices according to the text style and the semantic tags of the video material to obtain the most appropriate voice, and cloning the most appropriate voice using a preset voice cloning model;

[0082] According to the most suitable voice, the digital human model is used to drive the lip movements of the characters in the video material, generate subtitles, and obtain the oral video corresponding to the video material;

[0083] Based on the video copy, use the LLM model to generate a popular short video title for the short video.

[0084] In this embodiment, the synthesis of a short video with a product is mainly divided into three steps: voice selection, digital human lip shape driving, and using LLM to generate a hot short video title for the short video based on the video copy.

[0085] Specifically, according to the text style and the semantic tags of the video material, the appropriate sound is matched. The specific steps of sound matching are:

[0086] In the first step, we use the high-quality spoken-word material library constructed above to extract semantic labels of video characters [such as age, gender, etc.] and text content as text features, and use the sound corresponding to the video to form a text-sound data pair (select spoken-word videos with no background sound, no soundtrack, and no noise) as positive samples, and the combination of text and random speech as negative samples to complete the data set construction.

[0087] The second step is to use the data constructed above to train a text-sound matching model. Specifically: select a matching model, such as a twin network, or a multimodal model similar to clip; the model contains two modal data inputs, text (character semantic labels + copy content) and voice (a 3s to 10s audio); the model output is 0 (irrelevant) or 1 (relevant).

[0088] The third step is to use the above model to match the most suitable sound (assuming that there is a sound library, or so-called timbre library). Specifically, take the character semantic tags and text content of the current video. The semantic tags can be extracted by the multimodal model according to the specified character semantic tag system; match the character semantic tags and text content with all the sounds in the sound library, take the sounds with a correlation higher than 80%, and randomly select one of the sounds.

[0089] Finally, use the emotional voice cloning model to clone the above-mentioned matching voice and generated text (if you are not satisfied with the voice, you can also record your own voice to clone).

[0090] In this embodiment, the digital human lip shape driving mainly uses a high-quality digital human model to drive the lip shape of the character in the video material, and the input data is the sound data generated above. Subtitles are generated to obtain a video.

[0091] In summary, the method for automatically generating multi-storyboard spoken-word videos that integrates product selection results provides a solution for generating video materials based on product selection results, avoiding the problem of incomplete scene coverage caused by generating video materials first. In addition, the video content integrates product content, rather than pure spoken-word without physical objects, providing more diverse video presentation forms. At the same time, you can choose to generate multiple storyboards, which improves the diversity of materials and solves the problem of insufficient material length. In short, the method for automatically generating multi-storyboard spoken-word videos that integrates product selection results provides a new solution for generating spoken-word videos, which increases the diversity of video generation results; increases video length; multiple storyboards; and integrates the visual content of products.

[0092] See also Figure 2 The second embodiment of the present invention further provides a device for automatically generating a multi-storyboard oral video integrating product selection results, which includes:

[0093] The product selection unit 201 is used to select products based on sales volume and commission, or based on the influencer’s line and the influencer’s historical behavior on the platform;

[0094] The product information extraction unit 202 is used to extract product information of the selected product, wherein the product information includes product selling points, product categories, and copywriting;

[0095] The video storyboard generation unit 203 is used to pre-process the product information using the LLM big model and the video generation big model to obtain a storyboard video, wherein the LLM big model generates a video text for each storyboard, and the video generation big model generates the storyboard video;

[0096] A copywriting and polishing unit 204, used for creating and polishing the copy of each of the storyboard videos;

[0097] The product-carrying short video synthesis unit 205 is used to match the appropriate voice in the preset tone library according to the text, drive the digital human lip shape, generate the short video title, and synthesize the product-carrying short video.

[0098] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for automatically generating a multi-storyboard oral video integrating product selection results, characterized in that: include: Select products based on sales volume and commission, or based on influencer lines and platform influencer historical behavior; Extracting product information of the selected product, wherein the product information includes product selling points, product categories, and copywriting; The product information is preprocessed using the LLM big model and the video generation big model to obtain a storyboard video, wherein the LLM big model generates a video text for each storyboard, and the video generation big model generates the storyboard video; Create and polish the copy for each of the storyboard videos; According to the text, the appropriate voice is matched in the preset tone library, the digital human lip shape is driven, the short video title is generated, and the short video with product promotion is synthesized.

2. The method for automatically generating a multi-storyboard oral video integrating product selection results according to claim 1, characterized in that: Extract the product information of the selected product, specifically: Extract the selling point t1 of the selected product and obtain the product category label; Retrieve short video materials of the same product with sales volume higher than the preset value, and extract video text t2.

3. The method for automatically generating a multi-storyboard oral video integrating product selection results according to claim 1, characterized in that: The product information is preprocessed using the LLM large model to generate a video storyboard, specifically: Generating character feature requirements based on the product information; Using the LLM big model to generate video storyboard requirements based on the character feature requirements, wherein the video storyboard requirements include the number of storyboards, the character in each storyboard, and the product performance; Call the LLM big model to generate the video copy for each storyboard according to the product content and the video storyboard requirements; Get the starting picture of the video storyboard; A storyboard video is generated according to the copy content of each storyboard, wherein a video generation model is used to generate a storyboard video using a starting image and storyboard copy that have been integrated into the product image.

4. The method for automatically generating a multi-storyboard oral video integrating product selection results according to claim 3, characterized in that: Get the starting image of the video storyboard, specifically: According to the video storyboard requirements, each storyboard generates n starting pictures, where n is an integer; Using the multimodal big model to select the starting picture, wherein, for the purpose of authenticity and in combination with the multimodal big model's own aesthetic evaluation ability, a photo that best meets the requirements is selected as the starting picture for each storyboard; Incorporate product images into your starter image.

5. The method for automatically generating a multi-storyboard oral video integrating product selection results according to claim 4, characterized in that: Integrate the product image into the starting image, specifically: Use the preset integration model to insert the specified objects into the starting image, specifically: Establish a virtual 3D scene including virtual objects and proxy planes; Use a physically based renderer to simulate the interaction between ambient lighting and inserted virtual objects, and how this affects the background scene; Render virtual objects using a physically based path tracing algorithm to generate a foreground image that is consistent with the scene lighting; Optimize rendered images using a custom-tuned diffusion model; Introducing the fractional distillation loss LDS based on the diffusion model to guide the optimization in the rendering process by calculating the difference between the personalized model output and the non-personalized model output; Use a spherical Gaussian illumination model to represent scene illumination, initialize dual ambient illumination maps, regularize ambient illumination fusion, and differentiate tone mapping curves to ensure that virtual objects appear most realistic in images or videos. Handle virtual object insertion in dynamic scenes; Insert virtual objects into scenes shot from different perspectives to complete the integration of product images.

6. The method for automatically generating a multi-storyboard oral video integrating product selection results according to claim 4, characterized in that: Integrate the product image into the starting image, specifically: After the character image is generated, the product image is pasted on the desktop of the picture. Specifically: Cut out the specified product image and cut out the product; Requirements for generating character images, including the position of the characters in the images; Reduce the cut-out product to the ratio of the actual size and the real person; Use the segmentation model to find the desktop or counter, get the coordinates, draw the smallest rectangle that can completely contain the desktop or counter, and calculate the coordinates of the center point of the rectangle; Place the cut-out product in the center of the rectangle to complete the integration of the product image.

7. The method for automatically generating a multi-storyboard oral video integrating product selection results according to claim 1, characterized in that: Create and polish the copy for each of the storyboard videos, specifically: Receive the copy for each storyboard and call the LLM model to rewrite the copy content; Adjust the length of each storyboard to ensure that the length of each storyboard is basically consistent with the length of the copy.

8. The method for automatically generating a multi-storyboard oral video integrating product selection results according to claim 1, characterized in that: According to the copy, match the appropriate voice in the preset sound library, drive the digital human lip shape, generate the short video title, and synthesize the short video with goods, specifically: According to the text style and the semantic tags of the video material, a suitable voice is matched to obtain the most suitable voice, and the most suitable voice is cloned using a preset voice cloning model; According to the most suitable voice, the digital human model is used to drive the lip movements of the characters in the video material, generate subtitles, and obtain the oral video corresponding to the video material; Based on the video copy, use the LLM model to generate a popular short video title for the short video.

9. A device for automatically generating a multi-storyboard oral video integrating product selection results, characterized in that: include: Product selection unit, used to select products based on sales volume and commission, or based on influencer lines and historical behaviors of influencers on the platform; A product information extraction unit, used to extract product information of the selected product, wherein the product information includes product selling points, product categories, and copywriting; A video storyboard generation unit, configured to pre-process the product information using the LLM big model and the video generation big model to obtain a storyboard video, wherein the LLM big model generates a video text for each storyboard, and the video generation big model generates the storyboard video; A copywriting and polishing unit, used to create and polish the copy of each of the storyboard videos; The product-selling short video synthesis unit is used to match the appropriate voice in the preset tone library according to the text, drive the digital human lip shape, generate the short video title, and synthesize the product-selling short video.

Images