Promotion video synthesis method, system and device, storage medium and program product

Abstract

The invention discloses a promotion video synthesis method, system and device, a storage medium and a program product, and the method comprises the steps: receiving a video synthesis request sent by a client, the video synthesis request comprises material information of a to-be-synthesized material, and the to-be-synthesized material comprises a plurality of target video clips; at least one target video clip in the plurality of target video clips comprises promotion information of the subject to be promoted; according to the material information, obtaining the to-be-synthesized material from a material library; dividing the to-be-synthesized materials into at least one material combination according to a preset combination rule; and for any material combination, synthesizing the materials in the material combination to obtain a promotion video corresponding to the material combination. According to the invention, automatic synthesis of the promotion video is realized, and the synthesis efficiency of the promotion video is improved.

Description

Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a method, system, device, storage medium and program product for synthesizing a promotional video. Background Art

[0002] With the continuous development of multimedia technology, short videos have attracted the attention of many users. Promoting a company's products or services through short videos has gradually become one of the typical ways of corporate publicity. At present, companies usually produce short videos through manual editing. However, manual editing of short videos has problems such as long time consumption and low efficiency. Summary of the invention

[0003] In response to the above technical problems, the present application provides a promotional video synthesis method, system, device, storage medium and program product.

[0004] A first aspect of the present application provides a method for synthesizing a promotional video, the method comprising:

[0005] receiving a video synthesis request sent by a client, the video synthesis request including material information of a material to be synthesized, the material to be synthesized including a plurality of target video segments, at least one of the plurality of target video segments including promotion information of a subject to be promoted;

[0006] According to the material information, obtain the material to be synthesized from the material library;

[0007] According to a preset combination rule, the materials to be synthesized are divided into at least one material combination;

[0008] For any material combination, each material in the material combination is synthesized to obtain a promotional video corresponding to the material combination.

[0009] A second aspect of the present application provides a promotional video synthesis device, the device comprising:

[0010] A receiving module, configured to receive a video synthesis request sent by a client, wherein the video synthesis request includes material information of a material to be synthesized, the material to be synthesized includes a plurality of target video segments, and at least one of the plurality of target video segments includes promotion information of a subject to be promoted;

[0011] The acquisition module is used to obtain the material to be synthesized from the material library according to the material information;

[0012] A division module, used to divide the material to be synthesized into at least one material combination according to a preset combination rule;

[0013] The synthesis module is used to synthesize the materials in any material combination to obtain a promotional video corresponding to the material combination.

[0014] A third aspect of the present application provides a promotional video synthesis system, the system comprising: a client and a server;

[0015] The client is used to send a video synthesis request to the server according to the material information of the material to be synthesized input by the user; the material to be synthesized includes multiple target video clips, and at least one target video clip of the multiple target video clips includes the promotion information of the subject to be promoted;

[0016] The server is used to receive a video synthesis request, obtain the material to be synthesized from the material library according to the material information; divide the material to be synthesized into at least one material combination according to a preset combination rule; for any material combination, synthesize each material in the material combination to obtain a promotional video corresponding to the material combination.

[0017] A fourth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect above.

[0018] A fifth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method as described in the first aspect above.

[0019] A sixth aspect of the present application provides a computer program product, including a computer program, wherein the computer program is executed by a processor to implement the method described in the first aspect above.

[0020] This application has at least the following beneficial effects or advantages:

[0021] In the embodiment of the present application, by pre-building a material library, and when the server receives a video synthesis request sent by the client, the material to be synthesized is obtained from the material library according to the material information of the material to be synthesized included in the video synthesis request; and the material to be synthesized is divided into at least one material combination according to a preset combination rule; and for any material combination, each material in the material combination is synthesized to obtain a promotional video corresponding to the material combination. In this way, the automatic synthesis of the promotional video is realized, and the synthesis efficiency of the promotional video is improved. Furthermore, since the material combination is divided according to the combination rule, and the video synthesis is performed on each material combination, it is possible to avoid irrelevant video clips from being synthesized into the same promotional video, which is conducive to improving the overall quality of the promotional video.

[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0024] Figure 1 A schematic diagram of an application scenario of a promotional video synthesis method provided in an embodiment of the present application;

[0025] Figure 2 A first flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0026] Figure 3 A second flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0027] Figure 4 A third flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0028] Figure 5 A fourth flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0029] Figure 6 A fifth flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0030] Figure 7 A schematic diagram of a tree structure provided in an embodiment of the present application;

[0031] Figure 8 A sixth flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0032] Fig. 9 A seventh flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0033] Fig.10 An eighth flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0034] Fig.11 A schematic diagram of the processing flow of the video synthesis engine provided in an embodiment of the present application;

[0035] Fig.12A schematic diagram of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0036] Fig.13 A ninth flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application;

[0037] Fig.14 A schematic diagram of the structure of a promotional video synthesis device provided in an embodiment of the present application;

[0038] Fig.15 A schematic diagram of the composition of a promotional video synthesis system provided in an embodiment of the present application;

[0039] Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0042] Large Language Model (LLM): A language model with a large number of parameters that learns language structure and patterns through unsupervised pre-training on massive text data. LLM is able to perform a variety of natural language processing tasks such as text generation, question answering, translation, etc. without the need for additional task-specific training.

[0043] Vision Language Model (VLM): A deep learning model that combines vision and language understanding, which improves the model's understanding and generation capabilities by fusing information from the two modalities. They are usually used to process and generate the relationship between images and text, and are widely used in tasks such as image description, image question answering, and visual search.

[0044] Stable Video Diffusion (SVD): A video generation technology based on a diffusion model. It generates high-quality video content by gradually processing video frames, gradually adding noise, and then reversely denoising. This technology can achieve diverse styles and content generation while maintaining inter-frame coherence.

[0045] Supervised Fine-Tuning (SFT): Optimizing a model by using labeled data to perform a specific task more accurately. This process generally involves adjusting the parameters of a pre-trained model to improve its performance in a specific domain or task, and is usually used to improve the practicality and reliability of the model.

[0046] Few-shot learning: It is a machine learning paradigm that aims to enable the model to learn effectively with very few labeled samples. For example, the model only needs to see a few examples and add guidance before and after to understand the context of a specific task.

[0047] Chain of Thought (CoT): is a technique used in the field of artificial intelligence to enhance logical reasoning capabilities, especially in large language models. By guiding the model to generate answers in a step-by-step manner, its understanding and logical reasoning capabilities are enhanced. CoT encourages the model to think in steps, allowing it to perform better on complex problems, improving accuracy and consistency. This approach not only promotes a clearer thinking process, but also helps the model to be more systematic and comprehensive when dealing with open-ended problems.

[0048] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0049] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination", etc.

[0050] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0051] At present, there are two main ways for enterprises to produce videos. One is to have professional video editors in-house, and the other is to cooperate with professional video production agencies and use the video editors of video production agencies to produce videos. However, no matter which of the above two methods is adopted, it is ultimately a manual editing method to produce videos. Manual editing is not only time-consuming and inefficient, but also costly and requires high professional skills of the editor.

[0052] Based on this, the embodiment of the present application provides a method for synthesizing a promotional video. Figure 1 Schematic diagram of an application scenario of a method for synthesizing a promotional video provided in an embodiment of the present application. Figure 1 As shown, the scenario includes a client and a server.

[0053] The client can be a mobile phone, tablet computer, desktop computer, portable notebook, wearable device, vehicle-mounted terminal and other terminal devices. The client can install and run the video synthesis application, which can be an independent application (Application, referred to as App), a small program embedded in other applications, or a web application.

[0054] The client is integrated with a display screen, which is configured to display the user interface (UI) of the client, and provides an interface for interaction and information exchange between the client and the user. The UI involved in the embodiment of the present application can be configured as a medium interface for user operation to interact with the video synthesis application. As an interactive interface with the user, the UI can convert the computer language of the video synthesis application into a form that the user can accept and recognize, including displayed pictures, text, buttons, etc. The commonly used form of expression of UI can be a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed in the display screen of the client, wherein the control can include visual interface elements such as icons, buttons, menus, tabs, text boxes, etc. The control can be implemented as a visual function interface of the video synthesis application. When the control receives a corresponding trigger operation from the user, the corresponding function interface of the video synthesis application receives a corresponding processing instruction, so that the video synthesis application responds to the processing instruction for functional processing. For example, each operation of the user (such as clicking, double-clicking, editing, etc.) inputs corresponding instructions to the video synthesis application through the corresponding GUI control, thereby triggering the video synthesis application to perform related processing and update the GUI based on the processed results.

[0055] The server can be an independent server such as a physical server or a cloud server, or a server cluster composed of multiple servers. The server can maintain a material library construction strategy, a material acquisition strategy, a combination strategy, a video synthesis strategy, etc. Among them, the material library construction strategy is used to indicate that a video library is constructed based on multiple target images and multiple historical promotional videos of the subject to be promoted. The target image may include the object to be promoted of the subject to be promoted. The material acquisition strategy is used to indicate that when a video synthesis request sent by the client is received, the corresponding material to be synthesized is obtained from the material library according to the material information included in the video synthesis request. The combination strategy is used to indicate that the material to be synthesized is divided into at least one material combination according to a preset combination rule. The video synthesis strategy is used to indicate that for any material combination, each material in the material combination is synthesized to obtain a promotional video corresponding to the material combination.

[0056] The client and the server can exchange data based on the network, which can be a wired network or a wireless network. For the data exchange process between the client and the server, please refer to the relevant description below.

[0057] It should be understood that Figure 1 In the figure, the first client is a desktop computer and the server is an independent physical server. Figure 1 The illustration is only a schematic representation of the application scenario of the promotion video synthesis method involved in this application, and does not constitute a limitation on the technical solution of this application. In some other implementations, the application scenario of the screening condition creation method involved in this application may also include more or fewer components.

[0058] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0059] Figure 2 This is a flow chart of a method for synthesizing a promotional video provided in an embodiment of the present application. Figure 2 The method shown can be Figure 1 The server in the implementation. Figure 2 As shown, the method includes the following steps 101-104.

[0060] Step 101: receiving a video synthesis request sent by a client, the video synthesis request including material information of a material to be synthesized, the material to be synthesized including a plurality of target video segments, at least one of the plurality of target video segments including promotion information of a subject to be promoted.

[0061] Step 102: Obtain the material to be synthesized from the material library according to the material information.

[0062] Step 103: divide the materials to be synthesized into at least one material combination according to a preset combination rule.

[0063] Step 104, for any material combination, synthesize the materials in the material combination to obtain a promotional video corresponding to the material combination.

[0064] The subject to be promoted may be an enterprise, institution, organization, group or individual. The promotion information of the subject to be promoted may vary depending on the type of service provided by the subject to be promoted. For example, if the subject to be promoted provides takeaway services, the promotion information may include the download address of the corresponding takeaway platform, preferential information, etc.; for another example, if the subject to be promoted provides fitness services, the promotion information may include the address of the fitness venue, the type of exercise, etc. The specific content of the promotion information is not specifically limited in this application, and it can be set as needed in actual application.

[0065] In some embodiments, the user (e.g., an employee of an enterprise) included in the subject to be promoted can edit the material information of the material to be synthesized in the video synthesis application installed on the client and operate the submission control to submit. Accordingly, the client sends a video synthesis request to the server according to the acquired material information in response to the user's submission operation. When the server receives the video synthesis request sent by the client, it obtains the corresponding material to be synthesized from the pre-built material library according to the material information. Among them, the material information may include video attributes and audio attributes. The video attributes may include the segment identifier of the target video segment, or the template identifier of the template corresponding to the target video segment, the attributes of the promotion object of the subject to be promoted (e.g., name, etc.), etc. The promotion object may be food, electronic equipment, service platform, clothing, people, daily necessities, etc. The audio attributes may include a voice identifier, a background music identifier, or the language type of the voice (e.g., local dialect, English, Japanese, etc.), the type of background music (e.g., cheerful, dynamic, etc.), etc. Correspondingly, the material library may include a video library and an audio library, wherein the video library includes multiple video segments, and the audio library includes multiple voices and multiple background music.

[0066] Furthermore, corresponding to the material information, the material to be synthesized may include multiple target video clips corresponding to the video attributes, and may also include at least one target audio corresponding to the audio attributes, and the target audio may include at least one of voice and background music. In order to avoid the video viewer from feeling that the material is randomly pieced together, in the implementation of the present application, a material combination rule is preset, and the acquired material to be synthesized is divided into at least one material combination according to the combination rule. And, for each material combination, each material in the material combination is synthesized to obtain a promotional video corresponding to the material combination.

[0067] In the method for synthesizing a promotional video provided in an embodiment of the present application, a material library is pre-constructed, and when a video synthesis request sent by a client is received at a server, the material to be synthesized is obtained from the material library according to the material information of the material to be synthesized included in the video synthesis request; and the material to be synthesized is divided into at least one material combination according to a preset combination rule; and for any material combination, each material in the material combination is synthesized to obtain a promotional video corresponding to the material combination. Thus, the automatic synthesis of the promotional video is realized and the efficiency of video synthesis is improved. Furthermore, since the material combinations are divided according to the combination rules and video synthesis is performed on each material combination, it is possible to avoid irrelevant video clips being synthesized into the same promotional video, which is beneficial to improving the overall quality of the promotional video.

[0068] Considering that in actual applications, some users may have clear demands for the video content and hope to synthesize multiple target video segments specified by them. Based on this, in some embodiments, the material information may include the segment identifier of each target video segment. Figure 3 As shown, step 102 may include the following steps 1021 to 1023:

[0069] Step 1021: identify each segment in the material information and the corresponding video segment in the material library as the target video segment.

[0070] In some implementations, the video synthesis application may include a material information configuration interface, and the user may operate a query control in the material information configuration interface to query the video clips included in the material library and select multiple target video clips therefrom. Accordingly, in response to the user's operation on the submit control, the client determines the information such as the clip identifiers of the multiple target video clips selected by the user as the material information, and sends a video synthesis request to the server according to the material information.

[0071] Step 1022: If the material information also includes audio attributes, then at least one corresponding target audio is acquired from the material library according to the audio attributes.

[0072] In some embodiments, when the target video clip selected by the user does not contain a voice, the user can also query the voices in the audio library and select at least one target voice from them. The user can also query the background music in the background music library and select at least one target background music from them. Accordingly, the audio attributes may include a voice identifier of at least one target voice and / or a music identifier of at least one target background music. It should be noted that when the user selects multiple target voices, the playback order of the target voices can also be specified. When the user selects multiple target background music, the playback order of the target background music, the start and end playback time of each target background music, etc. can also be specified.

[0073] In some embodiments, when the target video clip selected by the user contains voice, the user can query multiple languages ​​that support conversion and select at least one target language from them. At this time, the user can also select at least one target background music in the above manner. Accordingly, the audio attributes can include the language type of at least one target language and / or the music identifier of at least one background music. It should be noted that when the user selects multiple target languages, the start and end speech time corresponding to each target language can also be specified.

[0074] Correspondingly, when the server determines that the material information includes audio attributes, it obtains at least one corresponding target audio from the material library according to the audio attributes, and the target audio includes at least one target voice corresponding to a target language, and / or at least one background music.

[0075] Step 1023: determine each target video segment and at least one target audio as materials to be synthesized.

[0076] It is understandable that when the material information does not include audio attributes, step 1022 may not be performed, and in step 1023, each target video segment is determined as the material to be synthesized.

[0077] Therefore, by determining the target video clips corresponding to the clip identifiers in the material information and at least one target audio corresponding to the audio attributes as the materials to be synthesized, while realizing the automatic synthesis of the promotional video, it can meet the needs of users to specify the video content by themselves.

[0078] Considering that in actual applications, some users may not have clear requirements for the video content, as long as the promotional video conforms to the specified template. Based on this, in some implementations, the material information may include template identifiers corresponding to multiple target video clips. Specifically, Figure 4 As shown, step 102 may include the following steps 1024 and 1025:

[0079] Step 1024: determine the template corresponding to the template identifier in the material information among the preset multiple templates as the reference video template.

[0080] Specifically, multiple templates can be preset, and the user can operate the template selection control in the material information configuration interface to select any template from the multiple templates displayed. Accordingly, the client determines the template identifier and other information of the template selected by the user as the material information in response to the user's operation on the submit control.

[0081] Among them, the multiple templates may include at least one first template and at least one second template. Any first template may include multiple promotional scenes. Taking the takeaway service provided by the subject to be promoted as an example, the multiple promotional scenes may be any of pre-sticker, short play, click link, download, last shot, post-sticker, etc. Among them, the pre-sticker may be used to attract the attention of the audience and may also prompt the content to be played. The short play is a short scene or story used to attract the interest of the audience and may contain humorous and fascinating plots. The guide may contain interactive elements such as links, buttons, etc., to encourage the audience to click to obtain more information, participate in activities or receive rewards, etc. Download is a link to promote the promotion object (such as a takeaway platform) of the subject to be promoted, and may display the download link, functions, advantages, etc. of the promotion object. The end shot can be used to strengthen the impression or information of the promotion object and may contain logos, slogans, calls to action, etc. The post-sticker may be used to further strengthen the promotion object, etc.

[0082] As an example, a first template may be “previous post-situation drama-download-end screen”, another first template may be “previous post-situation drama-guidance-download-end screen-next post”, and so on.

[0083] Any second template may include multiple second description texts and the playback duration of each second description text. The multiple second description texts are used as the voice content of the promotional video, describing multiple promotional scenes. The multiple promotional scenes may be any multiple of the aforementioned pre-post, situational drama, guide, download, end screen, post-post, etc.

[0084] As an example, if the promotion target is a food delivery platform, a second template is as follows: [

[0086] {"text":"Let's see who doesn't know about this event!",

[0087] "duration":2.05},

[0088] {"text":"Now you can get a 10 yuan red packet with no threshold when ordering takeout!",

[0089] "duration":3.0375},

[0090] {"text":"Click the link below",

[0091] "duration":1.2125},

[0092] {"text":"Old friends of [promotion target] can receive up to 10 yuan",

[0093] "duration":2.6},

[0094] {"text":"No threshold for red envelopes",

[0095] "duration":1.125},

[0096] {"text":"You can deduct directly when placing an order",

[0097] "duration":2.1625},

[0098] {"text":"Super save money",

[0099] "duration":0.85},

[0100] {"text":"Moreover, [promotion targets] have a wide variety of takeaways,"

[0101] "duration":2.675},

[0102] {"text":"Order whatever you want to eat,"

[0103] "duration":1.4125},

[0104] {"text":"Don't hesitate,"

[0105] "duration":0.9875},

[0106] {"text":"Family members who are interested, please come to [promotion target],"

[0107] "duration":2.3},

[0108] {"text":"Cheap takeout",

[0109] "duration":1.3125} ]

[0111] Among them, text represents the second description text, duration represents the playback duration, and [promotion object] can be filled with the name of the food delivery platform and other information. The second description text "I see who doesn't know about this event. Now you can get a 10 yuan red envelope with no threshold for ordering food delivery" can describe a situational drama; the second description text "Click the link below, "old friends of [promotion object] can receive a red envelope of up to 10 yuan, no threshold, and can be directly deducted when placing an order, which is super cost-saving" can describe guidance; the second description text "[promotion object] has a rich variety of food delivery, you can order whatever you want, don't hesitate, family members who are interested can quickly come to [promotion object], and order takeout at a cheaper price" can describe the ending screen.

[0112] Step 1025: If the material information also includes audio attributes, the material to be synthesized is obtained from the material library according to the reference video template and the audio attributes.

[0113] In some embodiments, Figure 5 As shown, step 1025 may include the following steps 1025-1 to 1025-3:

[0114] Step 1025-1, determining a plurality of video nodes included in the reference video template, wherein the plurality of video nodes correspond one-to-one to the plurality of video clips, and the arrangement order of the plurality of video nodes is used to represent the playback order of the plurality of video clips.

[0115] Specifically, when the reference video template is the first template, the multiple promotion scenes included in the first template are determined as video nodes. When the reference video template is the second template, the multiple promotion scenes described in the second template are determined as video nodes. The connection order of each video node in the first template is the arrangement order of the video nodes, and the description order of each video node in the second template is the arrangement order of the video nodes.

[0116] As an example, the reference video template is the first template of the aforementioned example "previous post - situational drama - download - ending screen", which represents the sequential playback of the target video clip corresponding to the previous post, the target video clip corresponding to the situational drama, the target video clip corresponding to the download, and the target video clip corresponding to the ending screen.

[0117] As another example, the reference video template is the second template of the aforementioned example, which represents the sequential playback of the target video segment corresponding to the situation drama, the target video segment corresponding to the introduction, and the target video segment corresponding to the ending screen.

[0118] Step 1025-2: determine each video node as the first node according to the arrangement order, and obtain at least one target material matching the first node and the audio attribute from the material library.

[0119] In order to facilitate the acquisition of target video clips that match each video node, in some implementations, any video clip in the video library carries a tag, and the tag is used to characterize the video content in the video clip. Figure 5 As shown, step 1025-2 may include the following steps A1 to A4:

[0120] Step A1, according to the arrangement order, each video node is determined as the first node, and the matching text, sub-audio attribute and multiple candidate video segments corresponding to the first node are determined.

[0121] Considering that in actual applications, users may want different video nodes to use different language types or different background music, based on this, in some embodiments, the audio attribute in the material information may also include multiple sub-audio attributes, and the multiple sub-audio attributes correspond one-to-one to the multiple video nodes in the video reference template. Accordingly, if the server determines that the audio attribute includes multiple sub-audio attributes, the sub-audio attribute corresponding to the first node is obtained from the material information; if it is determined that there is only one audio attribute, the audio attribute is determined as the sub-audio attribute corresponding to the first node.

[0122] In order to improve the efficiency of acquiring the target video clips, in some embodiments, a first correspondence between each video clip in the video library and each promotion scene can be pre-established. For example, the video library can be pre-divided into multiple sub-video libraries, and the multiple sub-video libraries correspond to multiple video nodes one by one. For another example, the label of any video clip can include the node name of the corresponding video node, etc. The specific implementation form of the first correspondence is not specifically limited in this application. Accordingly, the server determines each video clip corresponding to the first node as a candidate video clip.

[0123] The method of determining the matching text varies with the reference video template, and the details can be found in the relevant description below.

[0124] Step A2, matching the matching text with the labels of each candidate video segment, and determining the candidate video segment corresponding to at least one label that successfully matches as the target video segment.

[0125] In some embodiments, the label of the video clip includes a first description text, and the first description text is used to describe the content of the video clip, such as the name of the promotion object, whether a person is included, whether the person is speaking, the content of the speech, whether the task has a preset action (such as eating, running, etc.), etc. Accordingly, matching the matching text with the label of each candidate video clip in step A2 may include:

[0126] For any candidate video segment, call the pre-trained matching model to extract the first description text of the candidate video segment and the first semantic feature of the matching text respectively; determine the first score according to the first semantic feature, and the first score is used to characterize the degree of relevance between the first description text and the matching text; determine the label corresponding to the target first score as the label of successful matching, and the target first score is at least one first score that satisfies the first preset condition. The higher the first score, the higher the degree of relevance between the corresponding first description text and the matching text.

[0127] Among them, the structure of the matching model, the data processing process and the training process can all refer to the matching model in the relevant technology, which will not be described in detail in this application.

[0128] Step A3: acquiring at least one corresponding target audio from the audio library according to the sub-audio attribute.

[0129] In some implementations, each audio in the audio library carries a label. The label of any voice in the audio library may include the language type, the text corresponding to the voice, etc. The label of any background music may include the type of music (e.g., cheerful, dynamic, etc.), the name of the music, etc. Accordingly, the sub-audio attributes are matched with the labels of each audio in the audio library, and the audio corresponding to at least one label that is successfully matched is determined as the target audio.

[0130] It should be pointed out that when there is only one audio attribute, it is only necessary to obtain the target audio once from the audio library. For example, when the first node is the first video node, obtain at least one corresponding target audio from the audio library; or, when the first node is the last video node, obtain at least one corresponding target audio from the audio library; and determine the obtained at least one target audio as the target audio corresponding to each video node.

[0131] Step A4: determine the target video clip and target audio as the target material corresponding to the first video node.

[0132] Therefore, in the above steps A1 to A4, the target material is obtained based on the matching text and sub-audio attributes corresponding to the first node, which ensures that the target material can match the first node, thereby ensuring the quality of the promotional video.

[0133] Step 1025-3, determining the target materials corresponding to the multiple video nodes as the materials to be synthesized.

[0134] In the above steps 1025 - 1 to 1025 - 3 , the material to be synthesized is obtained from the material library according to the reference video template and audio attributes, thereby ensuring that the obtained material to be synthesized can match the user's needs, thereby ensuring that the promotional video can meet the user's needs.

[0135] In some implementations, in order to ensure that multiple target video clips with high correlation can be synthesized into a promotional video, in some implementations, such as Figure 6 As shown, step 103 may include the following steps 1031 to 1034:

[0136] Step 1031, obtaining at least one tree structure formed by target video clips corresponding to multiple video nodes, wherein any tree structure includes at least one branch, and any branch includes multiple target video clips, and the multiple target video clips correspond to the multiple video nodes one by one.

[0137] It is understandable that the tree structure includes at least one branch, and any branch includes multiple tree nodes. In the embodiment of the present application, each tree node corresponds to a video node, and each tree node is used to represent a target video segment of the corresponding video node, that is, the number of tree nodes included in each branch is consistent with the number of video nodes included in the reference video template. As an example, the reference video template includes three video nodes, and a schematic diagram of a tree structure is shown in FIG. Figure 7 As shown, the tree structure includes 3 branches, each of which includes 3 target video segments (i.e., 3 tree nodes). The 3 branches are target video segment 3-target video segment 5-target video segment 10, target video segment 3-target video segment 6-target video segment 8, and target video segment 3-target video segment 6-target video segment 16.

[0138] Step 1032: Determine a second score corresponding to each branch according to the target first score corresponding to the target video segment, where the second score is used to characterize the combined quality of the multiple target video segments included in the corresponding branch.

[0139] According to the above description, the target first score corresponding to the target video clip is the target first score corresponding to the successfully matched tag. In some embodiments, for any branch, the target first scores corresponding to the target video clips included in the branch can be multiplied, and the multiplication result can be determined as the second score. Alternatively, the target first scores corresponding to the target video clips included in the branch are added, and the addition result is determined as the second score. Alternatively, the target first scores corresponding to the target video clips included in the branch are weightedly added, and the result is determined as the second score. The method for determining the second score is not specifically limited in this application.

[0140] Since the first score can characterize the correlation between the first description text and the matching text of the video segment, and the correlation between the first description text and the matching text can characterize the correlation between the video segment corresponding to the first description text and the target video segment corresponding to the matching text, and the higher the correlation, the higher the combination quality. Therefore, the second score determined based on the first score can characterize the combination quality of the multiple target video segments included in the corresponding branch, and the higher the second score, the higher the combination quality, that is, the higher the correlation between the corresponding multiple target video segments.

[0141] Step 1033: determine the branch corresponding to the target second score as the target branch, where the target second score is at least one second score that satisfies the second preset condition.

[0142] In some embodiments, the second scores may be sorted in descending order, and the first number of second scores ranked in front in the sorting result are determined as at least one second score that meets the second preset condition. In some embodiments, the second scores may be sorted in ascending order, and the first number of second scores ranked in the back in the sorting result are determined as at least one second score that meets the second preset condition. The first number may be set as needed in actual applications, for example, the first number is 5.

[0143] Step 1034: for any target branch, the target video segment in the target branch and the target audio corresponding to the target branch are determined as a material combination.

[0144] Therefore, by acquiring at least one tree structure corresponding to the target video segment, the material combination can be quickly and accurately acquired based on the tree structure, and a high correlation can be ensured between multiple target video segments in the material combination.

[0145] According to the above description, the first template and the second template contain different contents, so the specific implementation methods of some of the above operation steps vary with the reference video template. Specifically, when the reference video template is the first template, Figure 8 As shown, the aforementioned step A1 may include the following steps A11:

[0146] Step A11, according to the arrangement order, determine each video node as the first node, determine the sub-audio attribute and multiple candidate video clips corresponding to the first node, and when the reference video template is the first template, determine the matching text of the first video node according to the video category corresponding to the first node, the arrangement position of the first node in multiple video nodes, and the labels of each video clip in the video library.

[0147] Specifically, if the video category corresponding to the first node is the target category and the arrangement position is the first, the attribute of the promoted object in the material information is determined as the matching text, and the target category indicates that the video content of the first node contains the object to be promoted. If the video category corresponding to the first node is the target category and the arrangement position is not the first, the matching text is determined based on the attribute of the object to be promoted and the target label; the target label is a label that satisfies the third preset condition among the labels of each target video clip currently obtained. If the video category corresponding to the first node is not the target category and the arrangement position is the first, the attribute of the object to be promoted is determined as the matching text. If the video category corresponding to the first node is not the target category and the arrangement position is not the first, the matching text is determined based on the target label.

[0148] That is to say, when the first node is used to display the promotion object, in order to obtain the target video clip containing the object to be promoted, the matching text includes the attributes of the object to be promoted. In order to ensure that the target video clips corresponding to different video nodes have a certain correlation, the arrangement position of the first node among multiple video nodes is also taken into account when determining the matching text, and when the arrangement position is the first, since the target video clip is not currently obtained, only the attributes of the object to be promoted are determined as matching text. When the arrangement position is not the first, since at least one target video clip is currently obtained, in order to ensure that there is a certain correlation between the current target video clip and the target video clip corresponding to the first node, the attributes of the promotion object and the labels of the currently obtained target video clips, the labels that meet the third preset condition are determined as matching text.

[0149] Among them, the attributes of the object to be promoted may be the name of the object to be promoted, the presentation method, etc. As an example, the object to be promoted is a hamburger, and the attributes of the object to be promoted may be the name of the hamburger, or the presentation method of the hamburger, such as a white-collar worker eating a hamburger or a deliveryman delivering a hamburger. When the video reference template is the first template, it can be pre-set that the video content of the situation drama includes the object to be promoted. Accordingly, in the case where the video reference template is the first template, if the server determines that the first node is the promotion scene of the situation drama, it determines that the video category corresponding to the first node is the target category.

[0150] Therefore, when the reference video template is the first template, the matching text is determined according to the video category and arrangement position corresponding to the first node, which can ensure that the target video clip suitable for the first node is matched based on the matching text, and can also ensure that the target video clip of the first node has a certain correlation with the target video clips of other video nodes, thereby improving the video quality of the promotional video.

[0151] Furthermore, in order to ensure the accuracy of the matching text, before determining the matching text according to the target tag, the method may further include:

[0152] If the arrangement position of the first node is the second position, each target video segment corresponding to the first video node is determined as the root node of the corresponding tree structure, and the label of each target video segment corresponding to the first video node is determined as the target label; if the arrangement position of the first node is any position after the second position, the label of the target video segment in each branch of the tree structure is determined as the target label;

[0153] Correspondingly, Figure 8 As shown, after determining the target video segment in the aforementioned step A2, the method may further include the following step A0:

[0154] Step A0, if the target video clip is obtained based on the matching text corresponding to the target tag, then the target video clip is added to the branch corresponding to the target tag, and when the arrangement position of the first node is the last, the current tree structure is saved. The target tag is determined based on the branch in the tree structure.

[0155] Among them, the saved tree structure is used to determine the material combination. Specifically, considering that in practical applications, each video node may correspond to multiple target video clips, and the correlation between different target video clips between different video nodes is often different. Based on this, in order to facilitate the acquisition of the target video clip of the next video node that is relevant to any target video clip, in one or more embodiments of the present application, a greedy search is performed in the material library for each target video clip to obtain at least one target video clip that is relevant to the corresponding target video clip. In order to effectively record the correlation relationship between each target video clip, multiple tree structures are constructed in the present application, and each node in any tree structure corresponds to a target video clip, and each child node in the tree structure has a correlation with its parent node. It can be understood that the number of target video clips corresponding to the first video node is the number of tree structures. In other words, by constructing a tree structure, the combination relationship between each target video clip can be clearly and accurately reflected based on the tree structure, so that when the video is synthesized, multiple target video clips with a combination relationship are synthesized to improve the video quality of the promotion video.

[0156] Further, when the reference video template is the first template, Figure 8 As shown, the aforementioned step 1031 may include the following steps 1031-1:

[0157] Step 1031 - 1 : Obtain at least one saved tree structure.

[0158] That is to say, when the reference video template is the first template, the target video segment is obtained based on the correlation between the video segments. And in the process of obtaining the target video segment, at least one tree structure is constructed based on the correlation between the target video segments. That is, the target tag is determined according to the branch of the tree structure, and the target video segment is obtained according to the matching text corresponding to the target tag, and the target video segment is added to the branch corresponding to the target tag, each branch includes multiple target video segments with correlation; after adding each target video segment corresponding to the last video node to the corresponding tree structure, the final tree structure is obtained and saved. And, when dividing the material combination, at least one saved tree structure is obtained.

[0159] The above is a specific description of the relevant steps when the reference video template is the first template. According to the above specific description, for each video node in the first template, at least one corresponding target video clip can be obtained from the material library, and when the material information includes sub-audio attributes, at least one target audio can be obtained. And according to the tree structure constructed in the process of obtaining the target video clip, the material combination can be determined, so as to perform video synthesis on the material combination to obtain the promotion video. Not only the automatic synthesis of the promotion video is realized, but also the quality of the promotion video is guaranteed.

[0160] Further, in the case where the reference video template is the second template, such as Fig. 9 As shown, the aforementioned step A1 may include the following steps A12:

[0161] Step A12, according to the arrangement order, each video node is determined as the first node, the sub-audio attribute and multiple candidate video clips corresponding to the first node are determined, and when the reference video template is the second template, the pre-trained large language model is called to determine the matching text according to at least one second description text corresponding to the first node in the second template.

[0162] Among them, calling a pre-trained large language model and determining the matching text according to at least one second description text corresponding to the first node in the second template can include: generating a first prompt word according to the at least one second description text corresponding to the first node and the first prompt template, the first prompt word being used to prompt the large language model to extract the key information of the second description text; inputting the first prompt word into the large language model to obtain the key information of the second description text, and determining the key information as the matching text.

[0163] When the reference video template is the second template, since each second description text in the second template is the voice of the promotional video to be synthesized, in order to ensure that the acquired target video segment matches the second description text corresponding to the first node, it is avoided that the second description text describes a hamburger, while the acquired target video segment is about spicy crayfish. In some implementations of the present application, a pre-trained large language model is used to extract key information from the second description text, and the extracted key information is determined as the matching text. Therefore, when the target video segment is acquired according to the matching text, it is ensured that the content of the target video segment matches the corresponding second description text, that is, the second description text describes a hamburger, and the target video segment is also about a hamburger.

[0164] Since the target video segments are obtained based on the key information of the second description text corresponding to each video node when the reference video template is the second template, the target video segments of different video nodes may not be closely related. At this time, in order to accurately identify the target video segments with related relationships, one or more embodiments of the present application construct a tree structure based on the obtained target video segments. Fig. 9 As shown, the aforementioned step 1031 may include the following steps 1031-2 to 1031-6:

[0165] Step 1031 - 2 , according to the arrangement order of the multiple video nodes, determine each video node as the second node.

[0166] Step 1031 - 3 , when the second node is the first video node, each target video segment corresponding to the second node is determined as the root node of the corresponding tree structure.

[0167] That is to say, the number of target video segments corresponding to the first video node is the number of tree structures. When the first video node corresponds to one target video segment, there is one tree structure; when the first video node corresponds to multiple target video segments, there are multiple tree structures.

[0168] Step 1031 - 4 , when the second node is not the first video node, determine the segment to be matched in the target video segment corresponding to the second node.

[0169] In some embodiments, when the second node is not the first video node, a target video segment may be randomly selected from the target video segments corresponding to the second node without repetition, and the selected target video segment may be determined as the segment to be matched. Alternatively, each target video segment may be determined as the segment to be matched in turn according to the order in which the target video segments corresponding to the second node are obtained.

[0170] Step 1031-5, respectively determine the third scores between the to-be-matched segment and each segment set, each segment set corresponds one-to-one to each branch of the current tree structure, any segment set includes the target video segment in the corresponding branch, and the third score represents the degree of correlation between the to-be-matched segment and the corresponding segment set.

[0171] Among them, determining the third score between the to-be-matched segment and each segment set may include: for any segment set, determining the target label according to the labels of each target video segment included in the segment set; calling the matching model to respectively extract the second semantic features of the label of the to-be-matched segment and the target label; and determining the third score between the to-be-matched segment and the segment set according to the second semantic features.

[0172] In some embodiments, for any segment set, the labels of the target video segments included in the segment set may be concatenated to obtain the target label; or the label of the target video segment corresponding to the leaf node may be determined as the target label. The label of the matching segment and the target label are input into the matching model, the second semantic feature of the label of the matching segment and the second semantic feature of the target label are extracted respectively, and the third score is determined according to the extracted second semantic feature.

[0173] Step 1031-6, determining the to-be-matched segment as a child node of the target branch corresponding to the target segment set, and adding the child node to the target branch, the target first segment is a segment set corresponding to the third score that meets the fourth preset condition.

[0174] In some embodiments, the third scores can be sorted in descending order to obtain a sorting result, and the second number of third scores arranged in front in the sorting structure are determined as the third scores that meet the first preset condition; the fragment set corresponding to the third scores that meet the first preset condition is determined as the target fragment set, and the branch corresponding to the target fragment set is determined as the target branch; and the fragment to be matched is determined as a child node of the last node of the target branch and added to the target branch.

[0175] In some implementations, the third scores may be sorted in ascending order to obtain a sorting result, and the third scores of the second number arranged at the end of the sorting structure are determined as the third scores that meet the first preset condition. The sorting method may be set as needed in actual applications, and this application does not make any specific restrictions on this.

[0176] That is, when the reference video template is the second template, the target video segment corresponding to each video node is obtained according to the second description text in the second template, and the correlation between the target video segments of different video nodes may not be high. In order to ensure that the multiple target video segments in the material combination have a high correlation, in one or more embodiments of the present application, after obtaining the target video segment corresponding to each target video node, the third score is calculated and multiple tree structures are constructed, and the material combination is determined according to each tree structure.

[0177] The above is a specific description of the relevant steps when the reference video template is the second template. According to the above specific description, for each video node in the second template, at least one corresponding target video clip can be obtained from the material library, and when the material information includes sub-audio attributes, at least one target audio can be obtained. After obtaining each target video clip, a tree structure can be constructed, and the material combination can be determined based on the tree structure, so as to perform video synthesis on the material combination to obtain a promotional video. Not only the automatic synthesis of the promotional video is achieved, but also the quality of the promotional video is guaranteed.

[0178] Based on any of the above implementations, Fig.10 As shown, step 104 may include the following steps 1041 and 1042:

[0179] Step 1041, generating a video synthesis protocol according to the access address of each material in the material combination in the material library.

[0180] In some implementations, the access address of each material can be filled into the protocol template in sequence according to the playback order of each material in the material set. In some implementations, the protocol template can also include the start playback time and end playback time of each material. The specific form of the protocol template can be set as needed in actual applications, and this application does not make specific restrictions on this.

[0181] It should be noted that when the material information includes the segment identifier of the target video segment specified by the user, the playback order of each material can be the order specified by the user. When the material information includes the template identifier, the playback order of each material can be determined according to the arrangement order of the video nodes included in the template.

[0182] Step 1042, calling the video synthesis engine, synthesizing the various materials according to the video synthesis protocol, and obtaining the corresponding promotional video.

[0183] In some implementations, the video synthesis engine may include a protocol parser, a material download module, a segment generator, a track fusion module, an audio track fusion module, an encoding synthesis module, a storage module, and a query module. Fig.11 As shown, the server calls the video synthesis engine, and after inputting the video synthesis protocol into the video synthesis engine, the video synthesis protocol can be parsed by the protocol parser to obtain the protocol parsing result. The corresponding materials are downloaded by the material download module according to the access address of each material included in the parsing result. The fragment generator generates one or more blank fragments, static fragments, dynamic pictures, text, transitions, special effects, videos, etc. according to the relevant algorithms based on the downloaded materials and other video synthesis requirements in the protocol parsing result. Among them, the blank fragment can be a fragment without any visual content, which is used for positioning, transition, etc.; the static fragment refers to a video fragment with fixed and continuous screen content, such as an image or a static scene, which can be used for background, title page, etc. The text can be subtitles that appear in the video, etc. And through the track fusion module, the generated content and the downloaded material are track fused. Through the audio track fusion module, the audio fusion is performed on the audios that need to be fused. And, through the encoding synthesis module, the materials to be synthesized are synthesized to obtain the corresponding promotional video. The obtained promotional video is assigned a video identifier, the promotional video and the video identifier are correspondingly saved in the storage module, and the video identifier is returned to the server. The server can send the video identifier to the client, and upon receiving the video query request sent by the client according to the video identifier, send the video query request to the video synthesis engine, so as to search for the corresponding target promotion video in the saved promotion videos through the query module of the video synthesis engine, and send the target promotion video to the client. In some embodiments, the server can also send a synthesis status query request to the video synthesis engine to query the synthesis status of the promotion video.

[0184] It should be pointed out that Fig.11 This is only for illustration and not for limitation. The structure and data processing process of the video synthesis engine can be set as needed in actual applications. In some embodiments, after receiving the video identifier, the client can also directly send a video query request to the video synthesis engine according to the video identifier.

[0185] By generating a video synthesis protocol and synthesizing the promotional video through a video synthesis engine, the synthesis speed and quality of the promotional video can be improved.

[0186] In order to ensure the diversity of promotional videos and determine video clips that are popular with the audience, in some implementations, the video synthesis of various materials according to the video synthesis protocol may include:

[0187] Add tracking points to each target video segment, and use the tracking points to collect interaction data between users and the corresponding target video segment, where the interaction data can be clicks, comments, forwarding, etc.

[0188] Correspondingly, after obtaining the promotional video corresponding to the material combination in step 104, the method may further include: delivering the promotional video to a designated platform; if interaction data collected by the embedding point is obtained, updating the selection weight of the corresponding target video segment according to the interaction data.

[0189] When the interaction data indicates that a certain video clip is popular with the public, the selection weight of the corresponding video clip can be appropriately increased in order to attract more viewers. When the interaction data indicates that a certain video clip has a high number of plays, the selection weight corresponding to the video clip can be appropriately reduced in order to ensure the diversity of the promoted videos.

[0190] Accordingly, obtaining the material to be synthesized from the material library in step 102 includes: obtaining multiple target video clips from the material library according to the selection weight of each video clip in the material library. Specifically, for each video clip, after obtaining its corresponding first score in the above manner, the first score is multiplied by the selection weight, and the multiplication result is determined as the final first score.

[0191] Therefore, by embedding points in each target video clip to collect user interaction data, the selection weight of each video clip in the material library can be dynamically adjusted according to the interaction data, thereby ensuring the diversity of the synthesized promotional videos and synthesizing promotional videos that are popular with the public.

[0192] In order to meet different user needs and realize dynamic synthesis of diversified promotional videos, in some implementations of the present application, for example Fig.12 As shown, a material library is constructed in advance based on multiple target images, multiple historical promotion videos of the subject to be promoted, and multiple pre-generated second templates. Fig.13 As shown, the above step 101 may further include the following steps 1001 to 1003:

[0193] Step 1001: construct a video library based on multiple target images and multiple historical promotional videos of the subject to be promoted, wherein the target image includes the promotion object of the subject to be promoted.

[0194] The promotion object may be food, people, service platforms, electronic devices, clothing, daily necessities, etc. In order to ensure the quality of the video library, in some implementations, such as Fig.12 As shown, for the target image, multiple models can be used to generate video clips; for the historical promotion video, slicing, extraction and trimming, subtitle deletion, video extension, subtitle calibration and other processing can be performed to obtain the video clip. Specifically, step 1001 can include the following steps 1001-1 to 1001-3:

[0195] Step 1001 - 1 : for any target image, call the video macro model and / or the video diffusion model to generate at least one corresponding first candidate segment.

[0196] In some embodiments, in order to highlight the details of the promotion object, for any target image, a pre-trained video model can be called to generate at least one first candidate segment corresponding to the target image according to the input target image and at least one motion information input. The motion information can be a flip angle, a rotation angle, etc. The specific processing and training process of the video model can refer to the relevant technology, which will not be described in detail in this application.

[0197] In some embodiments, in order to enrich the video material, for any target image, if it is determined that the target image includes a person or an animal, the video diffusion model can also be called to generate a corresponding first candidate segment according to the input target image and the pre-generated skeletal animation. The person or animal in the first candidate segment is moving according to the skeletal animation, such as dancing.

[0198] Step 1001 - 2 : for any historical promotion video, the historical promotion video is divided into a plurality of second candidate segments, and the plurality of second candidate segments correspond one to one to the plurality of video objects.

[0199] In some implementations, for any historical promotion video, a boundary detection model is called to detect the shot switching of the historical promotion video, and the historical promotion video is divided into a plurality of second candidate segments according to the shot switching. The shot switching represents that the shooting shot jumps from one video object to another video object. The video object may be the promotion object of the subject to be promoted, or may not be the promotion object of the subject to be promoted. For example, the promotion object of the subject to be promoted includes food, and the video object is a certain landscape, etc.

[0200] By switching the historical promotional video to a plurality of second candidate segments corresponding to a plurality of promotional objects, it is convenient to obtain the segments corresponding to the corresponding promotional objects when synthesizing the promotional video later, so as to synthesize a new promotional video.

[0201] Step 1001 - 3 , preprocessing the first candidate segment and the second candidate segment to obtain a video library, which includes a plurality of video segments carrying labels.

[0202] In some embodiments, for any first candidate segment, a label of the first candidate segment can be generated according to the promotion object corresponding to the first candidate segment. For any second candidate segment, the visual large model is called to generate an initial description text of the second candidate segment, and the initial description text is used to describe the content of the second candidate segment. For any target second candidate segment, the label of the target second candidate segment is determined according to the initial description text of the target second candidate segment; the target second candidate segment is the second candidate segment related to the subject to be promoted determined according to the initial description text; and if it is determined according to the initial description text that the target second candidate segment contains subtitles but does not contain characters, the subtitles are deleted; for any candidate segment carrying a label, the extension model is called to adjust the size of the candidate segment to obtain a video segment of the target size.

[0203] In order to facilitate the quick acquisition of target video clips that match user needs during promotional video synthesis, labels of each first candidate clip and labels of each target second candidate clip are generated during preprocessing. For the first candidate clip, since the first candidate clip is generated based on a single static image and has a relatively simple content, the label of the first candidate clip can be the name of the promotion object corresponding to the first candidate clip.

[0204] For the second candidate segment, since its content is relatively rich, in order to ensure the accuracy of its label, in some embodiments of the present application, for each second candidate segment, at least one key video frame of the second candidate segment is extracted (this process is the aforementioned abstract), and the key video frame and the second prompt word are input into the visual large model, so that the visual large model outputs the initial description text of each key video frame according to the second prompt word, and the initial description text of each key video frame is spliced ​​and determined as the initial description text of the corresponding second candidate segment. Among them, the pre-second prompt word can include multiple questions to guide the visual large model to generate the initial description text through the way of thinking chain, so as to fully describe the content of the second candidate segment through the initial description text. Among them, the key video frame can be average extraction, random extraction, etc., which is not specifically limited in this application. Multiple questions include but are not limited to "whether it contains characters", "if it contains characters, whether the characters have specific actions, whether they speak", "if there are specific actions, what are the actions (i.e. event detection)", "if speaking, what is the content of the speech, what is the emotion of the speech (i.e. sentiment analysis)", "whether it contains subtitles", "whether it contains promotion objects (such as food)".

[0205] Considering that the second candidate video clips may contain clips that are not related to the subject to be promoted, in order to improve the quality of the video library, in some embodiments, after obtaining the initial description text of each second candidate clip, a third prompt word is generated for each second candidate clip based on the initial description text of the second candidate clip and the third prompt template, and the large language model is called to determine whether the second candidate clip is related to the subject to be promoted based on the third prompt word, and the determination result is output. According to the determination result, the second candidate clips that are not related to the subject to be promoted are deleted to obtain the target second candidate clip (this process is the aforementioned fine tailoring). And according to the initial description text of the target second candidate clip, the label of the target second candidate clip is determined.

[0206] Furthermore, considering that when the target second candidate segment contains subtitles but no characters, in order to increase the application scenarios of the target second candidate segment, the subtitles contained in the target second candidate segment are also deleted so as to add different voices to it during video synthesis. In order to ensure the integrity and smoothness of the picture after deleting the subtitles, in some embodiments, the target area after deleting the subtitles can also be repaired to ensure a smooth transition between the target area and elements outside the target area.

[0207] Furthermore, in order to avoid promoting the situation where some video clips in the video are larger in size and some video clips are smaller in size, in some embodiments of the present application, for each candidate clip carrying a label (i.e., the first candidate clip and the target second candidate clip), the candidate clip and the target size are input into the extension model for size adjustment, and a video clip of the target size is input (this process is the aforementioned video extension).

[0208] Therefore, by preprocessing each candidate segment, the quality of each video segment and the reusability of multiple scenes are improved on the basis of obtaining the identification of each video segment.

[0209] When the target second candidate segment includes subtitles and characters, in order to ensure the accuracy of the label, the above-mentioned determining the label of the target second candidate segment according to the initial description text of the target second candidate segment may include:

[0210] If it is determined based on the initial description text that the target second candidate segment contains a speaking person and subtitles, the subtitles and the voice text of the person are extracted; based on the extracted subtitles and voice text, the subtitles in the initial description text are corrected, and the initial description text containing the corrected subtitles is determined as the first description text of the target second candidate segment; based on the first description text, a label of the target second candidate segment is generated.

[0211] In some embodiments, if it is determined that the initial description text includes description information representing the existence of speaking characters and subtitles, it is determined that the target second candidate segment contains speaking characters and subtitles. Optical Character Recognition (OCR) is performed on each video frame of the target second candidate segment to extract the subtitles in the video frame. And Automatic Speech Recognition (ASR) is performed on the target second candidate segment to obtain a speech text. If it is determined that the subtitles in the initial description text are different from the extracted subtitles or speech text, the subtitles in the initial description text are corrected according to the extracted subtitles and speech text to obtain a first description text containing accurate subtitles. In some embodiments, the first description text can be determined as a label of the target second candidate segment, and the node information of the first description text and the target video node can also be determined as a label of the target second candidate segment. Among them, the target video node is the video node corresponding to the target second candidate segment in the historical promotion video.

[0212] By correcting the subtitles in the initial description text, it is ensured that when the subtitles in the initial description text are re-dubbed, the corresponding audio is consistent with the lip movements of the characters in the video clip. It also ensures that when synthesizing the promotional video, the target video clip can be accurately obtained based on the label of the video clip.

[0213] Step 1002, call the large language model to generate multiple second templates, build an audio library based on the second templates and historical promotional videos, and any script template includes multiple second description texts.

[0214] In some embodiments, Fig.12 As shown, multiple high-quality historical promotional videos can be analyzed and summarized in advance to obtain multiple promotional scenarios contained in the high-quality promotional videos, thereby generating multiple first templates based on the obtained promotional scenarios. In addition, a fourth prompt template is set according to the obtained promotional scenarios, and a fourth prompt word is generated according to different promotion needs and the fourth prompt template, and the fourth prompt word is input into the large language model so that the large language model generates a second template that meets the promotion needs according to the fourth prompt word. In order to improve the quality of the second template generated by the large language model, a high-quality second template can also be determined based on the interaction data obtained by the aforementioned embedding point (for example, the second template corresponding to the promotional video with a number of interaction data greater than a preset number is determined as a high-quality second template), and the high-quality second template is used to perform Few-shot supervised fine-tuning on the large language model. And the corresponding second template is generated according to the promotion needs using the fine-tuned large language model. As shown in FIG. Fig.12 As shown, in some implementations, multiple second templates can also be manually compiled according to promotion needs.

[0215] In order to obtain rich and diverse audio, in some implementations, such as Fig.12 As shown, the audio library can be constructed based on the sound reproduction and emotion enhancement. Specifically, the audio library constructed based on the second template and the historical promotion video can include:

[0216] According to the second template, at least one first speech carrying the label is generated; according to the target second candidate segment, at least one second speech carrying the label is generated; the music generation model is called to generate multiple background music carrying the labels; the first speech, the second speech, and the background music are determined as an audio library.

[0217] Among them, generating at least one first voice carrying a label according to the second template may include: for any second template, dividing multiple second description texts in the second template into at least one text subset, and at least one text subset corresponds one-to-one to at least one promotion scenario; for any text subset, performing speech synthesis on the second description text in the text subset according to the first language feature to obtain the first voice with the first language feature, and determining the second description text and the first language feature as the label of the first voice.

[0218] Specifically, when generating the first speech according to the second template, the semantic features of the plurality of second description texts in the second template are first parsed, at least one promotion scenario described is determined according to the semantic features, and the plurality of second description texts are divided into at least one text subset corresponding to at least one promotion scenario. For each text subset, the second description text and the first language feature in the text subset are input into the pre-trained speech synthesis model for speech synthesis processing to obtain the first speech of the first language feature. Among them, the training method, structure and processing process of the speech synthesis model can refer to the relevant technology, and no specific limitation is made in this application. The first language feature may include language type (such as local dialect, English, Japanese, etc., to achieve sound reproduction), language emotion (such as happy, surprised, etc., to achieve emotion enhancement), etc. In some embodiments, for each text subset, multiple first speech corresponding to different first language features can be synthesized. In order to facilitate the subsequent accurate acquisition of the first speech, the second description text and the first language feature are determined as the label of the first speech.

[0219] Thus, according to the promotion scenario described by the second description text, a text subset is determined, and speech synthesis is performed on the second description text in the text subset, which ensures that when the promotion video is synthesized based on the second template, the speech in the promotion video matches the promotion scenario. Furthermore, since multiple speech corresponding to the second template is generated when the audio library is constructed, each time a promotion video is synthesized based on the second template, there is no need to repeat the speech synthesis operation, but the corresponding speech in the audio library can be reused, which is conducive to improving the synthesis rate of the promotion video.

[0220] Furthermore, the above-mentioned generation of at least one second speech carrying a label based on the target second candidate segment may include: for any target second candidate segment, if it is determined that the target second candidate segment contains the initial speech, extracting the initial speech; transforming the initial speech according to the second language feature to obtain the second speech with the second language feature, and determining the text corresponding to the initial speech and the second language feature as the label of the second speech.

[0221] Considering that in actual applications, for the same video clip, different users may have different needs for voice, for example, some users want the voice of Sichuan dialect, and some users want the voice of Northeastern Laotie. Based on this, in one or more embodiments of the present application, for each target second candidate segment containing the initial voice, the initial voice in the target second candidate segment is extracted, and the initial voice is transformed according to the second language feature to obtain the second voice with the second language feature. In order to facilitate the subsequent accurate acquisition of the second voice, the text corresponding to the initial voice and the second language feature are determined as the label of the second voice. Among them, the second language feature may include language type (such as local dialect, English, Japanese, etc.), language emotion (such as happy, surprised, etc.), etc. Therefore, by transforming the initial audio in the target second candidate segment, the diversity of voice can be enriched to meet different user needs.

[0222] Furthermore, the above-mentioned calling of the music generation model to generate multiple background music with labels may include: calling the music generation model to generate corresponding background music according to the input music type. The music type may include dynamic, soothing, cheerful, etc., so as to obtain rich and interesting background music. The specific structure, processing process and training process of the music generation model can all refer to the relevant technology, and this application does not make specific limitations on this.

[0223] Step 1003, determine the video library and the audio library as the material library.

[0224] By constructing a video library and an audio library, we obtain a material library, which provides rich materials for the synthesis of subsequent promotional videos, thereby improving the diversity of promotional videos.

[0225] Corresponding to the above-mentioned embodiment of the method for synthesizing a promotional video, the present application also provides an embodiment of a device for synthesizing a promotional video. Fig.14 FIG. 1 is a schematic diagram of a structure of a promotional video synthesis device according to an exemplary embodiment. The device can be configured to Figure 1 The device is used to execute the promotion video synthesis method provided in any of the above embodiments, such as Fig.14 As shown, the synthesis device of the promotional video includes:

[0226] The receiving module 201 is used to receive a video synthesis request sent by a client, the video synthesis request includes material information of a material to be synthesized, the material to be synthesized includes multiple target video segments, at least one of the multiple target video segments includes promotion information of a subject to be promoted;

[0227] The acquisition module 202 is used to acquire the material to be synthesized from the material library according to the material information;

[0228] A division module 203, used to divide the material to be synthesized into at least one material combination according to a preset combination rule;

[0229] The synthesis module 204 is used to synthesize the materials in any material combination to obtain a promotional video corresponding to the material combination.

[0230] In some implementations, the material information includes segment identifiers of each target video segment. Accordingly, the acquisition module 202 is specifically configured to:

[0231] The video clips corresponding to the clip marks in the material library are determined as target video clips;

[0232] If the material information also includes audio attributes, obtaining at least one corresponding target audio from the material library according to the audio attributes;

[0233] A plurality of target video clips and at least one target audio are determined as materials to be synthesized.

[0234] In some implementations, the material information includes template identifiers corresponding to a plurality of video clips. Accordingly, the acquisition module 202 is specifically configured to:

[0235] Determine a template corresponding to the template identifier in the preset multiple templates as a reference video template;

[0236] If the material information also includes audio attributes, the material to be synthesized is obtained from the material library according to the reference video template and the audio attributes.

[0237] In some implementations, the acquisition module 202 is further specifically configured to:

[0238] Determine a plurality of video nodes included in the reference video template, wherein the plurality of video nodes correspond one-to-one to the plurality of video clips, and an arrangement order of the plurality of video nodes is used to represent a playback order of the plurality of video clips;

[0239] According to the arrangement order, each video node is determined as a first node, and at least one target material matching the first node and the audio attribute is obtained from the material library;

[0240] The target materials corresponding to the multiple video nodes are determined as materials to be synthesized.

[0241] In some implementations, the material library includes a video library and an audio library, and any video clip in the video library carries a tag, which is used to characterize the video content in the video clip. Accordingly, the acquisition module 202 is further specifically used to:

[0242] Determine matching text, sub-audio attributes, and multiple candidate video segments corresponding to the first node;

[0243] Matching the matching text with the labels of each candidate video segment, and determining the candidate video segment corresponding to at least one label that successfully matches as the target video segment;

[0244] According to the sub-audio attribute, obtaining at least one corresponding target audio from the audio library;

[0245] The target video clip and target audio are determined as target materials.

[0246] In some implementations, the tag includes a first description text, and the first description text is used to describe the content of the video segment. Accordingly, the acquisition module 202 is further specifically used to:

[0247] For any candidate video segment, call the pre-trained matching model to extract the first description text of the candidate video segment and the first semantic feature of the matching text respectively;

[0248] Determine a first score according to the first semantic feature, the first score being used to characterize the degree of relevance between the first description text and the matching text;

[0249] The tag corresponding to the target first score is determined as a successfully matched tag, and the target first score is at least one first score that satisfies the first preset condition.

[0250] In some implementations, the partitioning module 203 is specifically configured to:

[0251] Acquire at least one tree structure formed by target video segments corresponding to the plurality of video nodes, wherein any tree structure includes at least one branch, and any branch includes a plurality of target video segments, and the plurality of target video segments correspond to the plurality of video nodes one by one;

[0252] Determine a second score corresponding to each branch according to the target first score, where the second score is used to characterize the combined quality of the multiple target video clips included in the corresponding branch;

[0253] Determine a branch corresponding to a target second score as a target branch, where the target second score is at least one second score that satisfies a second preset condition;

[0254] For any target branch, the target video clip in the target branch and the target audio corresponding to the target branch are determined as a material combination.

[0255] In some implementations, the acquisition module 202 is further specifically configured to:

[0256] When the reference video template is the first template, multiple promotion scenes included in the first template are determined as video nodes;

[0257] When the reference video template is the second template, multiple promotion scenes described by the second template are determined as video nodes, the second template includes multiple second description texts, and the second description texts are used as voice content of the promotion video.

[0258] In some implementations, the acquisition module 202 is further specifically configured to:

[0259] When the reference video template is the first template, determining the matching text according to the video category corresponding to the first node, the arrangement position of the first node in the plurality of video nodes, and the label;

[0260] In the case where the reference video template is the second template, a pre-trained large language model is called to determine a matching text according to at least one second description text corresponding to the first node in the second template.

[0261] In some implementations, the material information also includes the attributes of the object to be promoted; accordingly, the acquisition module 201 is further specifically used to:

[0262] If the video category is the target category and the ranking position is the first, the attribute of the object to be promoted is determined as the matching text, and the target category indicates that the video content of the first node contains the object to be promoted;

[0263] If the video category is the target category and the ranking position is not the first, the matching text is determined according to the attributes of the object to be promoted and the target tag; the target tag is the tag that meets the third preset condition among the tags of the currently obtained target video clips;

[0264] If the video category is not the target category and the ranking position is the first, the attribute of the object to be promoted is determined as the matching text;

[0265] If the video category is not the target category and the ranking position is not the first, the matching text is determined according to the target label.

[0266] In some embodiments, the apparatus further comprises a first determining module, an adding module, and a saving module;

[0267] The first determination module is used for, before the acquisition module 202 determines the matching text according to the target tag, if the arrangement position is the second position, then the target video segments corresponding to the first video node are respectively determined as the root nodes of the corresponding tree structure, and the tags of the target video segments are respectively determined as the target tags; if the arrangement position is any position after the second position, then the tags of the target video segments in each branch of the tree structure are respectively determined as the target tags;

[0268] An adding module, configured to add the target video segment to a branch corresponding to the target tag after the acquisition module 202 determines the target video segment, if the target video segment is obtained according to the matching text corresponding to the target tag;

[0269] The saving module is used to save the current tree structure when the arrangement position is the last one, and the saved tree structure is used to determine the material combination.

[0270] In some implementations, the acquisition module 202 is further specifically configured to:

[0271] Generate a first prompt word according to at least one second description text and a first prompt template, where the first prompt word is used to prompt a rule for the large language model to extract key information of the second description text;

[0272] Input the first prompt word into the large language model to obtain key information;

[0273] Identify key information as matching text.

[0274] In some implementations, the reference video template is the second template, and accordingly, the division module 203 is further specifically used for:

[0275] According to the arrangement order, each video node is determined as the second node;

[0276] In the case where the second node is the first video node, each target video segment corresponding to the second node is determined as a root node of the corresponding tree structure;

[0277] In the case where the second node is not the first video node, determining a segment to be matched in the target video segment corresponding to the second node;

[0278] Determine a third score between the to-be-matched segment and each segment set respectively, each segment set corresponds to each branch of the current tree structure one by one, any segment set includes the target video segment in the corresponding branch, and the third score represents the correlation between the to-be-matched segment and the corresponding first segment;

[0279] The to-be-matched segment is determined as a child node of a target branch corresponding to a target segment set, and the child node is added to the target branch, wherein the target first segment is a segment set corresponding to a third score that satisfies a fourth preset condition.

[0280] In some implementations, the partitioning module 203 is further specifically configured to:

[0281] For any segment set, determine a target label according to labels of each target video segment included in the segment set;

[0282] Calling the matching model to extract the second semantic features of the label of the to-be-matched segment and the target label respectively;

[0283] A third score of the to-be-matched segment and the segment set is determined according to the second semantic feature.

[0284] In some implementations, the synthesis module 204 is specifically configured to:

[0285] Generate a video synthesis protocol according to the access address of each material in the material combination in the material library;

[0286] The video synthesis engine is called to synthesize the various materials according to the video synthesis protocol to obtain the corresponding promotional video.

[0287] In some implementations, the apparatus further includes a delivery module and an update module, and the synthesis module 204 is further configured to:

[0288] Adding a tracking point to each target video segment, wherein the tracking point is used to collect interaction data between the user and the corresponding target video segment;

[0289] A delivery module, which is used for delivering the promotional video corresponding to the material combination to a designated platform after the synthesis module obtains the promotional video;

[0290] An updating module, for updating the selection weight of the corresponding target video segment according to the interaction data if the interaction data is obtained;

[0291] The acquisition module is used to acquire multiple target video clips from the material library according to the selection weights.

[0292] In some embodiments, the apparatus further comprises a first building module, a second building module, and a second determining module;

[0293] The first construction module is used to construct a video library according to multiple target images and multiple historical promotional videos of the subject to be promoted before the receiving module 201 receives the video synthesis request sent by the client, wherein the target image includes the promotion object of the subject to be promoted;

[0294] A second construction module is used to call the large language model to generate multiple second templates, build an audio library according to the second templates and historical promotion videos, and any script template includes multiple second description texts;

[0295] The second determining module is used to determine the video library and the audio library as a material library.

[0296] In some embodiments, the first building block is specifically used to:

[0297] For any target image, calling the video macro model and / or the video diffusion model to generate at least one corresponding first candidate segment;

[0298] For any historical promotion video, the historical promotion video is divided into a plurality of second candidate segments, and the plurality of second candidate segments correspond to the plurality of video objects one by one.

[0299] The first candidate segment and the second candidate segment are preprocessed to obtain a video library, which includes a plurality of video segments carrying labels.

[0300] In some embodiments, the first building block is further specifically configured to:

[0301] For any first candidate segment, generating a label of the first candidate segment according to a promotion object corresponding to the first candidate segment;

[0302] For any second candidate segment, calling the visual big model to generate an initial description text of the second candidate segment, where the initial description text is used to describe the content of the second candidate segment;

[0303] For any target second candidate segment, a label of the target second candidate segment is determined according to the initial description text of the target second candidate segment; the target second candidate segment is a second candidate segment related to the subject to be promoted and determined according to the initial description text; and

[0304] If it is determined according to the initial description text that the target second candidate segment contains subtitles but does not contain characters, deleting the subtitles;

[0305] For any candidate segment carrying a label, the extension model is called to adjust the size of the candidate segment to obtain a video segment of the target size.

[0306] In some embodiments, the first building block is further specifically configured to:

[0307] If it is determined according to the initial description text that the target second candidate segment includes a speaking person and subtitles, extracting the subtitles and the voice text of the person;

[0308] Correcting the subtitles in the initial description text according to the subtitles and the speech text, and determining the initial description text including the corrected subtitles as the first description text of the target second candidate segment;

[0309] A label of a target second candidate segment is generated according to the first description text.

[0310] In some embodiments, the second building block is specifically used to:

[0311] Generate at least one first speech carrying a label according to the second template;

[0312] Generate at least one second speech carrying a label according to the target second candidate segment;

[0313] Call the music generation model to generate multiple background music with tags;

[0314] The first voice, the second voice, and the background music are determined as an audio library.

[0315] In some embodiments, the second building block is further specifically configured to:

[0316] For any second template, multiple second description texts in the second template are divided into at least one text subset, and the at least one text subset corresponds to at least one promotion scenario one by one;

[0317] For any text subset, speech synthesis is performed on the second description text in the text subset according to the first language feature to obtain a first speech with the first language feature, and the second description text and the first language feature are determined as labels of the first speech.

[0318] In some embodiments, the second building block is further specifically configured to:

[0319] For any target second candidate segment, if it is determined that the target second candidate segment contains the initial speech, extracting the initial speech;

[0320] The initial speech is transformed according to the second language feature to obtain a second speech with the second language feature, and the text corresponding to the initial speech and the second language feature are determined as a label of the second speech.

[0321] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components illustrated as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application. A person of ordinary skill in the art may understand and implement it without creative work.

[0322] Corresponding to the aforementioned embodiment of the method for synthesizing a promotional video, the present application also provides an embodiment of a system for synthesizing a promotional video. Fig.15 FIG. 1 is a schematic diagram of a structure of a promotional video synthesis system according to an exemplary embodiment. Fig.15 As shown, the synthesis system of the promotional video includes: a client 301 and a server 302;

[0323] The client 301 is used to send a video synthesis request to the server 302 according to the material information of the material to be synthesized input by the user; the material to be synthesized includes multiple target video clips, at least one of the multiple target video clips includes the promotion information of the subject to be promoted;

[0324] The server 302 is used to receive a video synthesis request, obtain the material to be synthesized from the material library according to the material information; divide the material to be synthesized into at least one material combination according to a preset combination rule; for any material combination, synthesize each material in the material combination to obtain a promotional video corresponding to the material combination.

[0325] The promotional video synthesis system provided in the embodiment of the present application and the promotional video synthesis method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0326] It is understandable that the division of the above modules is only a division of logical functions. In actual implementation, the functions of the above modules can be integrated into hardware entities. For example, the function of the aforementioned receiving module 201 can be integrated into a transceiver, and the functions of the acquisition module 202, the division module 203 and the synthesis module 204 can be integrated into a processor.

[0327] Based on this, some embodiments of the present application further provide an electronic device corresponding to the method for synthesizing a promotional video provided in the aforementioned embodiment, and the electronic device can be configured as follows: Figure 1 The server shown is used to execute the above-mentioned promotional video synthesis method.

[0328] Fig.16 4 is a hardware structure diagram of an electronic device according to an exemplary embodiment, the electronic device includes: a communication interface 401, a processor 402, a memory 403 and a bus 404; wherein the communication interface 401, the processor 402 and the memory 403 communicate with each other through the bus 404. The processor 402 can execute the above-described promotional video synthesis method by reading and executing the machine executable instructions corresponding to the control logic of the promotional video synthesis method in the memory 403. The specific content of the method is referred to the above embodiment, which will not be repeated here.

[0329] The memory 403 mentioned in the embodiment of the present application can be any electronic, magnetic, optical or other physical storage device, and can contain storage information, such as executable instructions, data, etc. Specifically, the memory 403 can be RAM (Random Access Memory), flash memory, storage drive (such as hard disk drive), any type of storage disk (such as optical disk, etc.), or similar storage medium, or a combination thereof. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 401 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0330] The bus 404 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 403 is used to store a program, and the processor 402 executes the program after receiving an execution instruction.

[0331] The processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 402. The above-mentioned processor 402 can be a general-purpose processor, including a network processor (Network Processor, NP), a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware controls, etc. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute.

[0332] The electronic device provided in the embodiment of the present application and the method for synthesizing the promotional video provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented therein.

[0333] The embodiments of the present application also provide a computer-readable storage medium corresponding to the method for synthesizing a promotional video provided in the aforementioned embodiments, on which a computer program (i.e., a program product) is stored. When the computer program is executed by a processor, the method for synthesizing a promotional video provided in any of the aforementioned embodiments will be executed.

[0334] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0335] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the method for synthesizing a promotional video provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0336] The embodiments of the present application also provide a computer program product corresponding to the method for synthesizing a promotional video provided in the aforementioned embodiments. The computer program product includes a computer program, which is executed by a processor to implement the method for synthesizing a promotional video provided in the aforementioned embodiments.

[0337] The computer program product provided by the above-mentioned embodiments of the present application and the method for synthesizing a promotional video provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0338] The solutions in this application specification and embodiments, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0339] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0340] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0341] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for synthesizing a promotional video, characterized in that: The method comprises: receiving a video synthesis request sent by a client, wherein the video synthesis request includes material information of a material to be synthesized, wherein the material to be synthesized includes a plurality of target video segments, and at least one target video segment of the plurality of target video segments includes promotion information of a subject to be promoted; According to the material information, obtaining the material to be synthesized from a material library; According to a preset combination rule, the materials to be synthesized are divided into at least one material combination; For any material combination, each material in the material combination is synthesized to obtain a promotional video corresponding to the material combination.

2. The method according to claim 1, characterized in that The material information includes a segment identifier of each target video segment, and acquiring the material to be synthesized from a material library according to the material information includes: Determine the video segment corresponding to each segment identifier in the material library as the target video segment; If the material information further includes audio attributes, acquiring at least one corresponding target audio from the material library according to the audio attributes; The multiple target video clips and the at least one target audio are determined as the materials to be synthesized.

3. The method according to claim 1, characterized in that The material information includes template identifiers corresponding to the multiple video clips, and acquiring the material to be synthesized from a material library according to the material information includes: Determine a template corresponding to the template identifier in a plurality of preset templates as a reference video template; If the material information also includes audio attributes, the material to be synthesized is obtained from the material library according to the reference video template and the audio attributes.

4. The method according to claim 3, characterized in that: The acquiring the material to be synthesized from the material library according to the reference video template and the audio attribute includes: Determine a plurality of video nodes included in the reference video template, wherein the plurality of video nodes correspond one-to-one to a plurality of video clips, and an arrangement order of the plurality of video nodes is used to represent a playback order of the plurality of video clips; According to the arrangement order, each video node is determined as a first node, and at least one target material matching the first node and the audio attribute is acquired from the material library; The target materials respectively corresponding to the multiple video nodes are determined as the materials to be synthesized.

5. The method according to claim 4, characterized in that The material library includes a video library and an audio library, and any video clip in the video library carries a tag, and the tag is used to characterize the video content in the video clip; The acquiring at least one target material matching the first node and the audio attribute from the material library comprises: Determining matching text, sub-audio attributes, and multiple candidate video segments corresponding to the first node; Matching the matching text with the tags of each candidate video segment, and determining the candidate video segment corresponding to at least one tag that successfully matches as the target video segment; According to the sub-audio attribute, acquiring at least one corresponding target audio from the audio library; The target video clip and the target audio are determined as the target material.

6. The method according to claim 5, characterized in that The tag includes a first description text, and the first description text is used to describe the content of the video segment. The matching text is matched with the tag of each candidate video segment, including: For any candidate video segment, calling a pre-trained matching model to respectively extract the first description text of the candidate video segment and the first semantic feature of the matching text; Determine a first score according to the first semantic feature, where the first score is used to represent the degree of relevance between the first description text and the matching text; A tag corresponding to a target first score is determined as a successfully matched tag, wherein the target first score is at least one first score that satisfies a first preset condition.

7. The method according to claim 6, characterized in that The step of dividing the material to be synthesized into at least one material combination includes: Acquire at least one tree structure formed by the target video segments corresponding to the multiple video nodes, wherein any tree structure includes at least one branch, and any branch includes multiple target video segments, and the multiple target video segments correspond to the multiple video nodes one by one; Determine, according to the target first score, a second score corresponding to each branch, where the second score is used to characterize the combined quality of the multiple target video clips included in the corresponding branch; Determine a branch corresponding to a target second score as a target branch, wherein the target second score is at least one second score that satisfies a second preset condition; For any target branch, a target video segment in the target branch and a target audio corresponding to the target branch are determined as a material combination.

8. The method according to any one of claims 5 to 7, characterized in that: The determining the matching text corresponding to the first node includes: In the case where the reference video template is the first template, determining the matching text according to the video category corresponding to the first node, the arrangement position of the first node in the plurality of video nodes, and the label; In the case where the reference video template is the second template, a pre-trained large language model is called to determine the matching text according to at least one second description text corresponding to the first node in the second template.

9. The method according to any one of claims 1 to 7, characterized in that: Before receiving the video synthesis request sent by the client, the method further includes: Building a video library according to a plurality of target images and a plurality of historical promotional videos of the subject to be promoted, wherein the target image includes the promotion object of the subject to be promoted; Calling the large language model to generate multiple second templates, building an audio library based on the second templates and the historical promotion video, any script template includes multiple second description texts; The video library and the audio library are determined as the material library.

10. A promotional video synthesis system, characterized in that: The system comprises: a client and a server; The client is used to send a video synthesis request to the server according to the material information of the material to be synthesized input by the user; the material to be synthesized includes a plurality of target video segments, at least one of the plurality of target video segments includes the promotion information of the subject to be promoted; The server is used to receive the video synthesis request, obtain the material to be synthesized from the material library according to the material information; divide the material to be synthesized into at least one material combination according to a preset combination rule; for any material combination, synthesize each material in the material combination to obtain a promotional video corresponding to the material combination.

Images