Video synthesis method, device, computer equipment and storage medium

Abstract

The present disclosure provides a video synthesis method, apparatus, computer equipment and storage medium, wherein the method comprises: obtaining a video to be synthesized and text information corresponding to the video to be synthesized; generating a target template matching the video to be synthesized based on the video to be synthesized and the text information corresponding to the video to be synthesized; wherein the target template comprises a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format matching the text information determined based on the background image and the first text format, the second text format comprising font color information; synthesizing the target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a video synthesis method, apparatus, computer equipment and storage medium. Background Art

[0002] In recent years, video platforms and short video platforms have become more and more popular, and more and more people like to browse videos posted by video producers on the Internet. When posting videos, producers usually make videos according to a certain ratio and then post them to various video platforms. However, different video platforms have different standards for the display ratio of videos. If the video ratio of the video posted by the video producer does not meet the standards of the video platform, then the video will have black edges in the upper and lower or left and right directions when it is displayed, affecting the display effect of the video.

[0003] In the related art, a template is usually selected from multiple fixed templates according to the color of the video, and the template is merged with the video to be released to generate a video that meets the standard ratio of the video platform. However, the display style of the video synthesized using the fixed template is relatively single, and the display effect of the template selected only according to the color of the video and the video to be released after being merged is not good. Therefore, how to better synthesize videos has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a video synthesis method, apparatus, computer equipment and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a video synthesis method, including:

[0006] Acquire the video to be synthesized and text information corresponding to the video to be synthesized;

[0007] Based on the video to be synthesized and the text information corresponding to the video to be synthesized, a target template matching the video to be synthesized is generated; wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format matching the text information determined based on the background image and the first text format, wherein the second text format includes font color information;

[0008] A target video is synthesized based on the video to be synthesized, text information corresponding to the video to be synthesized, and the target template.

[0009] In a possible implementation manner, the method further includes determining a first text format matching the text information according to the following method:

[0010] Determining attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type;

[0011] Based on the attribute information of the text information and a plurality of preset text formats, a first text format matching the text information is determined from the plurality of preset text formats.

[0012] In a possible implementation manner, the determining, based on the attribute information of the text information and a plurality of pre-set text formats, a first text format matching the text information from the plurality of pre-set text formats includes:

[0013] Based on the number of text types included in the text information, determining a candidate text format that matches the number of text types from a plurality of pre-set text formats;

[0014] Based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, a first text format matching the text information is determined.

[0015] In a possible implementation manner, determining the first text format matching the text information based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format includes:

[0016] Based on the first number of characters and the second number of characters, determining the character number distance between each candidate text format and the text information; wherein the character number distance is used to represent the difference information between the second number of characters corresponding to each text type of the candidate text format and the first number of characters corresponding to each text type of the text information;

[0017] The first text format is determined from the candidate text formats based on the distance in number of characters between each candidate text format and the text information.

[0018] In a possible implementation manner, the first text format includes a font size;

[0019] After determining a first text format matching the text information from the plurality of preset text formats, the method further includes:

[0020] For any text type, when the number of first characters of the text type included in the text information exceeds the number of second characters of the text type included in the first text format, the font size of the text information is adjusted.

[0021] In a possible implementation manner, the method further includes determining a background image matching the video to be synthesized according to the following method:

[0022] Extracting first feature information of the video to be synthesized, and obtaining second feature information of multiple candidate background images;

[0023] Based on the first feature information and the second feature information, a background image matching the video to be synthesized is screened out from the multiple candidate background images.

[0024] In a possible implementation manner, the extracting the first feature information of the video to be synthesized includes:

[0025] Extracting first feature information of the video to be synthesized based on a pre-trained first network model;

[0026] The second feature information of the candidate background image is extracted based on a pre-trained second network model, and the first network model and the second network model are models trained based on contrastive learning.

[0027] In a possible implementation manner, the first text format includes display positions corresponding to characters of each text type; the second text format includes font color information corresponding to characters of each text type;

[0028] The method further includes determining a second text format matching the text information according to the following method:

[0029] For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image;

[0030] Based on the color information of the area image and a plurality of preset color information pairs, font color information corresponding to the characters of the text type is determined, wherein the color information pair includes font color information and first background color information corresponding to the font color information.

[0031] In a possible implementation manner, determining the font color information corresponding to the characters of the text type based on the color information of the regional image and a plurality of preset color information pairs includes:

[0032] Determining second background color information corresponding to the regional image based on color information of each pixel contained in the regional image;

[0033] The second background color information is compared with the first background color information in the multiple color information pairs, the target background color information is determined from the first background color information, and the font color information in the color information pair where the target background color information is located is used as the font color information corresponding to the characters of the text type.

[0034] In a possible implementation manner, synthesizing a target video based on the video to be synthesized, text information corresponding to the video to be synthesized, and the target template includes:

[0035] The video to be synthesized is added to the background image, and the text information is added according to the first text format and the second text format to obtain the target video.

[0036] In a second aspect, the present disclosure also provides a video synthesis device, including:

[0037] An acquisition module, used to acquire the video to be synthesized and text information corresponding to the video to be synthesized;

[0038] A generating module, configured to generate a target template matching the video to be synthesized based on the video to be synthesized and text information corresponding to the video to be synthesized; wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format matching the text information determined based on the background image and the first text format, wherein the second text format includes font color information;

[0039] The synthesis module is used to synthesize a target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template.

[0040] In a possible implementation manner, the generating module is further configured to determine a first text format matching the text information according to the following method:

[0041] Determining attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type;

[0042] Based on the attribute information of the text information and a plurality of preset text formats, a first text format matching the text information is determined from the plurality of preset text formats.

[0043] In a possible implementation manner, the generating module, when determining the first text format matching the text information from the plurality of pre-set text formats based on the attribute information of the text information and the plurality of pre-set text formats, is configured to:

[0044] Based on the number of text types included in the text information, determining a candidate text format that matches the number of text types from a plurality of pre-set text formats;

[0045] Based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, a first text format matching the text information is determined.

[0046] In a possible implementation manner, the generating module, when determining the first text format matching the text information based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, is configured to:

[0047] Based on the first number of characters and the second number of characters, determining the character number distance between each candidate text format and the text information; wherein the character number distance is used to represent the difference information between the second number of characters corresponding to each text type of the candidate text format and the first number of characters corresponding to each text type of the text information;

[0048] The first text format is determined from the candidate text formats based on the distance in number of characters between each candidate text format and the text information.

[0049] In a possible implementation manner, the first text format includes a font size;

[0050] After determining a first text format matching the text information from the plurality of preset text formats, the generating module is further configured to:

[0051] For any text type, when the number of first characters of the text type included in the text information exceeds the number of second characters of the text type included in the first text format, the font size of the text information is adjusted.

[0052] In a possible implementation manner, the generating module is further configured to determine a background image matching the video to be synthesized according to the following method:

[0053] Extracting first feature information of the video to be synthesized, and obtaining second feature information of multiple candidate background images;

[0054] Based on the first feature information and the second feature information, a background image matching the video to be synthesized is screened out from the multiple candidate background images.

[0055] In a possible implementation manner, the generating module, when extracting the first feature information of the video to be synthesized, is used to:

[0056] Extracting first feature information of the video to be synthesized based on a pre-trained first network model;

[0057] The second feature information of the candidate background image is extracted based on a pre-trained second network model, and the first network model and the second network model are models trained based on contrastive learning.

[0058] In a possible implementation manner, the first text format includes display positions corresponding to characters of each text type; the second text format includes font color information corresponding to characters of each text type;

[0059] The generating module is further configured to determine a second text format matching the text information according to the following method:

[0060] For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image;

[0061] Based on the color information of the area image and a plurality of preset color information pairs, font color information corresponding to the characters of the text type is determined, wherein the color information pair includes font color information and first background color information corresponding to the font color information.

[0062] In a possible implementation manner, the generating module, when determining the font color information corresponding to the characters of the text type based on the color information of the regional image and a plurality of preset color information pairs, is used to:

[0063] Determining second background color information corresponding to the regional image based on color information of each pixel contained in the regional image;

[0064] The second background color information is compared with the first background color information in the multiple color information pairs, the target background color information is determined from the first background color information, and the font color information in the color information pair where the target background color information is located is used as the font color information corresponding to the characters of the text type.

[0065] In a possible implementation manner, the synthesis module, when synthesizing the target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template, is used to:

[0066] The video to be synthesized is added to the background image, and the text information is added according to the first text format and the second text format to obtain the target video.

[0067] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0068] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0069] The video synthesis method, device, computer equipment and storage medium provided by the embodiments of the present disclosure can generate a target template matching the video to be synthesized based on the video to be synthesized and the text information corresponding to the video to be synthesized after obtaining the video to be synthesized and the text information corresponding to the video to be synthesized, wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format of the text information determined based on the background image and the first text format, wherein the second text format includes font color information, and then the target video can be synthesized based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template. Therefore, this method does not require a large number of templates to be set in advance, and since the target template is generated based on the video to be synthesized and the text information, the target template generated according to this method can better match the video to be synthesized, thereby improving the display effect of the synthesized target video.

[0070] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0072] Figure 1 A flow chart of a video synthesis method provided by an embodiment of the present disclosure is shown;

[0073] Figure 2 A schematic diagram showing a process of determining a background image provided by an embodiment of the present disclosure;

[0074] Figure 3 A schematic diagram showing a process of training a first network model and a second network model provided by an embodiment of the present disclosure;

[0075] Figure 4 A schematic diagram showing a display style of a target video provided by an embodiment of the present disclosure;

[0076] Figure 5 A schematic diagram showing the overall process of a video synthesis method provided by an embodiment of the present disclosure;

[0077] Figure 6 A schematic diagram of the architecture of a video synthesis device provided by an embodiment of the present disclosure is shown;

[0078] Figure 7 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0080] When users publish videos, they usually make them according to a certain ratio and then publish them to various video platforms. However, different video platforms have different standards for the display ratio of videos. For example, some platforms display videos in vertical screens, while some platforms display videos in horizontal screens. The display ratios of videos are different. If the video ratio of the video released by the video producer does not meet the standards of the video platform, then the video will have black edges in the upper and lower or left and right directions when it is displayed, affecting the display effect of the video.

[0081] In the related art, a template is usually selected from multiple fixed templates according to the color of the video, and the template is merged with the video to be released to generate a video that meets the standard ratio of the video platform. However, this method requires the design of a large number of templates including different background images and font formats, and the background image and font format in the same template are the same, and the display effect is relatively simple. Therefore, how to better generate video templates and synthesize videos has become an urgent problem to be solved.

[0082] Based on the above research, the present disclosure provides a video synthesis method, device, computer equipment and storage medium, which can generate a target template matching the video to be synthesized based on the video to be synthesized and the text information corresponding to the video to be synthesized after obtaining the video to be synthesized and the text information corresponding to the video to be synthesized, wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format of the text information determined based on the background image and the first text format, wherein the second text format includes font color information, and then the target video can be synthesized based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template. Therefore, this method does not require a large number of templates to be set in advance, and since the target template is generated based on the video to be synthesized and the text information, the target template generated according to this method can better match the video to be synthesized, thereby improving the display effect of the synthesized target video.

[0083] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0084] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0085] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0086] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to the computer device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0087] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the computer device.

[0088] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0089] To facilitate understanding of this embodiment, a video synthesis method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the video synthesis method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example, a user terminal or a server. In some possible implementations, the video synthesis method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0090] See also Figure 1 FIG. 1 is a flowchart of a video synthesis method provided by an embodiment of the present disclosure, wherein the method includes steps 101 to 103, wherein:

[0091] Step 101: Obtain a video to be synthesized and text information corresponding to the video to be synthesized;

[0092] Step 102: generating a target template matching the video to be synthesized based on the video to be synthesized and the text information corresponding to the video to be synthesized; wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format matching the text information determined based on the background image and the first text format, wherein the second text format includes font color information;

[0093] Step 103: synthesize a target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template.

[0094] Here are detailed instructions for the above steps:

[0095] For step 101,

[0096] Among them, the text information corresponding to the video to be synthesized can be content input by the user, for example, an input box is displayed when the video to be synthesized is released, so that the user can input the text information; or the text information can be determined based on the introduction, summary (the summary is the key information extracted from the introduction), title, author and other related information of the video to be synthesized. Usually, an input box for the relevant information will be displayed when the video is released, so that the user can enter the relevant information of the video to be synthesized. Therefore, the text information corresponding to the video to be synthesized can be directly extracted from the relevant information. For example, the title of the video to be synthesized can be used as the text information, and part of the content can be extracted from the introduction of the video to be synthesized as the text information.

[0097] In a possible application scenario, in response to a publishing operation on the video to be synthesized, the video to be synthesized and text information corresponding to the video to be synthesized can be obtained, so that the target video corresponding to the video to be synthesized can be automatically synthesized when the video to be synthesized is published.

[0098] In another possible application scenario, when it is detected that the target ratio of the video to be synthesized does not meet the first preset condition, the video to be synthesized and the text information corresponding to the video to be synthesized can be obtained. Specifically, the target ratio of the video to be synthesized can be detected when the video to be synthesized is produced or when the video to be synthesized is released. Since different media platforms have different video ratio standards (such as some media platforms display videos in vertical screens, and some media platforms display videos in horizontal screens), if the ratio of the published video does not meet the ratio standard, the published video may have black edges when it is displayed, thereby reducing the display effect. Therefore, when it is detected that the target ratio does not meet the first preset condition, a target video whose video ratio meets the first preset condition can be synthesized.

[0099] For step 102,

[0100] Among them, the first text format may include the display position, character font (such as Songti) and font size (such as 20) corresponding to the characters of each text type. The text type is used to indicate the type of relevant information of the video to be synthesized, such as title type, introduction type, summary type, author type, etc. The display position may be the display position of the text display area corresponding to the characters of each text type. Exemplarily, for any text type, the first text format may include the area coordinates of the text type (such as the coordinates of the vertices of the text display area), character font, and character size. The background image may be used to fill the video to be synthesized so that the proportion of the generated target video meets the proportion standard of the media platform. The second text format may include at least one font color information, and the font color information may be red, yellow, and blue. In actual applications, the font color information may be represented by an RGB value, such as (150, 20, 180). For any text type, the font color may also include at least one font color information, such as the font color information of the title type may include red and yellow.

[0101] The process of generating a target template that matches the video to be synthesized can be understood as a process of determining a first text format that matches the text information, and / or a background image that matches the video to be synthesized, and / or a second text format that matches the text information.

[0102] In a possible implementation manner, the first text format matching the text information may be determined according to the following steps A1 to A2:

[0103] A1. Determine attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type.

[0104] Specifically, the number of text types is used to indicate the number of text types contained in the text information. Exemplarily, if the text types contained in the text information include title type, author type, and abstract type, the number of text types contained in the text information is 3. If the content corresponding to the title type in the text information is "How to Cook Shredded Pork with Fish Flavor", the number of first characters corresponding to the title type is 7.

[0105] A2. Based on the attribute information of the text information and a plurality of pre-set text formats, determine a first text format matching the text information from the plurality of pre-set text formats.

[0106] In a possible implementation, when determining a first text format matching the text information from the plurality of preset text formats based on the attribute information of the text information and the plurality of preset text formats, the following steps B1 to B2 may be adopted:

[0107] B1. Based on the number of text types included in the text information, determine a candidate text format that matches the number of text types from a plurality of pre-set text formats.

[0108] Specifically, different text formats contain different numbers of text types. For example, text format one contains 3 text types, and text format two contains 2 text types. Therefore, in order to display all text types in the text information, it is necessary to first screen out candidate text formats with the same number of text types as in the text information, that is, the candidate text format that matches the number of text types is: a text format that contains the same number of text types as the number of text types contained in the text information, or the candidate text format can also be: a text format that contains no less than the number of text types contained in the text information.

[0109] Here, the candidate text format can be directly filtered from a plurality of pre-set text formats according to the number of text types. For example, if the number of text types contained in the text information is 3, a text format containing 3 text types is filtered out from the plurality of pre-set text formats as the candidate text format.

[0110] Alternatively, the plurality of pre-set text formats may be divided into a plurality of text format sets for storage according to the number of text types contained therein, such as the number of text types corresponding to text format set 1 is 2, and text format set 1 includes text format 1 and text format 2. Therefore, a text format set that matches the number of text types contained in the text information may be determined first, and then the text formats in the text format set may be used as the candidate text formats. Exemplarily, if the number of text types contained in the text information is 3, a target text format set corresponding to the number of text types of 3 is screened out from the plurality of text format sets, and the text formats in the target text format set are used as the candidate text formats.

[0111] B2. Determine a first text format that matches the text information based on a first number of characters corresponding to each text type of the text information and a second number of characters corresponding to each text type of the candidate text format.

[0112] In a possible implementation, when determining the first text format matching the text information based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, the following steps C1 to C2 may be used:

[0113] C1. Based on the first number of characters and the second number of characters, determine the character number distance between each candidate text format and the text information; wherein the character number distance is used to represent the difference information between the second number of characters corresponding to each text type of the candidate text format and the first number of characters corresponding to each text type of the text information.

[0114] Here, the character count distance can be determined based on the difference in the number of characters between the first number of characters and the second number of characters of each same text type. For example, when the text type of the text information includes a title type and an author type, the character count distance can be determined based on the difference in the number of characters between the first number of characters and the second number of characters of the title type, and the difference in the number of characters between the first number of characters and the second number of characters of the author type.

[0115] Exemplarily, the character number distance can be calculated using the following formula:

[0116]

[0117] Wherein, distance represents the distance between the number of characters, n represents the number of text types, i represents the number of the text type, l i represents the first character number, Indicates the second character number.

[0118] Exemplarily, if the first number of characters of the title type in the text information is 5, the first number of characters of the author type is 6, the second number of characters of the title type in the candidate text format is 6, and the second number of characters of the author type is 4, then the character number distance is |5-6|+|6-4|=3.

[0119] C2. Determine the first text format from the candidate text formats based on the distance in number of characters between each candidate text format and the text information.

[0120] Specifically, the candidate text format with the smallest character number distance can be determined as the first text format. Exemplarily, if the character number distance between candidate text format 1 and the text information is 5, the character number distance between candidate text format 2 and the text information is 3, and the character number distance between candidate text format 3 and the text information is 2, then candidate text format 3 is determined as the first text format.

[0121] Here, when there are multiple candidate text formats with the smallest distance in the number of characters, any candidate text format can be determined as the first text format, or multiple candidate text formats with the smallest distance in the number of characters can be displayed, and the user can select the first text format from the displayed multiple text formats.

[0122] By adopting this method, the difference between the second number of characters of each text type in the first text format screened out and the first number of characters is small, so that the text information of each text type can present the best display effect when displayed according to the designed font size.

[0123] Since the second number of characters of some text types in the filtered first text format may be less than the first number of characters, there may not be enough space to display the text information of this text type according to the designed character size. Therefore, in a possible implementation, after determining the first text format that matches the text information from the pre-set multiple text formats, for any text type, if the first number of characters of the text type contained in the text information exceeds the second number of characters of the text type contained in the first text format, the font size of the text information is adjusted (such as reducing the font size of the text information).

[0124] Exemplarily, if the second number of characters corresponding to the title format in the first text format is 5 and the font size is 20, and the first number of characters corresponding to the title format in the text information is 10, the font size corresponding to the title format in the text information can be adjusted to 10.

[0125] Specifically, when the font size of the text information is adjusted, it can be determined according to the first width of the text display area corresponding to the text type in the first text format and the first number of characters of the text type in the text information. For example, the second width of each character in the text information can be obtained by dividing the first width by the first number of characters, and the font size is determined by the second width.

[0126] Alternatively, a text image corresponding to the characters of the text type in the text information may be generated according to the first text format, and then the text image may be reduced to a size that can be displayed in the text display area corresponding to the text type in the first text format. In this way, the text information may be converted into a text information image of a suitable size, and when synthesizing the target video, the color of the characters in the text information image may be updated based on the font color information, and then the updated text information image may be added to the background image.

[0127] By adopting this method, when the second number of characters of certain text types is smaller than the first number of characters, the font size of the text information can be automatically adjusted so that the text information can be fully displayed.

[0128] In another possible implementation, when determining the first text format matching the text information based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, it is also possible to first determine a target candidate text format in which the second number of characters corresponding to each text type is greater than the first number of characters (such as the second number of characters corresponding to the title type is greater than the first number of characters corresponding to the title type, and the second number of characters corresponding to the author type is greater than the first number of characters corresponding to the author type), and then determine the character number distance between each target candidate text format and the text information based on the first number of characters and the second number of characters, and use the target candidate text format with the smallest corresponding character number distance as the first text format.

[0129] By adopting this method, there can be enough space in the text display area of ​​each text type in the filtered first text format, and the text information of the corresponding text type can be displayed according to the designed font size. The target candidate text format with the smallest corresponding character number distance can be used as the first text format. This can, to a certain extent, prevent the number of first characters of each text type from being too small, resulting in too small a display area and too sparse a display effect.

[0130] In a possible implementation, Figure 2 As shown, the background image matching the video to be synthesized can be determined according to the following steps D1 to D2:

[0131] D1. Extract the first feature information of the video to be synthesized, and obtain the second feature information of multiple candidate background images.

[0132] The first feature information may be extracted from multiple video frames in the video to be synthesized, and the multiple video frames may be a preset number of video frames, or the multiple video frames may be extracted from the video to be synthesized at preset time intervals (such as 1 minute). The second feature information of the candidate background image may be extracted from the candidate background image in advance and stored in a background image library, so that it can be directly taken out from the background image library when synthesizing the target video.

[0133] In a possible implementation, when extracting the first feature information of the video to be synthesized, the first feature information of the video to be synthesized can be extracted based on a pre-trained first network model. In addition, the second feature information of the candidate background image can be extracted based on a pre-trained second network model, and the first network model and the second network model are models trained based on contrastive learning.

[0134] Specifically, Figure 3 As shown, the first network model and the second network model may adopt the following steps E1 to E4 during training:

[0135] E1. Acquire multiple sample videos and multiple sample background images; wherein the multiple sample background images include sample background images that match the sample videos.

[0136] Here, the sample background image matching the sample video may be manually selected, and the display effect after synthesizing any sample video with the sample background image matching the sample video is better.

[0137] E2. Input the sample video into a first network model to be trained to extract a first training feature of the sample video; and input the sample background image into a second network model to be trained to extract a second training feature of the sample background image.

[0138] E3. For any sample video, calculate the feature similarity between the first training feature and the second training features of the multiple sample background images.

[0139] E4. Adjust the parameters of the first network model to be trained and the parameters of the second network model to be trained so that the feature similarity between the first training feature and the second training feature of the sample background image that matches the sample video increases, and the feature similarity between the first training feature and the second training feature of the sample background image that does not match the sample video decreases.

[0140] In this way, repeatedly performing the above steps E1 to E4 can enable the trained first network model and the trained second network model to accurately extract the first feature information and the second feature information, thereby improving the training effect.

[0141] D2. Based on the first feature information and the second feature information, select a background image that matches the video to be synthesized from the multiple candidate background images.

[0142] Specifically, the feature similarity between the first feature information and the second feature information of the plurality of candidate background images may be calculated, and then the candidate background image with the greatest feature similarity is used as the background image that matches the video to be synthesized.

[0143] Exemplarily, the feature similarity may be cosine similarity, and the cosine similarity between the first feature information and the second feature information may be calculated according to the following formula:

[0144]

[0145] Among them, s i represents the cosine similarity, q represents the first feature information, k i represents the second feature information, and i represents the number of the candidate background image.

[0146] In a possible implementation manner, the second text format includes font color information corresponding to characters of each text type; when the second text format of the text information is determined based on the background image and the first text format, the following steps F1 to F2 may be adopted:

[0147] F1. For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image.

[0148] Specifically, the coordinates of the display position corresponding to the characters of the text type in the first text format may be determined first, and then the image at the coordinates in the background image may be used as the regional image. For example, if the vertex coordinates of the display position corresponding to the characters of the title type are: (20,100)(80,100)(20,10)(80,10), then the image in the region with vertex coordinates of (20,100)(80,100)(20,10)(80,10) in the background image may be used as the regional image.

[0149] F2. Determine the font color information corresponding to the characters of the text type based on the color information of the area image and a plurality of preset color information pairs, wherein the color information pair includes the font color information and first background color information corresponding to the font color information.

[0150] Specifically, the color information may be an RGB value, and the color information pair may be manually set. For example, the first background information and the font color information may be extracted from a background image of a video with a better display effect, and the font color information and the first background color information may be stored correspondingly to obtain the color information pair.

[0151] Here, it should be noted that any color information pair may include at least one font color information, so that the text information can be displayed in multiple colors to improve the display effect. Since the display positions corresponding to the characters of each text type are different, the regional images corresponding to the characters of each text type are different. Therefore, the characters of each text type in the text information correspond to their own font color information.

[0152] In a possible implementation, when determining the font color information corresponding to the characters of the text type based on the color information of the regional image and a plurality of preset color information pairs, the following steps G1 to G2 may be adopted:

[0153] G1. Determine second background color information corresponding to the regional image based on color information of each pixel contained in the regional image.

[0154] Specifically, the average value of the color information of each pixel contained in the regional image can be used as the second background color information corresponding to the regional image, or the proportion of each color in the regional image can be determined first, and then the color with the highest proportion can be used as the second background color information.

[0155] G2. Compare the second background color information with the first background color information in the multiple color information pairs, determine the target background color information from the first background color information, and use the font color information in the color information pair where the target background color information is located as the font color information corresponding to the characters of the text type.

[0156] Specifically, the color difference distance between the second background color information and the first background color information in each color information pair can be calculated first, and the color difference distance is used to characterize the degree of difference between the first background color information and the second background color information. Then, the first background color information with the smallest color difference distance is determined as the target background color information. Finally, the font color information in the color information pair where the target background color information is located is used as the font color information corresponding to the characters of the text type.

[0157] Exemplarily, if the color difference distance between the second background color information and the first background color information 1 is 10, the color difference distance between the second background color information and the first background color information 2 is 5, and the color difference distance between the second background color information and the first background color information 3 is 8, then the first background color information 2 is determined as the target background color information, and then the font color information 2 in the color information pair 2 where the first background color information 2 is located is used as the font color information corresponding to the characters of the text type.

[0158] Here, the color difference distance can be calculated based on the following formula:

[0159]

[0160] Among them, CD i Indicates the color difference distance, RGB i represents the first background color information, RGB represents the second background color information, and i represents the numbers of the multiple color information pairs.

[0161] Using this method, you can first determine the target background color information that is closest to the color of the regional image in the background image, and then use the font color information that matches the target background color information as the font color information corresponding to the characters of this text type. In this way, you can ensure that the filtered font color information is consistent with the color of the regional image, thereby improving the display effect.

[0162] For step 103,

[0163] For example, Figure 4 As shown, two styles of the synthesized target videos are shown. Style one is that the video to be synthesized is a horizontal screen video, and the synthesized target video is a vertical screen video display style. Style two is that the video to be synthesized is a vertical screen video, and the synthesized target video is a horizontal screen video display style.

[0164] Specifically, the video frames to be synthesized in the video to be synthesized that need to be synthesized with the target template can be determined first, and then the video frames to be synthesized, the text information and the target template are synthesized. The video frames to be synthesized can be every video frame in the video to be synthesized, or can be video frames within a target time period, and the target time period can be preset or set by the user. Exemplarily, the video frames of the first minute of the video to be synthesized can be synthesized with the text information and the target template, so that the first minute of the generated target video displays the text information and the background image.

[0165] In a possible implementation, when synthesizing a target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template, the video to be synthesized can be first added to the background image, and then the text information can be added according to the first text format and the second text format to obtain the target video.

[0166] Specifically, when adding the video to be synthesized to the background image, the video to be synthesized can be added to a preset position on the background image (such as the center of the background image). Here, each background image can correspond to a unified preset position, or each background image can be set with a different preset position, such as the preset position of background image 1 can be above the background image 1, and the preset position of background image 2 can be in the center of the background image 2. Therefore, the preset position in the background image can be determined first, and then the video to be synthesized can be added to the preset position.

[0167] When the text information is added according to the first text format and the second text format, the display style of the text information can be determined according to the font size and character font of the characters of each text type in the first text format, and the font color information of the characters of each text type in the second text format, and then the text information is displayed on the background image according to the display position in the first text format and the display style of the text information, such as displaying the characters of the title type above the background image in Songti, size 4, and red, and displaying the characters of the introduction type below the background image in bold, size 5, and blue.

[0168] Finally, if Figure 5 FIG. 1 is a schematic diagram of the overall process of a video synthesis method. The overall process of a video synthesis method is introduced below:

[0169] 1. Obtain the video to be synthesized and the text information corresponding to the video to be synthesized;

[0170] 2. Determine the attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type;

[0171] 3. Based on the attribute information of the text information and a plurality of pre-set text formats, determining a first text format matching the text information from the plurality of pre-set text formats;

[0172] 4. Based on the pre-trained first network model, extract the first feature information of the video to be synthesized, and based on the pre-trained second network model, extract the second feature information of multiple candidate background images; based on the first feature information and the second feature information, select the background image matching the video to be synthesized from the multiple candidate background images;

[0173] Among them, step 3 and step 4 are executed in no particular order.

[0174] 5. For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image; based on the color information of the area image and a plurality of preset color information pairs, determine the font color information corresponding to the characters of the text type, wherein the color information pair includes the font color information and the first background color information corresponding to the font color information;

[0175] 6. Synthesize a target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template.

[0176] The video synthesis method provided by the embodiment of the present disclosure can generate a target template matching the video to be synthesized based on the video to be synthesized and the text information corresponding to the video to be synthesized after obtaining the video to be synthesized and the text information corresponding to the video to be synthesized, wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format of the text information determined based on the background image and the first text format, wherein the second text format includes font color information, and then the target video can be synthesized based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template. Therefore, this method does not require setting a large number of templates in advance, and since the target template is generated based on the video to be synthesized and the text information, the target template generated according to this method can better match the video to be synthesized, thereby improving the display effect of the synthesized target video.

[0177] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0178] Based on the same inventive concept, a video synthesis device corresponding to the video synthesis method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned video synthesis method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0179] Reference Figure 6 , which is a schematic diagram of the architecture of a video synthesis device provided by an embodiment of the present disclosure, the device comprises: an acquisition module 601, a generation module 602, and a synthesis module 603; wherein,

[0180] An acquisition module 601 is used to acquire a video to be synthesized and text information corresponding to the video to be synthesized;

[0181] A generating module 602 is used to generate a target template matching the video to be synthesized based on the video to be synthesized and the text information corresponding to the video to be synthesized; wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format matching the text information determined based on the background image and the first text format, wherein the second text format includes font color information;

[0182] The synthesis module 603 is used to synthesize a target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template.

[0183] In a possible implementation manner, the generating module 602 is further configured to determine a first text format matching the text information according to the following method:

[0184] Determining attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type;

[0185] Based on the attribute information of the text information and a plurality of preset text formats, a first text format matching the text information is determined from the plurality of preset text formats.

[0186] In a possible implementation manner, the generating module 602, when determining the first text format matching the text information from the plurality of pre-set text formats based on the attribute information of the text information and the plurality of pre-set text formats, is configured to:

[0187] Based on the number of text types included in the text information, determining a candidate text format that matches the number of text types from a plurality of pre-set text formats;

[0188] Based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, a first text format matching the text information is determined.

[0189] In a possible implementation manner, the generating module 602, when determining the first text format matching the text information based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, is configured to:

[0190] Based on the first number of characters and the second number of characters, determining the character number distance between each candidate text format and the text information; wherein the character number distance is used to represent the difference information between the second number of characters corresponding to each text type of the candidate text format and the first number of characters corresponding to each text type of the text information;

[0191] The first text format is determined from the candidate text formats based on the distance in number of characters between each candidate text format and the text information.

[0192] In a possible implementation manner, the first text format includes a font size;

[0193] After determining the first text format matching the text information from the plurality of preset text formats, the generating module 602 is further configured to:

[0194] For any text type, when the number of first characters of the text type included in the text information exceeds the number of second characters of the text type included in the first text format, the font size of the text information is adjusted.

[0195] In a possible implementation manner, the generating module 602 is further configured to determine a background image matching the video to be synthesized according to the following method:

[0196] Extracting first feature information of the video to be synthesized, and obtaining second feature information of multiple candidate background images;

[0197] Based on the first feature information and the second feature information, a background image matching the video to be synthesized is screened out from the multiple candidate background images.

[0198] In a possible implementation manner, the generating module 602, when extracting the first feature information of the video to be synthesized, is used to:

[0199] Extracting first feature information of the video to be synthesized based on a pre-trained first network model;

[0200] The second feature information of the candidate background image is extracted based on a pre-trained second network model, and the first network model and the second network model are models trained based on contrastive learning.

[0201] In a possible implementation manner, the first text format includes display positions corresponding to characters of each text type; the second text format includes font color information corresponding to characters of each text type;

[0202] The generating module 602 is further configured to determine a second text format matching the text information according to the following method:

[0203] For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image;

[0204] Based on the color information of the area image and a plurality of preset color information pairs, font color information corresponding to the characters of the text type is determined, wherein the color information pair includes font color information and first background color information corresponding to the font color information.

[0205] In a possible implementation manner, the generating module 602, when determining the font color information corresponding to the characters of the text type based on the color information of the regional image and a plurality of preset color information pairs, is configured to:

[0206] Determining second background color information corresponding to the regional image based on color information of each pixel contained in the regional image;

[0207] The second background color information is compared with the first background color information in the multiple color information pairs, the target background color information is determined from the first background color information, and the font color information in the color information pair where the target background color information is located is used as the font color information corresponding to the characters of the text type.

[0208] In a possible implementation manner, the synthesis module 603, when synthesizing the target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template, is used to:

[0209] The video to be synthesized is added to the background image, and the text information is added according to the first text format and the second text format to obtain the target video.

[0210] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0211] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 7As shown, it is a schematic diagram of the structure of a computer device 700 provided in an embodiment of the present disclosure, including a processor 701, a memory 702, and a bus 703. Among them, the memory 702 is used to store execution instructions, including a memory 7021 and an external memory 7022; the memory 7021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 701, and the data exchanged with the external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the memory 7021. When the computer device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the following instructions:

[0212] Acquire the video to be synthesized and text information corresponding to the video to be synthesized;

[0213] Based on the video to be synthesized and the text information corresponding to the video to be synthesized, a target template matching the video to be synthesized is generated; wherein the target template includes a first text format matching the text information, and / or a background image matching the video to be synthesized, and / or a second text format matching the text information determined based on the background image and the first text format, wherein the second text format includes font color information;

[0214] A target video is synthesized based on the video to be synthesized, text information corresponding to the video to be synthesized, and the target template.

[0215] In a possible implementation manner, in the instructions executed by the processor 701, the method further includes determining a first text format matching the text information according to the following method:

[0216] Determining attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type;

[0217] Based on the attribute information of the text information and a plurality of preset text formats, a first text format matching the text information is determined from the plurality of preset text formats.

[0218] In a possible implementation manner, in the instructions executed by the processor 701, the determining, based on the attribute information of the text information and a plurality of pre-set text formats, a first text format matching the text information from the plurality of pre-set text formats includes:

[0219] Based on the number of text types included in the text information, determining a candidate text format that matches the number of text types from a plurality of pre-set text formats;

[0220] Based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, a first text format matching the text information is determined.

[0221] In a possible implementation manner, in the instructions executed by the processor 701, determining the first text format matching the text information based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format includes:

[0222] Based on the first number of characters and the second number of characters, determining the character number distance between each candidate text format and the text information; wherein the character number distance is used to represent the difference information between the second number of characters corresponding to each text type of the candidate text format and the first number of characters corresponding to each text type of the text information;

[0223] The first text format is determined from the candidate text formats based on the distance in number of characters between each candidate text format and the text information.

[0224] In a possible implementation, in the instruction executed by the processor 701, the first text format includes a font size;

[0225] After determining a first text format matching the text information from the plurality of preset text formats, the method further includes:

[0226] For any text type, when the number of first characters of the text type included in the text information exceeds the number of second characters of the text type included in the first text format, the font size of the text information is adjusted.

[0227] In a possible implementation manner, in the instructions executed by the processor 701, the method further includes determining a background image matching the video to be synthesized according to the following method:

[0228] Extracting first feature information of the video to be synthesized, and obtaining second feature information of multiple candidate background images;

[0229] Based on the first feature information and the second feature information, a background image matching the video to be synthesized is screened out from the multiple candidate background images.

[0230] In a possible implementation manner, in the instruction executed by the processor 701, extracting the first feature information of the video to be synthesized includes:

[0231] Extracting first feature information of the video to be synthesized based on a pre-trained first network model;

[0232] The second feature information of the candidate background image is extracted based on a pre-trained second network model, and the first network model and the second network model are models trained based on contrastive learning.

[0233] In a possible implementation, in the instruction executed by the processor 701, the first text format includes display positions corresponding to characters of each text type; the second text format includes font color information corresponding to characters of each text type;

[0234] The method further includes determining a second text format matching the text information according to the following method:

[0235] For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image;

[0236] Based on the color information of the area image and a plurality of preset color information pairs, font color information corresponding to the characters of the text type is determined, wherein the color information pair includes font color information and first background color information corresponding to the font color information.

[0237] In a possible implementation manner, in the instructions executed by the processor 701, determining the font color information corresponding to the characters of the text type based on the color information of the regional image and a plurality of preset color information pairs includes:

[0238] Determining second background color information corresponding to the regional image based on color information of each pixel contained in the regional image;

[0239] The second background color information is compared with the first background color information in the multiple color information pairs, the target background color information is determined from the first background color information, and the font color information in the color information pair where the target background color information is located is used as the font color information corresponding to the characters of the text type.

[0240] In a possible implementation manner, in the instructions executed by the processor 701, synthesizing the target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template includes:

[0241] The video to be synthesized is added to the background image, and the text information is added according to the first text format and the second text format to obtain the target video.

[0242] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the video synthesis method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0243] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the video synthesis method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0244] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0245] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0246] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0247] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0248] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0249] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A video synthesis method, characterized in that: include: Acquire the video to be synthesized and text information corresponding to the video to be synthesized; Based on the video to be synthesized and the text information corresponding to the video to be synthesized, generating a target template matching the video to be synthesized; wherein the target template includes a first text format matching the text information; Synthesize a target video based on the video to be synthesized, text information corresponding to the video to be synthesized, and the target template; The method further comprises determining a first text format matching the text information according to the following method: Determining attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type; Based on the attribute information of the text information and a plurality of preset text formats, a first text format matching the text information is determined from the plurality of preset text formats.

2. The method according to claim 1, characterized in that: The determining, based on the attribute information of the text information and a plurality of pre-set text formats, a first text format matching the text information from the plurality of pre-set text formats comprises: Based on the number of text types included in the text information, determining a candidate text format that matches the number of text types from a plurality of pre-set text formats; Based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, a first text format matching the text information is determined.

3. The method according to claim 2, characterized in that The determining, based on the first number of characters corresponding to each text type of the text information and the second number of characters corresponding to each text type of the candidate text format, a first text format matching the text information includes: Based on the first number of characters and the second number of characters, determining the character number distance between each candidate text format and the text information; wherein the character number distance is used to represent the difference information between the second number of characters corresponding to each text type of the candidate text format and the first number of characters corresponding to each text type of the text information; The first text format is determined from the candidate text formats based on the distance in number of characters between each candidate text format and the text information.

4. The method according to any one of claims 1 to 3, characterized in that: The first text format includes font size; After determining a first text format matching the text information from the plurality of preset text formats, the method further includes: For any text type, when the number of first characters of the text type included in the text information exceeds the number of second characters of the text type included in the first text format, the font size of the text information is adjusted.

5. The method according to claim 1, characterized in that The target template also includes: a background image that matches the video to be synthesized.

6. The method according to claim 5, characterized in that The method further comprises determining a background image matching the video to be synthesized according to the following method: Extracting first feature information of the video to be synthesized, and obtaining second feature information of multiple candidate background images; Based on the first feature information and the second feature information, a background image matching the video to be synthesized is screened out from the multiple candidate background images.

7. The method according to claim 6, characterized in that The extracting the first feature information of the video to be synthesized includes: Extracting first feature information of the video to be synthesized based on a pre-trained first network model; The second feature information of the candidate background image is extracted based on a pre-trained second network model, and the first network model and the second network model are models trained based on contrastive learning.

8. The method according to claim 5, characterized in that The target template also includes a second text format determined based on the background image and the first text format and matching the text information, wherein the second text format includes font color information; The first text format includes the display positions corresponding to the characters of each text type; the second text format includes the font color information corresponding to the characters of each text type; The method further includes determining a second text format matching the text information according to the following method: For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image; Based on the color information of the area image and a plurality of preset color information pairs, font color information corresponding to the characters of the text type is determined, wherein the color information pair includes font color information and first background color information corresponding to the font color information.

9. The method according to claim 8, characterized in that The determining, based on the color information of the regional image and a plurality of preset color information pairs, the font color information corresponding to the characters of the text type comprises: Determining second background color information corresponding to the regional image based on color information of each pixel contained in the regional image; The second background color information is compared with the first background color information in the multiple color information pairs, the target background color information is determined from the first background color information, and the font color information in the color information pair where the target background color information is located is used as the font color information corresponding to the characters of the text type.

10. The method according to claim 1, characterized in that The synthesizing a target video based on the video to be synthesized, text information corresponding to the video to be synthesized, and the target template includes: The video to be synthesized is added to the background image, and the text information is added according to the first text format and the second text format to obtain the target video.

11. A video synthesis device, characterized in that: include: An acquisition module, used to acquire the video to be synthesized and text information corresponding to the video to be synthesized; A generating module, configured to generate a target template matching the video to be synthesized based on the video to be synthesized and text information corresponding to the video to be synthesized; wherein the target template includes a first text format matching the text information; A synthesis module, used to synthesize a target video based on the video to be synthesized, the text information corresponding to the video to be synthesized, and the target template; Wherein, the generating module is further configured as follows: Determining attribute information of the text information; wherein the attribute information includes the number of text types contained in the text information and the number of first characters corresponding to each text type; Based on the attribute information of the text information and a plurality of preset text formats, a first text format matching the text information is determined from the plurality of preset text formats.

12. A video synthesis method, characterized in that: include: Acquire the video to be synthesized and text information corresponding to the video to be synthesized; Based on the video to be synthesized and the text information corresponding to the video to be synthesized, a target template matching the video to be synthesized is generated; wherein the target template includes: a first text format matching the text information, a background image matching the video to be synthesized, and a second text format matching the text information determined based on the background image and the first text format, wherein the second text format includes font color information; Synthesize a target video based on the video to be synthesized, text information corresponding to the video to be synthesized, and the target template; The first text format includes the display positions corresponding to the characters of each text type; the second text format includes the font color information corresponding to the characters of each text type; The method further includes determining a second text format matching the text information according to the following method: For any text type, based on the display positions corresponding to the characters of the text type and the background image, determine the area image corresponding to the characters of the text type in the background image; Based on the color information of the area image and a plurality of preset color information pairs, font color information corresponding to the characters of the text type is determined, wherein the color information pair includes font color information and first background color information corresponding to the font color information.

13. The method according to claim 12, characterized in that The determining, based on the color information of the regional image and a plurality of preset color information pairs, the font color information corresponding to the characters of the text type comprises: Determining second background color information corresponding to the regional image based on color information of each pixel contained in the regional image; The second background color information is compared with the first background color information in the multiple color information pairs, the target background color information is determined from the first background color information, and the font color information in the color information pair where the target background color information is located is used as the font color information corresponding to the characters of the text type.

14. The method according to claim 12, characterized in that The synthesizing a target video based on the video to be synthesized, text information corresponding to the video to be synthesized, and the target template includes: The video to be synthesized is added to the background image, and the text information is added according to the first text format and the second text format to obtain the target video.

15. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the video synthesis method as described in any one of claims 1 to 14 are performed.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video synthesis method according to any one of claims 1 to 14 are executed.

Images