Video generation method and device and computing equipment

Abstract

The invention discloses a video generation method and apparatus, and a computing device. The method comprises the steps of obtaining an original material according to a video generation request; extracting object features of the client; screening out a target modification element category matched with the object features from multiple modification element categories; based on the material content of the original material, adopting a target processing algorithm corresponding to a target modification element category to generate a target modification element corresponding to the original material and a rendering parameter of the target modification element; and sending the target modification element and the rendering parameter to the client, so that the client generates a video according to the original material, the target modification element and the rendering parameter. According to the scheme, the difference of video generation strategies of different clients can be realized, and the matching degree of the generated video and the client is improved; and the generated target modification element is matched with the material content, so that the differentiation of the modification elements of different original materials in the same video is realized, the display effect of the generated video is improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a video generation method, apparatus, computing device, computer storage medium and computer program product. Background Art

[0002] With the continuous development of Internet technology, the emergence of various video platforms has greatly facilitated people's work and life. Among them, some video platforms provide users with video creation functions such as "one-click filming" and "smart filming", based on which the corresponding videos can be automatically generated from the material resources provided by users.

[0003] However, the inventors found in the implementation process that the prior art has the following defects: in the prior art, all clients use a unified fixed strategy to generate videos. However, this method has low flexibility and poor user experience. Summary of the invention

[0004] In view of the above problems, the present application is proposed to provide a video generation method, apparatus, computing device, computer storage medium and computer program product that overcome the above problems or at least partially solve the above problems.

[0005] According to a first aspect of the present application, a video generation method is provided, comprising:

[0006] Receive a video generation request sent by a client, and obtain original material according to the video generation request;

[0007] Extracting object features of the client;

[0008] Screening out a target modifying element category matching the object feature from a plurality of modifying element categories;

[0009] For each original material, based on the material content of the original material, a target processing algorithm corresponding to the target modification element category is adopted to generate a target modification element corresponding to the original material and rendering parameters of the target modification element;

[0010] The target modification element and the rendering parameter are sent to the client, so that the client generates a video according to the original material, the target modification element and the rendering parameter.

[0011] In an optional implementation, the step of selecting a target modifying element category matching the object feature from a plurality of modifying element categories includes:

[0012] Obtain the category characteristics of each modification element category;

[0013] respectively calculating the similarity between the object feature and the category feature of each modification element category;

[0014] A target modifying element category matching the object feature is screened out from the multiple modifying element categories according to the similarity.

[0015] In an optional implementation, the rendering parameters include:

[0016] The rendering position of the target modification element in the corresponding original material, and / or the splicing order of the original materials corresponding to the target modification element.

[0017] In an optional implementation, if the target modification element category is a copywriting category;

[0018] Then, for each original material, based on the material content of the original material, using the target processing algorithm corresponding to the target modification element category, generating the target modification element corresponding to the original material and the rendering parameters of the target modification element include:

[0019] For each original material, a material keyword recognition model is used to identify the material keywords of the original material;

[0020] Filter out target copywriting that matches the material keywords of each original material from the copywriting library;

[0021] Determine a text segment matching each original material from the target text;

[0022] For each original material, a copy modification element of the original material and rendering parameters of the copy modification element are generated according to the copy segment matched by the original material.

[0023] In an optional implementation, the copy modification element includes: a text copy element, an audio copy element, and / or a background image copy element;

[0024] The rendering parameters of the text modification element are determined as follows:

[0025] Determining a rendering position of the text modification element in the corresponding original material according to the configuration parameters of the text modification element;

[0026] According to the position of the text segment corresponding to the text modification element in the target text, the splicing order of the original materials corresponding to the text modification element is determined.

[0027] In an optional implementation, if the target modification element category is a sticker category;

[0028] Then, for each original material, based on the material content of the original material, using the target processing algorithm corresponding to the target modification element category, generating the target modification element corresponding to the original material and the rendering parameters of the target modification element include:

[0029] For each original material, a material keyword recognition model is used to identify the material keywords of the original material;

[0030] For each original material, a target sticker matching the material keyword of the original material is searched from the sticker library, the matching target sticker is used as a sticker modification element of the original material, and rendering parameters of the sticker modification element are generated.

[0031] In an optional embodiment, the method further includes:

[0032] If the original material is a non-image material, a background modification element is generated according to a material keyword of the original material.

[0033] According to a second aspect of the present application, a video generating device is provided, including:

[0034] A receiving module, used for receiving a video generation request sent by a client;

[0035] A material acquisition module, used to acquire original material according to the video generation request;

[0036] A feature extraction module, used to extract object features of the client;

[0037] A category determination module, used to select a target modifying element category matching the object feature from a plurality of modifying element categories;

[0038] A generating module, for generating, for each original material, a target modifying element corresponding to the original material and rendering parameters of the target modifying element by adopting a target processing algorithm corresponding to the target modifying element category based on the material content of the original material;

[0039] The sending module is used to send the target modification element and the rendering parameter to the client, so that the client can generate a video according to the original material, the target modification element and the rendering parameter.

[0040] In an optional implementation, the category determination module is used to: obtain category features of each modification element category;

[0041] Calculate the similarity between the object features and the category features of each modification element category respectively;

[0042] A target modifying element category matching the object feature is selected from a plurality of modifying element categories according to the similarity.

[0043] In an optional implementation, the rendering parameters include:

[0044] The rendering position of the target modification element in the corresponding original material, and / or the splicing order of the original materials corresponding to the target modification element.

[0045] In an optional implementation, if the target modification element category is a copywriting category;

[0046] The generation module is used to: for each original material, use a material keyword recognition model to identify the material keywords of the original material;

[0047] Filter out target copywriting that matches the material keywords of each original material from the copywriting library;

[0048] Determine the copy segments that match each source material from the target copy;

[0049] For each original material, a copy modification element of the original material and rendering parameters of the copy modification element are generated according to the copy segment matched with the original material.

[0050] In an optional implementation, the copy modification element includes: a text copy element, an audio copy element, and / or a background image copy element;

[0051] The generation module is used to: determine the rendering position of the copy modification element in the corresponding original material according to the configuration parameters of the copy modification element;

[0052] According to the positions of the copy fragments corresponding to the copy modifying elements in the target copy, the splicing order of the original materials corresponding to the copy modifying elements is determined.

[0053] In an optional implementation, if the target modification element category is a sticker category;

[0054] The generation module is used to: for each original material, use a material keyword recognition model to identify the material keywords of the original material;

[0055] For each original material, a target sticker matching the material keyword of the original material is searched from the sticker library, the matching target sticker is used as a sticker modification element of the original material, and rendering parameters of the sticker modification element are generated.

[0056] In an optional implementation, the generation module is used to: if the original material is a non-image material, generate background modification elements according to material keywords of the original material.

[0057] According to a third aspect of the present application, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0058] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned video generation method.

[0059] According to a fourth aspect of the present application, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned video generation method.

[0060] According to a fifth aspect of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned video generation method.

[0061] The embodiment of the present application selects a target modifying element category that matches the object features of the client from a variety of modifying element categories, and then uses a target processing algorithm of the target modifying element category to generate a target modifying element, thereby achieving differentiation of video generation strategies of different clients and improving the matching degree between the generated video and the client; moreover, the embodiment of the present application combines the target processing algorithm and the material content of the original material to generate the target modifying element, so that the generated target modifying element is adapted to the material content, thereby achieving differentiation of the modifying elements of different original materials in the same video, improving the display effect of the generated video, and improving the user experience.

[0062] The embodiment of the present application can accurately screen out a target modifying element category that matches the object feature based on the similarity between the category feature of the modifying element category and the object feature, thereby improving the accuracy of determining the target modifying element category.

[0063] The rendering parameters of the embodiment of the present application include the rendering position of the target modification element in the corresponding original material and / or the splicing order of the original material corresponding to the target modification element, thereby improving the accuracy of adding the target modification element and improving the quality of the generated video.

[0064] In the embodiment of the present application, when determining that the target modifying element category that matches the object feature of the client is a copy category, the matching target copy is determined based on the material keywords of the original material, and then the copy segments that match each original material are determined from the target copy, and the copy modifying elements are generated based on the copy segments, so that the generated video can correspond to the same copy, thereby ensuring the continuity of the generated video and improving the user experience.

[0065] The copy modification elements of the embodiments of the present application include text copy elements, audio copy elements, and / or background image copy elements, thereby enriching the presentation form of the copy modification elements and improving the video presentation effect.

[0066] The embodiment of the present application determines the splicing order of the original materials corresponding to the copy modification elements according to the positions of the copy fragments corresponding to the copy modification elements in the target copy, thereby further improving the coherence of the generated video and enhancing the user experience.

[0067] When the embodiment of the present application determines that the target modification element category that matches the object features of the client is a sticker category, the target stickers that match the material keywords of each original material are obtained from the sticker library, thereby generating corresponding sticker modification elements, improving the matching degree between the sticker modification elements and the original material, improving the display effect of the generated video, and improving the user experience.

[0068] In the case where the original material is non-image material, the embodiment of the present application also generates background modification elements according to material keywords of the original material, automatically adds an image background to the generated video, and improves the video display effect.

[0069] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0071] Figure 1 A schematic diagram of an operating environment provided for implementing at least one embodiment of the present application is shown;

[0072] Figure 2 A flowchart of a video generation method provided in Embodiment 1 of the present application is shown;

[0073] Figure 3 A flow chart of a method for screening target modifying element categories provided in Example 1 of the present application is shown;

[0074] Figure 4 A flowchart of a video generation method provided in Embodiment 2 of the present application is shown;

[0075] Figure 5A flow chart of a method for determining a target document provided in Embodiment 2 of the present application is shown;

[0076] Figure 6 A flowchart of a video generation method provided in Embodiment 3 of the present application is shown;

[0077] Figure 7 A flowchart of a video generation method provided in Embodiment 4 of the present application is shown;

[0078] Figure 8 A structural diagram of a video generating device provided in Embodiment 5 of the present application is shown;

[0079] Fig. 9 A structural diagram of a computing device provided in Example 6 of the present application is shown. DETAILED DESCRIPTION

[0080] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0081] It should be noted that the object features and object data involved in the embodiments of the present application are all authorized or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for choosing to authorize or reject.

[0082] Figure 1 The present invention is applicable to an application environment including, but not limited to, a client 2 , a server 4 , and a network 6 .

[0083] in:

[0084] The server 4 may be composed of a single or multiple computing devices. The multiple computing devices may include virtualized computing instances. Virtualized computing instances may include virtual machines, such as simulations of computer systems, operating systems, servers, etc. The computing device may load a virtual machine based on a virtual image and / or other data defining specific software (e.g., operating system, dedicated application, server) for simulation. As the demand for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices. A hypervisor may be implemented to manage the use of different virtual machines on the same computing device.

[0085] The server 4 may be configured to communicate with the client 2, etc., via a network 6. The network 6 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 6 may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, etc., or wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0086] The server 4 can provide storage, reading, downloading, writing, querying, deleting and other services, such as providing static resource download services to clients through multiple domain names.

[0087] Client 2 can be running Windows, Android TM ) or IOS and other operating systems, such as smart phones, tablet devices, laptop computers, virtual reality devices, gaming devices, set-top boxes, car terminals, smart TVs. Based on the above operating systems, various applications, such as browsers, can be run.

[0088] Embodiment 1

[0089] Figure 2 A flow chart of a video generation method provided in the first embodiment of the present application is shown. The video generation method provided in the embodiment of the present application can be executed on the server side.

[0090] Specifically, if Figure 2 As shown, the method comprises the following steps:

[0091] Step S201: receiving a video generation request sent by a client, and obtaining original material according to the video generation request.

[0092] A video generation request can be initiated through corresponding operations of the client. For example, a video generation request can be generated in the client by triggering controls such as "one-click filming" and "smart filming". The video generation request carries relevant information of the original material, which is the material provided by the client for video generation. The relevant information of the original material may include the original material itself, the method and / or address for obtaining the original material, etc.

[0093] A communication connection is established between the server and the client, for example, an end-to-end communication connection is established between the server and the client, and data interaction can be performed through a communication protocol agreed upon between the server and the client. The server receives a video generation request sent by the client, and parses the video generation request to obtain relevant information of the original material, and then obtains the original material according to the relevant information of the original material. Among them, the embodiment of the present application does not limit the specific material type of the original material, for example, the original material may include: pictures, videos, audio, and / or text, etc. The original material obtained may be one or more, for example, the original material may be one or more videos, one or more pictures, or a piece of text data, etc.

[0094] In an optional implementation, after obtaining the original material, the original material is checked to eliminate the original material that does not comply with the regulations, such as eliminating the original material with format errors, garbled characters, non-compliant materials, etc., so as to ensure the compliance of the subsequently generated video and avoid wasting system processing resources.

[0095] Step S202: extracting object features of the client.

[0096] The object features of the client are the relevant features of the object using the client. The object features include: object attribute features and / or object preference features, etc. In the specific implementation process, the object features can be extracted in advance based on the object attribute data and / or object behavior data of the client. For example, based on the data of the videos historically released by the object and the videos that have been historically favorited (such as like operations, favorite operations, forwarding operations, positive evaluation operations), the modifying element categories of the object preference can be parsed to obtain the object preference features. The embodiment of the present application does not limit the method of extracting object features. For example, the object data can be processed using a feature extraction model to obtain the corresponding object features, or the object features can be extracted using a rule extraction algorithm.

[0097] Step S203: Filter out a target modifying element category that matches the object feature from the multiple modifying element categories.

[0098] There are multiple pre-configured modification element categories, each of which contains at least one modification element, which is used to modify the original material to improve the display effect. The modification element category may include: text category, sticker category, and / or filter category, etc. For example, the text category contains at least one text, the sticker category contains at least one sticker, the filter category contains at least one filter, etc. The modification element category and the modification elements under each modification element category can be flexibly configured, thereby improving the scalability of the system.

[0099] According to the object characteristics of the client, a target modification element category is screened out from a plurality of modification element categories. That is, the target modification element category is a modification element category that matches the object characteristics of the client, and the target modification element category may be one or more.

[0100] In an optional implementation, specifically, Figure 3 The steps shown are to filter the target modification element categories:

[0101] S2031, obtaining category features of each modification element category.

[0102] The category characteristics of each modifier element category are recorded in the configuration data in advance, and the category characteristics include attribute characteristics and / or delivery group characteristics. The attribute characteristics may include category name, category identifier, etc. The delivery group characteristics are the object characteristics of the objects to which the modifier element of the category is applicable, such as the delivery group characteristics may be a certain gender, etc.

[0103] S2032, respectively calculating the similarity between the object feature and the category feature of each modification element category.

[0104] For any modifying element category, the similarity between the category feature of the modifying element category and the object feature of the client is calculated. The similarity can be text similarity, semantic similarity, etc. The embodiment of the present application does not limit the specific calculation process of the similarity.

[0105] S2033, selecting a target modifying element category that matches the object feature from the multiple modifying element categories according to the similarity.

[0106] Specifically, the modifying element category with the highest similarity may be selected as the target modifying element category.

[0107] use Figure 3 The steps shown can accurately filter out the target modification element category and improve the matching degree between the subsequently generated video and the client object.

[0108] Step S204 , for each original material, based on the material content of the original material, a target processing algorithm corresponding to the target modifying element category is adopted to generate a target modifying element corresponding to the original material and rendering parameters of the target modifying element.

[0109] Each modification element category has a processing algorithm that matches it, and the processing algorithm corresponding to the target modification element category is called the target processing algorithm.

[0110] For each original material, the target modification element corresponding to the original material is generated in combination with the material content of the original material and the target processing algorithm. Among them, each target modification element belongs to the target modification element category, and each original material has a target modification element that matches it. The target modification element can be added to the corresponding original material to improve the display effect, and the target modification element belongs to the target modification element category, so that it can be adapted to the client object; in addition, the target modification element is generated based on the material content of the original material, so that the target modification element can be adapted to the material content, improve the matching degree between the target modification element and the original material, and improve the display effect of the generated video.

[0111] In addition, rendering parameters of the target modification element are generated. Specifically, the rendering parameters include: the rendering position of the target modification element in the corresponding original material, and / or the splicing order of the original material corresponding to the target modification element, etc. Among them, the rendering position of the target modification element in the corresponding original material also includes the original material information corresponding to the target modification element.

[0112] Step S205 , sending the target modification element and rendering parameters to the client, so that the client can generate a video according to the original material, the target modification element and the rendering parameters.

[0113] The server sends the generated target modifier and the corresponding rendering parameters to the client, and the client generates the corresponding video based on the original material, the target modifier and the rendering parameters. Specifically, the client adds the target modifier to the corresponding original material according to the rendering position of the target modifier in the matched original material, and splices the material with the target modifier added according to the splicing order of the original material matched by the target modifier to obtain the corresponding video. The generated video not only contains the original material provided by the client, but also contains the automatically generated modifier elements, which improves the video display effect.

[0114] It can be seen that the video generation method provided in the embodiment of the present application selects a target modification element category that matches the object characteristics of the client from a variety of modification element categories, and then uses the target processing algorithm of the target modification element category to generate the target modification element, thereby achieving differentiation of video generation strategies of different clients and improving the matching degree between the generated video and the client; moreover, the embodiment of the present application combines the target processing algorithm and the material content of the original material to generate the target modification element, so that the generated target modification element is adapted to the material content, thereby achieving differentiation of the modification elements of different original materials in the same video, improving the display effect of the generated video, and improving the user experience.

[0115] Embodiment 2

[0116] Figure 4A flow chart of a video generation method provided in the second embodiment of the present application is shown. The video generation method provided in the embodiment of the present application can be executed on the server side.

[0117] Specifically, if Figure 4 As shown, the method comprises the following steps:

[0118] Step S401, receiving a video generation request sent by a client, obtaining original material according to the video generation request; extracting object features of the client; and determining that a target modifying element category matching the object features of the client is a text category.

[0119] This embodiment is used to optimize the processing method when the target modifying element category is the text category, that is, to determine that the client object prefers to use the text to optimize the material.

[0120] Step S402: for each original material, a material keyword recognition model is used to recognize material keywords of the original material.

[0121] A material keyword recognition model is pre-trained, and the material keyword recognition model is constructed and trained based on a machine learning algorithm. Among them, the material keyword recognition model may include an image keyword recognition model and / or a text keyword recognition model. For example, an image keyword recognition model can be generated and trained based on VGGNet, ResNet, etc., and a text keyword recognition model can be generated and trained based on TF-IDF, TextRank, etc. In short, the embodiments of the present application do not limit the specific structure and training algorithm of the material keyword recognition model. The material keyword recognition model can analyze the material content of the original material to obtain the material keyword of each original material. The material keyword can be one or more, and the material keyword can reflect the core characteristics of the original material.

[0122] For any original material, if the original material is an image material such as a picture or video, the image keyword recognition model is used to identify the material keywords of the original material; if the original material is an audio material, the audio original material is converted into a corresponding text material, and the text keyword recognition model is used to identify the material keywords of the original material; if the original material is a text material, the text keyword recognition model is used to identify the material keywords of the original material.

[0123] Step S403: Filter out target texts that match the material keywords of each original material from the text library.

[0124] A copy library is pre-configured, and the copy library contains at least one copy, and each copy contains at least one copy fragment. The copy in the copy library is compared with the material keywords of each original material, and the copy matching the material keywords of each original material is used as the target copy.

[0125] In an optional implementation, it can be specifically adopted Figure 5 The following steps are used to determine the target copy:

[0126] S4031, for any text, calculating the sub-matching degree between the text and each original material.

[0127] Specifically, for each original material, the similarity between each material keyword of the original material and the copy is calculated respectively, and the sub-matching degree between the copy and the original material is determined according to the similarity. For example, the highest value of the similarity can be used as the sub-matching degree between the copy and the original material, and the average value of the similarity can also be used as the sub-matching degree between the copy and the original material. Thus, the sub-matching degree corresponding to the original material can reflect the similarity between the original material and the copy.

[0128] S4032: Generate a total matching degree of the text according to the sub-matching degrees between the text and each original material.

[0129] For any copy, the sum or average of the sub-matching degrees between the copy and each original material is taken as the total matching degree of the copy, which can reflect the overall similarity between the copy and all the original elements. The higher the total matching degree, the more the copy fits the original elements as a whole.

[0130] S4033, determining the target copy according to the total matching degree of the copy.

[0131] Specifically, the copy with the highest overall matching degree can be used as the target copy.

[0132] use Figure 5 The target copy determined by the method shown can be more consistent with the original material content provided by the client, which is conducive to improving the display effect of the video.

[0133] Step S404: determine the text segment matching each original material from the target text.

[0134] For each original material, a text segment with the highest similarity to the target text is determined from the target text as the text segment matching the original material.

[0135] In the specific implementation process, for any original material, the similarity between the material keywords of the original material and each copy segment is calculated, and the copy segment with the highest similarity is determined as the candidate copy segment, and then the copy segment matching the original material is determined based on the candidate copy segment.

[0136] In an optional implementation, when the candidate text segments of multiple original materials are the same, the same text segment is used as the conflicting text segment, and the multiple original materials are used as the conflicting original materials. Then, the similarity between each conflicting original material and the conflicting text segment is determined respectively, and the conflicting text segment is used as the text segment that matches the conflicting original material with the highest similarity. For the conflicting original material that is not the most similar to the conflicting text segment, the text segment with the second highest similarity to the conflicting original material is used as the text segment that matches it. This method can avoid conflicts in text segments and ensure the accuracy of text addition.

[0137] Step S405 , for each original material, generating the copy modification elements of the original material and rendering parameters of the copy modification elements according to the copy segments matched by the original material.

[0138] Each original material has a copywriting segment matching it, and the copywriting modification element and the rendering parameters of the copywriting modification element of each original material are generated according to the copywriting segment matching it. Therefore, in this embodiment, the target modification element is the copywriting modification element.

[0139] In an optional implementation, the copy modification elements include: text copy elements, audio copy elements, and / or background image copy elements. Among them, the text copy elements present the copy in text form (such as subtitles, etc.); the audio copy elements present the copy in audio form (such as converting the copy from text to audio); the background image copy elements are background images bound to the copy, for example, different copies are bound to images that match them according to their copy themes, and the images are background image copy elements.

[0140] In a specific implementation process, the presentation mode of the copy modification element can be determined according to the material type of the original material. For example, if the original material is an image material, the text copy element and / or audio copy element of the original material is generated according to the copy fragment matched by the original material; if the original material is an audio material, the text copy element and / or background image copy element of the original material is generated according to the copy fragment matched by the original material; if the original material is a text material, the text copy element and / or background image copy element of the original material is generated according to the copy fragment matched by the original material. For example, if the original material is a picture, a text subtitle (text copy element) is generated according to the copy fragment matched by the original material, and an audio copy can also be generated; if the original material is a piece of audio, a text subtitle is generated according to the copy fragment matched by the original material, and a background image (background image copy element) bound to the target copy can also be obtained by searching the configuration data; if the original material is a piece of text, a text subtitle is generated according to the copy fragment matched by the original material, and a background image bound to the target copy can also be obtained by searching the configuration data.

[0141] Furthermore, the rendering parameters of each text modification element can be determined in the following manner: the rendering position of the text modification element in the corresponding original material is determined according to the configuration parameters of the text modification element. Specifically, the configuration data usually records the default rendering positions of various forms (text copy form, audio copy form, background image copy form), such as the rendering position of the text copy modification element in the form of text copy is the set position below the canvas; the rendering position of the text modification element in the form of audio copy is the audio channel; the rendering position of the background image copy element is the Nth layer of the canvas, and so on. Therefore, according to the configuration data, the text modification elements of different images can be inserted into the default rendering position.

[0142] In addition, the splicing order of the original materials corresponding to the copy modification elements is determined according to the positions of the copy fragments corresponding to the copy modification elements in the target copy. Thus, the original materials superimposed with the copy modification elements can be sorted according to the positions of the copy fragments in the target copy, ensuring the coherence of the subsequently generated video.

[0143] Step S406, sending the text modification elements and rendering parameters to the client, so that the client can generate a video according to the original material, the text modification elements and the rendering parameters.

[0144] The client adds each text modification element to the corresponding original material according to the rendering parameters, and generates a video by splicing the original material with the superimposed text modification elements. During the splicing process, the original materials corresponding to the text modification elements are spliced ​​in the splicing order.

[0145] It can be seen that the video generation method provided in the embodiment of the present application, when determining that the target modification element category matching the object feature of the client is a copy category, determines the matching target copy based on the material keywords of the original material, and then determines the copy segments matching each original material from the target copy, and generates copy modification elements based on the copy segments, so that the generated video can correspond to the same copy, thereby ensuring the continuity of the generated video and improving the user experience.

[0146] Embodiment 3

[0147] Figure 6 A flowchart of a video generation method provided in Embodiment 3 of the present application is shown. The video generation method provided in the embodiment of the present application can be executed on the server side.

[0148] Specifically, if Figure 6 As shown, the method comprises the following steps:

[0149] Step S601, receiving a video generation request sent by a client, obtaining original material according to the video generation request; extracting object features of the client; and determining that a target modification element category matching the object features of the client is a sticker category.

[0150] This embodiment is used to optimize the processing method when the target modification element category is a sticker category, that is, to determine that the client object prefers to use stickers to optimize the material.

[0151] Step S602: for each original material, a material keyword recognition model is used to recognize material keywords of the original material.

[0152] The process of identifying the material keywords can refer to the relevant description in the second embodiment, which will not be repeated here.

[0153] Step S603, for each original material, search for a target sticker that matches the material keyword of the original material from the sticker library, use the matched target sticker as a sticker modification element of the original material, and generate rendering parameters of the sticker modification element.

[0154] The sticker library contains at least one sticker, which can be a small-size static picture or a GIF. Some stickers are also bound to corresponding audio such as onomatopoeia. Each sticker has at least one corresponding sticker label, and each sticker label is used to describe the sticker from a corresponding dimension. For example, if the content of a sticker is a dancing cat GIF, the labels assigned to the sticker can be "cat", "happy", "dance", etc. The target sticker of each original material is determined based on the matching degree between the material keywords of the original material and the sticker label of the sticker.

[0155] Specifically, taking any original material as an example, calculate the similarity between the material keywords of the original material and the sticker label of any sticker. The similarity can be text similarity or semantic similarity, etc., and take the sticker with the highest similarity as the target sticker of the original material.

[0156] The target sticker matched to each original material is used as the sticker modification element of the original material, so that the target modification element in this embodiment is the sticker modification element. And the rendering parameters of each sticker modification element are generated, and the rendering parameters are specifically the rendering position of the sticker modification element in the corresponding original material. In the actual implementation process, the default position can be used as the rendering position of the sticker modification element according to the sticker configuration parameters.

[0157] In an optional embodiment, if the original material is an image material, the sticker modification element of each original material is generated by the above method; if the original material is a non-image material (such as a text material and / or an audio material), in addition to using the above method to generate the sticker modification element of each original material, a background modification element is also generated according to the material keyword of the original material. The background modification element can be a background image. For example, a background image and label data of the background image are pre-configured, the similarity between the material keyword and the label data of the background image is calculated, and the background image with the highest similarity is used as the background image matching the original material, and then the corresponding background modification element is generated according to the matching background image.

[0158] Step S604, sending the sticker modification elements and rendering parameters to the client, so that the client can generate a video according to the original material, the sticker modification elements and the rendering parameters.

[0159] The client adds each sticker modification element to the corresponding original material according to the rendering parameters, and generates a video by splicing the original material with the sticker modification element superimposed thereon. The splicing process can be performed in the order in which the original materials are provided.

[0160] It can be seen that the video generation method provided in the embodiment of the present application, when determining that the target modification element category matching the object features of the client is a sticker category, obtains the target stickers matching the material keywords of each original material from the sticker library, thereby generating corresponding sticker modification elements, improving the matching degree between the sticker modification elements and the original material, improving the display effect of the generated video, and improving the user experience.

[0161] In addition, as an optional implementation, if the target modification element category matching the object feature of the client is determined to be a filter category, the material keyword of each original material is determined, and the target filter matching each original material is determined according to the similarity between the material keyword and the filter emotion label of each filter in the filter library, and then the target filter is used as the filter modification element of the corresponding original material, and the filter modification element is returned to the client, and the client applies the filter modification element to the corresponding original material, and generates a video after splicing the materials.

[0162] Embodiment 4

[0163] Figure 7 FIG. 4 shows a flow chart of a video generation method provided in Embodiment 4 of the present application. Specifically, Figure 7 As shown, the method comprises the following steps:

[0164] Step S701: configuring modifying elements of different modifying element categories.

[0165] The configuration center can configure the decorative elements of different decorative element categories, such as configuring different copywriting in the copywriting library, configuring different stickers in the sticker library, configuring different filters in the filter library, etc. Among them, the decorative element categories and the decorative elements under each decorative element category can be flexibly configured, thereby improving the scalability of the system. In addition, the default rendering position of decorative elements of different copywriting categories can also be configured in the configuration center.

[0166] Step S702: Send a video generation request.

[0167] The client sends a video generation request to the server.

[0168] Step S703, obtaining original material and extracting object features of the client.

[0169] The processing end on the server side extracts the original material provided by the client and extracts the object features of the client.

[0170] Step S704: determine the target modifying element category that matches the client object feature.

[0171] The processing end queries the configuration center to determine the categories of each modification element currently configured, and determines the target modification element category.

[0172] Step S705: using the model to identify the material keywords of the original material.

[0173] The processing end calls the pre-trained AI model to identify the content of the original material to determine the material keywords of the original material.

[0174] Step S706: Generate a target modifying element corresponding to each original material and rendering parameters of the target modifying element according to the material keywords.

[0175] The processing end generates the target modification element and the corresponding rendering parameters.

[0176] Step S707, returning the target modification element and rendering parameters.

[0177] The processing end returns the target modification element and rendering parameters to the client.

[0178] Step S708, generating a video according to the original material, the target modification element and the rendering parameters.

[0179] The client generates a video based on the original material, target modification elements, and rendering parameters.

[0180] It can be seen that the video generation method provided in the embodiment of the present application can realize the differentiation of video generation strategies of different clients and improve the matching degree between the generated video and the client; and the generated target modification elements are adapted to the material content, so as to realize the differentiation of modification elements of different original materials in the same video, improve the display effect of the generated video, and improve the user experience; and, the modification elements in the present application can be flexibly configured in the configuration center, thereby improving the flexibility and scalability of the embodiment of the method.

[0181] Embodiment 5

[0182] Figure 8 FIG. 5 shows a structural diagram of a video generation device provided in Embodiment 5 of the present application. Figure 8 As shown, the device 800 includes: a receiving module 810, a material acquisition module 820, a feature extraction module 830, a category determination module 840, a generation module 850, and a sending module 860.

[0183] Receiving module 810, used for receiving a video generation request sent by a client;

[0184] The material acquisition module 820 is used to acquire the original material according to the video generation request;

[0185] Feature extraction module 830, used to extract object features of the client;

[0186] A category determination module 840 is used to select a target modifying element category that matches the object feature from a plurality of modifying element categories;

[0187] A generating module 850 is used to generate, for each original material, a target modifying element corresponding to the original material and rendering parameters of the target modifying element based on the material content of the original material and using a target processing algorithm corresponding to the target modifying element category;

[0188] The sending module 860 is used to send the target modification elements and rendering parameters to the client, so that the client can generate a video according to the original material, the target modification elements and the rendering parameters.

[0189] In an optional implementation, the category determination module 840 is used to: obtain category features of each modification element category;

[0190] Calculate the similarity between the object features and the category features of each modification element category respectively;

[0191] A target modifying element category matching the object feature is selected from a plurality of modifying element categories according to the similarity.

[0192] In an optional implementation, the rendering parameters include:

[0193] The rendering position of the target modification element in the corresponding original material, and / or the splicing order of the original materials corresponding to the target modification element.

[0194] In an optional implementation, if the target modification element category is a copywriting category;

[0195] The generating module 850 is used to: for each original material, use a material keyword recognition model to identify the material keywords of the original material;

[0196] Filter out target copywriting that matches the material keywords of each original material from the copywriting library;

[0197] Determine the copy segments that match each source material from the target copy;

[0198] For each original material, a copy modification element of the original material and rendering parameters of the copy modification element are generated according to the copy segment matched with the original material.

[0199] In an optional implementation, the copy modification element includes: a text copy element, an audio copy element, and / or a background image copy element;

[0200] The generation module 850 is used to: determine the rendering position of the text modification element in the corresponding original material according to the configuration parameters of the text modification element;

[0201] According to the positions of the copy fragments corresponding to the copy modifying elements in the target copy, the splicing order of the original materials corresponding to the copy modifying elements is determined.

[0202] In an optional implementation, if the target modification element category is a sticker category;

[0203] The generating module 850 is used to: for each original material, use a material keyword recognition model to identify the material keywords of the original material;

[0204] For each original material, a target sticker matching the material keyword of the original material is searched from the sticker library, the matching target sticker is used as a sticker modification element of the original material, and rendering parameters of the sticker modification element are generated.

[0205] In an optional implementation, the generation module 850 is used to: if the original material is a non-image material, generate background modification elements according to material keywords of the original material.

[0206] It can be seen that the video generation device provided in the embodiment of the present application selects a target modification element category that matches the object characteristics of the client from a variety of modification element categories, and then uses the target processing algorithm of the target modification element category to generate the target modification element, thereby realizing the differentiation of video generation strategies of different clients and improving the matching degree between the generated video and the client; moreover, the target modification element is generated in combination with the material content of the original material, so that the generated target modification element is adapted to the material content, thereby realizing the differentiation of modification elements of different original materials in the same video, improving the display effect of the generated video, and improving the user experience.

[0207] Embodiment 6

[0208] Fig. 9 A structural diagram of a computing device provided in Example 6 of the present application is shown. The specific embodiments of the present application do not limit the specific implementation of the computing device.

[0209] like Fig. 9 As shown, the computing device may include: a processor (processor) 902 , a communication interface (Communications Interface) 904 , a memory (memory) 906 , and a communication bus 908 .

[0210] The processor 902, the communication interface 904, and the memory 906 communicate with each other via a communication bus 908. The communication interface 904 is used to communicate with other devices such as a client or other server network elements. The processor 902 is used to execute a program 910, which can specifically execute the relevant steps in the above-mentioned video generation method embodiment for a computing device.

[0211] Specifically, the program 910 may include program codes, which include computer operation instructions.

[0212] The processor 902 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0213] The memory 906 is used to store the program 910. The memory 906 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The program 910 can be specifically used to enable the processor 902 to perform the operations in the above method embodiment.

[0214] Embodiment 7

[0215] Embodiment 7 of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to execute operations corresponding to the video generation method in any of the above method embodiments.

[0216] Embodiment 8

[0217] Embodiment 8 of the present application provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to execute operations corresponding to the video generation method in any of the above method embodiments.

[0218] In summary, the computing device, computer storage medium and computer program product provided by the present application select a target modifying element category that matches the object features of the client from a variety of modifying element categories, and use a target processing algorithm for the target modifying element category to generate a target modifying element, thereby achieving differentiation of video generation strategies for different clients and improving the matching degree between the generated video and the client; the present application combines the target processing algorithm and the material content of the original material to generate the target modifying element, so that the generated target modifying element is adapted to the material content, thereby achieving differentiation of the modifying elements of different original materials in the same video, improving the display effect of the generated video, and improving the user experience.

[0219] The algorithm or display provided here are not inherently related to any specific computer, virtual system or other equipment. Various general systems can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present application described here, and the above description of specific languages ​​is to disclose the best mode of implementation of the present application.

[0220] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0221] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0222] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0223] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0224] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0225] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A video generation method, characterized in that: include: Receive a video generation request sent by a client, and obtain original material according to the video generation request; Extracting object features of the client; Screening out a target modifying element category matching the object feature from a plurality of modifying element categories; For each original material, based on the material content of the original material, a target processing algorithm corresponding to the target modification element category is adopted to generate a target modification element corresponding to the original material and rendering parameters of the target modification element; The target modification element and the rendering parameter are sent to the client, so that the client generates a video according to the original material, the target modification element and the rendering parameter.

2. The method according to claim 1, characterized in that The step of selecting a target modifying element category matching the object feature from a plurality of modifying element categories comprises: Obtain the category characteristics of each modification element category; respectively calculating the similarity between the object feature and the category feature of each modification element category; A target modifying element category matching the object feature is screened out from the multiple modifying element categories according to the similarity.

3. The method according to claim 1 or 2, characterized in that: The rendering parameters include: The rendering position of the target modification element in the corresponding original material, and / or the splicing order of the original materials corresponding to the target modification element.

4. The method according to any one of claims 1 to 3, characterized in that If the target modification element category is a copywriting category; Then, for each original material, based on the material content of the original material, using the target processing algorithm corresponding to the target modification element category, generating the target modification element corresponding to the original material and the rendering parameters of the target modification element include: For each original material, a material keyword recognition model is used to identify the material keywords of the original material; Filter out target copywriting that matches the material keywords of each original material from the copywriting library; Determine a text segment matching each original material from the target text; For each original material, a copy modification element of the original material and rendering parameters of the copy modification element are generated according to the copy segment matched with the original material.

5. The method according to claim 4, characterized in that The copy modification elements include: text copy elements, audio copy elements, and / or background image copy elements; The rendering parameters of the text modification element are determined as follows: Determining a rendering position of the text modification element in the corresponding original material according to the configuration parameters of the text modification element; According to the position of the text segment corresponding to the text modification element in the target text, the splicing order of the original materials corresponding to the text modification element is determined.

6. The method according to any one of claims 1 to 3, characterized in that If the target modification element category is a sticker category; Then, for each original material, based on the material content of the original material, using the target processing algorithm corresponding to the target modification element category, generating the target modification element corresponding to the original material and the rendering parameters of the target modification element include: For each original material, a material keyword recognition model is used to identify the material keywords of the original material; For each original material, a target sticker matching the material keyword of the original material is searched from the sticker library, the matching target sticker is used as a sticker modification element of the original material, and rendering parameters of the sticker modification element are generated.

7. The method according to claim 6, characterized in that The method further comprises: If the original material is a non-image material, a background modification element is generated according to a material keyword of the original material.

8. A video generating device, characterized in that: include: A receiving module, used for receiving a video generation request sent by a client; A material acquisition module, used to acquire original material according to the video generation request; A feature extraction module, used to extract object features of the client; A category determination module, used to select a target modifying element category matching the object feature from a plurality of modifying element categories; A generating module, for generating, for each original material, a target modifying element corresponding to the original material and rendering parameters of the target modifying element by adopting a target processing algorithm corresponding to the target modifying element category based on the material content of the original material; The sending module is used to send the target modification element and the rendering parameter to the client, so that the client can generate a video according to the original material, the target modification element and the rendering parameter.

9. A computing device, characterized in that include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the video generation method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The storage medium stores at least one executable instruction, and the executable instruction enables the processor to execute an operation corresponding to the video generation method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the video generation method according to any one of claims 1 to 7.

Images