Video generation method and device and storage medium

Abstract

The embodiment of the invention provides a video generation method and device and a storage medium, and the method comprises the steps: obtaining a material, and obtaining the feature information of the material; determining a target video generation mode according to a preset video generation mode determination rule; if the target video generation mode is a first generation mode, determining a target video template from a preset video template pool according to the feature information of the material, and generating a target video according to the material and the target video template; or if the target video generation mode is a second generation mode, determining a rendering effect combination according to the feature information of the material, and generating the target video according to the material and the rendering effect combination. According to the embodiment of the invention, the appropriate target video template can be selected to generate the target video, or the appropriate rendering effect combination is recommended based on the material features to generate the target video, so that the matching reasonability of the rendering effect combination is ensured, and the effect of the generated video is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer and network communication technologies, and in particular, to a video generation method, device, and storage medium. Background Art

[0002] With the development of network technologies and the improvement of the hardware performance of terminal devices, the demand for video creation and video editing on terminal devices is becoming increasingly strong. In order to lower the threshold of video creation and facilitate users to quickly produce video works, the function of automatically generating videos has emerged as the times require.

[0003] Some existing functions of automatically generating videos usually require users to upload materials, and then automatically add music, special effects, transitions, etc. to the materials for packaging. However, the existing functions of automatically generating videos can only achieve simple packaging processes and have poor effects. Summary of the Invention

[0004] Embodiments of the present disclosure provide a video generation method, device, and storage medium to better achieve automatic video editing and production.

[0005] In a first aspect, embodiments of the present disclosure provide a video generation method, including:

[0006] Obtaining materials and obtaining feature information of the materials;

[0007] Determining a target video generation method according to a preset rule for determining a video generation method;

[0008] If the target video generation method is a first generation method, determining a target video template from a preset video template pool according to the feature information of the materials, and generating a target video according to the materials and the target video template; or

[0009] If the target video generation method is a second generation method, determining a rendering effect combination according to the feature information of the materials, and generating a target video according to the materials and the rendering effect combination.

[0010] In a second aspect, embodiments of the present disclosure provide a video generation device, including:

[0011] An obtaining unit, configured to obtain materials and obtain feature information of the materials;

[0012] A determining unit, configured to determine a target video generation method according to a preset rule for determining a video generation method;

[0013] The first generation unit is configured to, if the target video generation method is the first generation method, determine a target video template from a preset video template pool according to the feature unit for information of the material, and generate a target video according to the material and the target video template; or

[0014] The second generation unit is configured to, if the target video generation method is the second generation method, determine a rendering effect combination according to the feature information of the material, and generate a target video according to the material and the rendering effect combination.

[0015] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0016] The memory stores computer-executable instructions;

[0017] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the video generation method described in the first aspect and various possible designs of the first aspect above.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored, and when a processor executes the computer-executable instructions, the video generation method described in the first aspect and various possible designs of the first aspect above is implemented.

[0019] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including computer-executable instructions, and when a processor executes the computer-executable instructions, the video generation method described in the first aspect and various possible designs of the first aspect above is implemented.

[0020] The video generation method, device, and storage medium provided by the embodiments of the present disclosure obtain a material and obtain the feature information of the material; determine a target video generation method according to a preset rule for determining a video generation method; if the target video generation method is the first generation method, determine a target video template from a preset video template pool according to the feature information of the material, and generate a target video according to the material and the target video template; or if the target video generation method is the second generation method, determine a rendering effect combination according to the feature information of the material, and generate a target video according to the material and the rendering effect combination. In the embodiments of the present disclosure, a suitable target video template can be selected to generate a target video, or a suitable rendering effect combination can be recommended based on the material features to generate a target video, ensuring the rationality of the rendering effect combination, avoiding conflicts in the rendering effect combination, improving the generated video effect, and meeting the needs of different users by selecting the above two target video generation methods, thereby improving the user experience. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present disclosure. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 It is a scenario example diagram of the video generation method provided by an embodiment of the present disclosure;

[0023] Figure 2 It is a schematic flowchart of the video generation method provided by an embodiment of the present disclosure;

[0024] Figure 3 It is a schematic flowchart of the video generation method provided by another embodiment of the present disclosure;

[0025] Figure 4 It is a structural block diagram of the video generation device provided by an embodiment of the present disclosure;

[0026] Figure 5 It is a schematic hardware structure diagram of the electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present disclosure.

[0028] Some existing functions for automatically generating videos usually require users to upload materials, and then automatically perform packaging processes such as adding music, special effects, and transitions to the materials. However, the existing functions for automatically generating videos can only achieve simple packaging processes and cannot achieve more complex automated editing and production. Moreover, the added music, special effects, transitions, etc. may not be appropriate, such as unreasonable matching and conflicts, resulting in poor video effects.

[0029] To solve the above technical problems, the present disclosure provides a video generation method, which includes obtaining materials and obtaining the feature information of the materials; determining a target video generation method according to a preset video generation method determination rule; if the target video generation method is the first generation method, determining a target video template from a preset video template pool according to the feature information of the materials, and generating a target video according to the materials and the target video template; or if the target video generation method is the second generation method, determining a rendering effect combination according to the feature information of the materials, and generating a target video according to the materials and the rendering effect combination. The present disclosure provides two target video generation methods, which can select a suitable target video template to generate a target video, or recommend a suitable rendering effect combination based on the material features to generate a target video, ensuring the rationality of the rendering effect combination, avoiding conflicts in the rendering effect combination, improving the generated video effect, and meeting the needs of different users by selecting the above two target video generation methods, thereby improving the user experience.

[0030] The video generation method provided by the present disclosure is applicable to electronic devices such as terminal devices or servers. For example, Figure 1 as shown, if the video generation method provided by the present disclosure is used in a terminal device, the terminal device can obtain materials and obtain the feature information of the materials; determine a target video generation method according to a preset video generation method determination rule; if the target video generation method is the first generation method, determine a target video template from a preset video template pool according to the feature information of the materials, adaptively adjust the target video template according to the materials, and generate a target video according to the materials and the target video template; or if the target video generation method is the second generation method, determine a rendering effect combination according to the feature information of the materials, and generate a target video according to the materials and the rendering effect combination.

[0031] In addition, after the second generation method determines the rendering effect combination, post-processing based on rules can also be performed, that is, determining whether each rendering effect in the target rendering effect sequence conforms to the preset rules in the preset rule pool; if any rendering effect does not conform to the rules, replacing or deleting the rendering effect, further ensuring the rationality of the rendering effect combination, avoiding conflicts in the rendering effect combination, and improving the generated video effect.

[0032] If the video generation method provided by the present disclosure is used in a server, the server may receive materials uploaded by a terminal, and the server obtains the feature information of the materials; determine the target video generation method according to the preset video generation method determination rules; if the target video generation method is the first generation method, then determine the target video template from the preset video template pool according to the feature information of the materials, and the target video template can be adaptively adjusted according to the materials, and generate the target video according to the materials and the target video template; or if the target video generation method is the second generation method, then determine the rendering effect combination according to the feature information of the materials, and generate the target video according to the materials and the rendering effect combination, and send it to the terminal device.

[0033] It should be noted that the user information and data involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0034] The video generation method of the present disclosure will be introduced in detail below in conjunction with specific embodiments.

[0035] Reference Figure 2 , Figure 2 is a schematic flowchart of a video generation method provided by an embodiment of the present disclosure. The method of this embodiment can be applied to a terminal device or a server. The video generation method includes:

[0036] S201. Obtain materials and obtain the feature information of the materials.

[0037] In this embodiment, the user can input one or more materials. The materials are multimedia content for generating videos, such as pictures, videos, audios, etc.; for each material, the feature information can be extracted.

[0038] Optionally, the feature information may include, but is not limited to, image feature information, audio feature information, the quantity information of the materials, the time information of the materials, etc.

[0039] The image feature information is the feature information extracted based on the image of the material, and may include, but is not limited to, the style, theme, mood, scene, characters, etc. of the picture / video; the audio feature information is the feature information extracted based on the audio of the material, and may include, but is not limited to, the beat points, chorus points (chorus start positions), musical phrases, rhythm (beats per minute bpm), total audio duration, etc.; the quantity information of the materials is also the number of materials; the time information of the materials may include the duration of each material, the total duration of all materials, the duration of the final video (finally generating the target video) determined according to the total audio duration, etc.; of course, the feature information may also include other feature information such as resolution, and will not be exemplified one by one here.

[0040] Optionally, obtaining the image feature information and audio feature information of the material can be achieved by means of a preset feature extraction model. For example, for each material, one or more video frames are extracted, and the video frames of the material are input into the preset feature extraction model. Feature extraction is performed through the preset feature extraction model to obtain the feature information of the material. The preset feature extraction model can be a neural network model or other machine learning models, and training samples can be collected and trained in advance, which will not be elaborated here.

[0041] It should be noted that when obtaining the feature information of the material, some feature information can be omitted. The advantage is that the interface for obtaining the feature information of the material can be omitted, greatly reducing the time consumption, and decoupling the dependence of the final video effect on the material. That is, for some scenarios where the requirement for the final video effect is not high but the requirement for time consumption is high (hoping for the shortest possible time consumption), the process of obtaining the feature information of the material can also be omitted.

[0042] S202. Determine the target video generation method according to the preset video generation method determination rule.

[0043] In this embodiment, two video generation methods are pre-configured, including the first generation method and the second generation method. The first generation method is to generate a video using a preset video template. In the preset video template, fixed animation combinations, transition combinations, special effect combinations, etc. are pre-configured. Only by applying multiple materials to the preset video template can the target video be generated. The time consumption for generating the target video is shorter, the efficiency is higher, and because the fixed animation combinations, transition combinations, and special effect combinations are used in the preset video template, the combination is more appropriate and the final video effect is better. The second generation method is to recommend a rendering effect combination according to the feature information of the material through a recommendation algorithm, including but not limited to animation combinations, transition combinations, and special effect combinations, and then synthesize the material and the rendering effect combination into the target video. That is, the second generation method does not have a fixed video template and has diversity, which can meet the personalized and diverse needs of users for the target video.

[0044] The determination of the target video generation method can be achieved by a random method or by using a specific algorithm. For example, different target video generation methods can be adopted according to user needs. For users with the need for diverse effects, the second generation method can be selected, and the generated target video is more diverse and personalized. For users with the need for stable effects, the first generation method can be selected, and a target video with less diversity but better effects can be generated. More specifically, the mapping relationship between different user need information and different video generation methods can be pre-configured, and the target video generation method corresponding to the user need can be determined according to the mapping relationship.

[0045] S203. If the target video generation method is the first generation method, determine a target video template from a preset video template pool according to the feature information of the material, and generate a target video according to the material and the target video template.

[0046] In this embodiment, different relatively general and high-quality preset video templates are designed in advance to form a preset video template pool. Further, if it is determined that the target video generation method is the first generation method, a target video template can be selected from the preset video template pool to generate a target video. In this embodiment, the target video template can be determined from the preset video template pool based on the feature information of the material obtained in S201 above.

[0047] Specifically, when designing different preset video templates in advance, the feature information of the preset video template can be determined, such as the suitable duration, the number of materials, the style, the atmosphere, etc. of the preset video template. Further, when determining the target video template, the feature information of each preset video template in the preset video template pool can be matched with the feature information of the material, and the target video template can be determined according to the matching result. For example, if there is a preset video template that matches, the preset video template is determined as the target video template. If there are multiple preset video templates that match, one of the multiple matching preset video templates is selected as the target video template. Optionally, a matching degree index for each preset video template can be given, and the preset video template with the highest matching degree index is determined as the target video template.

[0048] Optionally, when matching the feature information of each preset video template in the preset video template pool with the feature information of the material, the unmatched preset video templates can be filtered out. For example, if the atmosphere of the material is sad, the preset video templates with a happy atmosphere are filtered out. Further, a preset video template is selected from the remaining preset video templates as the target video template. If there is only one preset video template among the remaining preset video templates, the preset video template is determined as the target video template. If there are multiple preset video templates among the remaining preset video templates, one of them is selected as the target video template.

[0049] After determining the target video template, a target video can be generated according to the material and the target video template. Specifically, the material is filled into the target video template, and the rendering effects such as the animation combination, transition combination, and special effect combination in the target video template are synthesized with the material to finally generate the target video.

[0050] S204. If the target video generation method is the second generation method, determine a rendering effect combination according to the feature information of the material, and generate a target video according to the material and the rendering effect combination.

[0051] In this embodiment, if it is determined that the target video generation method is the second generation method, a recommendation algorithm is used to recommend a rendering effect combination based on the feature information of the material, including but not limited to an animation combination, a transition combination, a special effect combination, etc. Then, the rendering effect combination is synthesized with the material, and finally the target video is generated.

[0052] When determining the rendering effect combination, the rendering effect can be recommended according to the feature information of the material, and then a rendering effect sequence is obtained as the rendering effect combination.

[0053] Optionally, a correlation index between the feature information of the material and the candidate rendering effect can be obtained, and a correlation matrix is constructed; according to the correlation matrix, a target rendering effect sequence is determined from the candidate rendering effects, and the target rendering effect sequence is used as the rendering effect combination.

[0054] In this embodiment, a correlation index between the feature information of each material and each candidate rendering effect can be obtained, and then a correlation matrix is constructed. Assuming there are M materials and N candidate rendering effects, the correlation matrix is an M×N matrix. The correlation index can be the similarity between the feature information of the material and the feature information of the candidate rendering effect, and the cosine similarity can be used, or other similarities can also be used. The feature information of the candidate rendering effect can be preset. Further, a target rendering effect sequence is determined based on the correlation matrix, and the target rendering effect sequence is used as the rendering effect combination.

[0055] Optionally, the beam search algorithm (Beam Search) can be used, or the greedy algorithm can also be used to determine the rendering effect that matches each material from the correlation matrix to obtain the target rendering effect sequence. Taking the beam search algorithm as an example, a beam width (referred to as bs in this embodiment) can be specified in advance. When searching for the rendering effect that matches any material from the correlation matrix, the total correlation index (total similarity) of the new rendering effect sequence after adding each candidate rendering effect to the current existing rendering effect sequence is obtained, and the bs new rendering effect sequences with the highest total correlation index are retained; next, when searching for the rendering effect that matches the next material, on the basis of the bs new rendering effect sequences retained in the previous step, search is carried out respectively, and the bs new rendering effect sequences with the highest total correlation index are retained again, and this step is looped until the matching ends. Finally, bs rendering effect sequences are obtained, and the one with the highest total correlation index among them is taken as the final target rendering effect sequence.

[0056] Optionally, different correlation matrices can be constructed for the candidate animation, candidate transition, and candidate special effect respectively, and the animation sequence, transition sequence, and special effect sequence can be obtained respectively through the above process.

[0057] Furthermore, the determined rendering effect combinations (animation sequences, transition sequences, special effect sequences) are combined with the materials to finally generate the target video.

[0058] The video generation method provided in this embodiment obtains materials and obtains the characteristic information of the materials; determines the target video generation method according to the preset rules for determining the video generation method; if the target video generation method is the first generation method, determines the target video template from the preset video template pool according to the characteristic information of the materials, and generates the target video according to the materials and the target video template; or if the target video generation method is the second generation method, determines the rendering effect combination according to the characteristic information of the materials, and generates the target video according to the materials and the rendering effect combination. In this embodiment, an appropriate target video template can be selected to generate the target video, or a suitable rendering effect combination can be recommended based on the material characteristics to generate the target video, ensuring the rationality of the combination of rendering effect combinations, avoiding conflicts in the rendering effect combinations, improving the generated video effect, and meeting the needs of different users by selecting the above two target video generation methods, thereby improving the user experience.

[0059] Based on any of the above embodiments, when generating the target video according to the materials and the target video template in S203, considering that the target video template may not fully match the materials, the target video template can be adaptively adjusted according to the materials, and the adaptively adjusted target template is fused with the materials to finally generate the target video.

[0060] Optionally, the adaptive adjustment includes but is not limited to the adaptive filling of the animation sequence in the target video template, the adaptive filling of the transition sequence, the adaptive filling of the special effects, and the configuration and distribution of the fine-grained parameters (such as duration, intensity, etc.) of each rendering effect (animation, transition, special effect).

[0061] Optionally, the adaptive filling of the animation sequence is to fill each animation in the animation sequence into a suitable position. The animation can be various deformations, displacements, etc. of the materials. Therefore, each animation corresponds to a material, and the action range of each animation is from the start to the end of a material. Therefore, each material from the start to the end can be used as a slot, and the animation sequence in the target video template is automatically added to each corresponding slot of the material in sequence.

[0062] If there are multiple materials, considering that the number of animations in the animation sequence may not be equal to the number of materials, it is necessary to adaptively adjust the animation sequence according to the number of animations in the animation sequence and the number of materials, so that the number of animations in the animation sequence is equal to the number of materials, and the animations in the animation sequence can just fill each corresponding slot of the materials.

[0063] Optionally, if the number of animations in the animation sequence is greater than the number of materials, the animations in the animation sequence are automatically added to the respective slots corresponding to the materials in sequence, and the extra animations are deleted; if the number of animations in the animation sequence is less than the number of materials, the animations in the animation sequence can be repeated in order so that the number of repeated animations is equal to the number of materials. For example, if there are 7 materials and there are 3 animations [A, B, C] in the animation sequence, the animation sequence can be repeated in order so that the repeated animation sequence has 7 animations, which are [A, B, C, A, B, C, A]; in another alternative embodiment, if the number of animations in the animation sequence is less than the number of materials and the number of animations in the animation sequence is odd, by analyzing a large number of videos, it can be found that the first animation is usually the opening animation. Therefore, the animations in the animation sequence except the first animation can be repeated in order so that the number of repeated animations is equal to the number of materials. For example, in the above example, the number of animations 3 in the animation sequence [A, B, C] is odd, so only B and C are repeated, and the repeated animation sequence has 7 animations, which are [A, B, C, B, C, B, C]; in another alternative embodiment, if the number of animations in the animation sequence is less than the number of materials and the number of animations in the animation sequence is even, by analyzing a large number of videos, it can be found that when the number of animations in the animation sequence is even, the animations usually appear in pairs. Therefore, the animations in the animation sequence can be repeated in order so that the number of repeated animations is equal to the number of materials. For example, in the above example, the number of animations 2 in the animation sequence [A, B] is even, so A and B are repeated, and the repeated animation sequence has 7 animations, which are [A, B, A, B, A, B, A]; it should be noted that if the number of animations in the animation sequence is 1, the animation in the animation sequence is probably the opening animation, and the animation sequence is not repeated, and the animation is added to the first slot.

[0064] Optionally, the adaptive filling of the transition sequence is to fill each transition in the transition sequence into a suitable position. Since the transition is usually located between two adjacent materials, the position between two adjacent materials can be used as a slot, and the transition sequence in the target video template is automatically added to each slot in sequence.

[0065] If there are multiple materials, considering that the number of transitions in the transition sequence may not be equal to the number of material intervals of the materials, that is, not equal to the number of slots, it is necessary to adaptively adjust the transition sequence according to the number of transitions in the transition sequence and the number of materials so that the number of transitions in the transition sequence is equal to the number of material intervals (that is, equal to the number of materials minus 1), and the transitions in the transition sequence can just fill each slot.

[0066] Optionally, if the number of transitions in the transition sequence is greater than the number of material intervals, the transition sequence is automatically added to each slot between adjacent materials in sequence, and the redundant transitions are deleted; if the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in sequence, so that the number of transitions after the repetition is equal to the number of material intervals, and the transitions in the transition sequence can just fill each slot. For example, if there are 7 materials, there are 6 slots, and there are 3 transitions in the transition sequence, which are [D, E, F]. The transition sequence can be repeated in sequence, so that the transition sequence after the repetition has 6 transitions, which are [D, E, F, D, E, F].

[0067] Optionally, the adaptive filling of special effects is to fill the special effects in the target video template to the appropriate position, and it is necessary to determine the setting position of the special effects included in the target video template in the material. The special effects may include but are not limited to global special effects, highlight special effects, card point special effects, opening special effects, closing special effects, etc.

[0068] If the target video template includes global special effects, the setting position of the global special effects is determined to be the entire time range of the material, that is, the scope of the global special effects is from the beginning to the end of the material. If there are multiple materials, the scope of the global special effects is from the beginning of the first material to the end of the last material. The global special effects can be set to take effect in the finished film.

[0069] If the target video template includes a highlight effect, the highlight effect can also be effective in the finished film. It can be determined whether the music attribute features included in the feature information of the material include the starting position of the music chorus. If so, the setting position of the highlight effect is determined to be the starting position of the music chorus; or, if not, the setting position of the highlight effect is determined to be the starting position of the second material.

[0070] If the target video template includes a card point effect, the card point effect sequence can be repeated in full, and the length of the repeated sequence is aligned with the number of materials. It is also necessary to determine the starting position of the card point effect. Specifically, the setting position of the card point effect can be determined as the starting position of the second material, or the setting position of the card point effect can be determined as the starting position of the second material after the highlight effect. That is, if there is no highlight effect, the starting position of the card point effect can start from the starting position of the second material. If there is a highlight effect, the starting position of the card point effect can start from the starting position of the second material after the highlight effect, that is, one slot is extended.

[0071] If the target video template includes an opening special effect, the setting position of the opening special effect is determined to be the starting position of the first material, and the opening special effect can be set to take effect in the finished film.

[0072] If the target video template includes a closing special effect, determine that the setting position of the closing special effect is the end position of the last material, and the closing special effect can be set to take effect in the finished video.

[0073] Based on any of the above embodiments, after determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix in S204, post-processing based on rules can be further performed to further ensure the rationality of the combination of rendering effects, avoid conflicts in the combination of rendering effects, and improve the generated video effect. For example, Figure 3 as shown, specifically including:

[0074] S301. Determine whether each rendering effect in the target rendering effect sequence conforms to a preset rule;

[0075] S302. If any rendering effect does not conform to the rule, replace or delete the rendering effect.

[0076] In this embodiment, considering that there may be unreasonable rendering effects in the target rendering effect sequence recommended by the recommendation algorithm, some preset rules can be pre-configured to build a preset rule pool to check the rationality of the rendering effects in the target rendering effect sequence. If it is determined that any rendering effect in the target rendering effect sequence does not conform to the rule, it means that the rendering effect is unreasonable, and the rendering effect can be replaced or deleted.

[0077] For example, category labels are added to different candidate rendering effects in advance, and the preset rule is that the category labels of the rendering effects in the target rendering effect sequence are the same. Therefore, after obtaining the target rendering effect sequence, it can be checked whether the category labels of the rendering effects in the target rendering effect sequence are the same. If there is a rendering effect in the target rendering effect sequence with a category label different from other rendering effects, the rendering effect with different categories can be replaced with another rendering effect with a category label the same as other rendering effects, so as to ensure the unity of the category labels of the rendering effects in the target rendering effect sequence.

[0078] Of course, the preset rules are not limited to the above examples, and can also be designed according to actual needs, and will not be listed one by one here. Optionally, different preset rules can also be set for the animation combination, transition combination, and special effect combination respectively.

[0079] Based on any of the above embodiments, if only one material is obtained when obtaining the materials in S201, that is, there is only one segment of material, the duration of the material can be judged. If the duration of the material does not exceed the preset duration threshold (for example, 15 seconds), the music variation point is determined according to the music attribute feature included in the feature information of the material, and the playback speed of the part of the material after the music variation point is reduced to achieve the effect of speed change at the beat, and then the above video generation process is continued with this one segment of material, and finally the target video is generated.

[0080] If the duration of the material exceeds a preset duration threshold (e.g., 15 seconds), the material is divided into multiple segments of materials, and the above video generation process is performed on these multiple segments of materials to finally generate a target video. The segmentation of the material can be performed according to a preset duration, for example, segmented by 10 seconds. If it cannot be evenly divided, the last remaining part is merged into the last segment of the material. The number of segments of the segmented material is the total duration of the material divided by 10. For example, if the total duration of the material is 33 seconds, it is segmented into 3 segments of materials: 10 seconds, 10 seconds, and 13 seconds.

[0081] Based on any of the above embodiments, when generating the target video, a target video can be generated through a pre-configured draft protocol, and a renderable configuration file is issued. The draft protocol includes various track information and configuration information required for video editing, ultimately ensuring the unified format of the target video.

[0082] Corresponding to the video generation method in the above embodiments, Figure 4 is a structural block diagram of a video generation device provided by an embodiment of the present disclosure. For the sake of illustration, only parts related to the embodiments of the present disclosure are shown. Referring to Figure 4 the video generation device 400 includes: an acquisition unit 401, a determination unit 402, a first generation unit 403, and a second generation unit 404.

[0083] Among them, the acquisition unit 401 is used to acquire the material and acquire the feature information of the material;

[0084] The determination unit 402 is used to determine the target video generation method according to the preset video generation method determination rules;

[0085] The first generation unit 403 is used to, if the target video generation method is the first generation method, determine a target video template from a preset video template pool according to the feature information of the material, and generate a target video according to the material and the target video template; or

[0086] The second generation unit 404 is used to, if the target video generation method is the second generation method, determine a rendering effect combination according to the feature information of the material, and generate a target video according to the material and the rendering effect combination.

[0087] In one or more embodiments of the present disclosure, when the determination unit 402 determines the target video generation method according to the preset video generation method determination rules, it is used to:

[0088] Acquire user requirement information and determine the target video generation method according to the user requirement information; or

[0089] Determine the target video generation method in a random manner.

[0090] In one or more embodiments of the present disclosure, when the first generation unit 403 determines a target video template from a preset video template pool according to the feature information of the material, it is used for:

[0091] Match the feature information of each preset video template in the preset video template pool with the feature information of the material, and determine the target video template according to the matching result.

[0092] In one or more embodiments of the present disclosure, when the first generation unit 403 matches the feature information of each preset video template in the preset video template pool with the feature information of the material and determines the target video template according to the matching result, it is used for:

[0093] Match the feature information of each preset video template in the preset video template pool with the feature information of the material, filter out the unmatched preset video templates, and select one preset video template from the remaining preset video templates as the target video template.

[0094] In one or more embodiments of the present disclosure, when the first generation unit 403 generates a target video according to the material and the target video template, it is used for:

[0095] Adaptively adjust the target video template according to the material, and fuse the adaptively adjusted target template with the material to generate the target video.

[0096] In one or more embodiments of the present disclosure, when the first generation unit 403 adaptively adjusts the target video template according to the material, it is used for:

[0097] Adaptively adjust the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials; and / or

[0098] Adaptively adjust the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of materials; and / or

[0099] Determine the setting position of the special effects included in the target video template in the material.

[0100] In one or more embodiments of the present disclosure, if there are multiple materials, when the first generation unit 403 adaptively adjusts the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials, it is used for:

[0101] If the number of animations in the animation sequence is less than the number of the materials, each animation in the animation sequence is repeated in order so that the number of repeated animations is equal to the number of the materials; or

[0102] If the number of animations in the animation sequence is less than the number of the materials and the number of animations in the animation sequence is odd, the remaining animations in the animation sequence except the first animation are repeated in order so that the number of repeated animations is equal to the number of the materials; or

[0103] If the number of animations in the animation sequence is less than the number of the materials and the number of animations in the animation sequence is even, each animation in the animation sequence is repeated in order so that the number of repeated animations is equal to the number of the materials.

[0104] In one or more embodiments of the present disclosure, if there are multiple materials, when the first generating unit 403 adaptively adjusts the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials, it is configured to:

[0105] If the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in order so that the number of repeated transitions is equal to the number of material intervals, where the number of material intervals is the number of the materials minus 1.

[0106] In one or more embodiments of the present disclosure, if there are multiple materials, when the first generating unit 403 determines the setting position of the special effect included in the target video template in the materials, it is configured to:

[0107] If the target video template includes a global special effect, determine that the setting position of the global special effect is the entire time range of the materials; or

[0108] If the target video template includes a highlight special effect, determine whether the music attribute feature included in the feature information of the material includes the starting position of the music chorus. If it includes, determine that the setting position of the highlight special effect is the starting position of the music chorus; or, if it does not include, determine that the setting position of the highlight special effect is the starting position of the second material; or

[0109] If the target video template includes a beat special effect, determine that the setting position of the beat special effect is the starting position of the second material, or determine that the setting position of the beat special effect is the starting position of the second material after the highlight special effect; or

[0110] If the target video template includes an opening special effect, determine the setting position of the opening special effect as the starting position of the first material; or

[0111] If the target video template includes a closing special effect, determine the setting position of the closing special effect as the ending position of the last material.

[0112] In one or more embodiments of the present disclosure, when determining the rendering effect combination according to the feature information of the material, the second generating unit 404 is configured to:

[0113] Obtain the correlation index between the feature information of the material and the candidate rendering effects, and construct a correlation matrix;

[0114] Determine a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and use the target rendering effect sequence as the rendering effect combination;

[0115] Wherein the rendering effects include one or more of animation, transition, and special effect.

[0116] In one or more embodiments of the present disclosure, when determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the second generating unit 404 is configured to:

[0117] Based on the correlation matrix, use the beam search algorithm to obtain the rendering effects matching the material, and obtain the target rendering effect sequence.

[0118] In one or more embodiments of the present disclosure, after determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the second generating unit 404 is further configured to:

[0119] Judge whether each rendering effect in the target rendering effect sequence conforms to a preset rule;

[0120] If any rendering effect does not conform to the rule, replace or delete the rendering effect.

[0121] In one or more embodiments of the present disclosure, the obtaining unit 401 is further configured to:

[0122] If there is only one piece of material, judge the duration of the material. If the duration of the material does not exceed the preset duration threshold, determine the music variation point according to the music attribute feature included in the feature information of the material, and reduce the playing speed of the part of the material after the music variation point; or

[0123] If the duration of the material exceeds the preset duration threshold, divide the material into multiple pieces of material.

[0124] In one or more embodiments of the present disclosure, when the obtaining unit 401 obtains the feature information of the material, it is configured to:

[0125] Extract video frames from the material;

[0126] Input the video frames of the material into a preset feature extraction model, and obtain the feature information of the material through the preset feature extraction model.

[0127] In one or more embodiments of the present disclosure, the feature information of the material includes: image feature information, audio feature information, the quantity information of the material, and the time information of the material.

[0128] The device provided in this embodiment can be used to execute the technical solutions of the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0129] Refer to Figure 5 , which shows a schematic structural diagram of an electronic device 500 suitable for implementing the embodiments of the present disclosure. The electronic device 500 can be a terminal device or a server. Among them, the terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable media players (PMPs), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present disclosure.

[0130] As Figure 5 shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0131] Typically, the following devices can be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 can allow the electronic device 500 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 5 the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices can be alternatively implemented or had.

[0132] Specifically, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0133] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0134] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; it can also exist separately without being assembled into the electronic device.

[0135] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device is caused to execute the method shown in the above-mentioned embodiments.

[0136] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0138] The units described in the embodiments of the present disclosure may be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation on the unit itself in some cases. For example, the first acquisition unit may also be described as "the unit for acquiring at least two Internet protocol addresses".

[0139] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and so on.

[0140] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] In a first aspect, according to one or more embodiments of the present disclosure, there is provided a video generation method, including:

[0142] Obtain materials and obtain the feature information of the materials;

[0143] Determine a target video generation method according to a preset rule for determining a video generation method;

[0144] If the target video generation method is a first generation method, determine a target video template from a preset video template pool according to the feature information of the materials, and generate a target video according to the materials and the target video template; or

[0145] If the target video generation method is a second generation method, determine a rendering effect combination according to the feature information of the materials, and generate a target video according to the materials and the rendering effect combination.

[0146] According to one or more embodiments of the present disclosure, the determining of the target video generation method according to the preset rule for determining a video generation method includes:

[0147] Obtain user requirement information and determine the target video generation method according to the user requirement information; or

[0148] Determine the target video generation method in a random manner.

[0149] According to one or more embodiments of the present disclosure, the determining of the target video template from the preset video template pool according to the feature information of the materials includes:

[0150] Match the feature information of each preset video template in the preset video template pool with the feature information of the materials, and determine the target video template according to the matching result.

[0151] According to one or more embodiments of the present disclosure, matching the feature information of each preset video template in the preset video template pool with the feature information of the material, and determining the target video template according to the matching result includes:

[0152] Match the feature information of each preset video template in the preset video template pool with the feature information of the material, filter out the unmatched preset video templates, and select one preset video template from the remaining preset video templates as the target video template.

[0153] According to one or more embodiments of the present disclosure, generating a target video according to the material and the target video template includes:

[0154] Adaptive adjustment of the target video template according to the material, and fusing the adaptively adjusted target template with the material to generate the target video.

[0155] According to one or more embodiments of the present disclosure, the adaptive adjustment of the target video template according to the material includes:

[0156] Adaptive adjustment of the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials; and / or

[0157] Adaptive adjustment of the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of materials; and / or

[0158] Determine the setting position of the special effects included in the target video template in the material.

[0159] According to one or more embodiments of the present disclosure, if there are multiple materials, the adaptive adjustment of the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials includes:

[0160] If the number of animations in the animation sequence is less than the number of materials, then repeat each animation in the animation sequence in order so that the number of repeated animations is equal to the number of materials; or

[0161] If the number of animations in the animation sequence is less than the number of materials and the number of animations in the animation sequence is odd, then repeat the remaining animations in the animation sequence except the first animation in order so that the number of repeated animations is equal to the number of materials; or

[0162] If the number of animations in the animation sequence is less than the number of the materials and the number of animations in the animation sequence is even, each animation in the animation sequence is repeated in order so that the number of repeated animations is equal to the number of the materials.

[0163] According to one or more embodiments of the present disclosure, if there are multiple materials, the adaptive adjustment of the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials includes:

[0164] If the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in order so that the number of repeated transitions is equal to the number of material intervals, where the number of material intervals is the number of the materials minus 1.

[0165] According to one or more embodiments of the present disclosure, if there are multiple materials, determining the setting positions of the special effects included in the target video template in the materials includes:

[0166] If the target video template includes global special effects, determining the setting position of the global special effects as the entire time range of the materials; or

[0167] If the target video template includes highlight special effects, determining whether the music attribute feature included in the feature information of the materials includes the starting position of the music chorus. If it includes, determining the setting position of the highlight special effects as the starting position of the music chorus; or, if it does not include, determining the setting position of the highlight special effects as the starting position of the second material; or

[0168] If the target video template includes beat special effects, determining the setting position of the beat special effects as the starting position of the second material, or determining the setting position of the beat special effects as the starting position of the second material after the highlight special effects; or

[0169] If the target video template includes opening special effects, determining the setting position of the opening special effects as the starting position of the first material; or

[0170] If the target video template includes closing special effects, determining the setting position of the closing special effects as the ending position of the last material.

[0171] According to one or more embodiments of the present disclosure, determining the rendering effect combination according to the feature information of the materials includes:

[0172] Obtaining the correlation index between the feature information of the materials and the candidate rendering effects, and constructing a correlation matrix;

[0173] Determine a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and use the target rendering effect sequence as the rendering effect combination;

[0174] Wherein the rendering effects include one or more of animations, transitions, and special effects.

[0175] According to one or more embodiments of the present disclosure, the determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix includes:

[0176] Based on the correlation matrix, use a beam search algorithm to obtain rendering effects that match the material, and obtain a target rendering effect sequence.

[0177] According to one or more embodiments of the present disclosure, after determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix, it further includes:

[0178] Judge whether each rendering effect in the target rendering effect sequence conforms to a preset rule;

[0179] If any rendering effect does not conform to the rule, replace or delete the rendering effect.

[0180] According to one or more embodiments of the present disclosure, the method further includes:

[0181] If there is only one piece of material, judge the duration of the material. If the duration of the material does not exceed a preset duration threshold, determine the music variation point according to the music attribute characteristics included in the feature information of the material, and reduce the playback speed of the part of the material after the music variation point; or

[0182] If the duration of the material exceeds the preset duration threshold, divide the material into multiple pieces of material.

[0183] According to one or more embodiments of the present disclosure, the obtaining the feature information of the material includes:

[0184] Extract video frames from the material;

[0185] Input the video frames of the material into a preset feature extraction model, and obtain the feature information of the material through the preset feature extraction model.

[0186] According to one or more embodiments of the present disclosure, the feature information of the material includes: image feature information, audio feature information, the quantity information of the material, and the time information of the material.

[0187] In a second aspect, according to one or more embodiments of the present disclosure, there is provided a video generation device, including:

[0188] An acquisition unit for acquiring materials and acquiring feature information of the materials;

[0189] A determination unit for determining a target video generation method according to a preset video generation method determination rule;

[0190] A first generation unit, if the target video generation method is the first generation method, for determining a target video template from a preset video template pool according to the feature information of the materials, and generating a target video according to the materials and the target video template; or

[0191] A second generation unit, if the target video generation method is the second generation method, for determining a rendering effect combination according to the feature information of the materials, and generating a target video according to the materials and the rendering effect combination.

[0192] According to one or more embodiments of the present disclosure, when the determination unit determines the target video generation method according to the preset video generation method determination rule, it is used for:

[0193] Acquiring user requirement information, and determining the target video generation method according to the user requirement information; or

[0194] Determining the target video generation method in a random manner.

[0195] According to one or more embodiments of the present disclosure, when the first generation unit determines a target video template from a preset video template pool according to the feature information of the materials, it is used for:

[0196] Matching the feature information of each preset video template in the preset video template pool with the feature information of the materials, and determining the target video template according to the matching result.

[0197] According to one or more embodiments of the present disclosure, when the first generation unit matches the feature information of each preset video template in the preset video template pool with the feature information of the materials, and determines the target video template according to the matching result, it is used for:

[0198] Matching the feature information of each preset video template in the preset video template pool with the feature information of the materials, filtering out the unmatched preset video templates, and selecting one preset video template from the remaining preset video templates as the target video template.

[0199] According to one or more embodiments of the present disclosure, when the first generation unit generates a target video according to the materials and the target video template, it is used for:

[0200] Adapting the target video template according to the material, and fusing the adaptively adjusted target template with the material to generate the target video.

[0201] According to one or more embodiments of the present disclosure, when the first generating unit adaptively adjusts the target video template according to the material, it is configured to:

[0202] Adapting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of the materials; and / or

[0203] Adapting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials; and / or

[0204] Determining the setting position of the special effects included in the target video template in the material.

[0205] According to one or more embodiments of the present disclosure, if there are multiple materials, when the first generating unit adapts the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of the materials, it is configured to:

[0206] If the number of animations in the animation sequence is less than the number of the materials, repeating each animation in the animation sequence in order so that the number of repeated animations is equal to the number of the materials; or

[0207] If the number of animations in the animation sequence is less than the number of the materials and the number of animations in the animation sequence is odd, repeating the remaining animations in the animation sequence except the first animation in order so that the number of repeated animations is equal to the number of the materials; or

[0208] If the number of animations in the animation sequence is less than the number of the materials and the number of animations in the animation sequence is even, repeating each animation in the animation sequence in order so that the number of repeated animations is equal to the number of the materials.

[0209] According to one or more embodiments of the present disclosure, if there are multiple materials, when the first generating unit adapts the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials, it is configured to:

[0210] If the number of transitions in the transition sequence is less than the number of material intervals, repeating each transition in the transition sequence in order so that the number of repeated transitions is equal to the number of material intervals, where the number of material intervals is the number of the materials minus 1.

[0211] According to one or more embodiments of the present disclosure, if there are multiple pieces of the material, when the first generating unit determines the setting positions of the special effects included in the target video template in the material, it is configured to:

[0212] If the target video template includes global special effects, determine the setting position of the global special effects as the entire time range of the material; or

[0213] If the target video template includes highlight special effects, determine whether the music attribute features included in the feature information of the material include the starting position of the music chorus. If so, determine the setting position of the highlight special effects as the starting position of the music chorus; or, if not, determine the setting position of the highlight special effects as the starting position of the second piece of material; or

[0214] If the target video template includes beat special effects, determine the setting position of the beat special effects as the starting position of the second piece of material, or determine the setting position of the beat special effects as the starting position of the second piece of material after the highlight special effects; or

[0215] If the target video template includes opening special effects, determine the setting position of the opening special effects as the starting position of the first piece of material; or

[0216] If the target video template includes closing special effects, determine the setting position of the closing special effects as the end position of the last piece of material.

[0217] According to one or more embodiments of the present disclosure, when the second generating unit determines the rendering effect combination according to the feature information of the material, it is configured to:

[0218] Obtain the correlation index between the feature information of the material and the candidate rendering effects, and construct a correlation matrix;

[0219] Determine a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and use the target rendering effect sequence as the rendering effect combination;

[0220] Wherein the rendering effects include one or more of animations, transitions, and special effects.

[0221] According to one or more embodiments of the present disclosure, when the second generating unit determines a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, it is configured to:

[0222] Based on the correlation matrix, use the beam search algorithm to obtain the rendering effects matching the material, and obtain a target rendering effect sequence.

[0223] According to one or more embodiments of the present disclosure, after determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the second generating unit is further configured to:

[0224] Determine whether each rendering effect in the target rendering effect sequence conforms to a preset rule;

[0225] If any rendering effect does not conform to the rule, replace or delete the rendering effect.

[0226] According to one or more embodiments of the present disclosure, the obtaining unit is further configured to:

[0227] If there is only one piece of material, determine the duration of the material. If the duration of the material does not exceed a preset duration threshold, determine the music variation point according to the music attribute feature included in the feature information of the material, and reduce the playback speed of the part of the material after the music variation point; or

[0228] If the duration of the material exceeds the preset duration threshold, divide the material into multiple pieces of material.

[0229] According to one or more embodiments of the present disclosure, when obtaining the feature information of the material, the obtaining unit is configured to:

[0230] Extract video frames from the material;

[0231] Input the video frames of the material into a preset feature extraction model, and obtain the feature information of the material through the preset feature extraction model.

[0232] According to one or more embodiments of the present disclosure, the feature information of the material includes: image feature information, audio feature information, the quantity information of the material, and the time information of the material.

[0233] In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one processor and a memory;

[0234] The memory stores computer execution instructions;

[0235] The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the video generation method as described in the first aspect above and various possible designs of the first aspect.

[0236] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the video generation method as described in the first aspect above and various possible designs of the first aspect is implemented.

[0237] Fifthly, according to one or more embodiments of the present disclosure, there is provided a computer program product including computer-executable instructions, which, when executed by a processor, implement the video generation method as described in the first aspect above and various possible designs of the first aspect.

[0238] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0239] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0240] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video generation method, characterized in that, it includes: obtaining materials and obtaining the characteristic information of the materials; determining a target video generation method according to the preset video generation method determination rules; if the target video generation method is the first generation method, determining a target video template from a preset video template pool according to the characteristic information of the materials, and generating a target video according to the materials and the target video template; or if the target video generation method is the second generation method, determining a rendering effect combination according to the characteristic information of the materials, and generating a target video according to the materials and the rendering effect combination.

2. The method according to claim 1, characterized in that, the determining the target video generation method according to the preset video generation method determination rules includes: obtaining user requirement information and determining the target video generation method according to the user requirement information; or determining the target video generation method in a random manner.

3. The method according to claim 1, characterized in that, the determining the target video template from a preset video template pool according to the characteristic information of the materials includes: matching the characteristic information of each preset video template in the preset video template pool with the characteristic information of the materials, and determining the target video template according to the matching result.

4. The method according to claim 3, characterized in that, the matching the characteristic information of each preset video template in the preset video template pool with the characteristic information of the materials and determining the target video template according to the matching result includes: matching the characteristic information of each preset video template in the preset video template pool with the characteristic information of the materials, filtering out the unmatched preset video templates, and selecting one preset video template from the remaining preset video templates as the target video template.

5. The method according to claim 1, characterized in that, the generating the target video according to the materials and the target video template includes: adapting the target video template according to the materials, and fusing the adaptively adjusted target template with the materials to generate the target video.

6. The method according to claim 5, characterized in that, the adapting the target video template according to the materials includes: adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials; and / or adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of materials; and / or determining the setting position of the special effects included in the target video template in the materials.

7. The method according to claim 6, characterized in that, if there are multiple materials, the adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials includes: If the number of animations in the animation sequence is less than the number of the materials, each animation in the animation sequence is repeated in order so that the number of animations after repetition is equal to the number of the materials; or If the number of animations in the animation sequence is less than the number of the materials and the number of animations in the animation sequence is odd, the remaining animations in the animation sequence except the first animation are repeated in order so that the number of animations after repetition is equal to the number of the materials; or If the number of animations in the animation sequence is less than the number of the materials and the number of animations in the animation sequence is even, each animation in the animation sequence is repeated in order so that the number of animations after repetition is equal to the number of the materials.

8. The method according to claim 6, wherein if there are multiple materials, adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the multiple materials, includes: If the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in order so that the number of transitions after repetition is equal to the number of material intervals, where the number of material intervals is the number of the materials minus 1.

9. The method according to claim 6, wherein if there are multiple materials, determining the setting positions of the special effects included in the target video template in the materials, includes: If the target video template includes global special effects, determining the setting position of the global special effects as the entire time range of the materials; or If the target video template includes highlight special effects, determining whether the music chorus start position is included in the music attribute features included in the feature information of the materials. If it is included, determining the setting position of the highlight special effects as the music chorus start position; or, if not included, determining the setting position of the highlight special effects as the start position of the second material; or If the target video template includes beat special effects, determining the setting position of the beat special effects as the start position of the second material, or determining the setting position of the beat special effects as the start position of the second material after the highlight special effects; or If the target video template includes opening special effects, determining the setting position of the opening special effects as the start position of the first material; or If the target video template includes closing special effects, determining the setting position of the closing special effects as the end position of the last material.

10. The method according to claim 1, wherein determining the rendering effect combination according to the feature information of the materials, includes: Obtaining the correlation index between the feature information of the materials and the candidate rendering effects, and constructing a correlation matrix; Determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and taking the target rendering effect sequence as the rendering effect combination; where the rendering effects include one or more of animations, transitions, and special effects.

11. The method according to claim 10, wherein Determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix includes: Based on the correlation matrix, using a beam search algorithm to obtain a rendering effect that matches the material, and obtaining a target rendering effect sequence.

12. The method according to claim 10, wherein, After determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix, it further includes: Judging whether each rendering effect in the target rendering effect sequence conforms to a preset rule; If any rendering effect does not conform to the rule, replace or delete the rendering effect.

13. The method according to claim 1, wherein, The method further includes: If there is only one piece of material, judge the duration of the material. If the duration of the material does not exceed a preset duration threshold, determine the music variation point according to the music attribute feature included in the feature information of the material, and reduce the playback speed of the part of the material after the music variation point; or If the duration of the material exceeds the preset duration threshold, divide the material into multiple pieces of material.

14. The method according to any one of claims 1-13, wherein, Obtaining the feature information of the material includes: Extracting video frames from the material; Inputting the video frames of the material into a preset feature extraction model, and obtaining the feature information of the material through the preset feature extraction model.

15. The method according to any one of claims 1-13, wherein, The feature information of the material includes: image feature information, audio feature information, the number information of the material, and the time information of the material.

16. A video generation device, wherein, including: An acquisition unit for acquiring materials and acquiring the feature information of the materials; A determination unit for determining a target video generation method according to a preset rule for the video generation method; A first generation unit for, if the target video generation method is the first generation method, determining a target video template from a preset video template pool according to the feature unit of the material, and generating a target video according to the material and the target video template; or A second generation unit for, if the target video generation method is the second generation method, determining a rendering effect combination according to the feature information of the material, and generating a target video according to the material and the rendering effect combination.

17. An electronic device, wherein, including: At least one processor and a memory; The memory stores computer execution instructions; The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the method according to any one of claims 1-15.

18. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer execution instructions, and when the processor executes the computer execution instructions, the method according to any one of claims 1-15 is implemented.

19. A computer program product, wherein, Including computer-executable instructions that, when executed by a processor, implement the method according to any one of claims 1-15.

Images