Video generation method and device, and storage medium

Abstract

Embodiments of the present disclosure provide a video generation method and device, and a storage medium. The video generation method comprises: acquiring a material and acquiring the feature information of the material; determining a target video generation mode according to a preset video generation mode determination rule; if the target video generation mode is a first generation mode, determining a target video template from a preset video template pool on the basis of the feature information of the material, and generating a target video on the basis of the material and the target video template; or if the target video generation mode is a second generation mode, determining a rendering effect combination on the basis of the feature information of the material, and generating the target video on the basis of the material and the rendering effect combination.

Description

Video generation method, device and storage medium

[0001] This application claims priority to Chinese patent application No. 202311597879.8 filed on November 27, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby cited in their entirety as part of this application. Technical Field

[0002] The embodiments of the present disclosure relate to a video generation method, device, and storage medium. Background Art

[0003] With the development of network technology and the improvement of terminal device hardware performance, the demand for video creation and video editing on terminal devices is becoming increasingly strong. In order to lower the threshold for video creation and facilitate users to quickly produce video works, the function of automatic video generation has come into being.

[0004] Some existing automatic video generation functions usually require users to upload materials, and then automatically perform packaging processes such as adding music, special effects, transitions, etc. to the materials. However, the existing automatic video generation functions can only achieve simple packaging processes and the effects are poor. Summary of the Invention

[0005] The embodiments of the present disclosure provide a video generation method, device, and storage medium to better implement automated video editing and production.

[0006] In a first aspect, an embodiment of the present disclosure provides a video generation method, comprising:

[0007] Acquire material and obtain characteristic information of the material;

[0008] Determine rules according to a preset video generation method to determine a target video generation method;

[0009] If the target video generation method is the first generation method, determining a target video template from a preset video template pool according to the feature information of the material, and generating a target video according to the material and the target video template; or

[0010] If the target video generation method is the second generation method, a rendering effect combination is determined according to the feature information of the material, and the target video is generated according to the material and the rendering effect combination.

[0011] In a second aspect, an embodiment of the present disclosure provides a video generation device, including:

[0012] An acquisition unit, configured to acquire a material and obtain characteristic information of the material;

[0013] A determination unit, configured to determine a rule according to a preset video generation method to determine a target video generation method;

[0014] a first generating unit configured to, if the target video generating mode is the first generating mode, determine a target video template from a preset video template pool based on the feature unit of the material, and generate a target video based on the material and the target video template; or

[0015] The second generating unit is configured to determine a rendering effect combination according to feature information of the material if the target video generating method is the second generating method, and generate the target video according to the material and the rendering effect combination.

[0016] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0017] The memory stores computer-executable instructions;

[0018] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the video generation method described in the first aspect and various possible designs of the first aspect.

[0019] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0020] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, comprising computer-executable instructions. When a processor executes the computer-executable instructions, the video generation method as described in the first aspect and various possible designs of the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, a brief introduction to the drawings required for the embodiments will be given below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] FIG1 is a scene example diagram of a video generation method provided by an embodiment of the present disclosure;

[0023] FIG2 is a flow chart of a video generation method according to an embodiment of the present disclosure;

[0024] FIG3 is a flow chart of a video generation method provided by another embodiment of the present disclosure;

[0025] FIG4 is a structural block diagram of a video generation device provided by an embodiment of the present disclosure;

[0026] FIG5 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0028] Some existing automatic video generation functions usually require users to upload materials, and then automatically perform packaging processes such as adding music, special effects, transitions, etc. to the materials. However, the existing automatic video generation functions can only achieve simple packaging processes and cannot achieve more complex automated editing and production. In addition, the added music, special effects, transitions, etc. may not be appropriate, such as unreasonable matching and conflicts, resulting in poor generated video effects.

[0029] In order to solve the above technical problems, the present disclosure provides a video generation method, which obtains material and obtains characteristic information of the material; determines rules according to a preset video generation method, and determines a target video generation method; if the target video generation method is the first generation method, determines a target video template from a preset video template pool according to the characteristic information of the material, and generates a target video according to the material and the target video template; or if the target video generation method is the second generation method, determines a rendering effect combination according to the characteristic information of the material, and generates a target video according to the material and the rendering effect combination. The present disclosure provides two target video generation methods, and can select a suitable target video template to generate a target video, or recommend a suitable rendering effect combination based on the material characteristics to generate a target video, thereby ensuring the rationality of the rendering effect combination, avoiding conflicts in the rendering effect combination, and improving the generated video effect. In addition, the selection of the above two target video generation methods can meet the needs of different users and improve the user experience.

[0030] The video generation method provided by the present disclosure is applicable to electronic devices such as terminal devices or servers. As shown in Figure 1, if the video generation method provided by the present disclosure is used for a terminal device, the terminal device can obtain material and obtain characteristic information of the material; determine rules according to a preset video generation method, and determine a target video generation method; if the target video generation method is the first generation method, determine the target video template from a preset video template pool according to the characteristic information of the material, and can adaptively adjust the target video template according to the material, and generate a target video according to the material and the target video template; or if the target video generation method is the second generation method, determine a rendering effect combination according to the characteristic information of the material, and generate a target video according to the material and the rendering effect combination.

[0031] In addition, after the rendering effect combination is determined in the second generation method, rule-based post-processing can also be performed, that is, to determine whether each rendering effect in the target rendering effect sequence complies with the preset rules in the preset rule pool; if any rendering effect does not comply with the rules, the rendering effect is replaced or deleted, further ensuring the rationality of the rendering effect combination, avoiding conflicts in the rendering effect combination, and improving the generated video effect.

[0032] If the video generation method provided by the present invention is used for a server, the server can receive the material uploaded by the terminal, and the server obtains the characteristic information of the material; determines the rules according to the preset video generation method, and determines the target video generation method; if the target video generation method is the first generation method, determines the target video template from the preset video template pool according to the characteristic information of the material, and can adaptively adjust the target video template according to the material, and generate the target video according to the material and the target video template; or if the target video generation method is the second generation method, determines the rendering effect combination according to the characteristic information of the material, and generates the target video according to the material and the rendering effect combination, and sends it to the terminal device.

[0033] It should be noted that the user information and data involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] The video generation method disclosed herein will be described in detail below with reference to specific embodiments.

[0035] Referring to FIG2 , FIG2 is a flow chart of a video generation method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device or a server, and the video generation method includes:

[0036] S201: Acquire material and obtain feature information of the material.

[0037] In this embodiment, the user may input one or more materials, which are multimedia contents used to generate a video, such as pictures, videos, audios, etc.; for each material, feature information may be extracted.

[0038] Optionally, the feature information may include but is not limited to image feature information, audio feature information, quantity information of the material, time information of the material, etc.

[0039] The image feature information is feature information extracted based on the image of the material, which may include but is not limited to the style, theme, emotion, scene, character, etc. of the picture / video; the audio feature information is feature information extracted based on the audio of the material, which may include but is not limited to beat points, chorus points (the starting position of the chorus), phrases, rhythm (beats per minute (bpm), total audio duration, etc.; the quantity information of the material is also the number of materials; the time information of the material may include the duration of each material, the total duration of all materials, the duration of the film (ultimately generating the target video) determined according to the total audio duration, etc.; of course, the feature information may also include other feature information such as resolution, which will not be listed here one by one.

[0040] Optionally, obtaining the image feature information and audio feature information of the material can be achieved with the help of a preset feature extraction model. For example, one or more video frames are extracted for each material, and the video frames of the material are input into the preset feature extraction model. Feature extraction is performed through the preset feature extraction model to obtain the feature information of the material. The preset feature extraction model can be a neural network model or other machine learning model, and training samples can be collected in advance for training, which will not be repeated here.

[0041] It should be noted that when obtaining the characteristic information of the material, some characteristic information can be omitted. The advantage is that the interface for obtaining the characteristic information of the material can be omitted, which greatly reduces the time consumption and decouples the dependence of the film effect on the material. That is, for some scenes that do not require high film effects but have high requirements for time consumption (hope that the time consumption is as short as possible), the process of obtaining the characteristic information of the material can also be omitted.

[0042] S202: Determine a target video generation method by determining a rule according to a preset video generation method.

[0043] In this embodiment, two video generation methods are pre-configured, including a first generation method and a second generation method. The first generation method is to generate a video using a preset video template. The preset video template is pre-configured with a fixed animation combination, transition combination, special effects combination, etc. The target video can be generated by applying the preset video template to multiple materials. The generation of the target video takes less time and is more efficient. Moreover, since the preset video template uses a fixed animation combination, transition combination, and special effects combination, the combination is more appropriate and the film effect is better. The second generation method is to recommend a rendering effect combination based on feature information based on the material through a recommendation algorithm, including but not limited to animation combination, transition combination, and special effects combination, and then combine the material and the rendering effect into a target video. That is, the second generation method does not have a fixed video template and is diverse, which can meet the user's personalized and diversified needs for the target video.

[0044] The target video generation method can be determined in a random manner or by using a specific algorithm. For example, different target video generation methods can be used according to user needs. For example, users who need diversified effects can choose the second generation method, and the generated target videos are more diversified and personalized. Users who need stable effects can choose the first generation method, which can generate target videos with less diversity but better effects. More specifically, the mapping relationship between different user demand information and different video generation methods can be pre-configured, and the target video generation method corresponding to the user needs can be determined based on the mapping relationship.

[0045] S203: If the target video generation method is the first generation method, determine a target video template from a preset video template pool according to the feature information of the material, and generate a target video according to the material and the target video template.

[0046] In this embodiment, different relatively general, high-quality preset video templates are pre-designed to form a preset video template pool. Furthermore, if the target video generation method is determined to be the first generation method, a target video template can be selected from the preset video template pool to generate the target video. In this embodiment, the target video template can be determined from the preset video template pool based on the feature information of the material obtained in the above S201.

[0047] Specifically, when pre-designing different preset video templates, the characteristic information of the preset video templates can be determined, such as the duration, number of materials, style, atmosphere, etc. that the preset video template is suitable for. Furthermore, when determining the target video template, the characteristic information of each preset video template in the preset video template pool can be matched with the characteristic information of the material, and the target video template can be determined based on the matching result. For example, if there is a matching preset video template, the preset video template is determined as the target video template. If there are multiple matching preset video templates, one of the multiple matching preset video templates is selected as the target video template. Optionally, a matching degree index can be given for each preset video template, and the preset video template with the highest matching degree index is determined as the target video template.

[0048] Optionally, when matching the characteristic information of each preset video template in the preset video template pool with the characteristic information of the material, unmatched preset video templates can be filtered out. For example, if the atmosphere of the material is sad, the preset video template with a happy atmosphere will be filtered out. Furthermore, one preset video template is selected from the remaining preset video templates as the target video template. If there is only one preset video template among the remaining preset video templates, the preset video template is determined as the target video template. If there are multiple preset video templates among the remaining preset video templates, one is selected from them as the target video template.

[0049] After determining the target video template, the target video can be generated based on the material and the target video template. Specifically, the material is filled into the target video template, and the animation combination, transition combination, special effects combination and other rendering effects in the target video template are synthesized with the material to finally generate the target video.

[0050] S204: If the target video generation method is the second generation method, determine a rendering effect combination according to the feature information of the material, and generate the target video according to the material and the rendering effect combination.

[0051] In this embodiment, if it is determined that the target video generation method is the second generation method, a recommendation algorithm is used to recommend a rendering effect combination based on feature information based on the material, including but not limited to animation combination, transition combination, special effects combination, etc., and then the rendering effect combination is synthesized with the material to finally generate the target video.

[0052] When determining the rendering effect combination, a rendering effect may be recommended based on the feature information of the material, thereby obtaining a rendering effect sequence as the rendering effect combination.

[0053] Optionally, a correlation index between feature information of the material and candidate rendering effects may be obtained to construct a correlation matrix; a target rendering effect sequence may be determined from the candidate rendering effects according to the correlation matrix, and the target rendering effect sequence may be used as the rendering effect combination.

[0054] In this embodiment, a correlation index between the feature information of each material and each candidate rendering effect can be obtained, and then a correlation matrix can be constructed. Assuming there are M materials and N candidate rendering effects, the correlation matrix is ​​an M×N matrix. The correlation index can be the similarity between the feature information of the material and the feature information of the candidate rendering effect, which can be cosine similarity or other similarity metrics. The feature information of the candidate rendering effect can be preset. Furthermore, a target rendering effect sequence is determined based on the correlation matrix, and the target rendering effect sequence is used as the rendering effect combination.

[0055] Optionally, a beam search algorithm (Beam Search) or a greedy algorithm can be used to determine the rendering effects that match each material one by one from the correlation matrix to obtain a target rendering effect sequence. Taking the beam search algorithm as an example, a beam width (beam size, replaced by bs in this embodiment) can be pre-specified. When searching for a rendering effect that matches any material from the correlation matrix, the total correlation index (total similarity) of the new rendering effect sequence after each candidate rendering effect is added to the current rendering effect sequence is obtained, and the bs new rendering effect sequences with the highest total correlation index are retained; next, when searching for the rendering effect that matches the next material, on the basis of retaining the bs new rendering effect sequences in the previous step, searches are performed again, and the bs new rendering effect sequences with the highest total correlation index are retained. This step is repeated until the matching is completed. Finally, bs rendering effect sequences are obtained, and the rendering effect sequence with the highest total correlation index is taken as the final target rendering effect sequence.

[0056] Optionally, different correlation matrices may be constructed for candidate animations, candidate transitions, and candidate special effects, respectively. The animation sequence, transition sequence, and special effect sequence may be obtained through the above process, respectively.

[0057] Furthermore, the determined rendering effect combination (animation sequence, transition sequence, special effect sequence) is synthesized with the material to finally generate the target video.

[0058] The video generation method provided in this embodiment obtains material and obtains characteristic information of the material; determines rules according to a preset video generation method and determines a target video generation method; if the target video generation method is the first generation method, determines a target video template from a preset video template pool according to the characteristic information of the material, and generates a target video according to the material and the target video template; or if the target video generation method is the second generation method, determines a rendering effect combination according to the characteristic information of the material, and generates a target video according to the material and the rendering effect combination. In this embodiment, a suitable target video template can be selected to generate a target video, or a suitable rendering effect combination can be recommended based on the material characteristics to generate a target video, thereby ensuring the rationality of the rendering effect combination, avoiding conflicts in the rendering effect combination, and improving the generated video effect. In addition, the selection of the above two target video generation methods can meet the needs of different users and improve the user experience.

[0059] Based on any of the above embodiments, when generating the target video according to the material and the target video template in S203, considering that the target video template may not be fully adapted to the material, the target video template can also be adaptively adjusted according to the material, and the adaptively adjusted target template is fused with the material to finally generate the target video.

[0060] Optionally, adaptive adjustment includes but is not limited to adaptive filling of animation sequences, adaptive filling of transition sequences, adaptive filling of special effects in the target video template, and configuration and distribution of fine-grained parameters (such as duration, intensity, etc.) of each rendering effect (animation, transition, special effect), etc.

[0061] Optionally, the adaptive filling of the animation sequence is to fill each animation in the animation sequence into the appropriate position, where the animation can be various deformations, displacements, etc. of the material. Therefore, each animation corresponds to a material, and the scope of each animation is a material from the beginning to the end. Therefore, each material from the beginning to the end can be regarded as a slot, and the animation sequence in the target video template can be automatically added to the slots corresponding to the material in sequence.

[0062] If there are multiple materials, considering that the number of animations in the animation sequence may not be equal to the number of materials, the animation sequence needs to be adaptively adjusted according to the number of animations in the animation sequence and the number of materials, so that the number of animations in the animation sequence is equal to the number of materials, and the animations in the animation sequence can just fill the slots corresponding to the materials.

[0063] Optionally, if the number of animations in the animation sequence is greater than the number of materials, the animation sequence will be automatically added to the slots corresponding to the materials in sequence, and the redundant animations will be deleted; if the number of animations in the animation sequence is less than the number of materials, the animations in the animation sequence can be repeated in sequence so that the number of animations after repetition is equal to the number of materials. For example, there are 7 materials, and there are 3 animations in the animation sequence, which are [A, B, C]. The animation sequence can be repeated in sequence so that the animation sequence after repetition has 7 animations, which are [A, B, C, A, B, C, A]. In another optional embodiment, if the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an odd number, by analyzing a large amount of videos, it can be found that the first animation is usually the opening animation. Therefore, the remaining animations in the animation sequence except the first animation can be repeated in sequence so that the number of animations after repetition is equal to the number of materials. For example, in the above example, The number of animations 3 in the animation sequence [A, B, C] is an odd number, so only B and C are repeated, and the animation sequence after repetition has 7 animations, which is [A, B, C, B, C, B, C]; in another optional embodiment, if the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an even number, by analyzing massive videos, it can be found that when the number of animations in the animation sequence is an even number, the animations usually appear in pairs, so each animation in the animation sequence can be repeated in sequence so that the number of animations after repetition is equal to the number of materials. For example, in the above example, the number of animations 2 in the animation sequence [A, B] is an even number, so A and B are repeated, and the animation sequence after repetition has 7 animations, which is [A, B, A, B, A, B, A]; it should be noted that if the number of animations in the animation sequence is 1, the animation in the animation sequence is most likely the opening animation, and the animation sequence is not repeated, and the animation is added to the first slot.

[0064] Optionally, adaptive filling of the transition sequence is to fill each transition in the transition sequence into the appropriate position, where the transition is usually located between two adjacent materials. Therefore, the position between two adjacent materials can be used as a slot, and the transition sequence in the target video template can be automatically added to each slot in sequence.

[0065] If there are multiple materials, considering that the number of transitions in the transition sequence may not be equal to the number of material intervals of the materials, that is, not equal to the number of slots, it is necessary to adaptively adjust the transition sequence according to the number of transitions in the transition sequence and the number of materials, so that the number of transitions in the transition sequence is equal to the number of material intervals (that is, equal to the number of materials minus 1), and the transitions in the transition sequence can just fill all the slots.

[0066] Optionally, if the number of transitions in a transition sequence is greater than the number of intervals between clips, the transition sequence will be automatically added to the slots between adjacent clips, and any extra transitions will be deleted. If the number of transitions in a transition sequence is less than the number of intervals between clips, the transitions in the sequence will be repeated in order, so that the number of repeated transitions equals the number of intervals between clips, and the transitions in the sequence just fill all the slots. For example, if there are 7 clips, there are 6 slots, and the transition sequence contains 3 transitions, namely [D, E, F]. The transition sequence can be repeated in order, so that the repeated transition sequence has 6 transitions, namely [D, E, F, D, E, F].

[0067] Optionally, adaptive special effects fill the target video template with the appropriate special effects. This requires determining the placement of the special effects in the target video template within the source material. Special effects can include, but are not limited to, global effects, highlight effects, key effects, opening effects, and closing effects.

[0068] If the target video template includes global special effects, the setting position of the global special effects is determined to be the entire time range of the material, that is, the scope of the global special effects is from the beginning to the end of the material. If there are multiple materials, the scope of the global special effects is from the beginning of the first material to the end of the last material. The global special effects can be set to take effect in the finished film.

[0069] If the target video template includes a highlight effect, the highlight effect will also be applied to the final film. The method determines whether the music attribute features included in the feature information of the source material include the start position of the music chorus. If so, the highlight effect is set to the start position of the music chorus. Otherwise, the highlight effect is set to the start position of the second source material.

[0070] If the target video template includes a card effect, the card effect sequence can be repeated in its entirety, with the length of the repeated sequence aligned with the number of assets. The starting position of the card effect also needs to be determined. Specifically, the card effect can be set to the starting position of the second asset, or the starting position of the second asset after the highlight effect. That is, if the highlight effect doesn't exist, the card effect can start from the starting position of the second asset. If the highlight effect exists, the card effect can start from the starting position of the second asset after the highlight effect, extending the start position by one slot.

[0071] If the target video template includes an opening special effect, the setting position of the opening special effect is determined to be the starting position of the first material, and the opening special effect can be set to take effect in the finished film.

[0072] If the target video template includes a closing effect, the setting position of the closing effect is determined to be the end position of the last material, and the closing effect can be set to take effect in the finished film.

[0073] Based on any of the above embodiments, after determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix in S204, rule-based post-processing may be performed to further ensure the rationality of the rendering effect combination, avoid conflicts in the rendering effect combination, and improve the generated video effect, as shown in FIG3 . Specifically, the process includes:

[0074] S301, determining whether each rendering effect in the target rendering effect sequence complies with a preset rule;

[0075] S302: If any rendering effect does not comply with the rules, replace or delete the rendering effect.

[0076] In this embodiment, considering that there may be unreasonable rendering effects in the target rendering effect sequence recommended by the recommendation algorithm, some preset rules can be pre-configured to build a preset rule pool to check the rationality of the rendering effects in the target rendering effect sequence. If it is determined that any rendering effect in the target rendering effect sequence does not comply with the rules, it means that the rendering effect is unreasonable and the rendering effect can be replaced or deleted.

[0077] For example, category labels are added to different candidate rendering effects in advance, and the preset rule is that the category labels of the rendering effects in the target rendering effect sequence are the same. Therefore, after obtaining the target rendering effect sequence, it is possible to check whether the category labels of the rendering effects in the target rendering effect sequence are the same. If there is a rendering effect in the target rendering effect sequence whose category label is different from that of other rendering effects, the rendering effect with a different category can be replaced with another rendering effect with the same category label as the other rendering effects, thereby ensuring that the category labels of the rendering effects in the target rendering effect sequence are unified.

[0078] Of course, the preset rules are not limited to the above examples, and can also be designed according to actual needs. Examples are not given here one by one. Optionally, different preset rules can be set for animation combinations, transition combinations, and special effect combinations respectively.

[0079] Based on any of the above embodiments, if only one material is obtained when obtaining material in S201, that is, there is only one segment of material, the duration of the material can be determined. If the duration of the material does not exceed the preset duration threshold (for example, 15 seconds), the music variation point is determined based on the music attribute characteristics included in the characteristic information of the material, and the playback speed of the part of the material after the music variation point is reduced to achieve the effect of card point speed change, and then the above video generation process is continued to be executed with this segment of material to finally generate the target video.

[0080] If the duration of the material exceeds a preset duration threshold (e.g., 15 seconds), the material is divided into multiple segments, and the above-mentioned video generation process is performed on the multiple segments to finally generate the target video. The material can be segmented according to the preset duration, for example, by 10 seconds. If the duration is not divisible, the last remaining part is merged into the last segment of the material. The number of segmented materials is the total duration of the material divided by 10. For example, if the total duration of the material is 33 seconds, it is segmented into 3 segments of 10 seconds, 10 seconds, and 13 seconds.

[0081] Based on any of the above embodiments, when generating a target video, the target video can be generated through a pre-configured draft protocol, and a renderable configuration file can be issued. The draft protocol contains various track information and configuration information required for video editing, ultimately ensuring that the format of the target video is unified.

[0082] Corresponding to the video generation method described in the preceding embodiment, FIG4 is a block diagram of a video generation device provided in an embodiment of the present disclosure. For ease of illustration, only portions relevant to the embodiment of the present disclosure are shown. Referring to FIG4 , the video generation device 400 includes an acquisition unit 401, a determination unit 402, a first generation unit 403, and a second generation unit 404.

[0083] The acquisition unit 401 is used to acquire a material and obtain characteristic information of the material;

[0084] A determination unit 402 is configured to determine a target video generation method according to a predetermined video generation method determination rule;

[0085] The first generating unit 403 is configured to, if the target video generating mode is the first generating mode, determine a target video template from a preset video template pool according to feature information of the material, and generate a target video according to the material and the target video template; or

[0086] The second generating unit 404 is configured to determine a rendering effect combination according to the feature information of the material if the target video generating mode is the second generating mode, and generate the target video according to the material and the rendering effect combination.

[0087] In one or more embodiments of the present disclosure, when determining a target video generation method based on a preset video generation method determination rule, the determination unit 402 is configured to:

[0088] Obtaining user demand information, and determining the target video generation method according to the user demand information; or

[0089] The target video generation method is determined in a random manner.

[0090] In one or more embodiments of the present disclosure, when the first generating unit 403 determines the target video template from the preset video template pool according to the feature information of the material, it is configured to:

[0091] The characteristic information of each preset video template in the preset video template pool is matched with the characteristic information of the material, and the target video template is determined according to the matching result.

[0092] In one or more embodiments of the present disclosure, when matching the feature information of each preset video template in the preset video template pool with the feature information of the material and determining the target video template according to the matching result, the first generating unit 403 is configured to:

[0093] Match the characteristic information of each preset video template in the preset video template pool with the characteristic information of the material, filter out unmatched preset video templates, and select one preset video template from the remaining preset video templates as the target video template.

[0094] In one or more embodiments of the present disclosure, when generating the target video according to the material and the target video template, the first generating unit 403 is configured to:

[0095] The target video template is adaptively adjusted according to the material, and the adaptively adjusted target template is fused with the material to generate the target video.

[0096] In one or more embodiments of the present disclosure, when adaptively adjusting the target video template according to the material, the first generating unit 403 is configured to:

[0097] Adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of the materials; and / or

[0098] Adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials; and / or

[0099] Determine the setting position of the special effects included in the target video template in the material.

[0100] In one or more embodiments of the present disclosure, if there are multiple materials, the first generation unit 403, when adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials, is configured to:

[0101] If the number of animations in the animation sequence is less than the number of materials, repeating each animation in the animation sequence in order so that the number of repeated animations is equal to the number of materials; or

[0102] If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an odd number, then the remaining animations in the animation sequence except the first animation are repeated in order so that the number of repeated animations is equal to the number of materials; or

[0103] If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an even number, then each animation in the animation sequence is repeated in sequence so that the number of repeated animations is equal to the number of materials.

[0104] In one or more embodiments of the present disclosure, if there are multiple materials, the first generation unit 403, when adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of materials, is configured to:

[0105] If the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in order so that the number of transitions after repetition is equal to the number of material intervals, where the number of material intervals is the number of materials minus 1.

[0106] In one or more embodiments of the present disclosure, if there are multiple materials, the first generating unit 403, when determining the setting position of the special effects included in the target video template in the materials, is configured to:

[0107] If the target video template includes a global special effect, determining the setting position of the global special effect to be the entire time range of the material; or

[0108] If the target video template includes a highlight effect, determine whether the music attribute feature included in the feature information of the material includes the starting position of the music chorus; if so, determine that the setting position of the highlight effect is the starting position of the music chorus; or, if not, determine that the setting position of the highlight effect is the starting position of the second material; or

[0109] If the target video template includes a card point special effect, determining the setting position of the card point special effect to be the starting position of the second material, or determining the setting position of the card point special effect to be the starting position of the second material after the highlight special effect; or

[0110] If the target video template includes an opening special effect, determining the setting position of the opening special effect to be the starting position of the first material; or

[0111] If the target video template includes a closing special effect, the setting position of the closing special effect is determined to be the end position of the last material.

[0112] In one or more embodiments of the present disclosure, when determining the rendering effect combination according to the feature information of the material, the second generating unit 404 is configured to:

[0113] Obtain the correlation index between the feature information of the material and the candidate rendering effects, and construct a correlation matrix;

[0114] determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and using the target rendering effect sequence as the rendering effect combination;

[0115] The rendering effects include one or more of animation, transition, and special effects.

[0116] In one or more embodiments of the present disclosure, when determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the second generating unit 404 is configured to:

[0117] Based on the correlation matrix, a beam search algorithm is used to obtain rendering effects that match the material, and a target rendering effect sequence is obtained.

[0118] In one or more embodiments of the present disclosure, after determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the second generating unit 404 is further configured to:

[0119] Determining whether each rendering effect in the target rendering effect sequence complies with a preset rule;

[0120] If any rendering effect does not comply with the rules, the rendering effect will be replaced or deleted.

[0121] In one or more embodiments of the present disclosure, the acquiring unit 401 is further configured to:

[0122] If there is only one segment of material, the duration of the material is determined. If the duration of the material does not exceed a preset duration threshold, a music variation point is determined based on the music attribute features included in the feature information of the material, and the playback speed of the portion of the material after the music variation point is reduced; or

[0123] If the duration of the material exceeds a preset duration threshold, the material is divided into multiple segments.

[0124] In one or more embodiments of the present disclosure, when acquiring the feature information of the material, the acquiring unit 401 is configured to:

[0125] Extracting video frames from the material;

[0126] The video frame of the material is input into a preset feature extraction model, and feature information of the material is obtained through the preset feature extraction model.

[0127] In one or more embodiments of the present disclosure, the feature information of the material includes: image feature information, audio feature information, quantity information of the material, and time information of the material.

[0128] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0129] Referring to FIG5 , there is shown a schematic diagram of the structure of an electronic device 500 suitable for implementing an embodiment of the present disclosure. The electronic device 500 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG5 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0130] As shown in FIG5 , the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0131] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 shows an electronic device 500 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0132] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0133] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0134] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0135] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0136] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0139] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0140] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] In a first aspect, according to one or more embodiments of the present disclosure, a video generation method is provided, comprising:

[0142] Acquire material and obtain characteristic information of the material;

[0143] Determine rules according to a preset video generation method to determine a target video generation method;

[0144] If the target video generation method is the first generation method, determining a target video template from a preset video template pool according to the feature information of the material, and generating a target video according to the material and the target video template; or

[0145] If the target video generation method is the second generation method, a rendering effect combination is determined according to the feature information of the material, and the target video is generated according to the material and the rendering effect combination.

[0146] According to one or more embodiments of the present disclosure, determining a target video generation method according to a preset video generation method determination rule includes:

[0147] Obtaining user demand information, and determining the target video generation method according to the user demand information; or

[0148] The target video generation method is determined in a random manner.

[0149] According to one or more embodiments of the present disclosure, determining a target video template from a preset video template pool based on feature information of the material includes:

[0150] The characteristic information of each preset video template in the preset video template pool is matched with the characteristic information of the material, and the target video template is determined according to the matching result.

[0151] According to one or more embodiments of the present disclosure, matching the feature information of each preset video template in the preset video template pool with the feature information of the material, and determining the target video template based on the matching result includes:

[0152] Match the characteristic information of each preset video template in the preset video template pool with the characteristic information of the material, filter out unmatched preset video templates, and select one preset video template from the remaining preset video templates as the target video template.

[0153] According to one or more embodiments of the present disclosure, generating a target video according to the material and the target video template includes:

[0154] The target video template is adaptively adjusted according to the material, and the adaptively adjusted target template is fused with the material to generate the target video.

[0155] According to one or more embodiments of the present disclosure, the adaptively adjusting the target video template according to the material includes:

[0156] Adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of the materials; and / or

[0157] Adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials; and / or

[0158] Determine the setting position of the special effects included in the target video template in the material.

[0159] According to one or more embodiments of the present disclosure, if there are multiple materials, adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of materials includes:

[0160] If the number of animations in the animation sequence is less than the number of materials, repeating each animation in the animation sequence in order so that the number of repeated animations is equal to the number of materials; or

[0161] If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an odd number, then the remaining animations in the animation sequence except the first animation are repeated in order so that the number of repeated animations is equal to the number of materials; or

[0162] If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an even number, then each animation in the animation sequence is repeated in sequence so that the number of repeated animations is equal to the number of materials.

[0163] According to one or more embodiments of the present disclosure, if there are multiple materials, adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of materials includes:

[0164] If the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in order so that the number of transitions after repetition is equal to the number of material intervals, where the number of material intervals is the number of materials minus 1.

[0165] According to one or more embodiments of the present disclosure, if there are multiple materials, determining the location of the special effects included in the target video template in the materials includes:

[0166] If the target video template includes a global special effect, determining the setting position of the global special effect to be the entire time range of the material; or

[0167] If the target video template includes a highlight effect, determine whether the music attribute feature included in the feature information of the material includes the starting position of the music chorus; if so, determine that the setting position of the highlight effect is the starting position of the music chorus; or, if not, determine that the setting position of the highlight effect is the starting position of the second material; or

[0168] If the target video template includes a card point special effect, determining the setting position of the card point special effect to be the starting position of the second material, or determining the setting position of the card point special effect to be the starting position of the second material after the highlight special effect; or

[0169] If the target video template includes an opening special effect, determining the setting position of the opening special effect to be the starting position of the first material; or

[0170] If the target video template includes a closing special effect, the setting position of the closing special effect is determined to be the end position of the last material.

[0171] According to one or more embodiments of the present disclosure, determining a rendering effect combination according to feature information of the material includes:

[0172] Obtain the correlation index between the feature information of the material and the candidate rendering effects, and construct a correlation matrix;

[0173] determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and using the target rendering effect sequence as the rendering effect combination;

[0174] The rendering effects include one or more of animation, transition, and special effects.

[0175] According to one or more embodiments of the present disclosure, determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix includes:

[0176] Based on the correlation matrix, a beam search algorithm is used to obtain rendering effects that match the material, and a target rendering effect sequence is obtained.

[0177] According to one or more embodiments of the present disclosure, after determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the method further includes:

[0178] Determining whether each rendering effect in the target rendering effect sequence complies with a preset rule;

[0179] If any rendering effect does not comply with the rules, the rendering effect will be replaced or deleted.

[0180] According to one or more embodiments of the present disclosure, the method further includes:

[0181] If there is only one segment of material, the duration of the material is determined. If the duration of the material does not exceed a preset duration threshold, a music variation point is determined based on the music attribute features included in the feature information of the material, and the playback speed of the portion of the material after the music variation point is reduced; or

[0182] If the duration of the material exceeds a preset duration threshold, the material is divided into multiple segments.

[0183] According to one or more embodiments of the present disclosure, obtaining feature information of the material includes:

[0184] Extracting video frames from the material;

[0185] The video frame of the material is input into a preset feature extraction model, and feature information of the material is obtained through the preset feature extraction model.

[0186] According to one or more embodiments of the present disclosure, the feature information of the material includes: image feature information, audio feature information, quantity information of the material, and time information of the material.

[0187] In a second aspect, according to one or more embodiments of the present disclosure, a video generating device is provided, including:

[0188] An acquisition unit, configured to acquire a material and obtain characteristic information of the material;

[0189] A determination unit, configured to determine a rule according to a preset video generation method to determine a target video generation method;

[0190] a first generating unit configured to, if the target video generating mode is the first generating mode, determine a target video template from a preset video template pool according to feature information of the material, and generate a target video according to the material and the target video template; or

[0191] The second generating unit is configured to determine a rendering effect combination according to feature information of the material if the target video generating method is the second generating method, and generate the target video according to the material and the rendering effect combination.

[0192] According to one or more embodiments of the present disclosure, when determining a target video generation method based on a preset video generation method determination rule, the determination unit is configured to:

[0193] Obtaining user demand information, and determining the target video generation method according to the user demand information; or

[0194] The target video generation method is determined in a random manner.

[0195] According to one or more embodiments of the present disclosure, when the first generating unit determines the target video template from a preset video template pool according to the feature information of the material, it is configured to:

[0196] The characteristic information of each preset video template in the preset video template pool is matched with the characteristic information of the material, and the target video template is determined according to the matching result.

[0197] According to one or more embodiments of the present disclosure, when the first generating unit matches the feature information of each preset video template in the preset video template pool with the feature information of the material and determines the target video template according to the matching result, it is configured to:

[0198] Match the characteristic information of each preset video template in the preset video template pool with the characteristic information of the material, filter out unmatched preset video templates, and select one preset video template from the remaining preset video templates as the target video template.

[0199] According to one or more embodiments of the present disclosure, when the first generating unit generates the target video according to the material and the target video template, it is configured to:

[0200] The target video template is adaptively adjusted according to the material, and the adaptively adjusted target template is fused with the material to generate the target video.

[0201] According to one or more embodiments of the present disclosure, when the first generating unit adaptively adjusts the target video template according to the material, it is configured to:

[0202] Adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of the materials; and / or

[0203] Adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials; and / or

[0204] Determine the setting position of the special effects included in the target video template in the material.

[0205] According to one or more embodiments of the present disclosure, if there are multiple materials, the first generation unit, when adaptively adjusting the animation sequence based on the number of animations in the animation sequence included in the target video template and the number of materials, is configured to:

[0206] If the number of animations in the animation sequence is less than the number of materials, repeating each animation in the animation sequence in order so that the number of repeated animations is equal to the number of materials; or

[0207] If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an odd number, then the remaining animations in the animation sequence except the first animation are repeated in order so that the number of repeated animations is equal to the number of materials; or

[0208] If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an even number, then each animation in the animation sequence is repeated in sequence so that the number of repeated animations is equal to the number of materials.

[0209] According to one or more embodiments of the present disclosure, if there are multiple materials, the first generation unit, when adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of materials, is configured to:

[0210] If the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in order so that the number of transitions after repetition is equal to the number of material intervals, where the number of material intervals is the number of materials minus 1.

[0211] According to one or more embodiments of the present disclosure, if there are multiple materials, the first generation unit, when determining the setting position of the special effects included in the target video template in the materials, is configured to:

[0212] If the target video template includes a global special effect, determining the setting position of the global special effect to be the entire time range of the material; or

[0213] If the target video template includes a highlight effect, determine whether the music attribute feature included in the feature information of the material includes the starting position of the music chorus; if so, determine that the setting position of the highlight effect is the starting position of the music chorus; or, if not, determine that the setting position of the highlight effect is the starting position of the second material; or

[0214] If the target video template includes a card point special effect, determining the setting position of the card point special effect to be the starting position of the second material, or determining the setting position of the card point special effect to be the starting position of the second material after the highlight special effect; or

[0215] If the target video template includes an opening special effect, determining the setting position of the opening special effect to be the starting position of the first material; or

[0216] If the target video template includes a closing special effect, the setting position of the closing special effect is determined to be the end position of the last material.

[0217] According to one or more embodiments of the present disclosure, when determining the rendering effect combination according to the feature information of the material, the second generating unit is configured to:

[0218] Obtain the correlation index between the feature information of the material and the candidate rendering effects, and construct a correlation matrix;

[0219] determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and using the target rendering effect sequence as the rendering effect combination;

[0220] The rendering effects include one or more of animation, transition, and special effects.

[0221] According to one or more embodiments of the present disclosure, when determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the second generating unit is configured to:

[0222] Based on the correlation matrix, a beam search algorithm is used to obtain rendering effects that match the material, and a target rendering effect sequence is obtained.

[0223] According to one or more embodiments of the present disclosure, after determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the second generating unit is further configured to:

[0224] Determining whether each rendering effect in the target rendering effect sequence complies with a preset rule;

[0225] If any rendering effect does not comply with the rules, the rendering effect will be replaced or deleted.

[0226] According to one or more embodiments of the present disclosure, the acquiring unit is further configured to:

[0227] If there is only one segment of material, the duration of the material is determined. If the duration of the material does not exceed a preset duration threshold, a music variation point is determined based on the music attribute features included in the feature information of the material, and the playback speed of the portion of the material after the music variation point is reduced; or

[0228] If the duration of the material exceeds a preset duration threshold, the material is divided into multiple segments.

[0229] According to one or more embodiments of the present disclosure, when acquiring the feature information of the material, the acquiring unit is configured to:

[0230] Extracting video frames from the material;

[0231] The video frame of the material is input into a preset feature extraction model, and feature information of the material is obtained through the preset feature extraction model.

[0232] According to one or more embodiments of the present disclosure, the feature information of the material includes: image feature information, audio feature information, quantity information of the material, and time information of the material.

[0233] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0234] The memory stores computer-executable instructions;

[0235] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the video generation method described in the first aspect and various possible designs of the first aspect.

[0236] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the video generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0237] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising computer-executable instructions. When a processor executes the computer-executable instructions, the video generation method as described in the first aspect and various possible designs of the first aspect is implemented.

[0238] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0239] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0240] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video generation method, comprising: Acquire material, and acquire characteristic information of the material; Determine rules according to a preset video generation method to determine a target video generation method; If the target video generation method is the first generation method, determining a target video template from a preset video template pool according to the feature information of the material, and generating a target video according to the material and the target video template; or If the target video generation method is the second generation method, a rendering effect combination is determined according to the feature information of the material, and the target video is generated according to the material and the rendering effect combination.

2. The video generation method according to claim 1, wherein: The step of determining a target video generation method according to a preset video generation method includes: Acquire user demand information, and determine the target video generation method according to the user demand information; or The target video generation method is determined in a random manner.

3. The video generation method according to claim 1 or 2, wherein: The step of determining a target video template from a preset video template pool according to the feature information of the material includes: The characteristic information of each preset video template in the preset video template pool is matched with the characteristic information of the material, and the target video template is determined according to the matching result.

4. The video generation method according to claim 3, wherein: The matching of the characteristic information of each preset video template in the preset video template pool with the characteristic information of the material and determining the target video template according to the matching result includes: The characteristic information of each preset video template in the preset video template pool is matched with the characteristic information of the material, unmatched preset video templates are filtered out, and a preset video template is selected from the remaining preset video templates as the target video template.

5. The video generation method according to any one of claims 1 to 4, wherein: The generating a target video according to the material and the target video template includes: The target video template is adaptively adjusted according to the material, and the adaptively adjusted target template is merged with the material to generate the target video.

6. The video generation method according to claim 5, wherein: The step of adaptively adjusting the target video template according to the material includes: Adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of the materials; and / or Adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the materials; and / or Determine the setting position of the special effects included in the target video template in the material.

7. The video generation method according to claim 6, wherein: If there are multiple materials, adaptively adjusting the animation sequence according to the number of animations in the animation sequence included in the target video template and the number of the materials includes: If the number of animations in the animation sequence is less than the number of materials, repeating each animation in the animation sequence in order so that the number of animations after the repetition is equal to the number of materials; or If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an odd number, the remaining animations in the animation sequence except the first animation are repeated in order so that the number of repeated animations is equal to the number of materials; or If the number of animations in the animation sequence is less than the number of materials, and the number of animations in the animation sequence is an even number, then each animation in the animation sequence is repeated in sequence so that the number of animations after repetition is equal to the number of materials.

8. The video generation method according to claim 6, wherein: If there are multiple materials, adaptively adjusting the transition sequence according to the number of transitions in the transition sequence included in the target video template and the number of the multiple materials includes: If the number of transitions in the transition sequence is less than the number of material intervals, each transition in the transition sequence is repeated in order so that the number of transitions after repetition is equal to the number of material intervals, where the number of material intervals is the number of materials minus 1.

9. The video generation method according to claim 6, wherein: If there are multiple materials, determining the setting position of the special effects included in the target video template in the materials includes: If the target video template includes a global special effect, determining the setting position of the global special effect to be the entire time range of the material; or If the target video template includes a highlight effect, determine whether the music attribute feature included in the feature information of the material includes the starting position of the music chorus, and if so, determine that the setting position of the highlight effect is the starting position of the music chorus; or, if not, determine that the setting position of the highlight effect is the starting position of the second material; or If the target video template includes a card point special effect, the card point special effect is set to a starting position of the second material, or the card point special effect is set to a starting position of the second material after the highlight special effect; or If the target video template includes an opening special effect, determining the setting position of the opening special effect to be the starting position of the first material; or If the target video template includes a closing special effect, the setting position of the closing special effect is determined to be the end position of the last material.

10. The video generation method according to any one of claims 1 to 9, wherein: The determining of the rendering effect combination according to the feature information of the material includes: Obtain the correlation index between the feature information of the material and the candidate rendering effects, and construct a correlation matrix; Determine a target rendering effect sequence from the candidate rendering effects according to the correlation matrix, and use the target rendering effect sequence as the rendering effect combination; The rendering effects include one or more of animation, transition, and special effects.

11. The video generation method according to claim 10, wherein: The step of determining a target rendering effect sequence from the candidate rendering effects according to the correlation matrix includes: Based on the correlation matrix, a beam search algorithm is used to obtain a rendering effect matching the material to obtain a target rendering effect sequence.

12. The video generation method according to claim 10 or 11, wherein: After determining the target rendering effect sequence from the candidate rendering effects according to the correlation matrix, the method further includes: Determine whether each rendering effect in the target rendering effect sequence complies with a preset rule; If any rendering effect does not comply with the rules, the rendering effect will be replaced or deleted.

13. The video generation method according to any one of claims 1 to 12, further comprising: If there is only one segment of material, the duration of the material is determined. If the duration of the material does not exceed a preset duration threshold, a music variation point is determined according to the music attribute features included in the feature information of the material, and the playback speed of the portion of the material after the music variation point is reduced; or If the duration of the material exceeds a preset duration threshold, the material is divided into multiple segments.

14. The video generation method according to any one of claims 1 to 13, wherein: The obtaining of characteristic information of the material includes: Extracting video frames from the material; The video frame of the material is input into a preset feature extraction model, and feature information of the material is obtained through the preset feature extraction model.

15. The video generation method according to any one of claims 1 to 14, wherein: The feature information of the material includes: image feature information, audio feature information, quantity information of the material, and time information of the material.

16. A video generating device, comprising: An acquisition unit, configured to acquire a material and acquire characteristic information of the material; A determination unit, configured to determine a rule according to a preset video generation method to determine a target video generation method; A first generating unit is configured to determine a target video template from a preset video template pool according to feature information of the material if the target video generating mode is the first generating mode, and generate a target video according to the material and the target video template; or The second generating unit is configured to determine a rendering effect combination according to the feature information of the material if the target video generating method is the second generating method, and generate the target video according to the material and the rendering effect combination.

17. An electronic device comprising: at least one processor and memory; wherein, The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 15.

18. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the video generation method according to any one of claims 1 to 15 is implemented.

19. A computer program product, comprising computer-executable instructions, and when a processor executes the computer-executable instructions, the video generation method according to any one of claims 1 to 15 is implemented.

Images