Video generation method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to a video generation method and apparatus, an electronic device, and a storage medium. The method comprises: displaying an animation effect configuration page, the animation effect configuration page comprising a canvas, a guide picture being displayed in the canvas, and the guide picture comprising entities; in response to a selection operation on a target entity in the guide picture, displaying an animation effect configuration option corresponding to the target entity; in response to a configuration operation on the animation effect configuration option corresponding to the target entity, determining a motion constraint condition of the target entity; and generating a target video on the basis of the motion constraint condition of the target entity and the guide picture, in the target video, the target entity moving in a motion mode indicated by the motion constraint condition. The essence of the method is to provide a method for assisting a user in accurately defining the motion of an entity in a guide picture. By using the method, a video generation model can be assisted in "understanding" the creation intention of the user, so that a generated target video better meets the requirements of the user.

Description

Video generation method and device, electronic device, and storage medium

[0001] Cross-reference to Related Applications

[0002] This application claims priority to Chinese Patent Application No. 202411075149.6, filed on August 6, 2024, the disclosure of which is incorporated herein in its entirety as part of the present application. TECHNICAL FIELD

[0003] The present disclosure relates to a video generation method, device, electronic device, and storage medium. BACKGROUND

[0004] With the development of artificial intelligence technology, especially the application of deep learning in the field of image processing, the technology of generating video based on a guide image has been realized. However, the current video generation technology still has some limitations. In particular, in the generated video, it is difficult to accurately control the entities in the guide image to move in a specific motion manner. SUMMARY

[0005] To solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a video generation method, device, electronic device, and storage medium.

[0006] In a first aspect, the present disclosure provides a video generation method, comprising:

[0007] displaying an animation effect configuration page; the animation effect configuration page comprises a canvas, and the canvas displays a guide image; the guide image comprises entities;

[0008] in response to a selection operation on a target entity in the guide image, displaying an animation effect configuration option corresponding to the target entity;

[0009] in response to a configuration operation on the animation effect configuration option corresponding to the target entity, determining a motion constraint condition of the target entity;

[0010] generating a target video based on the motion constraint condition of the target entity and the guide image; in the target video, the target entity moves in a motion manner indicated by the motion constraint condition.

[0011] In a second aspect, the present disclosure also provides a video generation device, comprising:

[0012] a configuration page display module configured to display an animation effect configuration page; the animation effect configuration page comprises a canvas, and the canvas displays a guide image; the guide image comprises entities;

[0013] The entity selection module is configured to, in response to a selection operation on a target entity in the guide image, display an animation effect configuration option corresponding to the target entity.

[0014] The configuration module is configured to, in response to a configuration operation on the animation effect configuration option corresponding to the target entity, determine a motion constraint condition of the target entity.

[0015] The generation module is configured to generate a target video based on the motion constraint condition of the target entity and the guide image, in which the target entity moves in a motion manner indicated by the motion constraint condition.

[0016] In a third aspect, the present disclosure provides an electronic device, which comprises:

[0017] One or more processors;

[0018] A storage device configured to store one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the video generation method as described above.

[0020] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video generation method as described above. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure, together with the description.

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the embodiment description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0023] FIG. 1 is a flowchart of a video generation method according to an embodiment of the present disclosure;

[0024] FIGS. 2-8 are schematic diagrams of several electronic device display interfaces according to embodiments of the present disclosure;

[0025] FIG. 9 is a schematic diagram of a video generation apparatus according to an embodiment of the present disclosure;

[0026] FIG. 10 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to enable a more clear understanding of the above-mentioned objects, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0028] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other manners different from those described herein; it is apparent that the embodiments described in the specification are only a part of the embodiments of the present disclosure, and not all the embodiments.

[0029] FIG. 1 is a flowchart of a video generation method according to an embodiment of the present disclosure. The embodiment can be applied to a case of generating a video in a client. The method can be executed by a video generation apparatus. The apparatus can be implemented in software and / or hardware. The apparatus can be configured in an electronic device, such as a terminal, for example, a smartphone, a palmtop computer, a tablet computer, a wearable device with a display, a desktop computer, a notebook computer, an all-in-one computer, a smart home device, and the like.

[0030] As shown in FIG. 1, the method can specifically include the following steps.

[0031] S110, a motion effect configuration page is displayed. The motion effect configuration page includes a canvas, and a guide image is displayed in the canvas. The guide image includes entities.

[0032] The guide image is an image that needs to be input into a video generation model, and is used to guide the video generation model to generate a video.

[0033] An entity may, for example, refer to an individual or unit in the guide image that has explicit semantic information, is identifiable, and is countable or uncountable. They jointly constitute the visual content of the guide image through respective attribute features, and are basic objects of image processing and analysis. In some scenarios, the entities in the guide image can be accurately extracted and separated. The entities in the guide image can specifically include a person, a building, a tree, a vehicle, an animal, a sky, or grass in the guide image. The number or type of entities included in the guide image can be one or more, and the present disclosure does not limit this.

[0034] The motion effect configuration page refers to a page used to configure motion constraint conditions of the entities in the guide image. The motion constraint conditions may, for example, be conditions used to limit how the entities move in a target video generated subsequently. Exemplarily, the motion constraint conditions can include one or more of the following: a starting point position, an end point position, a moving track, a moving speed, a moving acceleration, a size change, a posture change, and an action change of the entities during movement.

[0035] The canvas may be, for example, an area for displaying the guide image. In some scenarios, the canvas may also be used to assist the user in configuring the motion constraint condition, so that the user can intuitively understand the effect of the configured motion constraint condition in combination with the guide image.

[0036] Referring to FIG. 2, the motion effect configuration page includes a canvas in which a guide image is displayed. The guide image includes six entities, namely, a person, a puppy, a frisbee, the sky, the grass, and the sun.

[0037] S120, in response to a selection operation on a target entity in the guide image, display a motion effect configuration option corresponding to the target entity.

[0038] The target entity may be, for example, a specific entity in the guide image, which is selected by the user.

[0039] The selection operation on the target entity in the guide image may be, for example, a click operation on the target entity in the guide image or a circle selection operation on the target entity using a circle selection tool.

[0040] In one embodiment, the entities in the guide image are segmented using an entity segmentation model to obtain the areas occupied by each entity in the guide image. If a selection operation (such as a click operation or a sliding operation within a certain area) of the user on a certain area is detected, the entity in the area is taken as the target entity. The selection operation of the user on the area is the selection operation on the target entity in the guide image.

[0041] In another embodiment, for example, if a region circle operation of the user using a circle selection tool in the guide image is detected, the entity in the area indicated by the circle selection tool at the end of the circle operation is taken as the target entity.

[0042] Optionally, the method further includes: in response to the selection operation on the target entity in the guide image, marking the state of the target entity in the guide image as a selected state. This facilitates the user to quickly understand whether the selection of the target entity is successful.

[0043] Marking the state of the target entity in the guide image as a selected state may include, for example, displaying a selection box around the target entity, and / or adjusting the color tone of the area occupied by the target entity to a preset color. Further, adjusting the color tone of the area occupied by the target entity to a preset color may be achieved by superimposing a mask layer with a preset color on the guide image, the area indicated by the mask layer being the area occupied by the target entity.

[0044] S130, in response to a configuration operation on the motion effect configuration option corresponding to the target entity, determining the motion constraint condition of the target entity.

[0045] The animation effect configuration option corresponding to the target entity may be, for example, a tool for configuring a movement constraint condition of the target entity. Specifically, the animation effect configuration option is used to configure one or more of a movement end position, a movement start position, a movement trajectory, and a form of the target entity during movement of the target entity.

[0046] There are various specific implementation methods for this step, and the present disclosure does not limit this. In actual implementation, optionally, in some embodiments, if the animation effect configuration option includes an end position configuration option, S120 may include: in response to a triggering operation on the end position configuration option, displaying an end position identifier corresponding to the target entity; in response to a movement operation of the end position identifier in the canvas, taking a position indicated by the moved end position identifier as a movement end position of the target entity; and adding the movement end position of the target entity to the movement constraint condition of the target entity.

[0047] The end position configuration option may be, for example, an option for assisting a user in configuring a movement end position of a target object. The triggering operation on the end position configuration option may be, for example, a click or a drag operation on the end position configuration option.

[0048] The end position identifier corresponding to the target entity represents the target entity and is an icon for indicating to a user where the movement end position of the target entity is currently set. In actual implementation, it may be an icon of any style, such as a dot, a circle, a rectangle, or a water drop shape.

[0049] Optionally, in response to a selection operation on the target entity in the guide image, a selection box is displayed, which surrounds the target entity. When a selection box appears around an entity, the entity is determined to be the target entity. The end position identifier corresponding to the target entity has the same shape and size as the selection box.

[0050] The movement operation of the end position identifier in the canvas may be, for example, an operation of editing a position coordinate of the end position identifier in the canvas, an operation of dragging the end position identifier, or the like.

[0051] It should be noted that in actual implementation, the size of the canvas may be set to be greater than the size of the guide image, and the end position identifier may be moved to any position in the canvas. That is, the end position identifier may be moved within the guide image or outside the guide image. In this case, the target video generated subsequently appears to show that the target subject moves out of the frame.

[0052] For example, referring to FIG. 2, if the user clicks the puppy in the guide map, referring to FIG. 3, a selection box appears around the puppy, indicating that the puppy is the target entity, and a motion effect configuration option corresponding to the puppy is displayed above the puppy. The motion effect configuration option includes an "end position" option. The "end position" option is a kind of end point configuration option. If the user clicks the "end position" option, an end point identifier is displayed in the canvas, which has the same shape and size as the selection box around the puppy. The user can drag the end point identifier to any position in the canvas. After the user drags the end point identifier, the last position of the end point identifier is taken as the moving end position of the puppy, and thus the constraint condition about the moving end position of the target entity in the motion constraint condition of the target entity is obtained.

[0053] In some other embodiments, optionally, the implementation method of the step can further include: in response to the morphological adjustment operation on the end point identifier, determining the morphology of the target entity at the moving end position based on the morphology of the end point identifier after the morphological adjustment; and adding the morphology of the target entity at the moving end position to the motion constraint condition of the target entity.

[0054] The morphological adjustment operation on the end point identifier may, for example, be an adjustment to one or more aspects of the size, deformation, and rotation of the end point identifier. Since the end point identifier represents the target entity, the morphological adjustment of the end point identifier means the morphological adjustment of the target entity, which will cause changes in the size, posture, action, etc. of the target entity.

[0055] For example, based on FIG. 3, the user further adjusts the size of the end point identifier in the vertical direction, such as adjusting the end point identifier to one half of the current size, i.e. obtaining the constraint condition about the morphology of the target entity at the moving end position in the motion constraint condition of the target entity.

[0056] It should be noted that in practice, the position of the end point identifier of the target entity can be kept consistent with the position of the target entity in the guide map, and only the morphology of the end point identifier of the target entity is adjusted. That is, the target entity is set to change in size, posture, action, etc. in place. Alternatively, the position of the end point identifier of the target entity can be adjusted to be different from the position of the target entity in the guide map, but the morphology of the end point identifier of the target entity is not adjusted. That is, the target entity is set to move, but not to change in size, posture, action, etc. The position of the end point identifier of the target entity can also be set to be different from the position of the target entity in the guide map, and the morphology of the end point identifier of the target entity is adjusted. That is, the target entity is set to move and change in size, posture, action, etc.

[0057] In some embodiments, the animation configuration option can further include a moving track configuration option; the implementation method of the present step can include: in response to a triggering operation on the moving track configuration option corresponding to the target entity, displaying a moving track drawing tool; in response to a moving operation on the moving track drawing tool in the guide map, displaying a moving track; and adding the moving track to the motion constraint condition of the target entity.

[0058] The moving track configuration option can be an option for assisting the user to configure the moving track of the target object. The triggering operation on the moving track configuration option can be, for example, a clicking or dragging operation on the moving track configuration option.

[0059] The moving track drawing tool is an identifier representing the target entity and allowing the user to move the position of the target entity and record the specific moving path during the movement. The moving path recorded by the moving track drawing tool during the movement is the moving track. The moving track drawing tool can be any style of icon, such as a dot, a circle, a rectangle, a water droplet, a pen, etc.

[0060] For example, referring to FIG. 2, if the user clicks the puppy in the guide map, referring to FIG. 4, a selection box appears around the puppy, indicating that the puppy is the target entity, and the animation configuration option corresponding to the puppy is displayed above the puppy. The animation configuration option includes a “motion track” option. The “motion track” option is the moving track configuration option. If the user clicks the “moving track configuration option” option, a dot appears at the puppy, which is the moving track drawing tool. The user can click and drag the dot to move. During the dragging process, the moving path of the dot is shown as a dashed line, which is the moving track. In this way, the constraint condition about the moving path in the motion constraint condition of the target entity is obtained.

[0061] It should be noted that in practice, the animation configuration option can include both the end point configuration option and the moving track configuration option. In practice, the two are combined to configure the motion constraint condition. For example, first, the moving end point position of the target entity is configured by means of the end point configuration option, and then the moving track between the moving start point position and the moving end point position of the target entity is configured by means of the moving track configuration option, with the position of the target entity in the guide map as the moving start point position. Further, in this case, the moving end point position of the target entity configured by means of the end point configuration option can be determined as the first end point position; during the process of “in response to the moving operation on the moving track drawing tool in the guide map, displaying the moving track”, it further includes: if it is checked that the distance between the end point of the moving track drawn by the current moving track drawing tool and the first end point position is less than a set distance threshold, the moving track drawing tool is attracted to the first end point position, so that the end point of the obtained moving track coincides with the first end point position.

[0062] Exemplarily, referring to FIG. 5, if the user first selects the “end position” option, the determined moving end position (i.e., the first end position) of the target entity is point B by adjusting the end position identifier. Then, the user selects the “motion trajectory” option, and draws the motion trajectory by using the motion trajectory drawing tool. During the drawing process, if at a certain time, the end point of the drawn motion trajectory is point A, and the distance between point A and point B is less than the set distance threshold, point A and point B are automatically connected, so that after the connection, the end point of the motion trajectory becomes point B, so as to present the visual effect that the end point of the motion trajectory is attracted to the first end position. In this way, the difficulty of drawing the motion trajectory can be reduced, and the drawing efficiency of the motion trajectory can be improved.

[0063] If the motion trajectory of the target entity is configured by using the motion trajectory configuration option first, it is determined that the motion trajectory configured by using the motion trajectory configuration option is the first motion trajectory; then the end position configuration option is triggered, and the end position identifier corresponding to the target entity is optionally displayed; the display position of the end position identifier is consistent with the position of the end point of the first motion trajectory. In this case, the position of the end position identifier can be further moved subsequently, or the form of the end position identifier can be adjusted.

[0064] In S140, a target video is generated based on the motion constraint condition of the target entity and the guide map. In the target video, the target entity moves in the motion manner indicated by the motion constraint condition.

[0065] The essence of this step is that in the process of generating the target video, the guide map is used, and the motion constraint condition obtained by S130 is also used. In the target image, the motion characteristics of the target entity meet the requirements of the motion constraint condition.

[0066] Exemplarily, referring to FIG. 4, if the target entity is a puppy, the motion constraint condition obtained by performing S130 includes that the moving end position of the puppy is point B, and the moving path is as shown by the dashed arrow in FIG. 4. In the finally generated target image, the puppy moves to point B along the dashed arrow with an arrow in FIG. 4 at a certain speed.

[0067] It should be noted that in actual application, the description of the motion of an object can be started from multiple aspects, such as speed, acceleration, starting point, end point, trajectory, whether there is a flip, whether there is a deformation, etc. The motion constraint condition can limit part or all of these aspects. If the motion constraint condition only limits some aspects, the video generation model can be allowed to freely play or a default motion mode can be used to process these unconstrained aspects.

[0068] It should be noted that in practice, the guide language can also be obtained, and the step can include: generating the target video based on the motion constraint condition of the target entity, the guide map and the guide language.

[0069] The technical solution sets the response to the selection operation of the target entity in the guide map to display the motion effect configuration option corresponding to the target entity, determines the motion constraint condition of the target entity in response to the configuration operation of the motion effect configuration option corresponding to the target entity, generates the target video based on the motion constraint condition of the target entity and the guide map, and the target entity moves in the motion manner indicated by the motion constraint condition in the target video. The essence is to give a method for assisting users to accurately define the motion of the entity in the guide map. By using the method, the video generation model can better "understand" the creative intention of the user, and the generated target video is more in line with the needs of the user.

[0070] Based on the above technical solution, the method can also include: determining the category name of the target entity in response to the selection operation of the target entity in the guide map; S140 can include: generating the target video based on the motion constraint condition of the target entity, the category name of the target entity and the guide map.

[0071] The category name of the target entity may, for example, be the name of the category to which the target entity belongs.

[0072] Optionally, in response to the selection operation of the target entity in the guide map, a candidate name that can be used as the target entity is displayed; and in response to the selection operation of the target name in the candidate name, the target name is used as the category name of the target entity. The candidate name that can be used as the target entity can be obtained by an entity recognition algorithm after recognizing the target entity. For example, referring to FIG. 6, when the user clicks on the puppy in the guide map, the puppy is the target entity, and three candidate names are displayed, namely "puppy", "teddy" and "kitty". The user can select one of them as the category name of the selected target entity. If the user selects "puppy" as the category name of the target entity, the motion of the puppy in the generated target video will reflect the natural motion characteristics of the puppy. If the user selects "kitty", the motion of the target entity, which is actually a puppy, will be given the motion characteristics similar to those of a kitten.

[0073] In some scenarios, a category name input box of the target entity can also be displayed to allow the user to input the category name of the target entity.

[0074] By setting the step of determining the category name of the target entity and using the determined category name as the input of the video generation model, the purpose is to make the target entity exhibit natural actions in line with its category characteristics in the generated target video.

[0075] On the basis of the technical solution, the dynamic effect configuration page can further include an entity display area. The method can further include: displaying a label of an entity configured with the motion constraint condition in the entity display area; and in response to a selection operation on a first entity in the entity display area, displaying the motion constraint condition corresponding to the first entity in the canvas.

[0076] The entity display area can be used to display labels of all entities configured with the motion constraint condition. The label of an entity can be information for distinguishing the entity from other entities, such as the name of the entity, the representative color of the entity, the number of the entity, the characteristics of the configured motion constraint condition, and the like. The first entity can be one or more of the entities displayed in the entity display area.

[0077] There are various specific implementation methods for displaying the motion constraint condition corresponding to the first entity, which are not limited in the present disclosure. For example, the motion constraint condition corresponding to the first entity can be displayed in a textual description manner and / or in an image manner.

[0078] Further, the entity display area can further include a single-entity switch option. If the single-entity switch option is in an open state, the motion constraint condition corresponding to the first entity is displayed in response to the selection operation on the first entity in the entity display area. If the single-entity switch option is in a closed state, the motion constraint conditions corresponding to all entities are displayed.

[0079] For example, referring to FIG. 7, it is assumed that the user has configured motion constraint conditions for the puppy, the man, and the frisbee in the guide map. The labels of the three entities, i.e., the puppy, the man, and the frisbee, are displayed in the entity display area. The motion constraint condition of the puppy is configured by using only the end point identifier. The motion constraint condition of the man is configured by using the end point identifier and the movement trajectory drawing tool. In FIG. 7, the “only display the current subject” option is the single-entity switch option. It is assumed that the single-entity switch option is in the open state in FIG. 7. Then, the user clicks the “puppy” option in the entity display area. As shown in FIG. 7, the movement end point position of the configured puppy is displayed in the canvas by using the end point identifier. Then, if the user clicks the “man” option in the entity display area, the movement end point position of the configured man is displayed in the canvas by using the end point identifier, and the movement trajectory of the configured man is displayed in the canvas by using the dashed arrow. If the user sets the single-entity switch option to the closed state, the motion constraint conditions of the puppy, the man, and the frisbee are displayed simultaneously.

[0080] In practice, in the process of configuring the motion constraint condition of an entity, a representative color corresponding to the entity can be determined for each entity, and the representative colors of different entities are different. For example, the representative color of a puppy is pink. If the puppy is a target entity, when configuring the motion constraint condition of the puppy, the hue of the puppy can be adjusted to pink (which can be achieved by superimposing a mask of a specific color), and the color of one or more of the end identifier, the movement trajectory drawing tool, and the movement trajectory used is pink. In the entity display area, the representative color of the puppy is prompted by a pink color block or a pink color text.

[0081] Since the representative colors of different entities are different, the user can intuitively distinguish which entity the current end identifier, movement trajectory drawing tool, or movement trajectory reflects the motion constraint condition configuration of by color.

[0082] On the basis of the above technical solutions, the motion effect configuration page can also optionally include a prompt information display area. The method can also include: in the process of configuring the motion constraint condition of the target entity, displaying the use instruction of the currently used option in the prompt information display area.

[0083] The prompt information display area can be used to display the use instructions of some or all options in the motion effect configuration page. The content displayed in this area will be dynamically updated as the currently used option changes.

[0084] In practice, the user may not understand the use methods of the options, so the user needs to be prompted about the use instructions of the options. In practice, the use instructions of the options can be directly displayed near the options or can be uniformly displayed in the prompt information display area.

[0085] If the use instructions of the options are uniformly displayed in the prompt information display area, on the one hand, it is convenient for the user to quickly find the use instructions of the currently used option, and on the other hand, the use instructions will not block the canvas and will not affect the user's configuration of the motion constraint condition.

[0086] Illustratively, referring to FIG. 2, if the target entity needs to be selected, two target entity selection options, namely a “point selection” option and a “box selection” option, are provided on the left side of the prompt information display area. If the user selects the “point selection” option, the user is allowed to select the target entity by clicking on the area occupied by the entity in the guide map. If the user selects the “box selection” option, the user can use the circle selection tool to circle the target entity in the guide map. In FIG. 2, the “point selection” option is in a selected state, and the “point selection” option is the currently used option. The prompt information “please point to select the subject of the specified motion” is displayed in the prompt information display area, which is the use instruction of the “point selection” option.

[0087] Similarly, referring to FIG. 3, the "end position" option is in a selected state, the "end position" option is the option being used at present, and the prompt information "please move the end identifier to specify the end of the movement" is displayed in the prompt information display area, which is an instruction for use of the "end position" option.

[0088] On the basis of the technical solution described above, S110 can be replaced by: displaying a video generation configuration page; the video generation configuration page comprises a guide map display area, a movement constraint condition configuration entry, and a video generation option; in the case where the guide map is displayed in the guide map display area, in response to a triggering operation on the movement constraint condition configuration entry, a motion effect configuration page is displayed; and S140 can be replaced by: in response to a triggering operation on the video generation option in the video generation configuration page, generating a target video based on the movement constraint condition of the target entity and the guide map.

[0089] The video generation configuration page may, for example, refer to a page for configuring the limiting information for video generation. Optionally, the limiting information can comprise one or more of the following: a guide map, a guide sentence, a video generation model, a time length, a size, and a camera movement of the generated video.

[0090] The guide map display area may, for example, be an area for displaying a guide map in the video generation configuration page in the case where a guide map has been uploaded. The movement constraint condition configuration entry may, for example, be an entry for leading the user to a motion effect configuration page. The video generation option may, for example, be an option for triggering the video generation model to work and generating a target video.

[0091] Illustratively, referring to FIG. 8, the video generation configuration page comprises a guide map display area, a movement constraint condition configuration entry, and a video generation option. In the case where no guide map has been uploaded, a prompt "add a guide map" is displayed in the guide map display area to guide the user to upload a guide map. After the guide map is uploaded, the guide map is displayed in the guide map display area. The "motion effect palette" in FIG. 8 is the movement constraint condition configuration entry, and the "generate" option is the video generation option.

[0092] In the case where the guide map is displayed in the guide map display area, if the user clicks the "motion effect palette", a motion effect configuration page is displayed. The guide map displayed in the canvas of the motion effect configuration page is consistent with the guide map displayed in the guide map display area in the video generation configuration page.

[0093] After the limiting information required for video generation is all configured, if the user clicks the "generate" option in the video generation configuration page, a target video is generated based on the movement constraint condition of the target entity and the guide map.

[0094] It should be noted that in some scenarios, multiple functions can be provided in the video generation configuration page to set the guidance of video generation. Some of these functions can conflict with the entity motion constraint condition configuration function provided by the present disclosure. In this case, the user should be allowed to select one function from the conflicting functions. For example, if the first and last frame image limiting function and the lens movement limiting function conflict with the entity motion constraint condition configuration function, when the user selects the first and last frame image limiting function or the lens movement limiting function, the use of the entity motion constraint condition configuration function is limited; conversely, if the user selects the entity motion constraint condition configuration function, the use of the first and last frame image limiting function or the lens movement limiting function is limited.

[0095] For example, the first and last frame image limiting function can mean that the user can customize the screen content at the start (first frame) and end (last frame) of the target video. The lens movement limiting function can mean that the user can specify the way in which the lens moves or changes in the target video. The lens movement includes, but is not limited to, different types of lens movements such as push, pull, shake, shift, and lift.

[0096] By setting the video generation configuration page to include the motion constraint condition configuration portal, the essence is to set the portal of the dynamic effect configuration page in the video generation configuration page, so that the functions of the video generation configuration page are more concentrated, and the user can configure the limiting conditions of the generated video more conveniently.

[0097] On the basis of the above technical solutions, the method can further include: displaying a video display page; the video display page includes the target video and prompt information corresponding to the target video; the prompt information is used to prompt the guidance map used in the target video generation process and the motion constraint condition that has been configured.

[0098] The video display page can be, for example, a page for displaying the generated target video.

[0099] For example, referring to FIG. 8, the video display page displays the generation results of two video generations. Video 1 is generated based on guidance map 1, guidance text 1, and the motion constraint condition for the entity in guidance map 1. The prompt information corresponding to video 1 prompts the guidance map 1, guidance text 1, and the motion constraint condition that has been configured for the entity in guidance map 1. Video 2 is generated based on guidance map 2 and guidance text 2, and does not use the motion constraint condition. The prompt information corresponding to video 2 prompts the guidance map 2 and guidance text 2, and does not prompt the motion constraint condition that has been configured. This setting facilitates the user to quickly understand which information is used in each video generation process.

[0100] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the use range, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0101] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can voluntarily choose whether to provide personal information to the software or hardware such as an electronic device, an application program, a server or a storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0102] As an optional but non-limiting implementation manner, in response to receiving an active request of a user, the manner of sending a prompt information to the user may, for example, be a pop-up window manner, and the prompt information may be presented in the form of text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0103] It can be understood that the above notification and obtaining of user authorization process is only illustrative, and does not limit the implementation manner of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0104] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited to the action sequence described, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.

[0105] FIG. 9 is a structural schematic diagram of a video generation apparatus in an embodiment of the present disclosure. The video generation apparatus provided by the embodiment of the present disclosure can be configured in a client. Referring to FIG. 9, the video generation apparatus specifically includes:

[0106] The configuration page display module 310 is configured to display an animation effect configuration page; the animation effect configuration page includes a canvas, and the canvas displays a guide image; the guide image includes an entity;

[0107] The entity selection module 320 is configured to, in response to a selection operation on a target entity in the guide image, display animation effect configuration options corresponding to the target entity;

[0108] The configuration module 330 is configured to, in response to a configuration operation on the animation effect configuration options corresponding to the target entity, determine a motion constraint condition of the target entity.

[0109] The generation module 340 is configured to generate a target video based on the motion constraint condition of the target entity and the guide graph, and in the target video, the target entity moves in a manner indicated by the motion constraint condition.

[0110] Further, the apparatus further comprises a name determination module configured to:

[0111] determine a category name of the target entity in response to a selection operation on the target entity in the guide graph;

[0112] The generation module 340 is configured to:

[0113] generate a target video based on the motion constraint condition of the target entity, the category name of the target entity and the guide graph.

[0114] Further, the motion effect configuration option is used to configure one or more of a movement end position, a movement start position, a movement trajectory and a form of the target entity during movement.

[0115] Further, if the motion effect configuration option comprises an end position configuration option, the configuration module 330 is configured to:

[0116] display an end position identifier corresponding to the target entity in response to a triggering operation on the end position configuration option;

[0117] add, as the movement end position of the target entity, a position indicated by the end position identifier after movement in response to a movement operation of the end position identifier in the canvas;

[0118] add the movement end position of the target entity to the motion constraint condition of the target entity.

[0119] Further, the configuration module 330 is further configured to:

[0120] determine a form of the target entity at the movement end position based on a form of the end position identifier after form adjustment in response to a form adjustment operation on the end position identifier;

[0121] add the form of the target entity at the movement end position to the motion constraint condition of the target entity.

[0122] Further, the motion effect configuration option comprises a movement trajectory configuration option, and the configuration module 330 is configured to:

[0123] display a movement trajectory drawing tool in response to a triggering operation on the movement trajectory configuration option corresponding to the target entity;

[0124] in response to a movement operation of the movement trajectory drawing tool in the guide map, display a movement trajectory;

[0125] add the movement trajectory to a motion constraint condition of the target entity.

[0126] Further, the motion effect configuration page further includes an entity display area; the configuration page display module is further configured to:

[0127] display a label of the entity with the configured motion constraint condition in the entity display area;

[0128] in response to a selection operation on a first entity in the entity display area, display the motion constraint condition corresponding to the first entity in the canvas.

[0129] Further, the motion effect configuration page further includes a prompt information display area; the configuration page display module is configured to:

[0130] in a configuration process of the motion constraint condition of the target entity, display a usage instruction of a currently used option in the prompt information display area.

[0131] Further, the configuration page display module is configured to:

[0132] display a video generation configuration page; the video generation configuration page includes a guide map display area, a motion constraint condition configuration entry, and a video generation option;

[0133] in a case where the guide map is displayed in the guide map display area, in response to a triggering operation on the motion constraint condition configuration entry, display the motion effect configuration page;

[0134] generate a target video based on the motion constraint condition of the target entity and the guide map, including:

[0135] in response to a triggering operation on the video generation option in the video generation configuration page, generate a target video based on the motion constraint condition of the target entity and the guide map.

[0136] Further, the apparatus further includes a result display module, configured to:

[0137] display a video display page; the video display page includes the target video and prompt information corresponding to the target video; the prompt information is used to prompt the guide map used in the target video generation process and the motion constraint condition configured.

[0138] The video generation apparatus provided by the embodiments of the present disclosure can perform the steps performed by the client or the server in the video generation method provided by the embodiments of the method of the present disclosure, and has the execution steps and advantages, which will not be repeated here.

[0139] FIG. 10 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. Referring to FIG. 10, a structural schematic diagram of an electronic device 1000 according to an embodiment of the present disclosure is shown. The electronic device 1000 according to an embodiment of the present disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (e.g., a car navigation terminal), a wearable electronic device, and the like, and a fixed terminal such as a digital TV, a desktop computer, a smart home device, and the like. The electronic device shown in FIG. 10 is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0140] As shown in FIG. 10, the electronic device 1000 can include a processing device (e.g., a central processing unit, a graphic processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003 to implement the video generation method according to the embodiments of the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0141] Generally, the following devices can be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage device 1008 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 1009. The communication device 1009 can allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although FIG. 10 shows the electronic device 1000 having various devices, it should be understood that all the shown devices are not required to be implemented or possessed. More or less devices can be alternatively implemented or possessed.

[0142] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts, thereby implementing the video generation method as described above. In such embodiments, the computer program can be downloaded and installed from a network by the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-described functions defined in the methods of embodiments of the present disclosure are performed.

[0143] It should be noted that the computer-readable medium described above in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include an information signal in a baseband or an information signal transmitted as a carrier wave in a propagated transmission. Such a propagated transmission can take place in various forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that can be used to carry or store a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to, a wire, cable, optical fiber, RF (radio frequency), or any suitable combination thereof.

[0144] In some embodiments, the client, server can communicate using any known or later developed network protocols, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital information (e.g., communication networks). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any known or later developed network.

[0145] The computer readable medium described above can be included in the electronic device described above; or can exist separately, without being assembled into the electronic device.

[0146] The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0147] A motion effect configuration page is displayed; the motion effect configuration page includes a canvas, and a guide image is displayed in the canvas; the guide image includes an entity;

[0148] In response to a selection operation on a target entity in the guide image, a motion effect configuration option corresponding to the target entity is displayed;

[0149] In response to a configuration operation on the motion effect configuration option corresponding to the target entity, a motion constraint condition of the target entity is determined;

[0150] A target video is generated based on the motion constraint condition of the target entity and the guide image; in the target video, the target entity moves in a motion manner indicated by the motion constraint condition.

[0151] Optionally, when the one or more programs described above are executed by the electronic device, the electronic device can further perform other steps described in the above embodiments.

[0152] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0153] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0154] The units described in the embodiments of the present disclosure can be implemented by hardware, software, or a combination of hardware and software. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0155] The functions described in this specification can be implemented in part or in whole through one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0156] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores the program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, comprising:

[0158] one or more processors;

[0159] a memory for storing one or more programs;

[0160] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the video generation methods as provided by the present disclosure.

[0161] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements any of the video generation methods as provided by the present disclosure.

[0162] The embodiments of the present disclosure also provide a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the video generation method as described above is implemented.

[0163] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0164] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a video, comprising: displaying a motion effect configuration page; the motion effect configuration page comprises a canvas, and a guide map is displayed in the canvas; the guide map comprises entities; in response to a selection operation on a target entity in the guide map, displaying a motion effect configuration option corresponding to the target entity; in response to a configuration operation on the motion effect configuration option corresponding to the target entity, determining a motion constraint condition of the target entity; based on the motion constraint condition of the target entity and the guide map, generating a target video; in the target video, the target entity moves in a motion manner indicated by the motion constraint condition.

2. The method of claim 1, further comprising: in response to the selection operation on the target entity in the guide map, determining a category name of the target entity; wherein the generating a target video based on the motion constraint condition of the target entity and the guide map comprises: generating the target video based on the motion constraint condition of the target entity, the category name of the target entity, and the guide map.

3. The method of claim 1 or 2, wherein: the motion effect configuration option is used to configure one or more of a moving end position, a moving start position, a moving trajectory, and a form of the target entity during movement of the target entity.

4. The method of claim 3, wherein, if the motion effect configuration option comprises an end position configuration option; the determining the motion constraint condition of the target entity in response to the configuration operation on the motion effect configuration option corresponding to the target entity comprises: in response to a trigger operation on the end position configuration option, displaying an end position identifier corresponding to the target entity; in response to a movement operation of the end position identifier in the canvas, taking a position indicated by the end position identifier after movement as a moving end position of the target entity; adding the moving end position of the target entity to the motion constraint condition of the target entity.

5. The method of claim 4, further comprising: in response to a form adjustment operation on the end position identifier, determining a form of the target entity at the moving end position based on a form of the end position identifier after form adjustment; adding the form of the target entity at the moving end position to the motion constraint condition of the target entity.

6. The method of claim 3, wherein, the motion effect configuration option comprises a moving trajectory configuration option; the determining the motion constraint condition of the target entity in response to the configuration operation on the motion effect configuration option corresponding to the target entity comprises: in response to a trigger operation on the moving trajectory configuration option corresponding to the target entity, displaying a moving trajectory drawing tool; in response to a movement operation of the moving trajectory drawing tool in the guide map, displaying a moving trajectory; adding the moving trajectory to the motion constraint condition of the target entity.

7. The method according to any one of claims 1-6, wherein, the motion effect configuration page further comprises an entity display area; the method further comprises: displaying a label of an entity with a configured motion constraint condition in the entity display area. In response to a selection operation on a first entity in the entity display area, a motion constraint corresponding to the first entity is displayed in the canvas.

8. The method of any one of claims 1-7, wherein, The motion effect configuration page further includes a prompt information display area; and the method further includes: During configuration of the motion constraint of the target entity, a usage instruction of a currently used option is displayed in the prompt information display area.

9. The method of any one of claims 1-8, wherein, The motion effect configuration page includes: a video generation configuration page is displayed; the video generation configuration page includes a guide map display area, a motion constraint configuration entry, and a video generation option; In a case where the guide map is displayed in the guide map display area, in response to a triggering operation on the motion constraint configuration entry, a motion effect configuration page is displayed; The target video is generated based on the motion constraint of the target entity and the guide map, including: In response to a triggering operation on a video generation option in the video generation configuration page, the target video is generated based on the motion constraint of the target entity and the guide map.

10. The method of any one of claims 1-9, further comprising: a video display page is displayed; the video display page includes the target video and prompt information corresponding to the target video; The prompt information is used to prompt a guide map used in the target video generation process and a motion constraint configured.

11. A video generation apparatus, comprising: a configuration page display module configured to display a motion effect configuration page; The motion effect configuration page includes a canvas, and the canvas displays a guide map; the guide map includes entities; an entity selection module configured to display motion effect configuration options corresponding to a target entity in the guide map in response to a selection operation on the target entity; a configuration module configured to determine a motion constraint of the target entity in response to a configuration operation on the motion effect configuration options corresponding to the target entity; a generation module configured to generate a target video based on the motion constraint of the target entity and the guide map; In the target video, the target entity moves in a motion manner indicated by the motion constraint.

12. An electronic device, comprising: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-10.

13. A computer readable storage medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the method of any one of claims 1-10.

Images