Video generation method and device, electronic equipment and storage medium
Through the use of the motion-effect configuration page, users can accurately control the physical movement in the guide map, solving the problem that physical movement modes in the prior art are difficult to control, and the generated video is more in line with the user's intention.
Patent Information
- Application Number
- CN202411075149.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-06
AI Technical Summary
The existing video generation technology is difficult to accurately control the physical movement mode in the guide map, resulting in the generated video not meeting the user's creative intentions.
Provide a video generation method, by displaying the motion configuration page, allows users to select and configure the target entity in the guide map, determine their motion constraints, such as end point position, trajectory and morphological changes, and then generate target video.
It realizes a more accurate user intention understanding of the video generation model, the generated video is more in line with the user's creative needs, and the entity moves in the video in a specified motion manner.
Smart Images

Figure CN119094864A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a video generation method, device, electronic device and storage medium. Background Art
[0002] With the development of artificial intelligence technology, especially the application of deep learning in the field of image processing, the technology of video generation based on guidance graphs has been realized. However, there are still some limitations in the current video generation technology. In particular, it is difficult to accurately control the entities in the guidance graph to move in a specific way in the generated video. Summary of the invention
[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a video generation method, device, electronic device and storage medium.
[0004] In a first aspect, the present disclosure provides a video generation method, comprising:
[0005] Displaying a dynamic effect configuration page; the dynamic effect configuration page includes a canvas, and a guide map is displayed in the canvas; the guide map includes an entity;
[0006] In response to a selection operation on a target entity in the guide graph, displaying a motion effect configuration option corresponding to the target entity;
[0007] In response to a configuration operation on a motion effect configuration option corresponding to the target entity, determining a motion constraint condition of the target entity;
[0008] A target video is generated based on the motion constraint conditions of the target entity and the guide map; in the target video, the target entity moves in a motion manner indicated by the motion constraint conditions.
[0009] In a second aspect, the present disclosure further provides a video generation device, including:
[0010] A configuration page display module, used to display a dynamic effect configuration page; the dynamic effect configuration page includes a canvas, and a guide map is displayed in the canvas; the guide map includes an entity;
[0011] An entity selection module, configured to display, in response to a selection operation on a target entity in the guide graph, a motion effect configuration option corresponding to the target entity;
[0012] A configuration module, configured to determine a motion constraint condition of the target entity in response to a configuration operation on a motion effect configuration option corresponding to the target entity;
[0013] A generation module is used to generate a target video based on the motion constraint conditions of the target entity and the guide map; in the target video, the target entity moves in a motion manner indicated by the motion constraint conditions.
[0014] In a third aspect, the present disclosure further provides an electronic device, the electronic device comprising:
[0015] one or more processors;
[0016] A storage device for storing one or more programs;
[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the video generating method as described above.
[0018] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video generation method as described above.
[0019] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0020] The technical solution provided by the embodiment of the present disclosure is to display the motion effect configuration options corresponding to the target entity in response to the selection operation of the target entity in the guide map; determine the motion constraint conditions of the target entity in response to the configuration operation of the motion effect configuration options corresponding to the target entity; generate a target video based on the motion constraint conditions of the target entity and the guide map; in the target video, the target entity moves in the motion mode indicated by the motion constraint conditions. Its essence is to provide a method to assist users to accurately limit the movement of entities in the guide map. By adopting this method, the video generation model can be assisted to more clearly "understand" the user's creative intention, thereby making the generated target video more in line with the user's needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0023] Figure 1 A flowchart of a video generation method provided by an embodiment of the present disclosure;
[0024] Figure 2-Figure 8 Schematic diagrams of display interfaces of several electronic devices provided by embodiments of the present disclosure;
[0025] Fig. 9 is a structural schematic diagram of a video generating device in an embodiment of the present disclosure;
[0026] Fig.10 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0029] Figure 1 A flowchart of a video generation method provided in an embodiment of the present disclosure. This embodiment is applicable to situations where video generation is performed in a client. The method can be executed by a video generation device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a terminal, specifically including but not limited to smart phones, PDAs, tablet computers, wearable devices with display screens, desktop computers, laptop computers, all-in-one computers, smart home devices, etc.
[0030] like Figure 1 As shown, the method may specifically include:
[0031] S110, displaying a motion effect configuration page; the motion effect configuration page includes a canvas, and a guide map is displayed in the canvas; the guide map includes an entity.
[0032] The guide image is an image that needs to be input into the video generation model, which is used to guide the video generation model to generate a video.
[0033] For example, an entity may refer to an individual or unit in a guide map that has clear semantic information, is identifiable, and is countable or uncountable. They together constitute the visual content of the guide map through their respective attribute characteristics and are the basic objects of image processing and analysis. In some scenarios, the entities in the guide map can be accurately extracted and separated. The entities in the guide map may specifically include people, buildings, trees, vehicles, animals, sky, or grass in the guide map. The number or type of entities included in the guide map may be one or more, and this application does not limit this.
[0034] The motion effect configuration page refers to a page used to configure the motion constraints of entities in the guide image. Motion constraints can, for example, be conditions for limiting how the entity moves in the target video generated later. Exemplarily, the motion constraints can include limiting conditions for one or more aspects of the entity's starting position, end position, movement trajectory, movement speed, movement acceleration, changes in the entity's size during movement, changes in posture, and changes in movement.
[0035] The canvas may be, for example, an area for displaying a guide map. In some scenarios, the canvas may also be used to assist the user in configuring motion constraints, so that the user can intuitively understand the effect of combining the configured motion constraints with the guide map.
[0036] For example, see Figure 2 The animation configuration page includes a canvas, in which a guide map is displayed. The guide map includes six entities: a person, a puppy, a frisbee, the sky, the grass, and the sun.
[0037] S120: In response to a selection operation on a target entity in the guidance diagram, displaying motion effect configuration options corresponding to the target entity.
[0038] The target entity may be, for example, a specific entity in the guidance graph, which is the entity selected by the user.
[0039] The selection operation on the target entity in the guidance map may be, for example, a click operation on the target entity in the guidance map or a circle selection operation on the target entity using a circle selection tool.
[0040] In one embodiment, the entity segmentation model is used to segment the entities in the guide map to obtain the areas occupied by each entity in the guide map. If a user selection operation on a certain area is detected (such as a click operation or a slide operation in a certain area), the entity corresponding to the area is used as the target entity. The user's selection operation on the area is the selection operation on the target entity in the guide map.
[0041] In another embodiment, illustratively, it is detected that the user uses the circle selection tool to perform an area circle operation in the guide map, and the entity in the area indicated by the circle selection tool when the circle selection operation is completed is used as the target entity.
[0042] Optionally, the method further comprises: in response to a selection operation on the target entity in the guide map, marking the state of the target entity in the guide map as a selected state, so that the user can quickly understand whether the target entity is successfully selected.
[0043] Exemplarily, marking the state of the target entity as a selected state in the guide map may include displaying a selection box around the target entity, the selection box surrounding the target entity; and / or adjusting the color tone of the area occupied by the target entity to a preset color. Further, adjusting the color tone of the area occupied by the target entity to a preset color may be achieved by superimposing a mask layer with a preset color on the guide map, the area indicated by the mask layer being the area occupied by the target entity.
[0044] S130: In response to a configuration operation on a motion effect configuration option corresponding to a target entity, determine a motion constraint condition of the target entity.
[0045] The motion configuration options corresponding to the target entity may be, for example, a tool for configuring the motion constraints of the target entity. Specifically, the motion configuration options are used to configure one or more of the target entity's moving end position, moving start position, moving trajectory, and the target entity's form during movement.
[0046] There are many specific implementation methods for this step, and this application does not limit this. In practice, optionally, in some embodiments, if the motion effect configuration option includes an endpoint configuration option; S120 may include: in response to a trigger operation on the endpoint configuration option, displaying an endpoint identifier corresponding to the target entity; in response to a move operation on the endpoint identifier in the canvas, using the position indicated by the endpoint identifier after the move as the moving endpoint position of the target entity; adding the moving endpoint position of the target entity to the motion constraint condition of the target entity.
[0047] The destination configuration option may refer to, for example, an option for assisting the user in configuring the destination of the target object. The triggering operation on the destination configuration option may be, for example, an operation such as clicking or dragging the destination configuration option.
[0048] The destination identifier corresponding to the target entity represents the target entity and is used to indicate to the user where the destination of the current target entity is set to be. In practice, it can be an icon of any style, such as a dot, circle, rectangle, or teardrop.
[0049] Optionally, in response to a selection operation on a target entity in the guidance graph, a selection box is displayed, the selection box enclosing the target entity. When a selection box appears around an entity, it indicates that the entity is determined as the target entity. The endpoint identifier corresponding to the target entity has the same shape and size as the selection box.
[0050] The operation of moving the endpoint identifier in the canvas may be, for example, an operation of editing the position coordinates of the endpoint identifier in the canvas, an operation of dragging the endpoint identifier, and the like.
[0051] It should be noted that, in practice, the size of the canvas can be set larger than the size of the guide map, and the end point identifier can be moved to any position in the canvas. In other words, the end point identifier can be moved within the guide map or outside the guide map. In this case, the target video generated subsequently shows that the target subject moves out of the screen.
[0052] For example, see Figure 2 , if the user clicks on the puppy in the guide image, see Figure 3 , a selection box appears around the puppy, indicating that the current puppy is the target entity, and the dynamic configuration options corresponding to the puppy are displayed above the puppy. The dynamic configuration options include an "end position" option. The "end position" option is an end point configuration option. If the user clicks the "end position" option, the end point identifier is displayed in the canvas. The shape and size of the end point identifier are the same as the selection box around the puppy. The user can drag the end point identifier to any position in the canvas. After the user drags, the final position of the end point identifier is used as the end point position of the puppy's movement, thus obtaining the constraints on the end point position in the motion constraints of the target entity.
[0053] In other embodiments, optionally, the implementation method of this step may also include: in response to a morphological adjustment operation on the endpoint identifier, determining the morphology of the target entity at the moving endpoint position based on the morphology of the endpoint identifier after the morphological adjustment; and adding the morphology of the target entity at the moving endpoint position to the motion constraint conditions of the target entity.
[0054] The morphological adjustment operation on the endpoint identifier may be, for example, an adjustment of one or more aspects of the size, deformation, and rotation of the endpoint identifier. Since the endpoint identifier represents the target entity, the morphological adjustment of the endpoint identifier means adjusting the morphology of the target entity, which may cause changes in the size, posture, action, etc. of the target entity.
[0055] For example, in Figure 3 On the basis of this, the user further adjusts the size of the endpoint identifier in the vertical direction, such as adjusting the endpoint identifier to half of the current size, thereby obtaining the constraint conditions on the shape of the target entity at the moving endpoint position in the motion constraint conditions of the target entity.
[0056] It should be noted that, in practice, the position of the destination identifier of the target entity can be kept consistent with the position of the target entity in the guidance map, and only the shape of the destination identifier of the target entity can be adjusted. In other words, the target entity is set to change in size, posture, movement, etc. in situ. Alternatively, the position of the destination identifier of the target entity can be adjusted to be different from the position of the target entity in the guidance map, but the shape of the destination identifier of the target entity is not adjusted. In other words, the target entity is set to move, but no changes in size, posture, movement, etc. occur. It is also possible to set the position of the destination identifier of the target entity to be different from the position of the target entity in the guidance map, and adjust the shape of the destination identifier of the target entity. In other words, the target entity is set to move, and changes in size, posture, movement, etc. occur.
[0057] In some further embodiments, the motion effect configuration options may further include movement trajectory configuration options; the implementation method of this step may include: displaying a movement trajectory drawing tool in response to a trigger operation on the movement trajectory configuration option corresponding to the target entity; displaying the movement trajectory in response to a movement operation on the movement trajectory drawing tool in the guide map; and adding the movement trajectory to the motion constraint conditions of the target entity.
[0058] The movement track configuration option may refer to an option for assisting the user to configure the movement track of the target object. The triggering operation of the movement track configuration option may be, for example, an operation such as clicking or dragging the movement track configuration option.
[0059] A movement track drawing tool is an identifier that represents a target entity and allows the user to move its position and record its specific movement path during the movement. The movement path recorded by the movement track drawing tool during its movement is the movement track. The movement track drawing tool can be an icon of any style, such as a dot, circle, rectangle, teardrop, pen, etc.
[0060] For example, see Figure 2 , if the user clicks on the puppy in the guide image, see Figure 4 , a selection box appears around the puppy, indicating that the current puppy is the target entity, and the dynamic configuration options corresponding to the puppy are displayed above the puppy. The dynamic configuration options include the "motion trajectory" option. The "motion trajectory" option is the movement trajectory configuration option. If the user clicks the "movement trajectory configuration option" option, a dot appears at the puppy, and the dot is the movement trajectory drawing tool. The user can click and drag the dot to move. During the dragging process, the movement path of the dot is shown as the dotted line, and the dotted line is the movement trajectory. In this way, the constraints on the movement path in the motion constraints of the target entity are obtained.
[0061] It should be noted that, in practice, if the motion effect configuration options include both end point configuration options and moving trajectory configuration options. In practice, the two are combined to configure the motion constraints. For example, the moving end point position of the target entity is first configured with the help of the end point configuration option, and then the position of the target entity in the guide map is used as the moving starting point position, and the moving trajectory of the target entity is configured between the moving starting point position and the moving end point position with the help of the moving trajectory configuration option. Further, in this case, the moving end point position of the target entity configured with the help of the end point configuration option can be set as the first end point position; in the process of "displaying the moving trajectory in response to the moving operation of the moving trajectory drawing tool in the guide map", it also includes: if it is checked that the distance between the end point of the moving trajectory drawn by the current moving trajectory drawing tool and the first end point position is less than the set distance threshold, the moving trajectory drawing tool is adsorbed to the first end point position, so that the end point of the obtained moving trajectory coincides with the first end point position.
[0062] For example, see Figure 5 If the user selects the "End Position" option first, by adjusting the end point identifier, the moving end point position (i.e., the first end point position) of the target entity is determined to be point B. Then the user selects the "Motion Track" option and draws the moving track using the moving track drawing tool. During the drawing process, if at a certain moment, the end point of the drawn moving track is point A, and the distance between point A and point B is less than the set distance threshold, point A and point B are automatically connected, so that after the connection, the end point of the moving track becomes point B, so that the visual effect of the end point of the moving track being adsorbed to the first end point position is presented. This can reduce the difficulty of drawing the moving track and improve the efficiency of drawing the moving track.
[0063] If the moving track of the target entity is configured by means of the moving track configuration option, the moving track configured by means of the moving track configuration option is determined to be the first moving track; then the end point configuration option is triggered, and optionally, the end point identifier corresponding to the target entity is displayed; the display position of the end point identifier is consistent with the position of the end point of the first moving track. In this case, the position of the end point identifier may be further moved, or the shape of the end point identifier may be adjusted.
[0064] S140, generating a target video based on the motion constraint conditions of the target entity and the guide map; in the target video, the target entity moves in a motion manner indicated by the motion constraint conditions.
[0065] The essence of this step is that, in the process of generating the target video, in addition to using the guide map, the motion constraint condition obtained by S130 is also used. In the target image, the motion characteristics of the target entity meet the requirements of the motion constraint condition.
[0066] For example, see Figure 4 If the target entity is a puppy, by executing S130, the motion constraints obtained include the puppy's moving end point position being point B, and the moving path being Figure 4 In the final target image, the puppy moves along the Figure 4 The dashed line with an arrow in the middle moves to point B.
[0067] It should be noted that in practical applications, the description of the motion of an object can start from multiple aspects, such as speed, acceleration, starting point, end point, trajectory, whether it is flipped, whether it is deformed, etc. The motion constraint condition can limit some or all of these aspects. If the motion constraint condition only limits certain aspects, then for the unrestricted parts, the video generation model can be allowed to play freely, or the default motion mode can be used to handle these unrestricted aspects.
[0068] It should be noted that, in practice, a guide statement may also be obtained, and this step may include: generating a target video based on the motion constraint conditions of the target entity, the guide map, and the guide statement.
[0069] The above technical solution is to display the motion configuration options corresponding to the target entity in response to the selection operation of the target entity in the guide map; determine the motion constraints of the target entity in response to the configuration operation of the motion configuration options corresponding to the target entity; generate the target video based on the motion constraints of the target entity and the guide map; in the target video, the target entity moves in the motion mode indicated by the motion constraints. Its essence is to provide a method to assist users to accurately limit the movement of entities in the guide map. This method can assist the video generation model to more clearly "understand" the user's creative intent, thereby making the generated target video more in line with the user's needs.
[0070] Based on the above technical solution, optionally, the method may also include: determining the category name of the target entity in response to a selection operation on the target entity in the guide map; S140 may include: generating a target video based on the motion constraint conditions of the target entity, the category name of the target entity and the guide map.
[0071] The category name of the target entity may be, for example, the name of the category to which the target entity belongs.
[0072] Optionally, in response to the selection operation of the target entity in the guidance graph, candidate names that can be used as the target entity are displayed; in response to the selection operation of the target name in the candidate names, the target name is used as the category name of the target entity. The candidate name that can be used as the target entity can be obtained after the entity recognition algorithm recognizes the target entity. For example, see Figure 6When the user clicks on the puppy in the guide image, the puppy is selected as the target entity and three candidate names are displayed, namely "puppy", "teddy" and "kitten". The user can select one of them as the category name of the target entity. If the user selects "puppy" as the category name of the target entity, the movement of the puppy in the generated target video will reflect the natural movement characteristics of the puppy. If the user selects "kitten", even if the target entity is a puppy, its movement will be given movement characteristics similar to those of a kitten.
[0073] In some scenarios, a category name input box for the target entity may also be displayed to allow the user to input the category name of the target entity by themselves.
[0074] By setting a step of determining the category name of the target entity and using the determined category name as the input of the video generation model, the purpose is to allow the target entity to show natural movements that are consistent with its category characteristics in the generated target video.
[0075] Based on the above technical solution, optionally, the motion effect configuration page may also include an entity display area; the method may also include: displaying the label of the entity for which the motion constraint conditions have been configured in the entity display area; in response to a selection operation of the first entity in the entity display area, displaying the motion constraint conditions corresponding to the first entity in the canvas.
[0076] The entity display area may be used, for example, to display labels of all entities that are currently configured with motion constraints. The labels of entities may be, for example, information that distinguishes one entity from other entities, such as the name of the entity, the representative color of the entity, the number of the entity, the characteristics of the configured motion constraints, etc. The first entity may be, for example, one or more entities among the entities displayed in the entity display area.
[0077] There are many specific implementation methods for "displaying the motion constraint conditions corresponding to the first entity", and this application does not limit this. For example, the motion constraint conditions corresponding to the first entity can be displayed in the form of text description, and / or, the motion constraint conditions corresponding to the first entity can be displayed in the form of images.
[0078] Furthermore, the entity display area may include a single entity switch option. If the single entity switch option is in an on state, in response to a selection operation of a first entity in the entity display area, the motion constraint conditions corresponding to the first entity are displayed. If the single entity switch option is in an off state, the motion constraint conditions corresponding to all entities are displayed.
[0079] For example, see Figure 7, assuming that the user has configured motion constraints for the puppy, man, and Frisbee in the guide map, the three entity labels "puppy", "man", and "frisbee" are displayed in the entity display area. Among them, the motion constraint of the puppy is only configured with the help of the end point identifier. The motion constraint of the man is configured with the help of the end point identifier and the movement trajectory drawing tool. Figure 7 In the , the "Show only current entity" option is a single entity switch option. Figure 7 The single entity switch option is turned on, then click the "Puppy" option in the entity display area, see Figure 7 , in the canvas, the configured moving end position of the puppy is displayed with the help of the end point identifier. Then, if the user clicks the "man" option in the entity display area, in the canvas, the configured moving end position of the man is displayed with the help of the end point identifier, and the configured moving trajectory of the man is displayed with the help of the dotted arrow. If the user turns the single entity switch option off, the motion constraints of "puppy", "man" and "frisbee" are displayed at the same time.
[0080] In practice, when configuring the motion constraints of an entity, a representative color corresponding to it can be determined for each entity, and different entities have different representative colors. For example, the representative color of a puppy is pink. If the puppy is used as the target entity, when configuring its motion constraints, the color of the puppy can be adjusted to pink (which can be achieved by superimposing a mask of a specific color), and the color of one or more of the end point identifier, the moving trajectory drawing tool, and the moving trajectory used is pink. In the entity display area, a pink color block or pink text is used to indicate that the representative color of the puppy is pink.
[0081] Since different entities have different representative colors, users can intuitively distinguish by color which entity's motion constraint configuration is reflected by each current endpoint identifier, movement trajectory drawing tool or movement trajectory.
[0082] On the basis of the above technical solution, optionally, the motion effect configuration page may also include a prompt information display area; the method may also include: during the configuration of the motion constraint conditions of the target entity, displaying the instructions for use of the currently used option in the prompt information display area.
[0083] The prompt information display area can be used to display instructions for some or all options in the animation configuration page. The content displayed in this area will be dynamically updated as the options currently used by the user change.
[0084] In practice, the user may not understand how to use each option, so it is necessary to prompt the user with instructions for using the options. In practice, the instructions for using each option can be directly displayed near the option, or can be uniformly displayed in the prompt information display area.
[0085] If the instructions for using the options are uniformly displayed in the prompt information display area, on the one hand, it is convenient for users to quickly find the instructions for using the currently used options; on the other hand, the instructions will not block the canvas and will not affect the user's configuration of the motion constraint conditions.
[0086] For example, see Figure 2 If you need to select a target entity, two target entity selection options are provided on the left side of the prompt information display area, namely "click" and "box". If the user selects the "click" option, the user can select the target entity by clicking the area occupied by the entity in the guide map. If the user selects the "box" option, the user can use the circle selection tool to circle the target entity in the guide map. Figure 2 In the figure, the "Click" option is in selected state. The "Click" option is the option currently being used. The prompt message "Please click the subject of the specified movement" is displayed in the prompt message display area. The prompt message is the instruction for using the "Click" option.
[0087] Similarly, see Figure 3 The "End Position" option is selected. The "End Position" option is the option currently being used. The prompt information display area displays the prompt information "Please move the end point identifier and specify the end point of the movement". The prompt information is the instruction for using the "End Position" option.
[0088] On the basis of the above technical solution, optionally, S110 can be replaced by: displaying a video generation configuration page; the video generation configuration page includes a guide map display area, a motion constraint configuration entry, and a video generation option; when a guide map is displayed in the guide map display area, in response to a trigger operation on the motion constraint configuration entry, displaying a motion effect configuration page; S140 can be replaced by: in response to a trigger operation on the video generation option in the video generation configuration page, generating a target video based on the motion constraints of the target entity and the guide map.
[0089] The video generation configuration page may refer to a page for configuring the limited information for video generation, for example. Optionally, the limited information may include one or more of the following: a guide image, a guide word, a video generation model, and the duration, size, and camera movement of the generated video.
[0090] The guide image display area may be, for example, an area in the video generation configuration page for displaying the guide image when the guide image has been uploaded. The motion constraint configuration entry may be, for example, an entry for bringing the user to the motion effect configuration page. The video generation option may be, for example, an option for triggering the video generation model to generate a target video.
[0091] For example, see Figure 8 The video generation configuration page includes a guide map display area, a motion constraint configuration entry, and video generation options. If a guide map is not uploaded, a prompt "+ Add guide map" is displayed in the guide map display area to guide the user to upload a guide map. After the guide map is uploaded, the guide map is displayed in the guide map display area. Figure 8 The "Animation Drawing Board" is the entry for configuring motion constraints, and the "Generate" option is the video generation option.
[0092] When a guide image is displayed in the guide image display area, if the user clicks on the "animation drawing board", the animation configuration page is displayed. The guide image displayed in the canvas of the animation configuration page is different from the guide image displayed in the guide image display area of the video generation configuration page. Figure 1 To.
[0093] After all the limiting information required for video generation is configured, if the user clicks the "Generate" option in the video generation configuration page, the target video is generated based on the motion constraints of the target entity and the guide map.
[0094] It should be noted that in some scenarios, a variety of functions can be provided in the video generation configuration page to set the guidance of video generation. Some of these functions may conflict with the entity motion constraint configuration function provided in this application. In this case, the user should be allowed to choose one of the conflicting functions to use. For example, if the first and last frame image limitation function and the camera movement limitation function conflict with the entity motion constraint configuration function, then when the user selects the first and last frame image limitation function or the camera movement limitation function, the use of the entity motion constraint configuration function is restricted; conversely, if the user selects the entity motion constraint configuration function, the use of the first and last frame image limitation function or the camera movement limitation function is restricted.
[0095] The first and last frame image limitation function may, for example, mean that the user can customize the image content at the beginning (first frame) and the end (last frame) of the target video. The camera movement limitation function may, for example, mean that the user can specify how the camera moves or changes in the target video. Camera movement includes but is not limited to different types of camera movements such as push, pull, shake, shift, and lift.
[0096] By setting the video generation configuration page to include the motion constraint configuration entry, the essence is to set the entry of the motion effect configuration page on the video generation configuration page. This can make the functions of the video generation configuration page more centralized, making it easier for users to configure the limiting conditions for generating videos.
[0097] Based on the above technical solution, optionally, the method may also include: displaying a video display page; the video display page includes a target video and prompt information corresponding to the target video; the prompt information is used to prompt the guide map used in the target video generation process and the motion constraints that have been configured.
[0098] The video display page may be, for example, a page for displaying a generated target video.
[0099] For example, see Figure 8 The video display page shows the results of two video generation. Video 1 is based on the guidance Figure 1 , introductory words 1 and for the guidance Figure 1 The prompt information corresponding to video 1 indicates the guidance used. Figure 1 , boot 1, and boot Figure 1 The motion constraints of the entities in the video are based on the guidance Figure 2 and the guide 2, which does not use motion constraints. The prompt information corresponding to video 2 indicates the guide used. Figure 2 and guide 2, without indicating that motion constraints have been configured. This setting makes it easy for users to quickly understand what information is used in the generation process of each video.
[0100] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0101] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0102] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0103] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0104] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.
[0105] Fig. 9 FIG. 1 is a schematic diagram of the structure of a video generating device in an embodiment of the present disclosure. The video generating device provided in an embodiment of the present disclosure may be configured in a client. Fig. 9 , the video generating device specifically comprises:
[0106] The configuration page display module 310 is used to display the dynamic effect configuration page; the dynamic effect configuration page includes a canvas, and a guide map is displayed in the canvas; the guide map includes an entity;
[0107] An entity selection module 320 is used to display the motion effect configuration options corresponding to the target entity in the guidance diagram in response to the selection operation of the target entity in the guidance diagram;
[0108] A configuration module 330, configured to determine a motion constraint condition of the target entity in response to a configuration operation on a motion effect configuration option corresponding to the target entity;
[0109] The generating module 340 is used to generate a target video based on the motion constraint conditions of the target entity and the guiding map; in the target video, the target entity moves in a motion manner indicated by the motion constraint conditions.
[0110] Furthermore, the device also includes a name determination module, which is used to:
[0111] In response to a selection operation on a target entity in the guidance graph, determining a category name of the target entity;
[0112] The generating module 340 is used to:
[0113] A target video is generated based on the motion constraint condition of the target entity, the category name of the target entity and the guide map.
[0114] Furthermore, the motion effect configuration option is used to configure one or more of the moving end position, the moving starting position, the moving trajectory, and the shape of the target entity during the moving process.
[0115] Furthermore, if the motion effect configuration options include an endpoint configuration option; the configuration module 330 is used to:
[0116] In response to a triggering operation on the endpoint configuration option, displaying an endpoint identifier corresponding to the target entity;
[0117] In response to a moving operation of the end point identifier in the canvas, taking the position indicated by the end point identifier after the movement as the moving end point position of the target entity;
[0118] The moving end point position of the target entity is added to the motion constraint condition of the target entity.
[0119] Furthermore, the configuration module 330 is also used to:
[0120] In response to a shape adjustment operation on the end point identifier, determining a shape of the target entity at the moving end point position based on the shape of the end point identifier after the shape adjustment;
[0121] The shape of the target entity at the moving end position is added to the motion constraint condition of the target entity.
[0122] Furthermore, the motion effect configuration options include movement track configuration options; the configuration module 330 is used to:
[0123] In response to a triggering operation on a movement trajectory configuration option corresponding to the target entity, displaying a movement trajectory drawing tool;
[0124] In response to a movement operation of the movement trajectory drawing tool in the guide map, displaying a movement trajectory;
[0125] The movement trajectory is added to the motion constraint condition of the target entity.
[0126] Furthermore, the animation configuration page also includes a physical display area; the configuration page display module is also used to:
[0127] Displaying labels of entities for which motion constraints have been configured in the entity display area;
[0128] In response to a selection operation on a first entity in the entity display area, a motion constraint condition corresponding to the first entity is displayed in the canvas.
[0129] Furthermore, the animation configuration page also includes a prompt information display area; a configuration page display module, which is used to:
[0130] During the configuration of the motion constraint conditions of the target entity, the instructions for use of the currently used option are displayed in the prompt information display area.
[0131] Furthermore, the configuration page display module is used to:
[0132] Displaying a video generation configuration page; the video generation configuration page includes a guide map display area, a motion constraint configuration entry, and video generation options;
[0133] When the guide map is displayed in the guide map display area, in response to a trigger operation on the motion constraint condition configuration entry, a motion effect configuration page is displayed;
[0134] The generating a target video based on the motion constraint condition of the target entity and the guide graph includes:
[0135] In response to a triggering operation on a video generation option in the video generation configuration page, a target video is generated based on the motion constraints of the target entity and the guide map.
[0136] Furthermore, the device also includes a result display module, which is used to:
[0137] Display a video display page; the video display page includes the target video and prompt information corresponding to the target video; the prompt information is used to prompt the guide map used in the target video generation process and the motion constraints that have been configured.
[0138] The video generating device provided in the embodiment of the present disclosure can execute the steps executed by the client or server in the video generating method provided in the embodiment of the method of the present disclosure, and has the execution steps and beneficial effects, which will not be repeated here.
[0139] Fig.10 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Fig.10, which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0140] like Fig.10 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003 to implement the video generation method of the embodiment described in the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0141] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although Fig.10 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0142] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the video generation method as described above. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0143] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include an information signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated information signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0144] In some embodiments, the client and the server may communicate using any known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any known or future developed network.
[0145] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0146] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:
[0147] Displaying a dynamic effect configuration page; the dynamic effect configuration page includes a canvas, and a guide map is displayed in the canvas; the guide map includes an entity;
[0148] In response to a selection operation on a target entity in the guide graph, displaying a motion effect configuration option corresponding to the target entity;
[0149] In response to a configuration operation on a motion effect configuration option corresponding to the target entity, determining a motion constraint condition of the target entity;
[0150] A target video is generated based on the motion constraint conditions of the target entity and the guide map; in the target video, the target entity moves in a motion manner indicated by the motion constraint conditions.
[0151] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.
[0152] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0153] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0154] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.
[0155] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0156] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0157] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:
[0158] one or more processors;
[0159] A memory for storing one or more programs;
[0160] When the one or more programs are executed by the one or more processors, the one or more processors implement any video generating method provided in the present disclosure.
[0161] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the video generating method as described in any one of the above-described methods is implemented.
[0162] The embodiments of the present disclosure further provide a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the video generation method described above is implemented.
[0163] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0164] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A video generation method, characterized in that: include: Displaying a dynamic effect configuration page; the dynamic effect configuration page includes a canvas, and a guide map is displayed in the canvas; the guide map includes an entity; In response to a selection operation on a target entity in the guide graph, displaying a motion effect configuration option corresponding to the target entity; In response to a configuration operation on a motion effect configuration option corresponding to the target entity, determining a motion constraint condition of the target entity; Generate a target video based on the motion constraint conditions of the target entity and the guide map; In the target video, the target entity moves in a motion manner indicated by the motion constraint condition.
2. The method according to claim 1, characterized in that Also includes: In response to a selection operation on a target entity in the guidance graph, determining a category name of the target entity; The generating a target video based on the motion constraint condition of the target entity and the guide graph includes: A target video is generated based on the motion constraint condition of the target entity, the category name of the target entity and the guide map.
3. The method according to claim 1, characterized in that The motion effect configuration options are used to configure one or more of the moving end position, moving starting position, moving trajectory, and the shape of the target entity during the moving process.
4. The method according to claim 3, characterized in that If the animation configuration options include an endpoint configuration option; The step of determining the motion constraint condition of the target entity in response to the configuration operation of the motion effect configuration option corresponding to the target entity includes: In response to a triggering operation on the endpoint configuration option, displaying an endpoint identifier corresponding to the target entity; In response to a moving operation of the end point identifier in the canvas, taking the position indicated by the end point identifier after the movement as the moving end point position of the target entity; The moving end point position of the target entity is added to the motion constraint condition of the target entity.
5. The method according to claim 4, characterized in that Also includes: In response to a shape adjustment operation on the end point identifier, determining a shape of the target entity at the moving end point position based on the shape of the end point identifier after the shape adjustment; The shape of the target entity at the moving end position is added to the motion constraint condition of the target entity.
6. The method according to claim 3, characterized in that The motion effect configuration options include movement trajectory configuration options; and the determining of the motion constraint conditions of the target entity in response to the configuration operation of the motion effect configuration options corresponding to the target entity includes: In response to a triggering operation on a movement trajectory configuration option corresponding to the target entity, displaying a movement trajectory drawing tool; In response to a movement operation of the movement trajectory drawing tool in the guide map, displaying a movement trajectory; The movement trajectory is added to the motion constraint condition of the target entity.
7. The method according to claim 1, characterized in that The animation configuration page also includes a physical display area; the method also includes: Displaying labels of entities for which motion constraints have been configured in the entity display area; In response to a selection operation on a first entity in the entity display area, a motion constraint condition corresponding to the first entity is displayed in the canvas.
8. The method according to claim 1, characterized in that The animation configuration page also includes a prompt information display area; the method also includes: During the configuration of the motion constraint conditions of the target entity, the instructions for use of the currently used option are displayed in the prompt information display area.
9. The method according to claim 1, characterized in that: The display animation configuration page includes: Displaying a video generation configuration page; the video generation configuration page includes a guide map display area, a motion constraint configuration entry, and video generation options; When the guide map is displayed in the guide map display area, in response to a trigger operation on the motion constraint condition configuration entry, a motion effect configuration page is displayed; The generating a target video based on the motion constraint condition of the target entity and the guide graph includes: In response to a triggering operation on a video generation option in the video generation configuration page, a target video is generated based on the motion constraints of the target entity and the guide map.
10. The method according to claim 1, characterized in that Also includes: Displaying a video display page; the video display page includes the target video and prompt information corresponding to the target video; The prompt information is used to prompt the guide map used in the target video generation process and the motion constraint conditions that have been configured.
11. A video generating device, characterized in that: include: A configuration page display module, used to display a dynamic effect configuration page; the dynamic effect configuration page includes a canvas, and a guide map is displayed in the canvas; the guide map includes an entity; An entity selection module, configured to display, in response to a selection operation on a target entity in the guide graph, a motion effect configuration option corresponding to the target entity; A configuration module, configured to determine a motion constraint condition of the target entity in response to a configuration operation on a motion effect configuration option corresponding to the target entity; A generating module, used for generating a target video based on the motion constraint condition of the target entity and the guiding graph; In the target video, the target entity moves in a motion manner indicated by the motion constraint condition.
12. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 10.
13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.
Citation Information
Patent Citations
Information display method and device, computer equipment and storage medium
CN114254153A
Method, device and equipment for content shooting and storage medium
CN114979495A
Marking method and device based on live broadcast
CN115037985A
Method, apparatus, device and computer program product for generating video
CN117956246A
Method and apparatus for processing special effect video, and electronic device and storage medium
WO2023140786A2
Cited By
Video generation method and apparatus, electronic device, and storage medium
WO2026032301A1
