Method and device for generating video content, equipment and storage medium

Abstract

The embodiment of the invention relates to a method and device for generating video content, equipment and a storage medium. The method provided by the invention comprises the following steps: presenting a generation interface, wherein the generation interface comprises at least one mirror operation control; receiving a first interaction operation on a first mirror operation control in the at least one mirror operation control; on the basis of at least one operation attribute of the first interaction operation, mirror operation control information is determined; and presenting a first video clip in the generation interface, wherein the first video clip is generated based on the reference image and the mirror operation control information. In this way, the video content generation efficiency can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for generating video content. Background Art

[0002] As network technology becomes increasingly mature, more and more users use network platforms to assist in generating video content. For example, network platforms can generate video content based on images provided by users. However, when generating video content based on existing network platforms, it is not possible to support users to set camera information associated with video content based on interactive operations, resulting in low efficiency in generating video content and the quality of generated video content failing to meet user needs. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for generating video content is provided. The method includes: presenting a generation interface, the generation interface including at least one camera movement control; receiving a first interactive operation on a first camera movement control of the at least one camera movement control; determining camera movement control information based on at least one operation attribute of the first interactive operation; and presenting a first video clip in the generation interface, the first video clip being generated based on a reference image and the camera movement control information.

[0004] In a second aspect of the present disclosure, a device for generating video content is provided. The device includes: an interface presentation module configured to present a generation interface, the generation interface including at least one camera movement control; a receiving module configured to receive a first interactive operation on a first camera movement control of the at least one camera movement control; a determination module configured to determine camera movement control information based on at least one operation attribute of the first interactive operation; and a video presentation module configured to present a first video clip in the generation interface, the first video clip being generated based on a reference image and the camera movement control information.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments according to the present disclosure may be implemented;

[0010] FIG. 2A to FIG. 2F shows an example interface according to some embodiments of the present disclosure;

[0011] FIG. 3A to FIG. 3F shows an example interface according to some embodiments of the present disclosure;

[0012] Figure 4 A flowchart illustrating an example process of generating video content according to some embodiments of the present disclosure;

[0013] Figure 5 A schematic structural block diagram showing an example apparatus for generating video content according to some embodiments of the present disclosure; and

[0014] Figure 6 A block diagram of an electronic device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0018] The embodiments of the present disclosure may involve user data, data acquisition and / or use, etc. These aspects are subject to the corresponding laws, regulations and relevant provisions. In the embodiments of the present disclosure, all data collection, acquisition, processing, processing, forwarding, use, etc. are carried out on the premise that the user knows and confirms. Accordingly, when implementing each embodiment of the present disclosure, the type, scope of use, usage scenario, etc. of the data or information that may be involved should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with the relevant laws and regulations. The specific notification and / or authorization method can vary according to the actual situation and application scenario, and the scope of the present disclosure is not limited in this respect.

[0019] In this specification and the embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or it is necessary to perform a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.

[0020] As mentioned above, with the increasing maturity of network technology, more and more users use network platforms to assist in generating video content. For example, network platforms can generate video content based on images provided by users. However, when generating video content based on existing network platforms, it is impossible to support users to set camera information associated with video content based on interactive operations, resulting in low efficiency in generating video content and the quality of generated video content failing to meet user needs.

[0021] The embodiment of the present disclosure proposes a scheme for generating video content. According to the scheme, a generation interface can be presented, the generation interface includes at least one camera movement control; a first interactive operation on a first camera movement control of the at least one camera movement control is received; camera movement control information is determined based on at least one operation attribute of the first interactive operation; and a first video clip is presented in the generation interface, the first video clip being generated based on a reference image and the camera movement control information.

[0022] In this way, the embodiment of the present disclosure can provide at least one camera movement control in the generation interface, and can receive the user's interactive operation on the camera movement control. Moreover, the embodiment of the present disclosure can determine the camera movement control information based on the operation attributes of the interactive operation, so as to generate a video clip based on the reference image and the camera movement control information. In this way, the embodiment of the present disclosure can support the user to set the camera movement control information based on the interactive operation, thereby improving the efficiency and quality of generating video content.

[0023] Various example implementations of the solution are described in detail below in conjunction with the accompanying drawings.

[0024] Example Environment

[0025] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, example environment 100 may include electronic device 110 .

[0026] In this example environment 100, the electronic device 110 may run an application 120 that supports interface interaction. The application 120 may be any appropriate type of application for interface interaction, examples of which may include, but are not limited to, a call application, a conference application, a social application, or other appropriate applications. The user 140 may interact with the application 120 via the electronic device 110 and / or its attached device.

[0027] exist Figure 1 In the environment 100, if the application 120 is in an active state, the electronic device 110 can present an interface 150 for supporting interface interaction through the application 120.

[0028] In some embodiments, the electronic device 110 communicates with the server 130 to provide services for the application 120. The electronic device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a handheld computer, a portable game terminal, a VR / AR device, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio receiver, an e-book device, a game device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the electronic device 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.).

[0029] The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server 130 may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, etc. The server 130 may provide background services for the application 120 that supports virtual scenes in the electronic device 110.

[0030] A communication connection may be established between the server 130 and the electronic device 110. The communication connection may be established in a wired manner or a wireless manner. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and the embodiments of the present disclosure are not limited in this respect. In the embodiments of the present disclosure, the server 130 and the electronic device 110 may implement signaling interaction through the communication connection between the two.

[0031] It should be understood that the structure and function of the various elements in the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0032] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0033] Example Interaction

[0034] FIG. 2A to FIG. 2F 200A to 200F according to some embodiments of the present disclosure. The interfaces 200A to 200F may be composed of, for example, Figure 1 The electronic device 110 is provided as shown.

[0035] In some embodiments, Figure 2A As shown, the electronic device 110 may present the generation interface 200A. As an example, the electronic device 110 may receive the user's operation in the generation interface 200A, and generate video content for the user based on the user's operation.

[0036] As an example, the electronic device 110 may provide an add entry 205 in the generation interface 200A. Further, the electronic device 110 may present a plurality of candidate images in response to a trigger (eg, a click operation 206) on the add entry 205.

[0037] In some embodiments, Figure 2B As shown, the electronic device 110 can present multiple candidate images. The multiple candidate images can include, for example, candidate image 207-1, candidate image 207-2, and candidate image 207-3, etc. As an example, the multiple candidate images can include images provided by the application 120 and / or images contained in the user's local media library (e.g., photo album). As an example, the image can include static image content and dynamic image content. Further, the electronic device 110 can respond to receiving the user's selection of a target candidate image (e.g., candidate image 207-2) among the multiple candidate images and use the target candidate image as a reference image. As an example, the electronic device 110 can call the camera component to capture an image as a reference image based on the user's operation.

[0038] In some embodiments, Figure 2C As shown, the electronic device 110 may present the reference image in the generation interface 200C. As an example, the electronic device 110 may present the reference image in a preview area (eg, area 208).

[0039] As an example, the electronic device 110 may present at least one camera movement control in the generated interface.

[0040] For example, the at least one camera movement control may include a camera movement control associated with multiple camera movement parameters and / or a camera movement control associated with a single camera movement parameter.

[0041] An exemplary description is given by taking a camera movement control associated with multiple camera movement parameters as an example.

[0042] like Figure 2C As shown, the at least one camera movement control may include, for example, a camera movement control 210. As an example, the camera movement control 210 is a camera movement control associated with a plurality of camera movement parameters.

[0043] As an example, the camera movement control 210 may include a virtual joystick control 212. The virtual joystick control 212 may be implemented as a floating ball style, for example. The virtual joystick control 212 may be associated with multiple moving directions. For example, the virtual joystick operation 212 may move based on a preset operation of the user (e.g., a sliding operation). As an example, multiple moving directions may correspond to multiple preset types of camera movement parameters. For example, the electronic device 110 may determine the camera movement parameter corresponding to the moving direction based on the moving direction of the virtual joystick control 212. Further, the electronic device 110 may determine the value (or numerical value) of the camera movement parameter corresponding to the moving direction based on the moving distance of the virtual joystick control 212 in the moving direction.

[0044] In some embodiments, the camera movement parameters may include one or more parameters associated with the virtual camera. The virtual camera may provide screen contents at different angles by simulating the movement of a real camera. For example, the virtual camera may correspond to different positions and postures in space. In addition, the virtual camera may also correspond to different focal lengths to determine the corresponding screen range.

[0045] For example, the camera movement parameters associated with the camera movement control 210 may include position parameters associated with the virtual camera. As an example, the position parameters may indicate the positional relationship between the virtual camera and the reference image. Such position parameters may include, for example, the left shift distance, the right shift distance, the upward shift distance, the downward shift distance, and the plane distance between the plane where the virtual camera is located and the plane where the reference image is located, etc., based on the default position (the position of the virtual camera opposite the geometric center of the reference image). For example, the preset types associated with the position parameters may include, for example, left shift 211-1 (for example, moving the virtual camera to the left), right shift 211-3 (for example, moving the virtual camera to the right), push lens 211-8 (for example, moving the virtual camera forward), pull lens 211-4 (for example, moving the virtual camera backward), fixed camera position 211-2 (for example, fixing the position of the current virtual camera or adjusting the position of the virtual camera to the default position).

[0046] Additionally, the camera movement control 210 may be associated with an orientation parameter associated with the virtual camera. As an example, the orientation parameter may indicate an angle between the lens direction of the virtual camera and the plane where the reference image is located. Such an angle may include, for example, a pitch angle. For example, the preset types associated with the orientation parameter may include, for example: up 211-1 (e.g., raising the elevation angle of the virtual camera), down 211-5 (e.g., raising the depression angle of the virtual camera).

[0047] Additionally or alternatively, the camera control 210 may be associated with a focal length parameter associated with the virtual camera. As an example, the focal length parameter may indicate the focal length of the lens of the virtual camera. The focal length of the lens may indicate the distance from the optical center of the lens to the imaging plane (e.g., the virtual camera sensor). For example, the preset types associated with the focal length parameter may include, for example: extreme close-up 211-6 (e.g., increasing the focal length of the lens of the virtual camera).

[0048] In some embodiments, the electronic device 110 may receive a first interactive operation on a first camera movement control (eg, the camera movement control 210 ) among the at least one camera movement control.

[0049] As an example, the first interactive operation includes a sliding operation associated with the first camera movement control. For example, the sliding operation can be implemented as a sliding operation relying on a mouse (or other external device connected to the electronic device 110) or a touch screen. For example, the sliding operation can also be implemented as a suspended gesture operation that does not contact the screen. For example, the sliding operation can also be determined based on the eye movement information of the user followed by the sensor.

[0050] In some embodiments, the electronic device 110 may determine the mirror control information based on at least one operation attribute of the first interactive operation.

[0051] Additionally, at least one operation attribute of the sliding operation may include a sliding direction of the sliding operation. Further, the electronic device 110 may determine a parameter type of at least one lens movement parameter indicated by the lens movement control information based on the sliding direction of the sliding operation.

[0052] As an example, Figure 2D As shown, the electronic device 110 can determine the parameter type of at least one camera movement parameter indicated by the sliding direction of the sliding operation (eg, the zoom lens 211-4 associated with the position parameter) in response to receiving a sliding operation from the user (eg, operation 213).

[0053] Additionally or alternatively, at least one operation attribute of the sliding operation may include a sliding distance of the sliding operation. Further, the electronic device 110 may determine the parameter value of at least one camera movement parameter indicated by the camera movement information based on the sliding distance of the sliding operation. For example, the electronic device 110 may move the virtual rocker control 212 based on operation 213. The electronic device 110 may determine the parameter value of the pull lens 211-4 based on the moving distance of the virtual rocker control 212. For example, the moving distance may be proportional to the parameter value of the pull lens 211-4. The electronic device 110 may determine the parameter value of the pull lens 211-4 based on a preset proportional relationship and moving distance. Exemplarily, the moving distance of the virtual rocker control 212 is 1 cm, and the distance of the pull lens 211-4 (e.g., the distance to move the virtual camera backward) is 1 meter.

[0054] Additionally, at least one operation attribute of the sliding operation may include the duration of the sliding operation. Further, the electronic device 110 may determine the duration of the segment associated with the camera movement control information based on the duration of the sliding operation. For example, the electronic device 110 may determine the duration of the segment associated with the pull shot 211-4 based on the duration of operation 213. For example, the duration may be proportional to the duration of the segment. The electronic device 110 may determine the duration of the segment associated with the pull shot 211-4 based on a preset proportional relationship and the duration. Exemplarily, the duration of operation 213 is 5 seconds, and the duration of the segment associated with the pull shot 211-4 is also 5 seconds.

[0055] In some embodiments, Figure 2E As shown, the electronic device 110 can adjust the virtual joystick control 212 to a default position in response to the end of the sliding operation.

[0056] Additionally, the electronic device 110 may present a description text (e.g., text 214) in the generation interface 200E. The description text may indicate the camera movement control information. As an example, the description text is a prompt word generated based on one of the four operation attributes of the first interactive operation. For example, the description text is generated based on at least one operation attribute of operation 213. Further, the prompt word corresponding to the description text may be provided to the video generation model to generate the first video clip. As an example, the video generation model may be implemented as a generative model that generates video content based on text (e.g., prompt words) and / or reference images. Such a generative model may be implemented, for example, based on a reference image and a prompt word (e.g., pull the lens 5 meters for 3 seconds), to generate a video clip based on the reference image, with the viewing angle adjusted to move back 5 meters for 3 seconds. This is only an exemplary description, and the present disclosure is not intended to limit the specific content and training process of the generation model.

[0057] Alternatively, one or more camera movement parameters corresponding to the camera movement control information may be encoded as a vector, and provided as a control signal of a video generation model, so that the video generation model generates a corresponding video segment based on the control signal.

[0058] Additionally, the electronic device 110 can update the camera control information based on the editing operation for the description text. As an example, the electronic device 110 can present an editing window for editing the description text in response to a click operation on the text 214. Further, the electronic device 110 can obtain the content (e.g., text content) input by the user based on the editing window. The electronic device 110 can update the camera control information based on the content input by the user. That is to say, in addition to supporting the user to determine the prompt words for generating a video clip through an interactive operation (e.g., a sliding operation), the present disclosure can also support the user to edit the prompt words, thereby improving the efficiency of generating video content and meeting the needs of users to generate video content.

[0059] Additionally, the electronic device 110 may generate a corresponding video clip based on the prompt word in response to receiving a prompt word input by the user before receiving an interactive operation (eg, a sliding operation) from the user.

[0060] In some embodiments, the electronic device 110 may present the first video clip in the generation interface. The first video clip is generated based on the reference image and the camera control information. For example, the reference image may be a static image. For example, the reference image may be the ending image frame (or the last frame) of a dynamic image (e.g., the reference video clip). In this way, the first video clip may maintain content coherence relative to the reference image.

[0061] For example, Figure 2F As shown, the electronic device 110 may present the first video clip in the generated interface 200F. For example, the electronic device 110 may present the first video clip in a preset area (eg, area 208).

[0062] Additionally, the electronic device 110 may also present a media track 215 in the generated interface 200F. The electronic device 110 may indicate the playback progress of the video content in the area 208 based on the media track 215. Further, the electronic device 110 may adjust the playback progress of the video content in the area 208 based on a preset operation (e.g., a sliding operation or a clicking operation) of the user on the media track 215.

[0063] Alternatively, in response to the first video segment being generated, the electronic device 110 may present prompt information 216 (eg, “the last frame has been acquired, you may continue the operation”) in the generation interface 200E to indicate that the first video segment has been generated.

[0064] In some embodiments, at least one of the aforementioned camera movement controls is a first group of camera movement controls. The electronic device 110 may receive a mode switching operation associated with generation. For example, the electronic device 110 may provide multiple mode options in a generation interface. The electronic device 110 may receive a user's selection of a target mode option from the multiple mode options. As an example, the multiple mode options may correspond to different camera movement controls to meet the needs of different users. Further, the electronic device 110 may present a second group of camera movement controls in the generation interface in response to receiving a mode switching operation associated with generation. The first group of camera movement controls is different from the second group of camera movement controls.

[0065] The following is based on FIG. 3A to FIG. 3F The second set of camera movement controls is introduced as an example.

[0066] FIG. 3A to FIG. 3F 300A to 300F according to some embodiments of the present disclosure. Figure 1 The electronic device 110 is provided as shown.

[0067] In some embodiments, the electronic device 110 may present a generation interface 300A. The electronic device 110 may present a plurality of camera controls in the generation interface 300A. The plurality of camera controls may include, for example, a camera control 301-1, a camera control 301-2, and a camera control 301-3.

[0068] As an example, the camera movement control 301-1 can be associated with the horizontal rotation-vertical tilt (e.g., Pan-Tilt) control of the virtual camera. For example, in the Pan-Tilt technology, Pan can represent panning (i.e., angle adjustment in the horizontal direction, for example, rotating around an axis perpendicular to the ground), and Tilt can represent pitching (i.e., angle adjustment in the vertical direction, for example, rotating around an axis perpendicular to the line of sight, the camera tilts up or down). The electronic device 110 can provide a virtual rocker control 302 in the camera movement control 301-1. Further, the electronic device 110 can receive an interactive operation (e.g., a sliding operation) on the camera movement control 301-1 and adjust the position of the virtual rocker control 302. The electronic device 110 can determine the camera movement information with the camera movement control 301-1 based on the operation direction of the interactive operation. For example, if the operation direction of the interactive operation is to the left (e.g., the direction indicated by the arrow 303), the type of camera movement information corresponding to the interactive operation can be to shake the virtual camera to the left in the horizontal direction. Furthermore, the electronic device 110 may determine the angle of the left-pan virtual camera based on the operation distance of the interactive operation.

[0069] As an example, the mirror control 301-2 can be associated with the zoom (e.g., zoom) control of the virtual camera. The electronic device 110 receives an interactive operation (e.g., a sliding operation) on the mirror control 301-2, and determines that the interactive operation is associated with the zoom control of the virtual camera. Further, the electronic device 110 can determine to increase or decrease the lens focal length of the virtual camera based on the operation direction of the interactive operation. For example, the electronic device 110 can determine to increase the lens focal length of the virtual camera based on the operation direction of the interactive operation being upward (e.g., the direction indicated by the arrow 304). Conversely, the electronic device 110 can determine to reduce the lens focal length of the virtual camera. Further, the electronic device 110 determines the amplitude of adjusting the lens focal length of the virtual camera based on the operation distance of the interactive operation. For example, the greater the operation distance, the greater the amplitude.

[0070] As an example, the mirror control 301-3 can be associated with the roll (e.g., roll) control of the virtual camera. For example, roll can represent roll (e.g., rotation around an axis parallel to the line of sight). The electronic device 110 receives an interactive operation (e.g., a sliding operation) on the mirror control 301-3, and determines that the interactive operation is associated with the roll control of the virtual camera. Further, the electronic device 110 can determine the direction of the roll virtual camera based on the operation direction of the interactive operation. For example, the electronic device 110 can determine to roll the virtual camera clockwise perpendicular to the line of sight based on the operation direction of the interactive operation being upward (e.g., the direction indicated by the arrow 304). Conversely, the electronic device 110 can determine to roll the virtual camera counterclockwise perpendicular to the line of sight. Further, the electronic device 110 determines the angle of the roll virtual camera based on the operation distance of the interactive operation. For example, the greater the operation distance, the greater the angle.

[0071] In some embodiments, the electronic device 110 may receive a first interactive operation (e.g., a sliding operation) on a first camera movement control (e.g., camera movement control 301-1). Further, the electronic device 110 may determine the camera movement control information based on at least one operation attribute (e.g., sliding distance, sliding direction, and / or duration, etc.) of the first interactive operation.

[0072] Additionally, the electronic device 110 may also receive a second interactive operation on a second camera movement control (e.g., camera movement control 301-2). The first interactive operation and the second interactive operation are at least partially performed in parallel. For example, the first interactive operation and the second interactive operation are performed simultaneously, or the operation time of the first interactive operation and the second interactive operation overlap (e.g., the first interactive operation is performed in 0-10 seconds, and the second interactive operation is performed in 5-13 seconds, and the operation time of the first interactive operation and the second interactive operation overlaps within 5-10 seconds.). For example, the first interactive operation includes a first sliding operation for the camera movement control 301-1. The second interactive operation may include a second sliding operation for the camera movement control 301-2.

[0073] For example, the first interaction operation and the second interaction operation are at least partially performed in parallel, for example, the user can use the left finger to slide (or toggle) the virtual joystick control of the camera movement control 301-1 to rotate the virtual camera in the horizontal direction, and use the right finger to slide (or toggle) the camera movement control 301-2 to adjust the focal length of the virtual camera. In this way, the embodiment of the present disclosure can support the user to operate multiple camera movement controls at the same time to simulate the real camera movement process, so that the user has a more realistic camera movement experience, and improves the efficiency of the user's control of the camera movement process.

[0074] Further, the electronic device 110 may determine the mirror control information based on a first set of operation attributes of the first interactive operation (e.g., the sliding direction, sliding distance and / or duration of the first interactive operation, etc.) and a second set of operation attributes of the second interactive operation (e.g., the sliding direction, sliding distance and / or duration of the second interactive operation, etc.). For example, the electronic device 110 may determine parameter information (e.g., direction and angle) corresponding to horizontal rotation or vertical tilt based on the first set of operation attributes. The electronic device 110 may determine parameter information corresponding to zoom (e.g., the amplitude of increasing or decreasing the focal length) based on the second set of operation attributes. Further, the electronic device 110 may determine the mirror control information based on the parameter information corresponding to horizontal rotation or vertical tilt and the parameter information corresponding to zoom.

[0075] As an example, the electronic device 110 may provide tactile feedback in response to receiving the first interactive operation. For example, the electronic device 110 may trigger the vibration sensor to vibrate in response to receiving the first interactive operation to provide tactile feedback. In this way, the embodiments of the present disclosure may provide vibration feedback to the user's finger when the user slides the camera control with his finger, thereby improving the user's interactive experience.

[0076] Additionally, the electronic device 110 may provide visual feedback in response to receiving the first interaction operation.

[0077] Additionally or alternatively, the electronic device 110 may present a set of indicator elements in response to receiving the first interactive operation. A set of indicator elements is used to indicate a change process of a picture area corresponding to the mirror movement control information. As an example, the electronic device 110 may present a dynamic change process of a set of indicator elements. The dynamic change process may be associated with the mirror movement control information.

[0078] In some embodiments, the electronic device 110 may provide a preview area (eg, area 315 ) in which the electronic device 110 may present the reference image and / or the generated video content.

[0079] As an example, Figure 3B As shown, the electronic device 110 may present a set of indication elements (e.g., indication elements 305) in response to receiving an interactive operation (e.g., operation 306) for the area 315. For example, operation 306 may indicate that the user slides the area 315 in a first direction (e.g., the direction indicated by arrow 306-1) based on a finger. For example, if the camera control information determined in operation 306 indicates that the virtual camera is moved by 1 meter, the indication element 305 may present a picture adjustment effect associated with the rotation process.

[0080] As an example, Figure 3CAs shown, the electronic device 110 may present a set of indication elements (e.g., indication element 308) in response to receiving an interactive operation (e.g., operation 307) for the camera movement control 301-2. For example, operation 307 may indicate that the user slides the camera movement control 301-2 in a second direction (e.g., the direction indicated by arrow 307-1) based on a finger. For example, if the camera movement control information determined in operation 307 indicates that the focal length of the lens of the virtual camera is increased by 10 mm, the indication element 308 may present a picture adjustment effect associated with the focal length adjustment process.

[0081] For example, Figure 3D As shown, the electronic device 110 may present a set of indication elements (e.g., indication element 310) in response to receiving an interactive operation (e.g., operation 309) for the camera movement control 301-3. For example, operation 309 may indicate that the user slides the camera movement control 301-3 in a third direction (e.g., the direction indicated by arrow 309-1) based on a finger. For example, based on the camera movement control information determined in operation 309, the indication element 310 may present a picture adjustment effect associated with the rolling process.

[0082] For example, Figure 3E As shown, the electronic device 110 may present a set of indication elements (e.g., indication element 312) in response to receiving an interactive operation (e.g., operation 311) for the camera movement control 301-1. For example, operation 311 may indicate that the user slides the camera movement control 301-1 in a fourth direction (e.g., the direction indicated by arrow 311-1) based on a finger. For example, if the camera movement control information determined based on operation 311 indicates that the virtual camera is rotated 10 degrees around an axis perpendicular to the ground, the indication element 312 may present a picture adjustment effect associated with the rotation process.

[0083] For example, Figure 3F As shown, the electronic device 110 may present a set of indication elements (e.g., indication element 314) in response to receiving an interactive operation (e.g., operation 313) for the camera movement control 301-1. For example, operation 313 may indicate that the user slides the camera movement control 301-1 in a fifth direction (e.g., the direction indicated by arrow 313-1) based on a finger. For example, if the camera movement control information determined based on operation 313 indicates that the virtual camera is rotated 10 degrees around an axis perpendicular to the line of sight, the indication element 314 may present a picture adjustment effect associated with the rotation process.

[0084] Alternatively, the electronic device 110 may highlight (e.g., highlight) the first camera movement control associated with the first interaction operation in response to receiving the first interaction operation. As an example, the electronic device 110 may highlight the portion of the first camera movement control associated with the first interaction operation in response to receiving the first interaction operation (e.g., if the first interaction operation is a sliding operation to the left, the display portion on the left side of the first camera movement control is highlighted).

[0085] As an example, continue to refer to Figure 3B The electronic device 110 may present a first video clip in the generation interface 300B (e.g., area 315), where the first video clip is generated based on the reference image and the camera movement control information. The process of generating the first video clip based on the reference image and the camera movement control information is described above and will not be repeated here.

[0086] Additionally, the aforementioned camera movement control information is the first camera movement control information. The electronic device 110 can determine the second camera movement information based on the third interactive operation on the third camera movement control in the at least one camera movement control; and present the second video clip in the generation interface, the second video clip is generated based on the second reference image and the second camera movement information, and the second reference image is determined based on the first video clip. As an example, the second reference image can be the end image frame of the first video clip. In this way, the embodiment of the present disclosure can continue to generate new video content based on the generated first video clip and the user's interactive operation, thereby meeting the needs of the user.

[0087] In this way, the embodiment of the present disclosure can provide at least one camera movement control in the generation interface, and can receive the user's interactive operation on the camera movement control. Moreover, the embodiment of the present disclosure can determine the camera movement control information based on the operation attributes of the interactive operation, so as to generate a video clip based on the reference image and the camera movement control information. In this way, the embodiment of the present disclosure can support the user to set the camera movement control information based on the interactive operation, thereby improving the efficiency and quality of generating video content.

[0088] Example Process

[0089] Figure 4 4 is a flowchart showing an example process 400 of live interaction according to some embodiments of the present disclosure. The process 400 may be implemented at the electronic device 110. Figure 1 4. The process 400 is described below.

[0090] like Figure 4 As shown, in box 410, the electronic device 110 presents a generated interface, and the generated interface includes at least one camera movement control.

[0091] In box 420, the electronic device 110 receives a first interactive operation on a first camera control among the at least one camera control.

[0092] In block 430 , the electronic device 110 determines the mirror control information based on at least one operation attribute of the first interaction operation.

[0093] In box 440, the electronic device 110 presents a first video clip in a generation interface, where the first video clip is generated based on the reference image and the camera control information.

[0094] In some embodiments, the first interactive operation includes a sliding operation associated with the first camera movement control, and at least one operation attribute includes at least one of the following: a sliding direction of the sliding operation; a sliding distance of the sliding operation; and a duration of the sliding operation.

[0095] In some embodiments, based on at least one operation attribute of the first interactive operation, determining the camera movement control information includes at least one of the following: based on the sliding direction, determining the parameter type of at least one camera movement parameter indicated by the camera movement control information; based on the sliding distance, determining the parameter value of at least one camera movement parameter indicated by the camera movement control information; based on the duration, determining the duration of the segment associated with the camera movement control information.

[0096] In some embodiments, the first camera movement control is associated with at least one of the following camera movement parameters: a position parameter associated with the virtual camera; an orientation parameter associated with the virtual camera; a focal length parameter associated with the virtual camera.

[0097] In some embodiments, the first camera movement control includes a virtual joystick control, which is associated with multiple moving directions, the multiple moving directions correspond to multiple preset types of camera movement parameters, and the moving distance of the virtual joystick control indicates the value of the corresponding camera movement parameter.

[0098] In some embodiments, at least one camera movement control also includes a second camera movement control, and determining the camera movement control information based on at least one operation attribute of the first interaction operation includes: receiving a second interaction operation on the second camera movement control, the first interaction operation and the second interaction operation being at least partially executed in parallel; and determining the camera movement control information based on a first set of operation attributes of the first interaction operation and a second set of operation attributes of the second interaction operation.

[0099] In some embodiments, process 400 further includes providing visual feedback and / or tactile feedback in response to receiving the first interaction operation.

[0100] In some embodiments, providing visual feedback includes: in response to receiving the first interaction operation, presenting a group of indicator elements, the group of indicator elements being used to indicate a change process of a screen area corresponding to the mirror movement control information.

[0101] In some embodiments, the camera movement control information is first camera movement control information, the reference image is a first reference image, and process 400 also includes: determining second camera movement information based on a third interactive operation on a third camera movement control in at least one camera movement control; and presenting a second video clip in a generation interface, wherein the second video clip is generated based on the second reference image and the second camera movement information, and the second reference image is determined based on the first video clip.

[0102] In some embodiments, process 400 further includes: presenting description text in the generated interface, the description text indicating the camera movement control information.

[0103] In some embodiments, the process 400 further includes: updating the camera control information based on an editing operation on the description text.

[0104] In some embodiments, the description text is a prompt word generated based on at least one operation attribute of the first interactive operation, and the prompt word is provided to the video generation model to generate the first video segment.

[0105] In some embodiments, the reference image is an ending image frame of a reference video clip in the generation interface.

[0106] In some embodiments, at least one camera movement control is a first group of camera movement controls, and process 400 also includes: receiving a mode switching operation associated with generation; and presenting a second group of camera movement controls in the generation interface, the first group of camera movement controls being different from the second group of camera movement controls.

[0107] Example devices and equipment

[0108] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 5 A schematic structural block diagram of an example apparatus 500 for generating video content according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0109] like Figure 5 As shown, the device 500 includes: an interface presentation module 510, configured to present a generated interface, the generated interface includes at least one camera movement control; a receiving module 520, configured to receive a first interactive operation on a first camera movement control among the at least one camera movement control; a determination module 530, configured to determine camera movement control information based on at least one operation attribute of the first interactive operation; and a video presentation module 540, configured to present a first video clip in the generated interface, the first video clip being generated based on a reference image and the camera movement control information.

[0110] In some embodiments, the first interactive operation includes a sliding operation associated with the first camera movement control, and at least one operation attribute includes at least one of the following: a sliding direction of the sliding operation; a sliding distance of the sliding operation; and a duration of the sliding operation.

[0111] In some embodiments, the determination module 530 is also configured to: determine the parameter type of at least one camera movement parameter indicated by the camera movement control information based on the sliding direction; determine the parameter value of at least one camera movement parameter indicated by the camera movement control information based on the sliding distance; and determine the duration of the segment associated with the camera movement control information based on the duration.

[0112] In some embodiments, the first camera movement control is associated with at least one of the following camera movement parameters: a position parameter associated with the virtual camera; an orientation parameter associated with the virtual camera; a focal length parameter associated with the virtual camera.

[0113] In some embodiments, the first camera movement control includes a virtual joystick control, which is associated with multiple moving directions, the multiple moving directions correspond to multiple preset types of camera movement parameters, and the moving distance of the virtual joystick control indicates the value of the corresponding camera movement parameter.

[0114] In some embodiments, at least one camera movement control also includes a second camera movement control, and the determination module 530 is further configured to: receive a second interactive operation on the second camera movement control, the first interactive operation and the second interactive operation being at least partially executed in parallel; and determine the camera movement control information based on a first set of operating attributes of the first interactive operation and a second set of operating attributes of the second interactive operation.

[0115] In some embodiments, the apparatus 500 further includes a feedback module, and the feedback module is configured to provide visual feedback and / or tactile feedback in response to receiving the first interaction operation.

[0116] In some embodiments, the feedback module is further configured to: in response to receiving the first interactive operation, present a group of indicator elements, where the group of indicator elements are used to indicate a change process of the screen area corresponding to the mirror movement control information.

[0117] In some embodiments, the camera movement control information is first camera movement control information, the reference image is a first reference image, and the device 500 also includes an additional generation module, which is configured to: determine second camera movement information based on a third interactive operation on a third camera movement control in at least one camera movement control; and present a second video clip in a generation interface, wherein the second video clip is generated based on the second reference image and the second camera movement information, and the second reference image is determined based on the first video clip.

[0118] In some embodiments, the apparatus 500 further includes a description text module, which is configured to present a description text in the generation interface, wherein the description text indicates the camera movement control information.

[0119] In some embodiments, the apparatus 500 further includes an updating module, and the updating module is configured to update the camera movement control information based on an editing operation on the description text.

[0120] In some embodiments, the description text is a prompt word generated based on at least one operation attribute of the first interactive operation, and the prompt word is provided to the video generation model to generate the first video segment.

[0121] In some embodiments, the reference image is an ending image frame of a reference video clip in the generation interface.

[0122] In some embodiments, at least one camera movement control is a first group of camera movement controls, and the device 500 also includes a mode switching module, which is configured to: receive a mode switching operation associated with generation; and present a second group of camera movement controls in the generation interface, and the first group of camera movement controls is different from the second group of camera movement controls.

[0123] The units included in the device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 500 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0124] Figure 6 1 shows a block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6 The electronic device 600 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to implement Figure 1 An electronic device 110.

[0125] like Figure 6As shown, the electronic device 600 is in the form of a general electronic device. The components of the electronic device 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 600.

[0126] The electronic device 600 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 600.

[0127] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0128] The communication unit 640 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 600 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0129] The input device 650 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 660 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 600, or communicate with any device that allows the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0130] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0131] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0132] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0133] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0134] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0135] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating video content, comprising: Presenting a generation interface, wherein the generation interface includes at least one camera movement control; Receiving a first interactive operation on a first camera movement control among the at least one camera movement control; Determining mirror movement control information based on at least one operation attribute of the first interactive operation; as well as A first video clip is presented in the generation interface, where the first video clip is generated based on a reference image and the camera movement control information.

2. The method according to claim 1, wherein the first interactive operation comprises a sliding operation associated with the first camera movement control, and the at least one operation attribute comprises at least one of the following: a sliding direction of the sliding operation; A sliding distance of the sliding operation; The duration of the sliding operation.

3. The method according to claim 2, wherein determining the mirror control information based on at least one operation attribute of the first interactive operation comprises at least one of the following: Determining, based on the sliding direction, a parameter type of at least one lens movement parameter indicated by the lens movement control information; Determining, based on the sliding distance, a parameter value of the at least one lens movement parameter indicated by the lens movement control information; Based on the duration, a segment duration associated with the camera movement control information is determined.

4. The method according to claim 1, wherein the first camera movement control is associated with at least one of the following camera movement parameters: Position parameters associated with the virtual camera; an orientation parameter associated with the virtual camera; The focal length parameter associated with the virtual camera.

5. The method according to claim 1, wherein the first camera movement control comprises a virtual joystick control, the virtual joystick control is associated with multiple moving directions, the multiple moving directions correspond to multiple preset types of camera movement parameters, and the moving distance of the virtual joystick control indicates the value of the corresponding camera movement parameter.

6. The method according to claim 1, wherein the at least one camera movement control further comprises a second camera movement control, and based on at least one operation attribute of the first interactive operation, determining the camera movement control information comprises: receiving a second interactive operation on the second camera movement control, wherein the first interactive operation and the second interactive operation are at least partially performed in parallel; as well as The mirror movement control information is determined based on a first group of operation attributes of the first interactive operation and a second group of operation attributes of the second interactive operation.

7. The method according to claim 1, further comprising: In response to receiving the first interaction operation, visual feedback and / or tactile feedback is provided.

8. The method of claim 7, wherein providing visual feedback comprises: In response to receiving the first interactive operation, a group of indicator elements are presented, where the group of indicator elements are used to indicate a change process of a picture area corresponding to the mirror movement control information.

9. The method according to claim 1, wherein the camera movement control information is first camera movement control information, the reference image is a first reference image, and the method further comprises: Determining second camera movement information based on a third interactive operation on a third camera movement control among the at least one camera movement control; as well as A second video clip is presented in the generation interface, where the second video clip is generated based on a second reference image and the second camera movement information, and the second reference image is determined based on the first video clip.

10. The method according to claim 1, further comprising: A description text is presented in the generation interface, wherein the description text indicates the camera movement control information.

11. The method according to claim 10, further comprising: Based on the editing operation on the description text, the camera movement control information is updated. 12 . The method according to claim 10 , wherein the description text is a prompt word generated based on the at least one operation attribute of the first interactive operation, and the prompt word is provided to a video generation model to generate the first video clip.

13. The method according to claim 1, wherein the reference image is an end image frame of a reference video clip in the generation interface.

14. The method according to claim 1, wherein the at least one camera movement control is a first group of camera movement controls, the method further comprising: receiving a mode switching operation associated with said generating; as well as A second group of camera movement controls is presented in the generated interface, and the first group of camera movement controls is different from the second group of camera movement controls.

15. An apparatus for generating video content, comprising: An interface presentation module, configured to present a generated interface, wherein the generated interface includes at least one camera movement control; A receiving module, configured to receive a first interactive operation on a first camera movement control among the at least one camera movement control; A determination module, configured to determine the camera movement control information based on at least one operation attribute of the first interactive operation; as well as The video presentation module is configured to present a first video clip in the generation interface, where the first video clip is generated based on a reference image and the camera movement control information.

16. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 14 when executed by the at least one processing unit.

17. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 14.

Images