Three-dimensional video generation method and device

Abstract

The invention relates to a three-dimensional video generation method and device, and relates to the field of three-dimensional video generation. The method comprises the following steps: constructing a three-dimensional scene by utilizing multiple items of materials in a preset material resource library; acquiring mirror moving information of a virtual camera used for shooting a three-dimensional scene; determining a key frame according to the mirror moving information and the three-dimensional scene; performing difference operation on two adjacent key frames, and determining a motion picture of the virtual camera under the real-time frame rate; and determining a target three-dimensional video according to the motion picture. According to the invention, the three-dimensional scene can be directly utilized to generate the three-dimensional video, so that the generation efficiency of the three-dimensional video is improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional graphic image processing and three-dimensional video generation, and in particular to a three-dimensional video generation method and device. Background Art

[0002] With the maturity and promotion of 3D display technology, 3D content has gradually become a bottleneck for the development of the 3D industry. There are two main sources of 3D content: 3D video shooting and 2D video conversion to 3D video. Using a 3D camera to shoot new 3D videos is a feasible way to provide content, but this method is costly. Converting existing 2D videos to 3D videos usually involves increasing the thickness and rotation angle of the text to create a three-dimensional effect, and it is necessary to coordinate with spatial light and shadow effects to create a 3D effect. The processing efficiency of this method is related to the performance of the configured hardware, and it takes a long time to render. Summary of the invention

[0003] The embodiments of the present disclosure provide a three-dimensional video generation method and device, which are used to solve the problem of low efficiency in obtaining three-dimensional video generation in the prior art.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for generating a three-dimensional video, comprising: building a three-dimensional scene using multiple materials in a preset material resource library; obtaining camera movement information of a virtual camera used to shoot the three-dimensional scene; determining key frames based on the camera movement information and the three-dimensional scene; performing a difference operation on two adjacent key frames to determine a motion picture of the virtual camera at a real-time frame rate; and determining a target three-dimensional video based on the motion picture.

[0005] In a second aspect, an embodiment of the present disclosure provides a three-dimensional video generation device, comprising: a scene construction unit, configured to construct a three-dimensional scene using multiple materials in a preset material resource library; a camera movement acquisition unit, configured to acquire camera movement information of a virtual camera used to shoot the three-dimensional scene; a key frame determination unit, configured to determine a key frame based on the camera movement information and the three-dimensional scene; a picture determination unit, configured to perform a difference operation on two adjacent key frames to determine a motion picture of the virtual camera at a real-time frame rate; and a video generation unit, configured to determine a target three-dimensional video based on the motion picture.

[0006] By applying the technical solution of the present invention, an initial three-dimensional video can be obtained by constructing a three-dimensional scene and adding video materials to the three-dimensional scene. At the same time, the key frames are obtained by using the camera movement information of the virtual camera, and finally the target three-dimensional video is obtained. Compared with the prior art, the present invention does not need to use a 3D camera car to shoot a three-dimensional video, nor does it need to convert a 2D video into a three-dimensional video. Instead, the three-dimensional video is generated directly using the three-dimensional scene, thereby improving the generation efficiency of the three-dimensional video.

[0007] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0009] Figure 1 A schematic diagram of a flow chart of an embodiment of a three-dimensional video generation method disclosed in the present invention;

[0010] Figure 2a and Figure 2b A schematic diagram of a three-dimensional scene captured by virtual cameras of different viewing angles in an application scenario of the three-dimensional video generation method disclosed in the present invention;

[0011] Figure 3 A schematic diagram of a flow chart of another embodiment of a three-dimensional video generation method disclosed herein;

[0012] Figure 4 A schematic diagram of a camera movement information input control of the three-dimensional video generation method disclosed in the present invention;

[0013] Figure 5 A schematic diagram of a camera track of the three-dimensional video generation method disclosed in the present invention;

[0014] Figure 6 It is a structural schematic diagram of an embodiment of a three-dimensional video generating device disclosed in the present invention. DETAILED DESCRIPTION

[0015] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0016] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0017] In the absence of conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.

[0018] In order to make the technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0019] Figure 1 FIG. 1 shows a process 100 of an embodiment of a method for generating a three-dimensional video according to the present disclosure. Figure 1 As shown, the 3D video generation method of this embodiment may include the following steps:

[0020] Step 101, using multiple materials in a preset material resource library to build a three-dimensional scene.

[0021] In this embodiment, a material resource library may be pre-set. The material resource library may include two-dimensional materials and three-dimensional materials. Two-dimensional materials may include digital people, pictures, audio, video, animation, text and other contents. Three-dimensional materials may include three-dimensional models and other contents.

[0022] In this embodiment, by receiving a selection instruction for each material in the material resource library, the selected material can be added to the three-dimensional scene to achieve the construction of the three-dimensional scene. Alternatively, according to the needs of the three-dimensional scene, matching materials can be selected from the material resource library to achieve the construction of the three-dimensional scene. For example, if materials such as urban roads, motor vehicles, and non-motor vehicles are required in the three-dimensional scene, materials such as roads, motor vehicles, and non-motor vehicles can be selected from the material resource library according to the identification and / or type of each material.

[0023] It is understandable that after the three-dimensional scene is built, the relative position relationship between the materials is determined. Of course, the relative position of each material can be adjusted according to the needs of the three-dimensional scene.

[0024] Step 102: Obtain camera movement information of a virtual camera used to shoot a three-dimensional scene.

[0025] In the present embodiment, the above-mentioned three-dimensional scene can be photographed by a virtual camera to obtain an initial three-dimensional video. The camera movement information of the virtual camera may include multiple parameters, such as position, pitch angle, focal length, etc. The user can input the camera movement information by inputting the values ​​of the above-mentioned parameters. Alternatively, the camera movement information can be input by a control provided in a three-dimensional image rendering engine. Alternatively, the acquisition of the camera movement information can also be achieved by obtaining a camera movement information file. Specifically, the camera movement information of the virtual camera can be saved separately in a file of a specified format, and the acquisition of the camera movement information of the entire video can be achieved by reading the above-mentioned file.

[0026] Step 103, determining a key frame according to the camera movement information and the three-dimensional scene.

[0027] After obtaining the camera movement information, the key frame corresponding to the virtual camera at the real-time frame rate can be determined according to the relative position of the virtual camera and the three-dimensional scene. Specifically, the three-dimensional scene picture corresponding to the camera movement information can be used as a key frame every preset time. For example, the three-dimensional scene picture is taken as a key frame every 1 second.

[0028] Alternatively, the 3D scene image corresponding to the position where the mirror movement information changes can be used as a key frame. For example, if the mirror movement information at time A is a and the mirror movement information at time B is b, the 3D scene image corresponding to time B can be used as a key frame.

[0029] Step 104, performing a difference operation on two adjacent key frames to determine the motion picture of the virtual camera at the real-time frame rate.

[0030] After obtaining each key frame, a difference operation can be performed on two adjacent key frames to determine the motion picture of the virtual camera at the real-time frame rate between the two key frames. Specifically, the pixels of two adjacent key frames can be subtracted, and the obtained difference image can be used as the motion picture. At the same time, the real-time frame rate of the motion picture between the two key frames can be determined according to the interval time between the two adjacent key frames.

[0031] Step 105: determine the target 3D video according to the motion picture.

[0032] After obtaining the above motion picture, the above motion picture can be rendered in real time, and the video corresponding to the three-dimensional scene can be determined. Specifically, the real-time rendering can be achieved by using a three-dimensional image rendering engine.

[0033] See FIG2, which shows a schematic diagram of an application scenario of the 3D video generation method according to the present disclosure. In the application scenario of FIG2, a user can build a 3D scene through a resource library in a 3D image rendering engine, such as Figure 2a Then, the camera movement information of the virtual camera shooting the three-dimensional scene is obtained. The key frame is determined by the camera movement information and the three-dimensional scene, such as Figure 2b Then, by performing difference calculation on two adjacent key frames, the motion picture and real-time frame rate between the two adjacent key frames are determined. Finally, the target three-dimensional model is obtained.

[0034] The three-dimensional video generation method provided by the above-mentioned embodiment of the present disclosure can obtain an initial three-dimensional video by constructing a three-dimensional scene and adding video materials to the three-dimensional scene. At the same time, the key frame is obtained by using the camera movement information of the virtual camera, and finally the target three-dimensional video is obtained. Compared with the prior art, the present disclosure does not need to use a 3D camera car to shoot a three-dimensional video, nor does it need to convert a 2D video into a three-dimensional video. Instead, the three-dimensional video is directly generated using a three-dimensional scene, thereby improving the generation efficiency of the three-dimensional video.

[0035] Continue to see Figure 3 , which shows a process 300 of another embodiment of the 3D video generation method according to the present disclosure. Figure 3 As shown, the method in this embodiment may include the following steps:

[0036] Step 301, obtaining operation commands for two-dimensional materials and / or three-dimensional materials in a material resource library; and building a three-dimensional scene according to the operation commands.

[0037] In this embodiment, the user's operation command for the two-dimensional material and / or three-dimensional material of the material resource library can be obtained. The above-mentioned operation commands may include dragging, clicking, etc. A three-dimensional scene can be built according to the above-mentioned operation commands. For example, if it is detected that the user drags a certain three-dimensional model, that is, a drag command is obtained, the three-dimensional model can be moved to the end position of the drag. Or if the user adjusts the spatial position parameters or angle of the three-dimensional model, the three-dimensional model is placed at the adjusted spatial position at the adjusted angle. If it is detected that the user double-clicks a certain three-dimensional model, that is, a double-click command is obtained, the three-dimensional model can be placed at a preset position. After detecting the operation command for the three-dimensional model, the three-dimensional model can be displayed on the time code track. It can be understood that when adjusting the three-dimensional model, the three-dimensional model can be first selected on the time code track, and then the parameters of the three-dimensional model can be adjusted in the rendering window.

[0038] In some specific practices, the above-mentioned three-dimensional scene may also include video material. Specifically, the content of the video material may match other materials in the three-dimensional scene. For example, if other materials in the three-dimensional scene are city roads, the content of the video material may describe the appearance of the city. Specifically, the video material in the material resource library may include a summary or text of the content in the video. By calculating the matching degree between the summary or text and the identification or type of the material, it can be determined whether the two match.

[0039] The video material here can be a video of any length and any content. It is understandable that the content of the video material can be selected according to the actual application scenario or according to the three-dimensional scene. The number of added video materials can be one or more. When there are multiple video materials added, each video can be played at the same time or not. Specifically, by adding video materials on different tracks, and the playback time of each video material can overlap, the simultaneous decoding of multiple videos can be achieved. Furthermore, animation can be added in the above-mentioned overlapping area to achieve effects such as page turning, image wiping, and fade-in and fade-out.

[0040] In some optional implementations of this embodiment, the above-mentioned operation command may include the time code position and the three-dimensional space position of the material. The appearance time of each material can be determined according to the time code position of each material in the operation command. In this way, the material can appear at the moment corresponding to the time code position. Further, according to the three-dimensional space position of each material in the operation command, each material can be added to the corresponding three-dimensional space position in the three-dimensional scene. According to the appearance time of each material and the three-dimensional space position of each material, the construction of the three-dimensional scene is realized.

[0041] In some optional implementations of the present embodiment, the above-mentioned operation command may also include a binding instruction between at least two materials. The binding instruction corresponds to at least two materials, namely the binding material and the bound material. It is understandable that the bound material is the carrier of the binding material. For example, the binding material is a video, and the bound material is a picture. Then the display position of the video is at the position where the picture is located, or the video directly references and replaces the picture. In the process of building a three-dimensional scene, if a binding instruction is detected, the binding material and the bound material corresponding to the binding instruction can be first determined. Then, after the bound material is displayed, the bound material can be replaced with the binding material. The binding relationship here can also be understood as replacing the bound material with the binding material. For example, a decorative painting with an aspect ratio of 16:9 is hung on the wall, and it is associated with a video file with the same aspect ratio by reference, so that the binding of the decorative painting and the video file is realized. When the time code pointer plays to the decorative painting on the time code track, the actual effect is to play the video as the first frame of the video.

[0042] Through this implementation, it is possible to trigger the bound material automatically or passively after the bound material is triggered. For example, the bound material is a TV picture, and the bound material is a video. By clicking on the TV picture, the video playback can be triggered. For the user, when the 3D video lens moves to the TV picture, the video on the TV can be automatically played, thereby improving the user's browsing experience of the 3D video.

[0043] Step 302: Obtain camera movement information through at least one control in a preset operation panel.

[0044] In this embodiment, an operation panel for inputting mirror movement information may be provided to the user, such as Figure 4As shown. The above-mentioned operation panel may include multiple controls for users to input different camera movement information. For example, the above-mentioned operation panel may include a control joystick, a Dolly control, a Pan control, a Tlit control and a Zoom control. Among them, the control joystick is used to control the vertical lifting and horizontal movement of the lens. The Dolly control is used to control the forward and backward movement of the camera. The Pan control is used to control the horizontal panning of the camera lens. The Tlit control is used to control the vertical pitch of the camera lens. For the Zoom control, Zoom In and Zoom Out indicate that the lens is pushed forward and pulled out, changing the lens framing range.

[0045] Step 303: Add the camera movement information to the camera track of the video.

[0046] In the process of inputting the camera movement information, the camera movement information can be added below the display interface of the 3D video in the form of a camera track, such as Figure 5 As shown. The camera track may include the values ​​of the camera movement parameters at each moment. The camera track may include the position parameters, rotation parameters and zoom parameters of the camera lens. Among them, the position parameters record the x / y / z spatial position and the displacement change caused by moving along the x / y / z axis. The rotation parameters record the angle change caused by rotating along the x / y / z axis. The zoom parameters record the focal length of the lens corresponding to the key frame and the change in focal length.

[0047] Step 304: determine key frames according to the camera movement information and the three-dimensional scene.

[0048] Step 305, performing a difference operation on two adjacent key frames to determine the motion picture of the virtual camera at the real-time frame rate.

[0049] Step 306: Generate a video corresponding to the three-dimensional scene based on the motion picture.

[0050] In this embodiment, the logic layer in the three-dimensional rendering engine can be used to perform difference calculations on each key frame to render the motion effect of the camera lens in real time.

[0051] In some optional implementations of this embodiment, in response to receiving adjustment information for the three-dimensional scene and / or video, real-time rendering is performed according to the adjustment information.

[0052] In this implementation, the user can adjust the 3D scene and / or video. And according to the above adjustment, real-time rendering can be performed without waiting for rendering time. The above adjustment can include adjustment of any material in the 3D scene. For example, the position and angle of any material can be adjusted. It can also include adjustment of the video, such as adjusting the pointer on the time code track to start playing from the position indicated by the pointer.

[0053] In some optional implementations of this embodiment, the three-dimensional scene includes a digital human. The method can also adjust the display information of the digital human according to the camera movement information. In this implementation, the digital human can appear as a host, and the digital human can always appear at a preset position of the virtual camera's perspective, or the digital human's appearance position can be kept in a certain material in the three-dimensional scene. In this way, the display information of the digital human can be adjusted according to the above-mentioned preset position and the above-mentioned camera movement information. The above-mentioned display information can also include the display special effects and display area of ​​the digital human. For example, through the above-mentioned camera movement information, a certain three-dimensional model appears in the perspective of the virtual camera, and the display area of ​​the digital human can be adjusted according to the position of the three-dimensional model. Specifically, the above-mentioned three-dimensional model can be a lecture table, and the upper body of the digital human can be displayed on the above-mentioned lecture table. Alternatively, the display special effects of the digital human can also be set to a special effect of fading from the bottom to the top.

[0054] The three-dimensional video generation method provided by the above-mentioned embodiments of the present disclosure can use graphics technology to realize the rapid assembly of characters, three-dimensional models, and three-dimensional scenes in three-dimensional space, and realize lens movement, perspective relationship light and shadow effects that truly restore the effects of the physical world; and can provide visualization operation of lens parameters, control of lens movement, horizontal pan, vertical pitch, and zoom; in addition, video editing operations, including camera movement, are all rendered in real time, what you see is what you get, and there is no need to spend time on pre-rendering; three-dimensional graphics and two-dimensional images are used as scenes to realize the comprehensive application of camera movement effects, without the need for multiple tools to cooperate to complete content production.

[0055] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a three-dimensional video generation device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0056] like Figure 6 As shown, the 3D video generating device 600 of this embodiment includes: a scene building unit 601, a camera movement acquiring unit 602, a key frame determining unit 603, a picture determining unit 604 and a video generating unit 605.

[0057] The scene building unit 601 is configured to build a three-dimensional scene using multiple materials in a preset material resource library.

[0058] The camera movement acquisition unit 602 is configured to acquire camera movement information of a virtual camera used to shoot a three-dimensional scene.

[0059] The key frame determination unit 603 is configured to determine the key frame according to the camera movement information and the three-dimensional scene.

[0060] The picture determination unit 604 is configured to perform a difference operation on two adjacent key frames to determine the motion picture of the virtual camera at a real-time frame rate.

[0061] The video generation unit 605 is configured to determine a target 3D video according to the motion picture.

[0062] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A three-dimensional video generation method, comprising: Use multiple materials in the preset resource library to build a 3D scene; Acquiring camera movement information of a virtual camera used to shoot the three-dimensional scene; Determining a key frame according to the camera movement information and the three-dimensional scene; Performing a difference operation on two adjacent key frames to determine a motion picture of the virtual camera at a real-time frame rate; According to the motion picture, a target three-dimensional video is determined.

2. The method according to claim 1, wherein: The material resource library includes two-dimensional materials and three-dimensional materials; as well as The use of multiple materials in the preset material resource library to build a three-dimensional scene includes: Obtaining an operation command for the two-dimensional material and / or three-dimensional material in the material resource library; According to the operation command, a three-dimensional scene is constructed.

3. The method according to claim 2, wherein: The operation command includes the time code position and three-dimensional space position of the material; The step of building a three-dimensional scene according to the operation command includes: Determine the appearance time of each material according to the time code position of each material in the operation command; Determining the three-dimensional spatial position of each material according to the three-dimensional spatial position of each material in the operation command; Build a three-dimensional scene based on the appearance time and three-dimensional spatial position of each material.

4. The method according to claim 3, wherein: The operation command includes a binding instruction between at least two materials; as well as The step of building a three-dimensional scene according to the operation command includes: Determine the binding material and the bound material corresponding to the binding instruction; After the bound material is displayed, the bound material is replaced by the bound material.

5. The method according to claim 1, wherein: The obtaining of the camera movement information of the virtual camera for shooting the three-dimensional scene includes: The camera movement information is obtained through at least one control in a preset operation panel.

6. The method according to claim 5, wherein: The method further comprises: The camera movement information is added to the camera track of the video.

7. The method according to claim 5, wherein: The step of determining a key frame corresponding to the virtual camera according to the camera movement information and the initial three-dimensional video includes: The key frame is determined from the initial three-dimensional video according to the time code position of the change value of the camera movement parameter in the camera movement information.

8. The method according to claim 1, wherein: The three-dimensional scene includes a digital human; and The method further comprises: According to the camera movement information, the display information of the digital human is adjusted.

9. The method according to claim 1, wherein: The method further comprises: In response to receiving adjustment information for the three-dimensional scene and / or the video, real-time rendering is performed according to the adjustment information.

10. A video generating device, comprising: The scene building unit is configured to build a three-dimensional scene using multiple materials in a preset material resource library; A camera movement acquisition unit, configured to acquire camera movement information of a virtual camera used to shoot the three-dimensional scene; A key frame determining unit, configured to determine a key frame according to the camera movement information and the three-dimensional scene; A picture determination unit is configured to perform a difference operation on two adjacent key frames to determine a motion picture of the virtual camera at a real-time frame rate; The video generating unit is configured to determine a target three-dimensional video according to the motion picture.

Images