Real-time shooting system and method for digital animation

Abstract

The invention provides a digital animation real-time shooting system and method. The real-time shooting system is constructed based on a motion capture system, a UE engine and one-screen display application software. According to the real-time shooting system, a plurality of second virtual cameras with adjustable parameters are generated by adopting a UE engine to control corresponding entity virtual cameras (constructed based on a marked physical camera) in a motion capture system in real time, and shot pictures are transmitted to one-screen display application software in real time through the UE engine to be displayed in real time. Through the real-time shooting system provided by the invention, a complete digital animation effect can be captured and displayed in real time, the delay caused by multi-role shooting and software adaptability in a digital animation shooting process is solved, and meanwhile, camera parameters and role resources can be conveniently and timely adjusted according to the displayed shooting effect.

Description

Technical Field

[0001] The present invention relates to the field of shooting and producing digital film and television works, and in particular to a real-time shooting system and method for digital animation. Background Art

[0002] The traditional 3D animation production process usually begins with an animator building the corresponding character resources according to the plot, and the relevant production staff making motion capture data bound to the corresponding character, and then recording in the corresponding animation engine. From the entire production process, it can be seen that the level and experience of the animator directly affect the animation quality of the lens. At the same time, the recording or shooting process also has a serious impact on the final visual effect. On the one hand, it is difficult to show more animation details due to the influence of production time, and the manual frame-by-frame production mode is inefficient; on the other hand, although traditional motion capture technology has been popularized, it can only provide simple motion data, and the complete performance effect cannot be directly seen. Whether it is the director's judgment on the lens effect, the actor's performance quality, the lens composition, or the camera movement method, they cannot be displayed in real time on the spot, and only basic data collection can be done.

[0003] In summary, the existing 3D animation production process has serious effect lag (i.e., it is impossible to obtain the effect of shooting or recording in real time), and the effect often needs to be repeatedly modified after it is displayed, which is extremely inefficient. Therefore, it is urgent to provide a complete real-time virtual shooting system to break through the bottleneck of traditional 3D production and motion capture processes, and solve the delay caused by multiple roles and software adaptability according to the existing workflow. Summary of the invention

[0004] In order to solve the above-mentioned shortcomings of the traditional three-dimensional animation production process, the present invention provides a technical solution that can display the shooting / recording effects of digital animation in real time. The present invention achieves the above-mentioned purpose by redeveloping the Unreal Engine and making full use of the virtual camera function provided by the Unreal Engine in combination with a motion capture system. The virtual camera reflects the movement of the real camera to the virtual camera by positioning the position and angle of the camera, and previews the changes in the virtual content in real time. As long as the camera can be tracked, the lens movement and correct perspective relationship that are completely synchronized with the real camera can be obtained in a real-time rendering environment. In addition, the virtual camera system can also easily adjust the focal length and simulate the depth of field changes of the real camera to ensure accurate and realistic restoration of the shooting effect of the real camera.

[0005] The technical solution provided by the present application is implemented as: a real-time shooting system for digital animation, including: a motion capture module, a UE Unreal Engine module and a same-screen display module.

[0006] The motion capture module is used to generate a number of first virtual cameras by marking the motion capture points of a number of physical cameras, and place the first virtual cameras in a virtual motion capture field; the physical cameras and the first virtual cameras correspond one to one.

[0007] Preferably, the motion capture module is Vicon Shogun Live software running on a computer device.

[0008] UE Unreal Engine is used to receive the data captured by the motion capture module in real time through the built-in Live Link plug-in; construct a plurality of second virtual cameras with controllable parameters, and the real-time shooting pictures of the plurality of second virtual cameras and the first virtual camera are bound one by one.

[0009] Furthermore, the plurality of second virtual cameras are constructed based on the data structure BP_Switch of the virtual camera built-in plug-in in the UE Unreal Engine. By selecting the corresponding second virtual camera in the Bp_Switch data structure in the VCS of the UE Unreal Engine, the real-time connection node of the props added by setting its details is realized to bind the real-time shooting picture of the corresponding second virtual camera to the corresponding first virtual camera.

[0010] The same-screen display module is connected to the UE Unreal Engine module through the virtual camera Virtual Camera in the remote session plug-in in the UE Unreal Engine, so as to display the real-time shooting picture of the corresponding second virtual camera in the UE Unreal Engine module on the same screen.

[0011] Furthermore, there may be one or more on-screen display modules, and each on-screen display module displays a different real-time shooting picture of the second virtual camera.

[0012] Preferably, the same-screen display module is an application software Live Link Vcam running on an IOS device.

[0013] Accordingly, another aspect of the technical solution provided by the present application is implemented as: a real-time shooting method for digital animation. The real-time shooting method comprises:

[0014] Marking motion capture points of a plurality of physical cameras by a motion capture system to construct a plurality of first virtual cameras, and placing the plurality of first virtual cameras in a virtual motion capture field; the plurality of physical cameras and the plurality of first virtual cameras correspond one to one;

[0015] Setting the built-in Live Link plug-in in the UE Unreal Engine to receive the data captured by the motion capture system in real time, constructing a plurality of second virtual cameras with controllable parameters, and binding the real-time captured images of the plurality of second virtual cameras to the plurality of first virtual cameras in a one-to-one correspondence;

[0016] The display module is connected through the remote session plug-in Virtual Camera in the UE Unreal Engine to display the real-time shooting picture of the corresponding second virtual camera in the UE Unreal Engine module on the same screen.

[0017] Furthermore, the plurality of second virtual cameras are constructed based on the data structure BP_Switch of the virtual camera built-in plug-in in the UE Unreal Engine. By selecting the corresponding second virtual camera in the Bp_Switch data structure in the VCS of the UE Unreal Engine, the real-time connection node of the props added by setting its details is realized to bind the real-time shooting picture of the corresponding second virtual camera to the corresponding first virtual camera.

[0018] By shooting digital animation through the technical solution provided by this application, it is possible to display the shooting images of each camera in real time and comprehensively, so that relevant production / shooting personnel can grasp the lens effects and the performance quality of the actors in a timely manner, so as to timely adjust the role resources, lens composition, and camera movement methods.

[0019] That is, the technical solution provided by this application can effectively solve the problems existing in the existing 3D animation production process, such as effect display lag, excessive delay in shooting / recording effect modification (especially delay caused by multiple roles and software adaptability), and low efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a real-time shooting system for digital animation provided by the present invention in one embodiment. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations.

[0022] like Figure 1 As shown, a real-time shooting system for digital animation provided by the present invention includes: a motion capture module 100, a UE Unreal Engine 200 and a same-screen display module 300.

[0023] Among them, the motion capture module 100 is used to generate a number of first virtual cameras by marking the motion capture points of a number of physical cameras (i.e., physical virtual cameras), and place the number of first virtual cameras in a virtual motion capture field (virtual motion capture scene); the number of physical cameras and the number of first virtual cameras correspond one to one.

[0024] The same-screen display module is connected to the UE Unreal Engine module through the virtual camera VirtualCamera in the remote session plug-in in the UE Unreal Engine to display the real-time shooting picture of the corresponding second virtual camera in the UE Unreal Engine module on the same screen.

[0025] exist Figure 1 In the illustrated embodiment, the motion capture module 100 is Vicon Shogun Live software running on a computer device. The process of constructing the first virtual camera in Vicon Shogun Live software is as follows:

[0026] S10. Open the Vicon Shogun Live software, make several physical cameras according to the storyboard requirements and place them in the motion capture site.

[0027] S11. Mark motion capture points for several physical cameras (infrared light may be used to track the motion trajectory of the motion capture points). At this time, the marked physical cameras are displayed in the form of points on the software screen.

[0028] S12, connecting the motion capture points to generate the first virtual camera skeleton. Obviously, the first virtual camera corresponds to the physical camera one by one.

[0029] The UE Unreal Engine 200 is used to receive the data captured by the motion capture module in real time through the built-in LiveLink plug-in, and use the data structure BP_Switch of the built-in plug-in of the virtual camera of the UE Unreal Engine 200 to construct several second virtual cameras with controllable parameters. The real-time shooting pictures of the several second virtual cameras and the first virtual camera are bound one by one.

[0030] The data structure BP_Switch is used to adjust the focal length, depth of field, and upper, lower, left, and right offset parameters of the second virtual camera (the second virtual camera is bound to the first virtual camera) during shooting, as well as the blueprint for switching the shooting screen. The second virtual camera built in the UE Unreal Engine can make full use of the powerful image simulation and reasoning capabilities of the UE Unreal Engine, and obtain more and more comprehensive image detail information by setting the relevant parameters in BP_Switch.

[0031] exist Figure 1 In the illustrated embodiment, the operation process related to the UE Unreal Engine 200 and the same-screen display module 300 includes:

[0032] S20. Open the Live Link plug-in + Visual real-time plug-in in the UE Unreal Engine;

[0033] S21. Receive data from the motion capture module Shogun Live through the LiveLive plug-in, add and modify the real-time IP address, set the Port Number band to 801, and set the channel of the real-time motion capture computer and the computer running the UE Unreal Engine to the same signal for linking and transmitting real-time animation, otherwise real-time cannot be performed).

[0034] S22, the number structure BP_Switch in the virtual camera built-in plug-in is placed in the scene, and "Build Camera" is clicked in the detail setting interface to generate a second virtual camera with controllable parameters. Repeating this step in sequence can build multiple second virtual cameras with controllable parameters.

[0035] S23, by selecting the corresponding second virtual camera in the Bp_Switch data structure in the VCS of the UE Unreal Engine, and by setting its details to add a real-time connection node of the props to bind the real-time shooting picture of the corresponding first virtual camera to the corresponding second virtual camera.

[0036] S24. Click Settings-Plugins to open the plug-in menu, search for Virtual Camera, and enable Live Link and Remote Session plug-ins. Virtual Camera is a plug-in that links the virtual object of the physical camera and displays it in the outline. Live Link refers to the signal that links the real-time motion capture module so that the signal captured by the motion capture module can be transmitted to the computer of the UE Unreal Engine in real time. Enabling the remote session plug-in refers to the plug-in that links the signal of the same-screen display module (Live Link Vcam) to the UE Unreal Engine.

[0037] S25. Open Project Settings, set the computer's IP address and port in UDP; connect the device in the static endpoint list and add the device's IP address. This step is to make the computer running the UE Unreal Engine as the signal source for the camera, and to make the IP address of the computer running the UE Unreal Engine and the IP address of the camera link under the same signal.

[0038] S26. Set the pixel frame buffer format to 8-bit RGBA in the default settings option and restart the editor.

[0039] exist Figure 1In the embodiment shown, the same-screen display module is Live Link Vcam running in an IOS device. Of course, any application software or device that can be interconnected with UE4 to achieve real-time display can be used as the same-screen display module in this application, and there is no limitation here. The following only takes Live Link Vcam running in an IOS device as an example.

[0040] S27. Launch the Live Link Vcam APP on the iOS device and enter the IP address of the computer running the UE Unreal Engine.

[0041] S28. Click and drag the second virtual camera into the scene in the UE Unreal Engine panel. Place the second virtual camera plug-in into the scene to display the captured scene, which is convenient for subsequent scene shooting.

[0042] S29. Connect iPhone or iPad through UE Unreal Engine's Live Link (green means successful connection). The signal source link signal logo is automatically displayed after the plug-in is inserted. The information source comes from the iPhone app signal; yellow means standby and not connected, and green means connected.

[0043] S30. After linking the iPhone app, further settings are made on the computer running the UE Unreal Engine to accurately link it to the Live Link Vcam application on the iOS device to achieve same-screen display.

[0044] Furthermore, the same-screen display module 300 may be one or more, and each same-screen display module may display a different real-time shooting picture of the second virtual camera. In this way, the needs of multi-role performance, comprehensive, multi-angle real-time shooting can be easily met through multiple second virtual cameras.

[0045] Corresponding to the above-mentioned real-time shooting system of digital animation, the present application also provides a real-time shooting method of digital animation. The real-time shooting method includes:

[0046] Marking motion capture points of a plurality of physical cameras by a motion capture system to construct a plurality of first virtual cameras, and placing the plurality of first virtual cameras in a virtual motion capture field; the plurality of physical cameras and the plurality of first virtual cameras correspond one to one;

[0047] Setting the built-in Live Link plug-in in the UE Unreal Engine to receive the data captured by the motion capture system in real time, constructing a plurality of second virtual cameras with controllable parameters, and binding the real-time captured images of the plurality of second virtual cameras to the plurality of first virtual cameras in a one-to-one correspondence;

[0048] The display module is connected through the remote session plug-in Virtual Camera in the UE Unreal Engine to display the real-time shooting picture of the corresponding second virtual camera in the UE Unreal Engine module on the same screen.

[0049] Furthermore, the plurality of second virtual cameras are constructed based on the data structure BP_Switch of the virtual camera built-in plug-in in the UE Unreal Engine. By selecting the corresponding second virtual camera in the Bp_Switch data structure in the VCS of the UE Unreal Engine, the real-time connection node of the props added by setting its details is realized to bind the real-time shooting picture of the corresponding second virtual camera to the corresponding first virtual camera.

[0050] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A real-time shooting system for digital animation, characterized in that: The real-time shooting system includes: a motion capture module, a UE Unreal Engine module and a same-screen display module; The motion capture module is used to generate a plurality of first virtual cameras by marking motion capture points of a plurality of physical cameras, and place the plurality of first virtual cameras in a virtual motion capture field; the plurality of physical cameras and the plurality of first virtual cameras correspond one to one; UE Unreal Engine, used to receive the data captured by the motion capture module in real time through the built-in LiveLink plug-in; construct a plurality of second virtual cameras with controllable parameters, and bind the real-time captured images of the plurality of second virtual cameras to the plurality of first virtual cameras in a one-to-one correspondence; The same-screen display module is used to connect to the UE Unreal Engine module through the virtual camera VirtualCamera in the remote session plug-in in the UE Unreal Engine to display the real-time shooting picture of the corresponding second virtual camera in the UE Unreal Engine module on the same screen.

2. The real-time shooting system according to claim 1, characterized in that: The motion capture module is Vicon Shogun Live software running on a computer device.

3. The real-time shooting system according to claim 2, characterized in that: The plurality of second virtual cameras are constructed based on the data structure BP_Switch of the built-in plug-in of the virtual camera in the UE Unreal Engine.

4. The real-time shooting system according to claim 3, characterized in that: By selecting the corresponding second virtual camera in the Bp_Switch data structure in the VCS of the UE Unreal Engine, and setting its details to add a real-time connection node of the props, the corresponding second virtual camera and the real-time shooting picture of the corresponding first virtual camera are bound.

5. The real-time shooting system according to claim 4, characterized in that: There are one or more on-screen display modules, and each on-screen display module displays a different real-time shooting picture of the second virtual camera.

6. The real-time shooting system according to claim 5, characterized in that: The same-screen display module is the application software Live Link Vcam running on the IOS device.

7. A real-time shooting method for digital animation, characterized in that: The real-time shooting method comprises: Marking motion capture points of several physical cameras through a motion capture system to construct several first virtual cameras, and placing the first virtual cameras in a virtual motion capture field; the physical cameras and the first virtual cameras are multiple and correspond one to one; In the UE Unreal Engine, the data captured by the motion capture system is received in real time through the built-in LiveLink plug-in, a plurality of second virtual cameras with controllable parameters are constructed, and the real-time shooting images of the plurality of second virtual cameras and the plurality of first virtual cameras are bound one by one; The display module is connected through the remote session plug-in VirtualCamera in the UE Unreal Engine to display the real-time shooting picture of the corresponding second virtual camera in the UE Unreal Engine module on the same screen.

8. The real-time shooting method according to claim 7, characterized in that: The multiple second virtual cameras are constructed based on the data structure BP_Switch of the virtual camera built-in plug-in in the UE Unreal Engine.

Images