Animation production system

Abstract

To provide an animation production system which reduces the burden of animation production.SOLUTION: An animation production system comprises a virtual camera 3 for photographing characters 4-1 and 4-2 placed in a virtual space, and a parameter set list 6 for displaying a list of parameter items 62 available for photographing in the virtual space, and applies parameters selected from the parameter set list to the virtual space and performs photographing.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to an animation production system. [Background technology]

[0002] A virtual camera is arranged in a virtual space (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-146651 A Summary of the Invention [Problem to be solved by the invention]

[0004] When producing animation, the composition of the scene (for example, the placement of characters and objects relative to the camera) is important, but it is not always easy for the creator to compose the scene in a virtual space and place characters and other objects as they imagined, and this can take a lot of time.

[0005] The present invention has been made in view of the above background, and has an object to provide a technique that can reduce the burden of animation production. [Means for solving the problem]

[0006] The main invention of the present invention for solving the above problems is an animation production system comprising: a virtual camera for photographing a character placed in a virtual space; and a parameter list for displaying a list of parameter sets available for the photographing in the virtual space, and wherein a parameter set selected from the parameter list is applied to the virtual space to perform the photographing.

[0007] Other problems and solutions disclosed in this application will be made clear in the description of the preferred embodiments of the invention and the drawings. Effect of the Invention

[0008] According to the present invention, the burden of animation production can be reduced. [Brief description of the drawings]

[0009] [Figure 1] 2 is a diagram showing an example of a virtual space displayed on an HMD worn by a user in the animation production system of the present embodiment. FIG. [Diagram 2] FIG. 1 is a diagram showing an example of the overall configuration of an animation production system 300 according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram showing the appearance of an HMD 110 according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of the functional configuration of an HMD 110 according to the present embodiment. [Diagram 5] 2 shows a schematic diagram of the external appearance of a controller 210 according to the present embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of the functional configuration of a controller 210 according to the present embodiment. [Figure 7] FIG. 3 is a diagram illustrating an example of a functional configuration of an image generating device 310 according to the present embodiment. [Figure 8] FIG. 11 is a diagram for explaining the purchase of a parameter set from a parameter list 6 in the animation production system 300 of this embodiment. [Figure 9] FIG. 11 is a diagram for explaining the purchase of a parameter set from a parameter list 6 in the animation production system 300 of this embodiment. [Figure 10] FIG. 11 is a diagram for explaining the purchase of a parameter set from a parameter list 6 in the animation production system 300 of this embodiment. [Figure 11]FIG. 13 is a diagram illustrating a retained parameter set from a parameter list 6 in the animation production system 300 of this embodiment. [Figure 12] FIG. 13 is a diagram illustrating a retained parameter set from a parameter list 6 in the animation production system 300 of this embodiment. [Figure 13] FIG. 13 is a diagram illustrating a retained parameter set from a parameter list 6 in the animation production system 300 of this embodiment. [Figure 14] FIG. 13 is a diagram illustrating a retained parameter set from a parameter list 6 in the animation production system 300 of this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The present invention will be described below with reference to the preferred embodiments thereof.

[0011] [Item 1] a virtual camera for photographing a character placed in a virtual space; a parameter list displaying a list of parameter sets available for the image capturing in the virtual space; applying a parameter set selected from the parameter list to the virtual space and performing the photographing; An animation production system comprising: [Item 2] The animation production system according to item 1, The selected parameter set relates to the positioning of the camera and a 3DCG model of a character or object; An animation production system that features: [Item 3] The animation production system according to item 1, the selected parameter set relates to a position of the camera and a background model; An animation production system that features: [Item 4] The animation production system according to item 1, the selected parameter set is related to an effect of the camera image; An animation production system that features: [Item 5] The animation production system according to item 1, When the parameter set is selected, an image or video to which the parameter set is applied is displayed; An animation production system that features: [Item 6] The animation production system according to item 1, The parameter set in the parameter list is provided from a server connected via a network; An animation production system that features: [Item 7] The animation production system according to item 1, The parameter sets in the parameter list are registered by a user; An animation production system that features:

[0012] A specific example of an animation production system 300 according to an embodiment of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. In the following description, the same elements in the description of the drawings will be given the same reference numerals, and duplicated descriptions will be omitted.

[0013] <Summary> FIG. 1 is a diagram showing an example of a virtual space displayed on a head mounted display (HMD) worn by a user in an animation production system 300 of this embodiment. In the animation production system 300 of this embodiment, a virtual character 4, a virtual camera 3, a background model 5, and objects (e.g., objects such as a chair or a desk, not shown) are arranged in a virtual space 1, and the character 4, the background model 5, and the objects are photographed using the camera 3. Also, a cameraman 2 (photographer character) is arranged in the virtual space 1, and the camera 3 is virtually operated by the cameraman 2. In the animation production system 300 of this embodiment, a user creates animation while overlooking the virtual space 1 from a TPV (Third Person View) as shown in FIG. 1, placing the character 4 and the camera 3, photographing the character 4 from a FPV (First Person View) as the cameraman 2, and acting as the character 4 from the FPV. Furthermore, multiple characters 4 (characters 4-1 and 4-2 in the example of FIG. 1) can be placed in the virtual space 1, and the user can perform an action while possessing the characters 4. When multiple characters 4 are placed, the user can also switch the object possessed by each character 4 (for example, characters 4-1 and 4-2). That is, in the animation production system 300 of this embodiment, one person can play multiple roles. Furthermore, since the user can shoot while virtually operating the camera 3 as the cameraman 2, it is possible to realize natural camera work and enrich the expression of the video to be shot.

[0014] <Overall composition> FIG. 2 is a diagram showing an example of the overall configuration of an animation production system 300 according to an embodiment of the present invention. The animation production system 300 can be configured to include, for example, the HMD 110, the controller 210, and an image generation device 310 functioning as a host computer. An infrared camera (not shown) for detecting the position, orientation, inclination, etc. of the HMD 110 and the controller 210 can also be added to the animation production system 300. These devices can be connected to each other by wired or wireless means. For example, each device can be provided with a USB port and communication can be established by connecting with a cable, or communication can be established by wired or wireless means such as HDMI, wired LAN, infrared, Bluetooth (registered trademark), WiFi (registered trademark), etc. The image generation device 310 may be any device having a calculation processing function, such as a PC, a game machine, or a mobile communication terminal.

[0015] <hmd110> Fig. 3 is a schematic diagram showing the external appearance of the HMD 110 according to this embodiment. Fig. 4 is a diagram showing an example of the functional configuration of the HMD 110 according to this embodiment.

[0016] The HMD 110 is attached to the user's head and includes a display panel 120 arranged in front of the user's left and right eyes. The display panel may be an optically transmissive or non-transmissive display, but in this embodiment, a non-transmissive display panel capable of providing a greater sense of immersion is illustrated. The display panel 120 displays a left-eye image and a right-eye image, and a stereoscopic image can be provided to the user by utilizing the parallax between the two eyes. If it is possible to display a left-eye image and a right-eye image, it is possible to provide a left-eye display and a right-eye display separately, or to provide an integrated display for the left and right eyes.

[0017] The housing 130 of the HMD 110 includes a sensor 140. The sensor 140 may include, for example, a magnetic sensor, an acceleration sensor, or a gyro sensor, or a combination of these, though not shown, in order to detect movements such as the orientation and tilt of the user's head. When the vertical direction of the user's head is the Y axis, an axis that connects the center of the display panel 120 and the user and corresponds to the front-back direction of the user among the axes perpendicular to the Y axis is the Z axis, and an axis that is perpendicular to the Y axis and corresponds to the left-right direction of the user is the X axis, the sensor 140 can detect a rotation angle around the X axis (so-called pitch angle), a rotation angle around the Y axis (so-called yaw angle), and a rotation angle around the Z axis (so-called roll angle).

[0018] Instead of or in addition to the sensor 140, the housing 130 of the HMD 110 may be provided with a plurality of light sources 150 (e.g., infrared LEDs, visible LEDs). A camera (e.g., infrared camera, visible light camera) installed outside the HMD 110 (e.g., indoors, etc.) detects these light sources, thereby making it possible to detect the position, orientation, and inclination of the HMD 110 in a specific space. Alternatively, for the same purpose, the HMD 110 may be provided with a camera for detecting the light sources installed in the housing 130 of the HMD 110.

[0019] The housing 130 of the HMD 110 may also be equipped with an eye tracking sensor. The eye tracking sensor is used to detect the gaze direction and gaze point of the left and right eyes of the user. There are various possible methods for the eye tracking sensor, but for example, a method is possible in which the position of reflected light on the cornea created by irradiating the left and right eyes with weak infrared light is used as a reference point, the gaze direction is detected based on the position of the pupil relative to the position of the reflected light, and the intersection of the gaze directions of the left and right eyes is detected as the gaze point.

[0020] <Controller 210> Fig. 5 is a schematic diagram showing the external appearance of the controller 210 according to this embodiment. Fig. 6 is a diagram showing an example of the functional configuration of the controller 210 according to this embodiment.

[0021] The controller 210 can support the user in performing a predetermined input in a virtual space. The controller 210 can be configured as a set of a left-hand controller 220 and a right-hand controller 230. The left-hand controller 220 and the right-hand controller 230 can each have an operation trigger button 240, an infrared LED 250, a sensor 260, a joystick 270, and a menu button 280.

[0022] Operation trigger buttons 240 are arranged as 240a and 240b at positions that anticipate an operation of pulling the trigger with the middle finger and index finger when gripping grip 235 of controller 210. Frame 245 formed in a ring shape downward from both sides of controller 210 is provided with multiple infrared LEDs 250, and by detecting the positions of these infrared LEDs with a camera (not shown) provided outside the controller, the position, orientation, and tilt of controller 210 in a specific space can be detected.

[0023] The controller 210 may also include a sensor 260 for detecting movements such as the direction and tilt of the controller 210. Although not shown, the sensor 260 may include, for example, a magnetic sensor, an acceleration sensor, or a gyro sensor, or a combination of these. Furthermore, the controller 210 may include a joystick 270 and a menu button 280 on the upper surface thereof. The joystick 270 may be moved in 360 degrees around a reference point, and is assumed to be operated by the thumb when the grip 235 of the controller 210 is held. The menu button 280 is also assumed to be operated by the thumb in the same manner. Furthermore, the controller 210 may also include a vibrator (not shown) for applying vibrations to the hand of the user operating the controller 210. The controller 210 has an input / output unit and a communication unit in order to output information such as the contents of a user's input via a button or joystick, and the position, direction, and tilt of the controller 210 via a sensor, etc., and to receive information from a host computer.

[0024] Depending on whether the user is holding controller 210 and operating various buttons and joysticks, and on information detected by infrared LEDs and sensors, the system determines the movement and posture of the user's hands and can simulate the display and movement of the user's hands in the virtual space.

[0025] <Image Generating Device 310> 7 is a diagram showing a functional configuration of an image generating device 310 according to this embodiment. As the image generating device 310, a device such as a PC, a game machine, or a mobile communication terminal having a function of storing user input information transmitted from the HMD 110 or the controller 210 and information on the user's head movement or controller movement or operation acquired by a sensor or the like, performing a predetermined calculation process, and generating an image can be used. The image generating device 310 can include, for example, an input / output unit 320 for establishing a wired connection with peripheral devices such as the HMD 110 or the controller 210, and can include a communication unit 330 for establishing a wireless connection such as infrared, Bluetooth (registered trademark), or WiFi (registered trademark). Information on the movement of the user's head and the movement and operation of the controller received from the HMD 110 and / or the controller 210 via the input / output unit 320 and / or the communication unit 330 is detected as input contents including the user's position, gaze, posture, and other movements, speech, operation, etc., in the control unit 340, and processing such as controlling the character 4 and generating an image is performed by executing a control program stored in the storage unit 350 according to the user's input contents. The control unit 340 can be configured with a CPU, but by further providing a GPU specialized for image processing, information processing and image processing can be distributed to improve the efficiency of the overall processing. The image generating device 310 can also communicate with other calculation processing devices and have the other calculation processing devices share the information processing and image processing.

[0026] The control unit 340 includes a user input detection unit 410 that detects information on the user's head movement, the user's speech, and the movement and operation of the controller received from the HMD 110 and / or the controller 210, a character control unit 420 that executes a control program stored in a control program storage unit 460 for a character 4 stored in a character data storage unit 450 of the memory unit 350, a camera control unit 440 that controls a virtual camera 3 arranged in the virtual space 1 according to the character control, and an image generation unit 430 that generates an image of the virtual space 1 captured by the camera 3 based on the character control. Here, the control of the movement of the character 4 is realized by converting information such as the direction and inclination of the user's head and the movement of the hands detected via the HMD 110 and the controller 210 into the movement of each part of a bone structure created in accordance with the movement and constraints of the joints of the human body, and applying the movement of the bone structure by associating the bone structure with the character data stored in advance. The camera 3 is controlled, for example, by changing various settings for the camera 3 (for example, the position of the camera 3 in the virtual space 1, the line of sight direction of the camera 3, the focus position, zoom, etc.) in accordance with the hand movements of the character 4.

[0027] The memory unit 350 stores, in the above-mentioned character data storage unit 450, image data of the character 4 as well as information related to the character 4, such as the attributes of the character 4. Furthermore, the control program storage unit 460 stores programs for controlling the movements and expressions of the character 4 in the virtual space and for controlling objects such as the camera 3. The image data storage unit 470 stores images generated by the image generation unit 430. The background data storage unit 480 stores information related to the background model 5. The parameter data storage unit 490 stores parameter sets purchased on a parameter list or parameter sets registered by the user.

[0028] Figures 8-14 are diagrams for explaining the purchase of a parameter set from the parameter list 6 in the animation production system 300 of this embodiment. Note that the parameter types and the like included in the parameter list of this embodiment are not limited to those shown in Figure 8 and the like, and may be configured to include one or more of any of these, or may include other parameter types.

[0029] 8, the user opens the parameter list 6 in the virtual space 1 and selects, for example, a parameter set (e.g., Parameter CC2) related to the arrangement of the camera 3 and the character 4 in the parameter shop list. The parameter list 6 includes parameter type tabs 61, parameter items 62, a scroll bar 63, and buttons 64.

[0030] The parameter type tab 61 may include, for example, a placement parameter (Camera × Character) between the camera 3 and the character 4, a placement parameter (Character × Background) between the character 4 and the background model 5, and an effect setting parameter (Effect) as illustrated in FIG. 8. The effects may include, for example, a bloom effect, a depth of field effect, a vignetting effect, a color grading effect, a color curve effect, and a diffusion filter. The effect setting parameters may include, for example, parameters such as size and color. Note that the parameter types are not limited to these, and may be any type having parameters, such as a placement parameter between the camera 3 and an object, a placement parameter between the character 4 and an object, and settings of the camera 3 itself (for example, zoom magnification).

[0031] The parameter items 62 are changed according to the selection of the parameter type tab 61, and a list corresponding to each type is displayed. The items displayed in this list may be data acquired from a server connected via a network such as a cloud server. In FIG. 8, a list of placement parameters for the camera 3 and the character 4 is illustrated, and as an example, a composition sample image and a name are written in each item, but this is not limited to this, and for example, the price and composition setting may be displayed. In addition, when there are many display items, for example, all items may be made visible by sliding the scroll bar 63, and sorting may be possible in alphabetical order, price order, etc. Note that each item is not limited to a single parameter set, and may be sold as a set of multiple parameters of the same or different parameter types.

[0032] Buttons 64 can be arranged for various purposes, and in FIG. 8, a parameter set purchase button (SHOP), a list of owned parameter sets (MY LIST), and a button (BACK) for returning to the previous window display are exemplified.

[0033] Next, as shown in FIG. 9, when the user selects one of the parameter items 62 with the virtual right hand 21R in the virtual space 1, for example, item data 7 (e.g., a detailed description of the item, the purchase price, etc.) may be displayed.

[0034] Then, for example, after checking the contents of the item data 7, the purchase procedure may be carried out by pressing a purchase decision button (BUY) on the item data 7 or a button on the controller 210 to which a decision action is assigned.

[0035] As described above, the purchase of a parameter set has been described with reference to FIGS. 8 and 9, but rental can be configured in a substantially similar manner, except that the period of use after payment is limited.

[0036] Next, in the parameter list 6, parameters related to camera work may be provided. As shown in FIG. 10, the user can open the parameter list 6 in the virtual space 1 and select a parameter set (for example, Parameter CC2) related to the arrangement of the camera 3 and the character 4, for example, in the retained parameter set list. In the parameter set, a fixed position of the camera 3 may be set, or a change in position (including dolly and track, and does not have to be horizontal movement) can also be set assuming that the camera 3 moves. In addition, functions of the camera 3, such as pan, tilt, roll, and zoom, can also be set in the parameter set.

[0037] Next, as shown in FIG. 11, when the user selects one of the parameter items 62 with the virtual right hand 21R in the virtual space 1, for example, item data 7 (for example, a detailed explanation about the item) may be displayed.

[0038] Then, for example, after checking the contents of the item data 7, the selected parameter set may be applied to the virtual space 1 by pressing a confirmation button (OK) on the item data 7 or a button on the controller 210 to which a confirmation action is assigned. An example of the arrangement after application is shown in Fig. 12. It is also possible to register a parameter set created by the creator in the retained parameter list.

[0039] 13 illustrates an example of a list of parameter sets held by an effect, and shows a flare parameter set (e.g., size and color of the flare) as an example. Then, for example, as shown in FIG. 14, when a user selects one of the parameter items 62 with virtual right hand 21R in virtual space 1, sample window 8 may open in virtual space 1, and a video from camera 3 to which the selected effect has been applied may be displayed. Sample window 8 may also be displayed when effect setting parameters are purchased, etc., making it possible to check the video or image after the effect has been applied before purchase, etc.

[0040] In this embodiment, the effect is applied to the rendered moving image, but the effect that can be arranged in the virtual space 1 may be presented in the parameter list. For example, fog material, flare material, rain material, etc. can be presented in the parameter list 6 as a set of objects and effects that define their appearance or behavior. In this case, instead of the flare parameter set shown in FIG. 13, a parameter set (including an object) for the flare material can be presented in the parameter list. Then, as shown in FIG. 14, when the user selects one of the parameter items 62 with the virtual right hand 21R in the virtual space 1, for example, an object related to the flare material is placed in the virtual space 1, and the object can bring about a flare effect in the virtual space 1.

[0041] The parameter set can also include lighting-related parameters, such as where to place the light source (light), the color tone of the light, the brightness setting and brightness change (like a strobe light), and the light direction setting and change.

[0042] According to the animation production system 300 of this embodiment, the producer can obtain various parameter sets without having to prepare them himself, which can reduce the time spent considering the composition of a scene. In addition, it becomes possible to shoot with compositions that the producer had not anticipated, which broadens the range of expression of animated videos. Therefore, it is possible to reduce the burden of animation production and provide richer expression to animated videos.

[0043] For parameters that change over time, such as the camera 3, lighting, and character 4 movements, the parameter set can include time information. The time information can be a relative time from the start point (e.g., t = +10 seconds). The parameter change over time can be specified by changing the parameter linearly or according to a function corresponding to a predetermined time, including a first setting value at the start point of the change and a second setting value at the end point of the change. The user can also use multiple parameter sets in conjunction. For parameter sets that include a time-varying element, the start and end points of the change can be changed individually, or the time required for the change can be specified and the parameters can be automatically corrected so that the change occurs at the specified time.

[0044] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention, and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents thereof are also included in the present invention.

[0045] For example, in the present embodiment, the image generating device 310 is one computer, but this is not limiting, and the HMD 110 or the controller 210 may be provided with all or part of the functions of the image generating device 310. Also, some of the functions of the image generating device 310 may be provided in another computer communicably connected to the image generating device 310.

[0046] In addition, in this embodiment, a virtual space using virtual reality (VR) is assumed, but this is not limited to this, and the animation production system 300 of this embodiment can be directly applied to an augmented reality (AR) space or a mixed reality (MR) space. [Explanation of symbols]

[0047] 1. Virtual Space 2. Photographer 3. Camera 4. Characters 110 HMD 120 Display Panel 130 Housing 140 Sensors 150 light source 210 Controller 220 Left-handed controller 230 Right Hand Controller 235 Grip 240 Trigger Button 250 Infrared LEDs 260 Sensors 270 Joystick 280 Menu button 300 Animation Production System 310 Image generating device 320 Input / output section 330 Communications Department 340 Control Unit 350 Storage section 410 User Input Detection Unit 420 Character Control Unit 430 Image Generation Unit 440 Camera Control Unit 450 Character data storage section 460 Program Storage Unit 470 Image data storage unit

Claims

1. a virtual camera for photographing a character placed in a virtual space; a parameter list displaying a list of parameter sets available for the image capturing in the virtual space; applying a parameter set selected from the parameter list to the virtual space and performing the photographing; An animation production system comprising:

2. The animation production system according to claim 1, the selected parameter set relates to a position of the camera and a 3DCG model of a character or object; An animation production system that features:

3. The animation production system according to claim 1, the selected parameter set relates to a position of the camera and a background model; An animation production system that features:

4. The animation production system according to claim 1, the selected parameter set is related to an effect of the camera image; An animation production system that features:

5. The animation production system according to claim 1, When the parameter set is selected, an image or video to which the parameter set is applied is displayed; An animation production system that features:

6. The animation production system according to claim 1, The parameter set in the parameter list is provided from a server connected via a network; An animation production system that features:

7. The animation production system according to claim 1, The parameter sets in the parameter list are registered by a user; An animation production system that features:

Images