Animation creation system

Abstract

To allow for animation filming in a virtual space.SOLUTION: An animation creation method includes: arranging first and second objects and a virtual camera in a virtual space; controlling motion of the first object in accordance with operation from a first user; controlling motion of the second object in accordance with operation from a second user; controlling the camera in accordance with operation from the first or second user or operation from the second user; and imaging the first and second objects.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an animation production system.

Background Art

[0002] A virtual camera is placed in a virtual space (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There has been no attempt to shoot an animation in a virtual space.

[0005] The present invention has been made in view of such a background, and an object thereof is to provide a technique capable of shooting an animation in a virtual space.

Means for Solving the Problems

[0006] The main invention of the present invention for solving the above problems is an animation production method, comprising arranging a first and a second object and a virtual camera in a virtual space, controlling the movement of the first object in response to an operation from a first user, controlling the movement of the second object in response to an operation from a second user, and controlling the camera in response to an operation from the first or second user or the second user to shoot the first and second objects.

[0007] Regarding other problems disclosed in the present application and their solutions, they will be clarified by the embodiments of the invention and the drawings.

Effects of the Invention

[0008] According to the present invention, an animation can be photographed in a virtual space.

Brief Description of Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0010] The content of the embodiment of this invention is listed and demonstrated. The animation production method by the embodiment of this invention has the following structures.

[0011] [Item 1] A method for producing an animation, comprising: placing a first object, a second object, and a virtual camera in a virtual space; controlling the movement of the first object according to an operation from a first user; controlling the movement of the second object according to an operation from a second user; controlling the camera according to an operation from the first or second user or the second user to photograph the first and second objects; A method for producing an animation, characterized by the above. [Item 2] The method for producing an animation according to Item 1, comprising: recording the movement of the first object on a first track; controlling the movement of the second object while playing back the movement of the first object recorded on the first track; A method for producing an animation, characterized by the above. [Item 3] The method for producing an animation according to Item 1, comprising: the second object is the same object as the first object; recording the movement of the first object on a first track; changing the movement of the first object according to an operation of the second user while playing back the movement of the first object recorded on the first track; A method for producing an animation, characterized by the above.

[0012] A specific example of an animation production system according to an embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. In the following description, the same reference numerals are given to the same elements in the description of the drawings, and duplicate descriptions are omitted.

[0013] <Summary> FIG. 1 is a diagram showing an example of a virtual space displayed on a head-mounted display (HMD) worn by a user in the animation production system of the present embodiment. In the animation production system of the present embodiment, a character 4 and a camera 3 are arranged in a virtual space 1, and the character 4 is photographed using the camera 3. Also, a cameraman 2 is arranged in the virtual space 1, and the camera 3 is virtually operated by the cameraman 2. In the animation production system of the present embodiment, the user arranges the character 4 and the camera 3 while overlooking the virtual space 1 in a TPV (Third Person View), photographs the character 4 as a cameraman 2 in an FPV (First Person View), and performs the performance of the character 4 in the FPV to produce an animation. Also, a plurality of characters (in the example of FIG. 1, character 4 and character 5) can be arranged in the virtual space 1, and the user can perform a performance while possessing the character 4 and the character 5 respectively. That is, in the animation production system of the present embodiment, one person can play multiple roles. Also, since shooting can be performed while virtually operating the camera 3 as a cameraman 2, it is possible to realize natural camera work and enrich the expression of the video to be shot. Also, in the animation production system of the present embodiment, a plurality of users can collaborate to produce an animation. For example, one user can operate the character 4 to perform a performance, and another user can operate the cameraman 2 to photograph the performance of the character 4 with the camera 3. Furthermore, it is possible to share the work required for one animation by having a lighting engineer who performs lighting and a mixer who performs audio mixing performed by third and fourth users. In addition, in the animation production system of the present embodiment, these operations can be performed asynchronously.That is, after the first user operates character 4 to perform a performance, the second user operates cameraman 2 to take pictures in one or more cuts while changing the angle of camera 3, and then, the lighting engineer performs lighting while adjusting the color, position, lighting timing, etc. of the lights, so that it is also possible to perform the shooting work asynchronously.

[0014] <Overall Configuration> FIG. 2 is a diagram showing an example of the overall configuration of an animation production system 300 according to an embodiment of the present invention. The animation production system 300 can be configured to include a plurality of character operation systems 350 and an image generation device 310 that functions as a host computer. Further, the character operation system 350 can be configured to include, for example, an HMD 110, a controller 210, and a control device 310 that controls these. An infrared camera (not shown) or the like for detecting the position, orientation, inclination, etc. of the HMD 110 and the controller 210 can also be added to the character operation system 350. These devices can be connected to each other by wired or wireless means. For example, each device can be provided with a USB port and communication can be established by connecting with a cable. Alternatively, communication can also be established by wire or wirelessly such as HDMI, wired LAN, infrared, Bluetooth (registered trademark), WiFi (registered trademark), etc. The control device 350 may be a device having a calculation processing function such as a PC, a game machine, a mobile communication terminal, etc. The control device 350 is communicably connected to the image generation device 310 and can transmit input information from the user by the HMD 110 and the controller 210 to the image generation device 310.

[0015] <hmd110> FIG. 3 shows a schematic view of the appearance of a head-mounted display (hereinafter: HMD) 110 according to the present embodiment. FIG. 5 shows a functional configuration diagram of the HMD 110 according to the present embodiment. The HMD 110 is mounted on the user's head and includes a display panel 120 disposed in front of the user's left and right eyes. As the display panel, an optically transmissive type and a non-transmissive type display are conceivable, but in the present embodiment, a non-transmissive type display panel that can provide a greater sense of immersion is exemplified. On the display panel 120, a left-eye image and a right-eye image are displayed, and by utilizing the parallax of both eyes, a stereoscopic image can be provided to the user. If the left-eye image and the right-eye image can be displayed, it is possible to separately provide a left-eye display and a right-eye display, or it is also possible to provide an integrated display for both the left eye and the right eye.

[0016] The housing portion 130 of the HMD 110 includes a sensor 140. Although not shown, the sensor 140 can include, for example, any one of a magnetic sensor, an acceleration sensor, or a gyro sensor, or a combination thereof, in order to detect the user's head movements such as the orientation and tilt of the head. When the vertical direction of the user's head is defined as the Y-axis, and among the axes orthogonal to the Y-axis, the axis corresponding to the user's front-rear direction that connects the center of the display panel 120 and the user is defined as the Z-axis, and the axis orthogonal to the Y-axis and the Z-axis and corresponding to the user's left-right direction is defined as the X-axis, the sensor 140 can detect the rotation angle around the X-axis (so-called pitch angle), the rotation angle around the Y-axis (so-called yaw angle), and the rotation angle around the Z-axis (so-called roll angle).

[0017] Instead of or in addition to the sensor 140, the housing portion 130 of the HMD 110 can also include a plurality of light sources 150 (e.g., infrared light LEDs, visible light LEDs). By a camera (e.g., an infrared light camera, a visible light camera) installed outside the HMD 110 (e.g., in a room, etc.) detecting these light sources, the position, orientation, and tilt of the HMD 110 in a specific space can be detected. Alternatively, for the same purpose, the HMD 110 can also include a camera for detecting the light sources installed in the housing portion 130 of the HMD 110.

[0018] The housing portion 130 of the HMD 110 can also be provided with an eye tracking sensor. The eye tracking sensor is used to detect the line-of-sight directions and fixation points of the user's left and right eyes. Although various methods can be considered as the eye tracking sensor, for example, a method in which weak infrared light is irradiated onto the left and right eyes, the position of the reflected light on the cornea thus formed is used as a reference point, the line-of-sight direction is detected based on the position of the pupil relative to the position of the reflected light, and the intersection of the line-of-sight directions of the left and right eyes is detected as the fixation point, etc. can be considered.

[0019] <Controller 210> FIG. 4 shows a schematic diagram of the appearance of the controller 210 according to the present embodiment. FIG. 6 shows a functional configuration diagram of the controller 210 according to the present embodiment. The controller 210 can support a user to perform a predetermined input in the virtual space. The controller 210 can be configured as a set of a left-hand controller 220 and a right-hand controller 230. The left-hand controller 220 and the right-hand controller 230 can each have an operation trigger button 240, an infrared LED 250, a sensor 260, a joystick 270, and a menu button 280.

[0020] The operation trigger buttons 240 are arranged as 240a and 240b at positions assuming an operation of pulling the trigger with the middle finger and index finger when gripping the grip 235 of the controller 210. The frame 245 formed in a ring shape downward from both side surfaces of the controller 210 is provided with a plurality of infrared LEDs 250, and the position, orientation, and inclination of the controller 210 in a specific space can be detected by detecting the positions of these infrared LEDs with a camera (not shown) provided outside the controller.

[0021] In addition, the controller 210 can incorporate a sensor 260 to detect operations such as the orientation and inclination of the controller 210. Although not shown, the sensor 260 can be, for example, any one of a magnetic sensor, an acceleration sensor, or a gyro sensor, or a combination thereof. Further, a joystick 270 and a menu button 280 can be provided on the upper surface of the controller 210. The joystick 270 can be moved 360 degrees in all directions around a reference point and is assumed to be operated with the thumb when the grip 235 of the controller 210 is held. Similarly, the menu button 280 is also assumed to be operated with the thumb. Further, the controller 210 can also incorporate a vibrator (not shown) for vibrating the hand of the user who operates the controller 210. In order to output information such as the content of the user's input via buttons and the joystick and the position, orientation, and inclination of the controller 210 via sensors, etc., and to receive information from the host computer, the controller 210 has an input / output unit and a communication unit.

[0022] Based on whether the user holds the controller 210 and operates various buttons and the joystick, and the information detected by the infrared LED and the sensor, the system can determine the movement and posture of the user's hand and can virtually display and move the user's hand in the virtual space.

[0023] <Control device 350> FIG. 7 shows a functional configuration diagram of the control device 350 according to the present embodiment. As the control device 350, devices such as a PC, a game console, and a mobile communication terminal can be used, which have a function of storing information on user input, head movement of the user, movement and operation of the controller, etc. obtained from the HMD 110 and the controller 210 or sensors, performing predetermined calculation processing, and generating an image. The control device 350 can include, for example, an input / output unit 351 for establishing a wired connection with peripheral devices such as the HMD 110 and the controller 210, and can include a communication unit 352 for establishing a wireless connection such as infrared, Bluetooth (registered trademark), or WiFi (registered trademark). Information on the movement of the user's head, the movement and operation of the controller received from the HMD 110 and / or the controller 210 via the input / output unit 351 and / or the communication unit 352 is detected in the control unit 353 as input contents including movements such as the user's position, line of sight, and posture, speech and pronunciation, and operations. The control unit 353 can be configured by a CPU, but by further providing a GPU specialized for image processing, information processing and image processing can be decentralized, and the efficiency of the overall processing can be improved. The control unit 353 includes a user input detection unit 354 that detects information (input information) on the movement of the user's head, the user's speech and pronunciation, and the movement and operation of the controller received from the HMD 110 and / or the controller 210. These input information are transmitted to the image generation device 310 after passing through the communication unit 330. The communication unit 330 also receives information for reproducing the virtual space 1 from the image generation device 310 (which may be rendered video data or model information of objects arranged in the virtual space). Based on this information, the virtual space display unit 355 can display the virtual space 1 on the HMD 110.

[0024] <Image generation device 310> FIG. 8 shows a functional configuration diagram of the image generation device 310 according to the present embodiment. The image generation device 310 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing. When the image generation device 310 includes peripheral devices such as the HMD 110 and the controller 210, for example, it can include an input / output unit 320 for establishing a connection with the HMD 110 and the controller 210, and can include a communication unit 330 for establishing a connection by wire such as Ethernet (registered trademark) or a public telephone line or wirelessly such as infrared, Bluetooth (registered trademark), or WiFi (registered trademark). Information (input information) regarding the movement of the user's head, the movement and operation of the controller, which is input from the HMD 110 and / or the controller 210 in the character operation system 360, is transmitted to the control unit 340 via the input / output unit 320 and / or the communication unit 330. In the control unit 340, it is detected as input content including operations such as the user's position, line of sight, and posture, speech and pronunciation, and operations. According to the input content of the user, by executing the control program stored in the storage unit 350, processing such as controlling the character and generating an image is performed. The control unit 340 can be configured by a CPU, but by further providing a GPU specialized for image processing, information processing and image processing can be decentralized, and the efficiency of the overall processing can be improved. The image generation device 310 can also communicate with other calculation processing devices and share information processing and image processing with other calculation processing devices.

[0025] The control unit 340 includes a user input receiving unit 410 that acquires input information from the user, a character control unit 420 that executes the control program stored in the control program storage unit 520 for the characters stored in the character data storage unit 510 of the storage unit 350, and an image generation unit 430 that generates an image based on the character control. Here, regarding the control of the movement of the character, input information such as the orientation and inclination of the user's head and the movement of the hand detected via the HMD 110 and the controller 210 is converted into the movement of each part of the bone structure created in accordance with the movement and constraints of the joints of the human body, and the movement of the bone structure is applied by associating the bone structure with the pre-stored character data, which can be realized. The character control unit 420 can operate various objects in the virtual space 1 such as the character 4 and the camera 3 in response to operations by a plurality of users received from a plurality of character operation systems 350. For example, the movement of the character 4 is controlled based on the input information from the first user received from the first character operation system 350, and the camera 3 is controlled by operating the cameraman 2 based on the input information from the second user received from the second character operation system 350. Thereby, a division of labor can be performed in which the first user performs the performance of the character 4 and the second user shoots it. Furthermore, the control unit 340 includes a recording / playback execution unit 440 that records and plays back the generated character image on a track, and an editing execution unit 450 that edits each track and generates the final content.

[0026] The storage unit 350 has a character data storage unit 510 that stores information related to characters such as the attribute of the character in addition to the image data of the character. The control program storage unit 520 stores a program for controlling the movement and expression of the character in the virtual space. The storage unit 350 also has a track storage unit 530 that stores action data composed of parameters for controlling the movement of the character in the moving image generated by the image generation unit 430.

[0027] FIG. 9 is a flowchart for explaining an example of track generation processing according to an embodiment of the present invention.

[0028] First, the recording / reproducing execution unit 440 of the control unit 340 of the image generation apparatus 310 starts recording for storing action data of a moving image related to the operation by the first character in the virtual space in the first track of the track storage unit 530 (S101). Here, it is also possible to set the position of the camera that shoots the character and the viewpoint of the camera (for example, FPV, TPV, etc.). For example, in the virtual space 1 shown in FIG. 1, for the character 4 corresponding to the first character, the position where the cameraman 2 is arranged and the angle of the camera 3 can be set. The recording start operation may be instructed by a remote controller such as the controller 210, or may be instructed by another terminal. Further, this operation may be performed by a user who plays the character, wears the HMD 110, and operates the controller 210, or may be performed by a user other than the user who plays the character. Further, the recording process may be automatically started based on detecting the operation by the user who plays the character described later.

[0029] Subsequently, the user input detection unit 410 of the control unit 340 detects information regarding the movement of the user's head, the user's speech or pronunciation, and the movement and operation of the controller received from the HMD 110 and / or the controller 210 (S102). For example, when a user wearing the HMD 110 tilts their head, the sensor 140 provided in the HMD 110 detects the tilt and transmits information regarding the tilt to the image generation device 310. The image generation device 310 receives information regarding the user's movement via the communication unit 330, and the user input detection unit 410 detects the movement of the user's head based on the received information. Also, when the user performs a predetermined movement or operation such as lifting the controller 210 or pressing a button using the controller 210, the sensor 260 provided in the controller detects the movement and / or operation and transmits information regarding the movement and / or operation to the image generation device 310. The image generation device 310 receives information regarding the user's controller movement and operation via the communication unit 330, and the user input detection unit 410 detects the user's controller movement and operation based on the received information.

[0030] Subsequently, the character control unit 420 of the control unit 340 controls the movement of the first character in the virtual space based on the detected user movement (S103). For example, based on detecting that the user tilts their head, the character control unit 420 controls the first character to tilt its head. Also, based on detecting that the user lifts the controller and presses a predetermined button on the controller, the character control unit 420 controls the first character to stretch its arm upward and grab something. In this way, every time the user input detection unit 410 detects a user movement transmitted from the HMD 110 or the controller 210, the character control unit 420 controls the first character to perform the corresponding movement. The operations detected by the user input detection unit 410 and / or the parameters related to the operations are stored in the first track of the track storage unit 530. Also, without depending on user input, it is possible to control the character to execute the movements of a predetermined performance, and it is possible to store the action data related to the movements of this predetermined performance in the first track, or it is also possible to store both the action data related to the user's movement and the predetermined movement.

[0031] Subsequently, the recording and playback execution unit 440 checks whether it has received an instruction to end the recording from the user (S104). When it has received an instruction to end the recording, it completes the recording of the first track related to the first character (S105). The recording and playback execution unit 440 continues the recording process unless it receives an instruction to end the recording from the user. Here, the recording and playback execution unit 440 can also automatically perform a process to complete the recording when it stops detecting the movements of the user who plays the character. Also, instead of receiving an instruction from the user, it is possible to execute the process of ending the recording at a predetermined time by activating a timer.

[0032] Subsequently, the recording and playback execution unit 440 starts recording to store the action data of the moving image related to the operation by the second character in the virtual space in the second track of the track storage unit 530 (S106). Here, it is also possible to set the position of the camera that shoots the character and the viewpoint of the camera (for example, FPV, TPV, etc.). For example, in the virtual space 1 shown in FIG. 1, for the character 5 corresponding to the second character, the position where the cameraman 2 is arranged and the angle of the camera 3 can be set. For example, the user can set the camera viewpoint by FPV of the character 5, perform the playback process of the first track, and while checking the operation of the character 4, the user who plays the character 5 can perform a desired operation. The recording start operation may be instructed by a remote controller such as the controller 210, or may be instructed by another terminal. Further, this operation may be performed by the user who plays the character, wearing the HMD 110 and operating the controller 210, or may be performed by a user other than the user who plays the character. Further, the recording process may be automatically started based on detecting the operation and / or operation by the user who plays the character described later.

[0033] Subsequently, the user input detection unit 410 of the control unit 340 detects information regarding the movement of the user's head, the user's speech or pronunciation, and the movement and operation of the controller received from the HMD 110 and / or the controller 210 (S107). Here, the user may be the same user as the user who plays the above-described first character, or may be a different user. For example, when the user wearing the HMD 110 tilts their head, the sensor 140 provided in the HMD 110 detects the tilt and transmits information regarding the tilt to the image generation device 310. The image generation device 310 receives information regarding the user's movement via the communication unit 330, and the user input detection unit 410 detects the movement of the user's head based on the received information. Further, when the user performs a predetermined movement or operation such as lifting the controller 210 and shaking it left and right or pressing a button using the controller 210, the sensor 260 provided in the controller detects the movement and / or operation and transmits information regarding the movement and / or operation to the image generation device 310. The image generation device 310 receives information regarding the movement and operation of the user's controller via the communication unit 330, and the user input detection unit 410 detects the movement and operation of the user's controller based on the received information.

[0034] Subsequently, the character control unit 420 of the control unit 340 controls the movement of the second character in the virtual space based on the detected user movement (S108). For example, based on detecting that the user tilted their head, the character control unit 420 controls the second character to tilt its head. Also, based on detecting that the user lifted the controller and waved it left and right or pressed a predetermined button on the controller, the character control unit 420 controls the second character to wave its arm and open its fingers (i.e., perform the "bye-bye" gesture). In this way, every time the user input detection unit 410 detects a user movement transmitted from the HMD 110 or the controller 210, the character control unit 420 controls the second character to perform the corresponding movement. The operations detected by the user input detection unit 410 and / or the parameters related to the operations are stored in the second track of the track storage unit 530. Also, regardless of user input, it is possible to control the character to execute the movements of a predetermined performance, and it is possible to store the action data related to the movements of this predetermined performance in the second track, or it is possible to store both the action data related to the user's movement and the predetermined movement.

[0035] Subsequently, the recording and playback execution unit 440 checks whether it has received an instruction to end the recording from the user (S109). When it has received an instruction to end the recording, it completes the recording of the second track related to the second character (S110). The recording and playback execution unit 440 continues the recording process unless it receives an instruction to end the recording from the user. Here, the recording and playback execution unit 440 can also perform a process of automatically completing the recording when it stops detecting the movements of the user who plays the character.

[0036] FIG. 10 is a flowchart for explaining an example of the track editing process according to an embodiment of the present invention.

[0037] First, the editing execution unit 450 of the control unit 340 of the image generation device 310 performs a process of editing the first track stored in the track storage unit 530 (S201). For example, the user edits the first track (T1) related to the first character via a user interface for track editing as shown in FIG. 11(a). For example, in the user interface, an area where the first track is stored is displayed along the time series. By selecting a desired bar, the user can render the 3D model of the character based on the parameters of the stored moving image and the character data, and play the moving image of the character (e.g., character 4) arranged in the virtual space as shown in FIG. 1. Note that the user interface for track editing is not limited to the above display. For example, a method of displaying the track name and title (e.g., "first character") in a list format is also conceivable. In addition, as the user interface, it is possible to execute editing while directly playing the moving image of the character without showing the track or list as shown in FIG. 11, or to display the key frames of the moving image, and the user can select the key frame to be edited among them. Also, it is possible to execute a process such as changing only the color of the character selected as the editing target among the 3D characters displayed as the moving image for the time when the color is selected.

[0038] As the editing process, for example, in FIG. 1, it is conceivable to change the arrangement position of character 4 corresponding to the first character in the virtual space 1, change the shooting position and angle of character 4, and change the shooting viewpoints (FPV, TPV). It is also conceivable to change or delete the time scale.

[0039] Also, as an editing process, a part of the operation of Character 4 may be changed. For example, the user can specify a body part to be edited for the first character played on Track (T1), and only the operation of the specified part can be made to operate by an operation using the controller 210 or the HMD 110. For example, the user can specify the arm of the first character, and while playing the operation of the first character, only the arm of the first character can be operated by the operation of the controller 210 to correct the movement of the arm.

[0040] Next, the edit execution unit 450 performs a process of editing the second track stored in the track storage unit 530 (S202). For example, the user selects an option (not shown) to edit the second track (T2) related to the second character via a user interface for track editing, and arranges a bar indicating the area where the second track is stored along the time series via the user interface as shown in FIG. 11(b). As shown in FIG. 11(b), the user can relatively adjust such as synchronizing the playback timings of each track while editing the second track independently of the first track. Similarly, the user can edit other tracks (for example, the third track (T3)).

[0041] As an editing process, for example, in FIG. 1, in addition to changing the arrangement position of Character 5 corresponding to the second character in the virtual space 1, changing the shooting position and angle of Character 5, it is conceivable to change the shooting viewpoints (FPV, TPV). Also, it is conceivable to change or delete the time scale. Here, as an editing process, the user can change the arrangement position of Character 5 relative to Character 4 corresponding to the first character on the virtual space 1.

[0042] Furthermore, as an editing process, by playing each track, a moving image including the actions of the characters corresponding to each track is played in the virtual space, thereby creating an animated moving image realized in the virtual space as a whole. Also, by playing each track, not only can the movements of the characters be reproduced, but also the background set of the characters can be changed, the objects including the characters can be increased or decreased, the lighting settings can be changed, the costumes of the characters can be changed or accessories can be added, the amount and direction of the wind on the characters can be changed, or the characters can be changed while keeping the action data as it is, thereby editing the animated moving image that can be realized in the virtual space.

[0043] After the editing process is completed, the editing execution unit 450 saves the editing content in response to the user's request or automatically (S203).

[0044] Note that the editing processes of S201 and S202 can be executed in any order and can be repeated back and forth. The saving process of S203 can also be executed each time the editing of S201 and S202 is executed. Also, the editing can be ended with only the editing of S201.

[0045] Also, the above-described track generation process can be executed independently for each user. Therefore, after the track generation process by the first user is performed, the track generation process by the second user can be performed. Thus, for example, after the first user operates character 4 to perform an acting motion and stores it as a track in the first track storage unit 530, the second user controls camera 3 by operating cameraman 2 while playing the first track, shoots the acting of character 4 later, and stores the motion of camera 3 (or the motion of cameraman 2) in the second track. Thereby, by executing the first and second tracks, an image of character 4 acted by the first user can be created by the second user.

[0046] Also, the above-described track editing process can be executed independently for each user. Therefore, after the track generation process by the first user is performed, the second user can perform the editing process of the track created by the first user. Thus, the operation of one object can be defined by the operations contributed by the first user and the second user according to their respective areas of expertise. For example, after the first user operates character 4 and records a rapping motion on the track, the second user can operate character 4 to add a dancing motion to be performed between the rapping motions. As a result, even for the same character 4, when performing different motions, by having the user with expertise in the motion perform the operation according to the motion, it becomes possible to perform a high-quality motion as a whole.

[0047] As described above, according to the present embodiment, by applying the multi-track recording (MTR) method to animation production, character motions linked to user motions are stored in each track, and by performing editing work within each track or between tracks, simple and efficient animation production can be realized.

[0048] Although the present embodiment has been described above, the above embodiment is for facilitating the understanding of the present invention and is not for limiting and interpreting the present invention. The present invention can be changed and improved without departing from its gist, and equivalents of the present invention are also included therein.

[0049] For example, in the present embodiment, the track generation method and the editing method have been described by taking a character as an example. However, the method disclosed in the present embodiment can be applied not only to characters but also to objects with motions (vehicles, structures, articles, etc.) including characters.

[0050] Also, for example, in the present embodiment, the image generation device 310 has been described as being separate from the HMD 110. However, all or part of the configuration and functions provided by the image generation device 310 can also be provided in the HMD 110.

Explanation of Reference Numerals

[0051] 1 Virtual space 2 Cameraman 3 Camera 4 Character 5 Character 110 HMD 210 Controller 310 Image generation device

Claims

1. A method of producing animation, comprising: Placing a first object and a second object and a virtual camera in a virtual space; Controlling an action of the first object in response to an operation from a first user; Controlling an action of the second object in response to an operation from a second user; Controlling the camera in response to an operation from the first or second user or a second user to photograph the first and second objects; An animation production method characterized by:

2. 2. The animation production method according to claim 1, recording a motion of the first object on a first track; controlling a motion of the second object while playing back the motion of the first object recorded on the first track; An animation production method characterized by:

3. The animation production method according to claim 1, the second object is the same object as the first object, recording a motion of the first object on a first track; changing a motion of the first object in response to an operation of the second user while playing back the motion of the first object recorded on the first track; An animation production method characterized by:

Images