Image generation device, image generation method, and image generation program

Abstract

An image generation device (1) according to an embodiment is provided with an acceptance unit (30), a first generation unit (32), and a display control unit (35). The acceptance unit (30) accepts a user operation. The first generation unit (32) moves a virtual camera (10) in a three-dimensional virtual space (Sv) on the basis of the user operation accepted by the acceptance unit (30) to thereby generate a camera viewpoint image obtained by photographing a three-dimensional object (100) in the three-dimensional virtual space (Sv) by the virtual camera (10). The display control unit (35) disposes a virtual screen (Sc) for displaying the camera viewpoint image generated by the first generation unit (32) in the three-dimensional virtual space (Sv), and displays a camera trajectory (T) of the virtual camera (10) in the three-dimensional virtual space (Sv).

Description

[Technical field]

[0001] The present invention relates to an image generation device, an image generation method, and an image generation program. [Background technology]

[0002] There is a technique for generating video content in which a space is viewed from any viewpoint based on real video images captured from multiple positions in the space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2016 / 088437 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has room for improvement in terms of facilitating the generation of video content. Conventional technology requires a lot of equipment to capture images in a space, making it difficult for general users to generate video content.

[0005] The present invention has been made in consideration of the above, and has an object to provide an image generation device, an image generation method, and an image generation program that can facilitate the generation of video content. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, an image generating device according to an embodiment includes a receiving unit, a first generating unit, and a display control unit. The receiving unit receives a user operation. The first generating unit generates a camera viewpoint image of a three-dimensional object in the three-dimensional virtual space captured by the virtual camera by moving a virtual camera in the three-dimensional virtual space based on the user operation received by the receiving unit. The display control unit places a virtual screen in the three-dimensional virtual space for displaying the camera viewpoint image generated by the first generating unit, and displays a camera trajectory of the virtual camera in the three-dimensional virtual space. Effect of the Invention

[0007] According to one aspect of the embodiment, the generation of video content may be facilitated. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an overview of a video generating method according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of an image displayed by the image generating method according to the first embodiment. [Diagram 3] 1 is a diagram showing an example of the appearance of a remote control according to a first embodiment. [Figure 4] 1 is a block diagram of an information processing device according to a first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of key frame information according to the first embodiment. [Figure 6] FIG. 2 is an explanatory diagram of a screen size. [Figure 7] FIG. 1 is an explanatory diagram of the depth of field. [Figure 8] FIG. 4 is a diagram showing an example of a camera trajectory according to the first embodiment. [Figure 9] FIG. 4 is a diagram showing an example of a camera trajectory according to the first embodiment. [Figure 10] FIG. 4 is a diagram showing an example of a point of interest according to the first embodiment. [Figure 11]FIG. 4 is a diagram showing an example of a prohibited area according to the first embodiment. [Figure 12] FIG. 1 is an explanatory diagram of an imaginary line. [Figure 13] FIG. 4 is a diagram showing an example of an auxiliary image according to the first embodiment. [Figure 14] FIG. 4 is a diagram showing an example of an auxiliary image according to the first embodiment. [Figure 15] FIG. 4 is a diagram showing an example of an auxiliary image according to the first embodiment. [Figure 16] 5A to 5C are schematic diagrams illustrating a process performed by a correction unit according to the first embodiment. [Figure 17] 5A to 5C are schematic diagrams illustrating a process performed by a correction unit according to the first embodiment. [Figure 18] 5A to 5C are schematic diagrams illustrating a process performed by a correction unit according to the first embodiment. [Figure 19] 4 is a flowchart showing a processing procedure executed by the image generating device according to the first embodiment. [Figure 20] FIG. 11 is a diagram showing an overview of a video processing method according to a second embodiment. [Figure 21] FIG. 11 is a diagram illustrating an example of a display device according to a second embodiment. [Figure 22] FIG. 11 is a block diagram of an image generating device according to a second embodiment. [Figure 23] FIG. 2 is a hardware configuration diagram showing an example of a computer that realizes the functions of the image generating device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0010] <First embodiment> First, an overview of the image generation method according to the first embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a diagram showing an overview of the image generation method according to the first embodiment. Fig. 2 is a diagram showing an example of an image displayed by the image generation method according to the first embodiment. The image generation method according to the embodiment is executed by an image generation device 1 described later with reference to Fig. 4.

[0011] In addition, although the following description will be given of a case where the three-dimensional virtual space is a VR (Virtual Reality) space, the three-dimensional virtual space may be an AR (Augmented Reality) space. Note that a case where the three-dimensional virtual space is an AR space will be described in a second embodiment.

[0012] FIG. 1 shows a scene in which a user wears a display device 50 in the real world Sr and operates a controller 60 to operate a virtual camera 10 arranged in a three-dimensional virtual space (hereinafter, virtual space Sv).

[0013] Incidentally, in the past, when producing video content such as movies, it required a lot of production time. For example, the creator of the video content would repeatedly check the shot video and reconstruct the camera work according to the image. In addition, when producing video content in the real world, there is a limit to the camera work, which becomes an obstacle in producing attractive video content.

[0014] In contrast, in the video generation method according to the embodiment, the video content is shot within the virtual space Sv, which makes it easier to generate the video content.

[0015] The display device 50 shown in FIG. 1 is a head mounted display (hereinafter also referred to as HMD (Head Mount Display)), and provides a virtual space Sv to a user (creator) by displaying an image generated by an image generation device 1 described later.

[0016] 1 is a remote control device with which a user performs various operations on a virtual camera 10 in a virtual space Sv in the real world Sr. For example, in the image generating method according to the embodiment, it is possible to move the virtual camera 10 in the virtual space Sv in accordance with the trajectory of the controller 60 in the real world Sr, and to change the direction (camera angle) of the virtual camera 10 in accordance with the inclination of the controller 60.

[0017] For example, a user creates a camera viewpoint image of a three-dimensional object 100 existing in the virtual space Sv captured by the virtual camera 10 by moving the virtual camera 10 with the controller 60 while viewing the virtual space Sv displayed on the display device 50. Note that while the image of the virtual space Sv seen by the user is 3D, the virtual viewpoint image is a 2D image.

[0018] In this case, as described above, if the user checks the camera viewpoint image and camera work each time a camera viewpoint image is shot, this task may take up a lot of time, which may result in a decrease in work efficiency.

[0019] For this reason, in the image generating method according to the embodiment, a virtual screen for displaying a camera viewpoint image is disposed in the virtual space Sv, and the camera trajectory of the virtual camera 10 is displayed in the virtual space Sv.

[0020] That is, in the video generation method according to the embodiment, the camera viewpoint video and the camera trajectory during shooting are displayed on the display device 50. This enables the user to check the camera viewpoint video and the camera trajectory (camera work) in parallel with shooting the camera viewpoint video.

[0021] In the example shown in Figure 2, a virtual screen Sc1 is installed on the display screen of the display device 50, and a current camera viewpoint video is displayed on the virtual screen Sc1, while a camera trajectory T is displayed within a virtual space Sv visible to the user.

[0022] The virtual screen Sc1 is, for example, a screen that moves following the user's posture and is always placed in a position directly facing the user. As shown in Fig. 2, the virtual screen Sc1 and the camera viewpoint image displayed on the virtual screen Sc1 are transparent images. This allows the user to view the virtual space Sv while checking the virtual camera image displayed on the virtual screen Sc1.

[0023] Note that a virtual screen Sc2 may also be set on the outer wall of the virtual space Sv, as indicated by the dashed line in Fig. 2. In this case, the virtual screen Sc2 appears to the user as a part of the virtual space Sv.

[0024] Moreover, the camera trajectory T is a line indicating the trajectory of the virtual camera 10. Therefore, the user can check the camera work of the virtual camera 10 by the camera trajectory T. Moreover, the example of Fig. 2 shows a case where an icon A is displayed on the camera trajectory T. As will be described later, the icon A indicates a point where a key frame was captured when the virtual camera video was captured.

[0025] For example, the user can set a key frame by operating a button on the controller 60, and when the key frame is set, an icon A is added to the camera trajectory T. Note that, although the example in Fig. 2 shows a case where the icon A is a diamond model, the icon A may be a reduced display of the key frame, or may indicate the orientation of the virtual camera 10 when capturing the key frame.

[0026] Also, when the user selects icon A, for example, it is possible to display a virtual camera image corresponding to the selected icon A on the virtual screen Sc1. That is, by displaying icon A together with the camera trajectory T, the set key frame can be easily understood.

[0027] In this way, the image generating method according to the embodiment displays the current virtual camera image and the camera trajectory T. This allows the user to check the current virtual camera image and the camera trajectory T in parallel when creating the camera image.

[0028] Furthermore, in the video generation method according to the embodiment, a virtual camera video is captured within the virtual space Sv, so that video content can be easily generated if data of the three-dimensional object 100 is available.

[0029] Therefore, according to the video generation method of the embodiment, it is possible to easily generate video content. In the example of Fig. 2, the case where the virtual camera video and the camera trajectory T are displayed on the display device 50 has been described, but the present invention is not limited to this, and the virtual camera video and the camera trajectory T may be displayed on separate display devices, or one or both of the virtual camera video and the camera trajectory T may be set to not be displayed.

[0030] Next, the controller 60 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the external appearance of the controller 60 according to the embodiment. In the example shown in Fig. 3, the controller 60 is composed of a left controller 60a and a right controller 60b.

[0031] 3, the left controller 60a and the right controller 60b are each provided with a plurality of buttons. Each of the plurality of buttons is associated with a command, and the user can press each button to instruct the image generating device to execute the corresponding command.

[0032] For example, the left controller 60a can be used to input various commands such as adjusting the playback time and playback speed, changing the scale of the user's viewpoint image, setting and canceling a focus point (described below), and changing the user's position within the virtual space Sv.

[0033] In addition, the right controller 60b can be used to input various commands such as moving the virtual camera 10, setting the camera angle of the virtual camera 10, changing the line of sight position in the user's viewpoint video, setting key frames, etc. In addition, for example, by operating the left controller 60a and the right controller 60b simultaneously, it is also possible to change the scale of the virtual camera video.

[0034] Note that the controller 60 shown in Fig. 3 is an example and is not limited thereto. Also, for example, instead of the controller 60, the image generating device may receive a gesture operation by the user as a user operation. Also, hereinafter, when there is no need to distinguish between the left controller 60a and the right controller 60b, they will simply be referred to as "controller 60".

[0035] Next, a configuration example of the image generating device according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram of the image generating device according to the embodiment. In addition to the image generating device 1, Fig. 4 also shows a display device 50 and a controller 60.

[0036] First, the controller 60 will be described. In the example shown in Fig. 4, the controller 60 includes a gyro sensor 61, an acceleration sensor 62, and an operation button 63. The gyro sensor 61 is a sensor that detects angular velocities about three axes to detect the movement of the controller 60. For example, the gyro sensor 61 is built into the left controller 60a and the right controller 60b.

[0037] For example, the user can change the user viewpoint set within the virtual space Sv by moving the left controller 60a, and can move the virtual camera 10 within the virtual space Sv by moving the right controller 60b.

[0038] The acceleration sensor 62 is a sensor that detects acceleration occurring in the controller 60. The operation buttons 63 are a plurality of buttons provided on the controller 60, and various commands are associated with each of the plurality of buttons.

[0039] Next, a description will be given of the display device 50. In the example shown in Fig. 4, the display device 50 includes a display unit 51, a gyro sensor 52, an acceleration sensor 53, a camera 54, and a speaker 55.

[0040] As described above, since the display device 50 is an HMD, the display unit 51 is disposed in front of the user when the user wears the HMD. The display unit 51 may be configured with a light-blocking display surface or a light-transmitting display surface. That is, the display device 50 is a VR goggle, but may also be an AR goggle. Needless to say, the display device 50 is not limited to an HMD.

[0041] The gyro sensor 52 and the acceleration sensor 53 are sensors that respectively detect the movement of the display device 50 and the acceleration occurring in the display device 50. Since the display device 50 is an HMD, the gyro sensor 52 and the acceleration sensor 53 detect changes in the posture of the user wearing the display device 50.

[0042] The camera 54 is, for example, a camera that captures an image in front of the display device 50, in other words, a camera that captures an image of the real world Sr as seen by the user. The speaker 55 outputs sound based on the sound signal output from the image generation device 1.

[0043] Next, a description will be given of the image generating device 1. As shown in Fig. 4, the image generating device 1 includes a storage unit 2 and a control unit 3. Note that the image generating device 1 may also include a communication unit (not shown) for wirelessly or wired communication with an external device, and an operation unit (not shown) for receiving user operations.

[0044] The storage unit 2 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. In the example shown in Fig. 4, the storage unit 2 stores three-dimensional video information 20, key frame information 21, and virtual camera video information 22.

[0045] The three-dimensional video information 20 is three-dimensional video content that is the source of the virtual camera video, and is information about the video to be played in the virtual space Sv. The three-dimensional video information 20 includes information about the video of the three-dimensional object 100, information about the three-dimensional background video that is the background of the three-dimensional object 100, and the like.

[0046] The key frame information 21 is information about a frame (image) that is a key in a virtual camera video. An example of the key frame information 21 will now be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the key frame information 21 according to the embodiment.

[0047] In the example shown in FIG. 5, the key frame information 21 is information in which "image ID", "playback time", "camera coordinates", "point of interest", "angle", "screen size", "depth of subject", and the like are associated with one another.

[0048] "Image ID" indicates an identifier for identifying a key frame. "Playback time" indicates the playback time of the three-dimensional object 100. "Camera coordinates" indicates the coordinates of the virtual camera 10 in the virtual space Sv when the key frame is captured.

[0049] The "point of interest" refers to a point of interest set in the virtual space Sv. As will be described later, when a point of interest is set, the attitude of the virtual camera 10 is controlled so that the center of the angle of view of the virtual camera 10 faces the point of interest.

[0050] "Angle" refers to the camera angle of virtual camera 10 when capturing an image of a key frame, i.e., the angle of view of virtual camera 10. "Screen size" refers to the size of three-dimensional object 100 captured in a key frame. A specific example of the screen size will be described later with reference to FIG. 6.

[0051] Moreover, the "depth of field" indicates the camera angle with respect to the three-dimensional object 100. A specific example of the depth of field will be described later with reference to Fig. 7. The key frame information 21 shown in Fig. 5 is an example, and is not limited thereto.

[0052] Next, "screen size" and "depth of subject" will be explained with reference to Fig. 6 and Fig. 7. Fig. 6 is an explanatory diagram of screen size. Fig. 7 is an explanatory diagram of depth of subject. As shown in the left diagram of Fig. 6, in the case of a shot in which the entire body of a subject is photographed, screen sizes are broadly classified into long shot (LS), full figure (FF), knee shot (KS), and waist shot (WS).

[0053] Also, as shown in the right image of Figure 6, in the case of a shot of the upper body of a subject, the screen sizes are broadly classified into bust shot (BS; Bust Shot), up shot (US; Up Shot), close up (CU; Close Up), big close up (BCU; Big Close Up), and detail shot (Detail; Detail Shot).

[0054] As shown in Figure 7, the depth of field is broadly classified into "bird's-eye view," "medaka," and "tilt." For example, bird's-eye view indicates a shot in which the subject's face is shot from above, while medaka indicates a shot in which the subject is shot from eye level. Also, tilt indicates a shot in which the subject's face is shot from below.

[0055] The image generating device 1 associates the screen size and subject depth with the key frame as metadata according to the size of the three-dimensional object 100 captured in the key frame and the imaging direction of the three-dimensional object 100.

[0056] For example, auxiliary lines indicating the screen size may be displayed in the virtual space Sv. Also, for example, a command for specifying the screen size may be set in advance, and when the user selects the command, the virtual camera 10 may be moved so as to capture the three-dimensional object 100 at the screen size corresponding to the command, or the zoom ratio of the virtual camera 10 may be changed.

[0057] Returning to the explanation of Fig. 4, the virtual camera video information 22 will be explained. The virtual camera video information 22 is information relating to a virtual camera video virtually captured by the virtual camera 10. In this case, the virtual camera video information 22 is information relating to a virtual camera video in a state where an auxiliary image for capturing the virtual camera video, such as a virtual screen Sc2 (see Fig. 1), has been erased.

[0058] Next, a description will be given of the control unit 3. The control unit 3 is realized, for example, by a central processing unit (CPU) or a micro processing unit (MPU) executing a program stored inside the image generating device 1 using a random access memory (RAM) or the like as a working area. The control unit 3 is a controller, and may be realized, for example, by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0059] As shown in Fig. 4, the control unit 3 has a reception unit 30, a detection unit 31, a first generation unit 32, a second generation unit 33, a setting unit 34, a display control unit 35, and a correction unit 36, and realizes or executes the functions and actions of information processing described below. The internal configuration of the control unit 3 is not limited to the configuration shown in Fig. 4, and may be other configurations as long as they perform the information processing described below. The control unit 3 may be connected to a predetermined network by wire or wirelessly using, for example, a NIC (Network Interface Card) or the like, and may receive various information from an external server or the like via the network.

[0060] The reception unit 30 receives user operations. Specifically, the reception unit 30 receives various operations performed by the user on the controller 60 as user operations. The reception unit 30 receives an attitude signal related to the attitude of the controller 60 from the gyro sensor 61 and the acceleration sensor 62 of the controller 60, and receives commands from the operation buttons 63 of the controller 60. The reception unit 30 also receives an attitude signal of the display device 50 from the gyro sensor 52 and the acceleration sensor 53 of the display device 50.

[0061] The detection unit 31 detects the orientation of each of the controller 60 and the display device 50. Specifically, the detection unit 31 detects the orientation of the controller 60 based on an orientation signal of the controller 60, and detects the orientation of the display device 50, i.e., the user, based on an orientation signal of the display device 50.

[0062] The first generation unit 32 moves the virtual camera 10 in the virtual space Sv based on the user operation accepted by the acceptance unit 30, thereby generating a camera viewpoint video in which a three-dimensional object 100 in the virtual space Sv is captured by the virtual camera 10.

[0063] Specifically, the first generator 32 moves the virtual camera 10 within the virtual space Sv in accordance with the attitude signal of the controller 60, thereby generating a virtual camera video captured by the virtual camera 10 within the virtual space Sv.

[0064] Here, a specific example of the processing by the first generation unit 32 will be described with reference to Fig. 8 to Fig. 12. Fig. 8 and Fig. 9 are diagrams showing an example of a camera trajectory T according to the first embodiment. In the example shown in Fig. 8, the camera trajectory T, which is the trajectory of the virtual camera 10, is drawn based on a movement operation of moving the controller 60 in the real world Sr of the user.

[0065] That is, virtual camera 10 moves within virtual space Sv in accordance with the trajectory drawn by the user with controller 60, and a camera trajectory T is drawn. Then, when virtual camera 10 is moved along camera trajectory T, first generator 32 generates an image of virtual space Sv seen from virtual camera 10 as a virtual camera image.

[0066] At this time, the first generation unit 32 determines the tilt of the virtual camera 10 according to the tilt of the controller 60. That is, the user can change the position of the virtual camera 10 by moving the controller 60 (right controller 60b), and can change the tilt of the virtual camera 10 by changing the tilt of the controller 60.

[0067] 9, the image generating device 1 according to the embodiment has a fixed mode in which the height of the virtual camera 10 is fixed. In the fixed mode, for example, the virtual camera 10 moves in the virtual space Sv in accordance with the trajectory of the controller 60 while the height of the virtual camera 10 is fixed. That is, in the fixed mode, the movement of the virtual camera 10 in the roll direction is prohibited.

[0068] Therefore, the virtual camera 10 moves on a virtual plane Pt that is at the same height in the virtual space Sv, and a camera trajectory T is drawn on the virtual plane Pt. That is, in the fixed mode, by disabling the vertical movement of the controller 60, the user can create a virtual camera image without worrying about the shaking of the controller 60 in the vertical direction.

[0069] Although the case where the height is fixed in the fixed mode has been described here, the present invention is not limited to this, and the pitch and yaw of virtual camera 10 may each be fixed.

[0070] Next, the attention point will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the attention point according to the embodiment. As shown in Fig. 10, the user can set an attention point P in the virtual space Sv.

[0071] More specifically, the user can call up a setting command to set a point of interest P on the left controller 60a (see FIG. 3), and set the point of interest P at any position within the virtual space Sv, for example, by moving the left controller 60a.

[0072] Then, when a focus point P has been set, the first generation unit 32 generates a virtual camera image by controlling the attitude of the virtual camera 10 so that the center of the angle of view of the virtual camera 10 faces the focus point P, while moving the virtual camera 10 in accordance with the trajectory of the controller 60.

[0073] That is, in this case, the user can obtain a virtual camera image in which the attention point P is photographed at any time by setting the attention point P. Also, in this case, the user can set the camera trajectory T without worrying about the direction of the virtual camera 10 (i.e., the inclination of the controller 60).

[0074] In addition, a prohibited area may be provided where virtual camera 10 is prohibited from entering, and virtual camera 10 may be prohibited from entering the prohibited area, or a warning image may be displayed when virtual camera 10 enters the prohibited area.

[0075] Fig. 11 is a diagram showing an example of a prohibited area. In the example shown in Fig. 11, the prohibited area Pa is the right half of the virtual space Sv. Therefore, in the example shown in Fig. 11, the virtual camera 10 moves within the left half of the virtual space Sv.

[0076] For example, an imaginary line can be given as a reason for providing such a prohibited area Pa. Fig. 12 is an explanatory diagram of an imaginary line. As shown in Fig. 12, for example, in a scene where person A and person B are facing each other, the positions of person A and person B are reversed in an image captured by camera 1 behind person A and an image captured by camera 2 behind person B.

[0077] For this reason, for example, when an image taken by camera 1 is switched to an image taken by camera 2, it becomes difficult for the viewer to instantly grasp the situation. For this reason, a line segment connecting person A and person B is set as an imaginary line. Note that an imaginary line is sometimes also called an imaginary line.

[0078] 11, the prohibited area Pa is set so as not to cross the imaginary line, thereby making it possible to generate a virtual camera image that is easy for the viewer to understand.

[0079] The prohibited area Pa may be set by the control unit 3, or may be set by the user. The prohibited area Pa may be dynamically set in consideration of the current position of the virtual camera 10, etc. The prohibited area Pa may be set in advance when the three-dimensional object 100 is created.

[0080] Returning to the explanation of Fig. 4, the second generation unit 33 will be explained. The second generation unit 33 generates a user viewpoint video in which the virtual space Sv is viewed from a user viewpoint set in the virtual space Sv. Specifically, the second generation unit 33 moves the user viewpoint in the virtual space Sv in response to an attitude signal of the display device 50, and generates the user viewpoint video by continuously connecting images in the virtual space Sv seen from the user viewpoint.

[0081] At this time, the receiving unit 30 can receive various commands such as changing the scale of the virtual space Sv or rotating the virtual space Sv, and the second generation unit 33 generates a user viewpoint video that reflects the commands received by the receiving unit 30.

[0082] For example, when changing the scale of the virtual space Sv, the user can set either the life-size mode or the miniature mode as the default. Here, the life-size mode is a mode in which the virtual space Sv is displayed in actual size. By setting the life-size mode, the user can view a user-perspective video as if the user were actually present in the virtual space Sv.

[0083] The miniature mode is a mode in which the entire virtual space Sv is displayed in a reduced size. By setting the miniature mode, the user can view a user's viewpoint video that provides a bird's-eye view of the virtual space Sv.

[0084] In addition, the receiving unit 30 can also accept user operations to fine-tune the reduction ratio of the virtual space Sv, and the second generating unit 33 can also generate a user viewpoint video in which the reduction ratio has been fine-tuned in accordance with the user operations accepted by the receiving unit 30.

[0085] The second generating unit 33 can also generate a user viewpoint video in which the user viewpoint is moved along the camera trajectory T of the virtual camera 10. In the user viewpoint video, the user can view the inside of the virtual space Sv seen from the virtual camera 10.

[0086] The setting unit 34 sets each key frame based on the user operation accepted by the accepting unit 30. When setting a key frame, the setting unit 34 adds information about the key frame to the key frame information 21.

[0087] The display control unit 35 places the virtual screens Sc1 and Sc2 that display the camera viewpoint images generated by the first generation unit 32 within the virtual space Sv, and displays the camera trajectory T of the virtual camera 10 within the virtual space Sv.

[0088] That is, the display control unit 35 sets virtual screens Sc1 and Sc2 in the user viewpoint video generated by the second generation unit 33, and displays the virtual camera video generated by the first generation unit 32 on the virtual screens Sc1 and Sc2.

[0089] In addition, the display control unit 35 displays the camera trajectory T in the user's viewpoint video by superimposing the camera trajectory T on the user's viewpoint video, and when a key frame is set, displays an icon A at the position where the key frame was captured.

[0090] Furthermore, the display control unit 35 can also display an auxiliary image that assists in capturing the virtual camera image in the user's viewpoint image or in the virtual screen Sc. Here, specific examples of the auxiliary image will be described with reference to Figs.

[0091] 13 to 15 are diagrams showing an example of an auxiliary image according to an embodiment. In addition to the menu screen Sm, which is the auxiliary image, a virtual screen Sc1 is also shown in Fig. 13. As shown in Fig. 13, the display control unit 35 displays the menu screen in the user's viewpoint video when the user performs a predetermined operation.

[0092] In the example shown in Fig. 13, the menu screen Sm displays six buttons each associated with a different command. Also, as shown in Fig. 13, the virtual screen Sc1 is displayed in a position directly facing the user, whereas the menu screen is displayed at the lower left of the virtual screen Sc1.

[0093] This is because it is assumed that various operations on the menu screen Sm will be performed with the left controller 60a (see FIG. 3). More specifically, the user can input commands by touching each button on the menu screen arranged in the virtual space Sv with the left controller 60a in the real world Sr.

[0094] In other words, by displaying the menu screen in a position that is easy to operate with the left controller 60a, it is possible to make it easier for the user to input commands from the menu screen.

[0095] 13, a seek bar B indicating the playback position of the camera viewpoint video is displayed on the virtual screen Sc1. This allows the user to easily grasp the current playback position (playback time) by using the seek bar B.

[0096] Furthermore, the image generating device 1 can also accept a user operation on the seek bar B, and can change the playback location in response to the user operation. Here, the user can perform an operation on the seek bar B by operating the operation button on the left controller 60a.

[0097] Changing the playback location includes displaying the virtual camera image at the playback time selected by the user, and displaying key frames sequentially in response to user operations. In other words, the user can change the playback location to a playback time of their choice, and check the set key frames sequentially.

[0098] 14, the auxiliary image may include a virtual camera 10 and angle-of-view images V1 and V2 indicating the current angle of view of the virtual camera 10. For example, the display control unit 35 places the virtual camera 10 at a set position in the virtual space Sv, and when the virtual camera 10 is included in the user-viewpoint video, superimposes the virtual camera 10 on the user-viewpoint image.

[0099] At this time, the display control unit 35 takes into consideration the orientation of the virtual camera 10 in the virtual space Sv, displays the virtual camera 10 in the user viewpoint video, and displays the angle-of-view images V1 and V2 according to the current angle of view.

[0100] In the example of FIG. 14, two types of angle-of-view images V1 and V2 are shown, but the angle-of-view images can be changed as appropriate in accordance with the actual angle of view of the virtual camera 10 in the virtual space Sv.

[0101] 15 shows a case where the auxiliary image is a guide line Lw. The guide line Lw is a line that assists the trajectory of the virtual camera 10, and the user can draw an ideal camera trajectory T by moving the virtual camera 10 along the guide line Lw.

[0102] 15 shows a case where the guide line Lw includes concentric circles formed along the contour of a hemisphere and a ridge line intersecting the concentric circles. Here, the intersection Cp between the concentric circles and the ridge line serves as an index for setting a key frame when moving the virtual camera 10 along the guide line Lw.

[0103] That is, when the user moves the virtual camera 10 along the guide line Lw, the user can set a final camera trajectory T along the guide line Lw by setting a key frame on the intersection point Cp. Note that the guide line Lw shown in FIG. 15 is an example, and is not limited thereto, and may include, for example, a straight line. Also, the concentric circles of the guide line Lw may be expanded or the shape may be arbitrarily changed by a user operation.

[0104] Returning to the description of Fig. 4, the correction unit 36 ​​will be described. The correction unit 36 ​​corrects the camera trajectory T based on a key frame set by a user operation. When the camera trajectory T is corrected by the correction unit 36, the first generation unit 32 regenerates the virtual camera viewpoint video based on the corrected camera trajectory T.

[0105] Then, the virtual camera video regenerated by first generation unit 32 is displayed, for example, on virtual screen Sc1. This allows the user to check the virtual camera video edited by video generation device 1. In addition, the user can save the virtual camera video by performing a predetermined operation, and the saved virtual camera video is stored in storage unit 2 as virtual camera video information 22.

[0106] Specifically, the correction unit 36 ​​corrects the camera trajectory T by smoothly connecting the imaging coordinates of successive key frames in a time series. That is, in this case, when the user sets the key frames, a camera viewpoint video with the corrected camera trajectory T can be obtained. That is, in this case, blurring of the camera trajectory T based on the user's operation is automatically corrected.

[0107] Furthermore, when a key frame is added or deleted by a user operation, the correction unit 36 ​​corrects the camera trajectory T based on the added or deleted key frame. Note that the addition of a key frame here means that a new key frame is inserted between the playback times of consecutive key frames.

[0108] The correction unit 36 ​​sets a new camera trajectory to a line that smoothly connects the imaging position of the added key frame and the imaging positions of the key frames before and after the added key frame in a chronological order. When a key frame is deleted, the correction unit 36 ​​sets a new camera trajectory to a line that smoothly connects the key frames before and after the deleted key frame.

[0109] Furthermore, the correction unit 36 ​​corrects the camera trajectory T based on the time series changes regarding successive key frames. Here, the time series changes regarding the key frames refer to the time interval between key frames and the distance interval between the imaging positions of the key frames.

[0110] When the user performs a predetermined operation while editing the virtual camera video, the correction unit 36 ​​corrects the imaging positions of one or more key frames so that the time intervals of the key frames become equal. That is, in this case, the correction unit 36 ​​sets the virtual camera video at the shooting time with the equal time intervals as the key frames on the corrected camera trajectory T.

[0111] Similarly, when the user performs a predetermined operation while editing the virtual camera video, the correction unit 36 ​​corrects the imaging positions of one or more key frames so that the distance intervals between the imaging positions of the key frames become equal. That is, in this case, the correction unit 36 ​​sets the virtual camera video at the shooting time when the distance intervals between the imaging positions become equal as a key frame on the corrected camera trajectory T. In this way, the correction unit 36 ​​can easily edit the virtual camera video by the user by correcting the key frame.

[0112] The correction unit 36 ​​may correct the camera trajectory T by correcting the imaging position of the key frame based on the guide line Lw shown in Fig. 15, for example. Here, an example of the processing by the correction unit 36 ​​will be described with reference to Figs. 15 to 17. Note that, in the following, the imaging position of the key frame is indicated by the corresponding icon A, and the number added to the end of the icon A indicates the order in which the key frame was imaged.

[0113] In the example of FIG. 15, for example, key frames corresponding to icons A1 to A3 are set along guide line Lw, and a key frame corresponding to icon A4 deviates from guide line Lw.

[0114] In this case, the correction unit 36 ​​corrects the imaging position of the key frame so that the icon A4 is set on the guide line Lw. That is, in this case, the camera trajectory T is corrected as if all the key frames were captured along the guide line Lw.

[0115] In this way, by correcting the camera trajectory T along the guide line Lw, it is possible to generate a virtual camera image with the camera trajectory T suitable for capturing an image of the three-dimensional object 100. In this case, the correction unit 36 ​​may correct the imaging position of the key frame so that the icon A is located on the intersection point Cp shown in FIG.

[0116] Furthermore, in the case where the above-mentioned correction is taken into consideration, when the user sets a key frame, the correction unit 36 ​​may correct the position of the virtual camera 10 so that the key frame is captured on the intersection point Cp that is closest to the current virtual camera 10. That is, in this case, the virtual camera 10 moves as if it is attracted to the intersection point Cp. The correction unit 36 ​​may perform the above-mentioned processing not only on the intersection point Cp but also on the guide line Lw.

[0117] Furthermore, as described above, when the camera trajectory T is corrected based on key frames, it is assumed that the corrected camera trajectory T may differ from the camera work intended by the user. For this reason, the correction unit 36 ​​may correct the camera trajectory T based on a trajectory drawn by a user operation.

[0118] Specifically, in the example shown in FIG. 17, the camera trajectory T1 from icons A1 to A3 is shown to be the camera trajectory T corrected by the correction unit 36 ​​through the processing already described, and the camera trajectory T2 from icon A3 to icon A4 is shown to be the camera trajectory T2 drawn by user operation.

[0119] In addition, the user draws a trajectory with the right controller 60b in the same way as when shooting a virtual camera image, and the camera trajectory T2 is set based on the trajectory. That is, in this case, the user can set the camera trajectory T2 freehand, so that the camera work in the virtual viewpoint image can be extended infinitely. In addition, the camera trajectory T2 can also be realized by the user setting key frames at very short intervals.

[0120] Furthermore, the correction unit 36 ​​may automatically add key frames from the virtual camera video, as shown in Fig. 18. The example of Fig. 18 shows a scene in which key frames corresponding to icons A1 to A4 have already been set.

[0121] For example, in this case, the correction unit 36 ​​extracts a recommended image from the virtual camera video based on the camera trajectory T before correction, and sets it as a key frame. Note that the extraction of the recommended image is performed using a predetermined image analysis algorithm. Also, the user may set in advance the conditions under which the recommended image is extracted.

[0122] 18 shows a scene in which three images corresponding to icons A5 to A7 are newly added as key frames. After adding the key frames, the correction unit 36 ​​corrects the camera trajectory T in accordance with the chronological order of the key frames.

[0123] In other words, in this case, a camera trajectory T is generated that includes key frames added by automatic extraction, so that virtual camera footage captured with appropriate camera work can be obtained without the user having to set key frames.

[0124] In addition, the correction unit 36 ​​can also change the lighting mode for the three-dimensional object 100 in response to a user operation. Here, the lighting mode includes the lighting position for the three-dimensional object 100, the range to be illuminated, the color temperature of the lighting, etc. In this way, by changing the lighting mode, it is possible to realize a variety of lighting effects.

[0125] Next, a process procedure executed by the image generating device 1 according to the embodiment will be described with reference to Fig. 19. Fig. 19 is a flowchart showing the process procedure executed by the image generating device 1 according to the embodiment. Note that the process procedure shown below is repeatedly executed by the control unit 3 when generating a virtual camera image.

[0126] 19, first, the image generating device 1 advances the playback time of the three-dimensional object 100 including the virtual space Sv (step S101), and then determines whether or not the Roll change is disabled (step S102). Note that the process of step S102 is a process of determining whether or not the mode is the fixed mode described in FIG.

[0127] If the image generation device 1 determines in step S102 that the Roll change is invalid (Yes in step S102), the image generation device 1 maintains the horizontal direction and updates the coordinates of the virtual camera 10 in the virtual space Sv in response to a user operation (Step S103). If the image generation device 1 determines in step S102 that the Roll change is valid (No in step S102), the image generation device 1 updates the coordinates of the virtual camera 10 in response to the trajectory of the controller 60 (Step S104).

[0128] Next, the image generation device 1 determines whether or not a focus point P has been set (step S105), and if a focus point P has been set (step S105, Yes), sets the angle of the virtual camera 10 to an angle for photographing the focus point P (step S106).

[0129] Furthermore, if it is determined in step S105 that the attention point P has not been set (No in step S105), the image generation device 1 sets the angle of the virtual camera 10 in accordance with the attitude of the controller 60 (step S107).

[0130] Next, the image generating device 1 determines whether or not a key frame setting has been accepted (step S108), and if a key frame setting has been accepted (step S108, Yes), it stores information about the key frame (step S109). Note that, if it is determined in step S108 that a key frame setting has not been accepted (step S108, No), the process of step S109 is omitted.

[0131] Next, based on the results of the processes in steps S101 to S107, the image generating device 1 generates a camera viewpoint image at the current playback time (step S110), and generates a user viewpoint image at the current playback time (step S111). At this time, the user viewpoint image includes the camera trajectory T.

[0132] Thereafter, the image generating device 1 outputs the images generated in steps S110 and S111 to the display device 50 (step S112), and ends the process.

[0133] <Second embodiment> Next, an overview of an image generating method according to the second embodiment will be described with reference to Fig. 20 and Fig. 21. Fig. 20 is a diagram showing an overview of an image generating method according to the second embodiment. Fig. 21 is a diagram showing an example of a display device according to the second embodiment.

[0134] It should be noted that the first embodiment differs from the second embodiment in that the virtual space Sv is an AR space, whereas the first embodiment uses a VR space.

[0135] In the example shown in Fig. 20, the display device 50A is a smartphone. The display device 50A is not limited to a smartphone, and may be any other device as long as it has a camera function capable of capturing an image of the real world Sr. Needless to say, the display device 50A may be Ar goggles.

[0136] As shown in FIG. 20, for example, when a user photographs the real world Sr with a display device 50A, a virtual space Sa is displayed on the display device 50A, making it appear as if a three-dimensional object 100 virtually positioned in the real world Sr exists in the real world Sr.

[0137] 21, for example, when a user moves the display device 50A, the display position of the three-dimensional object 100 is updated according to the angle of view of the real world Sr captured by the display device 50A. That is, in this case, the display device 50A functions as both the display device 50 and the controller 60 in the first embodiment.

[0138] Also, for example, the user can set the above-mentioned key frame by operating the display device 50A, and can cause the display device 50A to display the key frame. The user can then adjust the angle of view of the key frame by changing the inclination of the display device 50A.

[0139] In this way, in the image generating method according to the second embodiment, a device such as a smartphone, which is more popular than an HMD, is used as the display device 50A. This allows a variety of users to easily create camera viewpoint images. In addition, in the second embodiment, since the background of the virtual space Sa is the real world Sr, the user can create a virtual viewpoint image with a desired location as the background as long as the data of the three-dimensional object 100 is available.

[0140] Next, a configuration example of an image generating device according to a second embodiment will be described with reference to Fig. 22. Fig. 22 is a block diagram of an image generating device according to a second embodiment. An image generating device 1A shown in Fig. 22 differs from the image generating device 1 according to the first embodiment in that it includes a 3D data generating unit 37.

[0141] 22, only the 3D data generation unit 37 will be described, and descriptions of other components will be omitted. The 3D data generation unit 37 generates 3D data from the background image of the virtual camera image generated by the first generation unit 32.

[0142] In other words, the 3D data generator 37 generates 3D data from the real world Sr captured by the user using the display device 50A. That is, in the image generating device 1A according to the second embodiment, a virtual viewpoint image is generated for the user, and 3D data can be generated from the background image. The 3D data can be generated by appropriately using an existing algorithm.

[0143] This makes it possible to generate 3D data for various three-dimensional virtual spaces, and therefore to provide various three-dimensional spaces. The display device 50A and the device having the camera function may be separate devices. In this case, for example, the camera may be an aircraft (a so-called drone).

[0144] The information devices such as the image generation device, HMD, and controller according to each of the above-described embodiments are realized by a computer 1000 having a configuration as shown in FIG. 23, for example. The image generation device 1 according to the embodiments will be described below as an example. FIG. 23 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the image generation device 1. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The various components of the computer 1000 are connected by a bus 1050.

[0145] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0146] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000.

[0147] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records a program according to the present disclosure, which is an example of program data 1450.

[0148] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0149] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined recording medium. The medium may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0150] For example, when computer 1000 functions as image generating device 1 according to the embodiment, CPU 1100 of computer 1000 executes a program loaded onto RAM 1200 to realize functions of reception unit 30, etc. Also, HDD 1400 stores programs according to the present disclosure and data in storage unit 2. Note that CPU 1100 reads program data 1450 from HDD 1400 and executes it, but as another example, these programs may be obtained from other devices via external network 1550.

[0151] The present technology can also be configured as follows. (1) A reception unit that receives a user operation; a first generation unit that generates a camera viewpoint video obtained by capturing a three-dimensional object in the three-dimensional virtual space with a virtual camera by moving a virtual camera in the three-dimensional virtual space based on the user operation received by the reception unit; a display control unit that places a virtual screen that displays the camera viewpoint video generated by the first generation unit in the three-dimensional virtual space, and displays a camera trajectory of the virtual camera in the three-dimensional virtual space; An image generating device comprising: (2) a second generation unit for generating a user viewpoint video of a user viewing the three-dimensional virtual space from a user viewpoint set in the three-dimensional virtual space; Further equipped with The display control unit is displaying the camera trajectory in the user viewpoint image; The image generating device according to (1) above. (3) The display control unit is displaying a seek bar indicating a playback position of the camera viewpoint video on the virtual screen, and changing the playback position of the three-dimensional object and the camera viewpoint video in response to a user operation on the seek bar accepted by the acceptance unit; An image generating device according to (1) or (2) above. (4) The reception unit is A movement operation for moving the controller in the real space is received as a user operation, The first generation unit is generating the camera viewpoint image by moving the virtual camera in accordance with the trajectory of the controller; The image generating device according to any one of (1) to (3) above. (5) The first generation unit is generating the camera viewpoint video by moving the virtual camera in response to a user operation while fixing the height of the virtual camera in a fixed mode in which the height of the virtual camera is fixed; The image generating device according to any one of (1) to (4) above. (6) The reception unit is accepting a user operation to set a focus point on the three-dimensional object; The first generation unit is generating a camera viewpoint image by moving the virtual camera in response to a user operation while controlling an attitude of the virtual camera so that the center of the angle of view of the virtual camera faces the attention point; The image generating device according to any one of (1) to (5) above. (7) The display control unit is Displaying a menu screen in the three-dimensional virtual space; The reception unit is Accepting a user operation on the menu screen; The image generating device according to any one of (1) to (6) above. (8) The reception unit is Accepting a setting of a key frame in the camera viewpoint image; The display control unit is displaying an icon indicating an imaging position of the key frame on the camera trajectory; The image generating device according to any one of (1) to (7) above. (9) a correction unit that corrects the camera trajectory based on the key frame set by a user operation; Equipped with The first generation unit is regenerating the camera viewpoint video in which the virtual camera moves along the camera trajectory corrected by the correction unit when editing the camera viewpoint video; The image generating device according to (8) above. (10) The display control unit is displaying guide lines in the three-dimensional virtual space to assist the trajectory of the virtual camera; The image generating device according to (9) above. (11) The correction unit is correcting an imaging position and an imaging time of the key frame so that the key frame is set on the guide line, and correcting the camera trajectory in accordance with the corrected key frame; The image generating device according to (10) above. (12) The correction unit is correcting the imaging positions of the key frames so that the time intervals between two or more successive key frames or the distance intervals between the imaging positions of two or more successive key frames become equal; The image generating device according to any one of (9) to (11) above. (13) The correction unit is when the key frame is added or deleted by a user operation, correcting the camera trajectory based on the added or deleted key frame; The image generating device according to any one of (9) to (12) above. (14) The correction unit is correcting the camera trajectory based on a trajectory drawn by a user operation; The image generating device according to any one of (9) to (13) above. (15) The correction unit is changing a lighting state for the three-dimensional object in accordance with a lighting state for the three-dimensional object set by a user operation; The image generating device according to any one of (9) to (14) above. (16) The first generation unit is generating a camera viewpoint image in which a screen size of the three-dimensional object is associated with each frame of the camera viewpoint image as meta information; The image generating device according to any one of (1) to (15) above. (17) The reception unit is further receiving a user operation regarding a position and a line of sight of the user in the three-dimensional virtual space; The second generation unit is controlling a position of the user's viewpoint and a line of sight direction in the three-dimensional virtual space based on the user operation accepted by the accepting unit; The image generating device according to (2) above. (18) The display control unit is displaying the user's viewpoint video obtained by moving the user's viewpoint along the camera trajectory; The image generating device according to (2) above. (19) Computer Accepts user operations, generating a virtual viewpoint video obtained by capturing a three-dimensional object in a three-dimensional virtual space from the virtual camera by moving the virtual camera based on the received user operation; displaying the generated virtual viewpoint image to a user while displaying a camera trajectory of the virtual camera together with the three-dimensional object; Video generation method. (20) Computer A reception unit that receives a user operation; a generating unit that generates a virtual viewpoint video obtained by capturing a three-dimensional object, which is a three-dimensional video, from the virtual camera by moving the virtual camera based on the user operation received by the receiving unit; a display control unit that displays the virtual viewpoint video generated by the generation unit to a user while displaying a camera trajectory of the virtual camera together with the three-dimensional object; This is an image generation program that functions as a [Explanation of symbols]

[0152] 1. 1A Image generation device 10 Virtual Camera 30 Reception 31 Detection unit 32 1st generation part 33 Second generation part 34 Setting section 35 Display control section 36 Correction section 37 3D Data Generation Department 50, 50A display device 60 Controller T Camera trajectory Sc1 Virtual Screen

Claims

1. A reception unit that receives user operations including setting key frames in a camera viewpoint image; a first generation unit that generates a camera viewpoint video obtained by capturing a three-dimensional object in the three-dimensional virtual space with the virtual camera by moving a virtual camera in the three-dimensional virtual space based on the user operation received by the reception unit; a display control unit that places a virtual screen that displays the camera viewpoint video generated by the first generation unit in the three-dimensional virtual space, and displays, in the three-dimensional virtual space, a camera trajectory of the virtual camera, and guide lines that include concentric circles formed along an outline of a hemisphere that assists the camera trajectory, and ridge lines that intersect with the concentric circles; a correction unit that corrects an imaging position of the key frame to an intersection between the concentric circle and the ridge line based on the key frame set by a user operation, and corrects the camera trajectory in accordance with the corrected key frame; An image generating device comprising:

2. a second generation unit that generates a user viewpoint video of a user viewing the three-dimensional virtual space from a user viewpoint set in the three-dimensional virtual space; Further equipped with The display control unit is displaying the camera trajectory within the user viewpoint image; 2. The image generating device according to claim 1.

3. The display control unit is displaying a seek bar indicating a playback position of the camera viewpoint video on the virtual screen, and changing the playback position of the three-dimensional object and the camera viewpoint video in response to a user operation on the seek bar accepted by the acceptance unit; 2. The image generating device according to claim 1.

4. The reception unit is A movement operation for moving the controller in the real space is received as a user operation, The first generation unit is generating the camera viewpoint image by moving the virtual camera in accordance with the trajectory of the controller; 2. The image generating device according to claim 1.

5. The first generation unit is generating the camera viewpoint video by moving the virtual camera in response to a user operation while fixing the height of the virtual camera in a fixed mode in which the height of the virtual camera is fixed; 2. The image generating device according to claim 1.

6. The reception unit is accepting a setting of a focus point for the three-dimensional object as a user operation; The first generation unit is generating a camera viewpoint image by moving the virtual camera in response to a user operation while controlling an attitude of the virtual camera so that the center of the angle of view of the virtual camera faces the attention point; 2. The image generating device according to claim 1.

7. The display control unit is Displaying a menu screen in the three-dimensional virtual space; The reception unit is Accepting a user operation on the menu screen; 2. The image generating device according to claim 1.

8. The display control unit is displaying an icon indicating an imaging position of the key frame on the camera trajectory; 2. The image generating device according to claim 1.

9. The first generation unit, regenerating the camera viewpoint video in which the virtual camera moves along the camera trajectory corrected by the correction unit when editing the camera viewpoint video; 9. An image generating device according to claim 8.

10. The correction unit is correcting an imaging position and an imaging time of the key frame so that the key frame is set on the guide line, and correcting the camera trajectory in accordance with the corrected key frame; 10. An image generating device according to claim 9.

11. The correction unit is correcting the imaging positions of the key frames so that the time intervals between two or more successive key frames or the distance intervals between the imaging positions of two or more successive key frames become equal; 10. An image generating device according to claim 9.

12. The correction unit is when the key frame is added or deleted by a user operation, correcting the camera trajectory based on the added or deleted key frame; 10. An image generating device according to claim 9.

13. The correction unit is correcting the camera trajectory based on a trajectory drawn by a user operation; 10. An image generating device according to claim 9.

14. The correction unit is changing a lighting state for the three-dimensional object in accordance with a lighting state for the three-dimensional object set by a user operation; 10. An image generating device according to claim 9.

15. The first generation unit is generating a camera viewpoint image in which a screen size of the three-dimensional object is associated with each frame of the camera viewpoint image as meta information; 2. The image generating device according to claim 1.

16. The reception unit is further receiving a user operation regarding a position and a line of sight of the user in the three-dimensional virtual space; The second generation unit is controlling a position of the user's viewpoint and a line of sight direction in the three-dimensional virtual space based on the user operation accepted by the accepting unit; 3. The image generating device according to claim 2.

17. The display control unit is displaying the user's viewpoint video obtained by moving the user's viewpoint along the camera trajectory; 3. The image generating device according to claim 2.

18. Computer Accepting user operations including setting key frames in the camera viewpoint image; generating a virtual viewpoint video obtained by capturing an image of a three-dimensional object in a three-dimensional virtual space from the virtual camera by moving the virtual camera based on the received user operation; displaying the generated virtual viewpoint image to a user, together with the three-dimensional object, a camera trajectory of the virtual camera, and a guide line including concentric circles formed along an outline of a hemisphere that assists the camera trajectory and a ridge line that intersects with the concentric circles; correcting an imaging position of the key frame to an intersection of the concentric circle and the edge line based on the key frame set by a user operation, and correcting the camera trajectory in accordance with the corrected key frame; Video generation method.

19. Computer a reception unit that receives user operations including setting a key frame in a camera viewpoint image; a generating unit that generates a virtual viewpoint video obtained by capturing a three-dimensional object, which is a three-dimensional video, from the virtual camera by moving the virtual camera based on the user operation received by the receiving unit; a display control unit that displays, to a user, the virtual viewpoint video generated by the generation unit, a camera trajectory of the virtual camera together with the three-dimensional object, and a guide line including concentric circles formed along an outline of a hemisphere that assists the camera trajectory and a ridge line that intersects with the concentric circles; a correction unit that corrects an imaging position of the key frame to an intersection between the concentric circle and the ridge line based on the key frame set by a user operation, and corrects the camera trajectory in accordance with the corrected key frame; This is an image generation program that functions as a

Images