Image generating device and image generating method

Abstract

To provide an image generation apparatus configured to prevent immersion of a user who is not on a vehicle from being disturbed, and an image generation method.SOLUTION: An image generation apparatus includes: a vehicle information acquisition unit which acquires vehicle information including information on behavior of a vehicle, from a device disposed on the vehicle; an image generation unit which generates an image which is a view in a predetermined direction seen from a virtual viewpoint in a vehicle interior space of the vehicle and in which a subject wavers with the behavior of the vehicle, as a viewpoint image; a display control unit which outputs a control signal for displaying the image generated by the image generation unit on a display device arranged in a space different from the vehicle interior space; and a point-of-interest information acquisition unit which acquires information on a point of interest of a user who views the display device. The image generation unit generates a viewpoint image so that a waver level of an object corresponding to the point of interest of the user in the subject may be lower than waver levels of the others than the object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image generating device and an image generating method. [Background technology]

[0002] A content viewing device that suppresses motion sickness caused by the movement of a moving body for a passenger of the moving body viewing a viewing content is known (for example, Patent Document 1). In this content viewing device, a display control means causes an output device to display a behavior image corresponding to the behavior of the moving body in conjunction with the viewing content. In the behavior image, an object that allows the passenger to visualize the acceleration that occurs with the behavior of the moving body is displayed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-157319 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the content viewing device described in Patent Document 1, since the objects displayed in the behavior image are not related to the moving object itself, there is a problem that the sense of immersion is easily impaired for users who are not in the vehicle.

[0005] An object of the present invention is to provide an image generating device and an image generating method that can suppress the loss of the sense of immersion of a user who is not in a vehicle. [Means for solving the problem]

[0006] The present invention obtains vehicle information including information on the behavior of the vehicle from a device provided in the vehicle, generates a viewpoint image as an image viewed in a predetermined direction from a virtual viewpoint within the vehicle cabin, in which a subject sways due to the behavior of the vehicle, outputs a control signal for displaying the viewpoint image on a display device provided in a space different from the vehicle cabin, and obtains information regarding the gaze point of a user viewing the display device. When the vehicle surroundings image is set as the target object, a viewpoint image in which the shaking level of the vehicle surroundings image is lower than the shaking level of the interior image, or when the interior image is set as the target object, a viewpoint image in which the shaking level of the interior image is lower than the shaking level of the vehicle surroundings image. The above problem is solved by generating Effect of the Invention

[0007] According to the present invention, it is possible to prevent the sense of immersion of a user who is not in a vehicle from being hindered. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram of an image display system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an explanatory diagram for explaining a viewpoint image. [Diagram 3] 5 is a flowchart showing image display control executed by the image generating device according to the present embodiment. [Figure 4] 10 is a flowchart showing an image display control executed by an image generating device according to a modified example. [Diagram 5] 13 is an example of a virtual space when the display device is incorporated in a head mounted display. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, an image generating device and an image generating method according to the present invention will be described using an example in which the image generating device and the image generating method are applied to an image display system.

[0010] 1 is a block diagram of an image display system 100 according to this embodiment. The image display system 100 according to this embodiment is used in a scene where a virtual space that reproduces a space where a user B is present is displayed to a user A who is in a space different from the user B. In this embodiment, a user who is in a remote space 30 that is a place away from the vehicle 1 is taken as an example of the user A, and a user who is in the vehicle 1 and in the in-vehicle space 20 is taken as an example of the user B. In the following description, for convenience, the user A who is in the remote space 30 is referred to as the remote user A, and the user B who is in the in-vehicle space 20 is referred to as the in-vehicle user B.

[0011] In this embodiment, the states of the remote user A and the in-vehicle user B are not particularly limited. For example, in FIG. 1, the state of the remote user A is shown as a seated state, but the remote user A may be standing. Also, for example, in FIG. 1, the state of the in-vehicle user B is shown as a state in which the steering wheel of the vehicle 1 is being operated, but the in-vehicle user B may not be the driver, but may be a passenger sitting in the passenger seat or the back seat.

[0012] In this embodiment, the image display system 100 is used in a system that uses virtual reality technology. As shown in FIG. 1, the remote user A and the in-car user B are in different spaces, so the remote user A cannot confirm what kind of scene the in-car user B is viewing in the in-car space 20. However, in the virtual reality technology that allows a user to perceive a virtual space, which is a world created by a computer, as reality, the computer can create a virtual space for the remote user A as if he were in the in-car space 20. In the virtual space, the remote user A perceives as if he were in the in-car space 20 through at least his vision and hearing. Furthermore, the image generating device and image generating method according to the present invention change the virtual space according to the behavior of the vehicle 1, so that the remote user A can grasp the behavior of the vehicle 1 through his vision even while in the remote space 30. In addition, the target object that the remote user A is gazing at does not shake as compared to other objects, so that the remote user A can be prevented from getting sick due to the change in the virtual space caused by the behavior of the vehicle 1. Hereinafter, each component of the image display system 100 will be described with reference to FIG.

[0013] 1, the image display system 100 includes an image generating device 10, an in-vehicle space 20, and a remote space 30. The remote space 30 is a space separated from the in-vehicle space 20. The remote space 30 may be, for example, an interior of a vehicle other than the vehicle 1 or a room in a building, but the remote space 30 is not particularly limited as long as it is a space separate from the in-vehicle space 20.

[0014] The vehicle interior space 20 is a space showing the interior of the vehicle 1. The vehicle 1 is provided with an on-board camera 21, an on-board sensor 22, and an on-board communication device 23. In FIG. 1, these devices are shown as being provided in the vehicle interior space 20, but the on-board camera 21, the on-board sensor 22, and the on-board communication device 23 are assumed to be mounted on the vehicle 1. The devices provided in the vehicle 1 are connected to each other by, for example, a Controller Area Network (CAN) or other on-board network (such as Ethernet (registered trademark)) in order to transmit and receive information to and from each other.

[0015] The on-board camera 21 is an imaging device that captures the surroundings of the vehicle 1. The on-board camera 21 is preferably provided so as to capture the range visible to the occupants of the vehicle 1, and may be configured with multiple cameras, such as a front camera that captures the front of the vehicle, a side camera that captures the side of the vehicle, and a rear camera that captures the rear of the vehicle. The captured images captured by the on-board camera 21 are output to the on-board communication device 23 as captured image data of the surroundings of the vehicle 1. The on-board camera 21 may be a device or equipment that captures the range visible to the occupants of the vehicle 1, and its form, number, characteristics, installation location, etc. are not particularly limited. For example, the on-board camera 21 may be configured with multiple cameras or multiple types of cameras.

[0016] For example, the function of the in-vehicle camera 21 is an anti-shake function that prevents the captured image from shaking due to the shaking of the vehicle 1. The anti-shake function is a function that prevents the shaking of the vehicle 1 from being reflected in the captured image. When the in-vehicle camera 21 has the anti-shake function and the function is enabled, the in-vehicle camera 21 captures an image in which the shaking is suppressed even when the vehicle 1 is shaking. The in-vehicle camera 21 outputs information on whether or not the in-vehicle camera 21 has the anti-shake function, and if so, information on whether or not the function is enabled or disabled, to the in-vehicle communication device 23 as in-vehicle camera data.

[0017] The on-board sensor 22 is a sensor that detects the current state of the vehicle 1. The current state of the vehicle 1 is the current behavior of the vehicle 1. The current behavior of the vehicle 1 includes the current attitude of the vehicle 1. The type and number of the on-board sensors 22 are not particularly limited. In this embodiment, the acceleration and angular acceleration of the vehicle 1 are described as examples of parameters indicating the behavior of the vehicle 1, but the parameters indicating the behavior of the vehicle 1 may include at least one of the acceleration and angular velocity of the vehicle 1. As shown in FIG. 1, the on-board sensor 22 is composed of an acceleration sensor 22a and an angular acceleration sensor 22b.

[0018] The acceleration sensor 22a is a so-called three-axis acceleration sensor, which is a sensor that measures the acceleration on three axes of the vehicle 1. The three axes of the vehicle 1 are an axis (x-axis) along the longitudinal direction of the vehicle 1, an axis (y-axis) along the lateral direction of the vehicle 1, and an axis (z-axis) along the up-down direction of the vehicle 1. An example of the acceleration sensor 22a is a MEMS type acceleration sensor. Note that the method of measuring the acceleration in the acceleration sensor 22a is not particularly limited. The acceleration on each axis of the vehicle 1 measured by the acceleration sensor 22a is output to the in-vehicle communication device 23 as acceleration data of the vehicle 1. The acceleration sensor 22a may be a device or equipment that measures the acceleration on three axes of the vehicle 1, and the form, number, characteristics, installation location, etc. of the acceleration sensor 22a are not particularly limited. For example, the acceleration sensor 22a may be configured with a plurality of acceleration sensors or a plurality of types of acceleration sensors.

[0019] The angular acceleration sensor 22b is a sensor that measures the rotational motion of the vehicle 1. The rotational motion of the vehicle 1 includes the rotational motion of the vehicle 1 in the roll direction, the pitch direction, and the yaw direction. The angular acceleration sensor 22b is composed of, for example, a roll angular acceleration sensor that measures the angular acceleration of the vehicle 1 in the roll direction, a pitch angular acceleration sensor that measures the angular acceleration of the vehicle 1 in the pitch direction, and a yaw angular acceleration sensor that measures the angular acceleration of the vehicle 1 in the yaw direction. Note that the method of measuring the angular acceleration in the angular acceleration sensor 22b is not particularly limited. The angular acceleration in each direction of the vehicle 1 measured by the angular acceleration sensor 22b is output to the in-vehicle communication device 23 as angular acceleration data of the vehicle 1. The angular acceleration sensor 22b may be a device or equipment that measures the angular acceleration in three directions of the vehicle 1, and the form, number, characteristics, installation location, and the like of the angular acceleration sensor 22b are not particularly limited. For example, the angular acceleration sensor 22b may be composed of multiple angular acceleration sensors or multiple types of angular acceleration sensors.

[0020] The in-vehicle communication device 23 is a device having a wireless communication function for wirelessly transmitting and receiving data to and from the outside of the vehicle 1. An example of the in-vehicle communication device 23 is a telematics control unit (TCU). The in-vehicle communication device 23 is connected to the Internet using a communication standard such as 4G / LTE or Wifi (registered trademark) by the wireless communication function, and transmits and receives various data to and from a server or a system provided outside the vehicle 1. The in-vehicle communication device 23 receives peripheral captured image data of the vehicle 1 and in-vehicle camera data from the in-vehicle camera 21. The in-vehicle communication device 23 also receives acceleration data and angular acceleration data of the vehicle 1 from the in-vehicle sensor 22. The in-vehicle communication device 23 transmits peripheral captured image data, acceleration data, and angular acceleration data of the vehicle 1 to the image generating device 10 at predetermined intervals.

[0021] The remote space 30 will now be described. As shown in Fig. 1, the remote space 30 is provided with a remote communication device 31, a display device 32, a remote camera 33, and a remote controller 34. The devices provided in the remote space 30 are connected to each other by, for example, Ethernet (registered trademark) in order to transmit and receive information to each other.

[0022] The remote communication device 31 is a device equipped with a wireless communication function for wirelessly transmitting and receiving data to and from the outside of the remote space 30. For example, a router can be used as the remote communication device 31. The wireless communication function of the remote communication device 31 is similar to the wireless communication function of the in-vehicle communication device 23, and therefore the description of the wireless communication function of the in-vehicle communication device 23 is used. In this embodiment, the remote communication device 31 transmits and receives data to and from the image generation device 10. Image data is input to the remote communication device 31 from the image generation device 10, and the remote communication device 31 outputs the image data to the display device 32. Furthermore, gaze point information of the remote user A is input to the remote communication device 31 from the remote controller 34, and the remote communication device 31 transmits the gaze point information of the remote user A to the image generation device 10.

[0023] The display device 32 displays the image generated by the image generation device 10. Image data generated by the image generation device 10 is input to the display device 32 via the remote communication device 31. In this embodiment, the display device 32 is a screen display portion of a head mounted display worn by the remote user A. Note that the display device 32 is not limited to being provided in a head mounted display, and may be, for example, a television or a monitor having a display such as a liquid crystal panel.

[0024] The remote camera 33 captures an image of the remote user A viewing the display device 32. The remote camera 33 is preferably provided at a location where it can capture an image of an object viewed by the remote user A in an image displayed by the display device 32. For example, the remote camera 33 is provided on a head-mounted display worn by the remote user A together with the display device 32. There may be a plurality of remote cameras 33, and the remote camera 33 may be configured, for example, with a camera that captures an image displayed by the display device 32 (hereinafter, also referred to as a display image) and a camera that captures the eyes of the remote user A viewing the display device 32. Note that there may be a plurality of cameras that capture the image displayed by the display device 32 and the remote user A. The captured image captured by the remote camera 33 is output to the remote controller 34.

[0025] The remote controller 34 is a computer including a ROM (Read Only Memory) storing a program for identifying an object that the remote user A is gazing at in the display image of the display device 32, a CPU (Central Processing Unit) that executes the program stored in the ROM, and a RAM (Random Access Memory) that functions as an accessible storage device. The remote controller 34 is provided, for example, in a head mounted display worn by the remote user A together with the display device 32 and the remote camera 33. The remote controller 34 may be provided in a location separate from the display device 32 and the remote camera 33.

[0026] The remote controller 34 acquires an image captured by the remote camera 33, which includes the eyes of the remote user A, and an image displayed on the display device 32. The remote controller 34 identifies the gaze direction of the remote user A by image processing. The remote controller 34 also identifies an object located in the gaze of the remote user A in the display image on the display device 32 by image processing. As a result, the remote controller 34 determines that the identified object is an object gazed at by the remote user, and identifies the position of the object as the gaze point of the remote user A. Note that the method of identifying the gaze point of the remote user A by the remote controller 34 may be any identifying method known at the time of filing the present application, and is not limited to the above identifying method. The gaze point information of the remote user A, including the object gazed at by the remote controller 34 and the position of the object in the display image on the display device 32, is output to the remote communication device 31.

[0027] The image generating device 10 will be described. In this embodiment, the image generating device 10 is provided in a server (not shown) located outside the vehicle interior space 20 and the remote space 30. The image generating device 10 wirelessly transmits and receives data between the in-vehicle communication device 23 and the remote communication device 31 via a communication device (not shown) in the server. The installation location of the image generating device 10 is not particularly limited, and for example, the image generating device 10 may be mounted on the vehicle 1. Furthermore, for example, the image generating device 10 may be provided in the remote space 30.

[0028] The image generating device 10 is a computer including a ROM (Read Only Memory) storing a program for generating and processing an image to be viewed by a remote user A, a CPU (Central Processing Unit) executing the program stored in the ROM, and a RAM (Random Access Memory) functioning as an accessible storage device. The image generating device 10 includes a vehicle information acquiring unit 11, a storage unit 12, a gaze point information acquiring unit 13, an image generating unit 14, and a display control unit 15. The image generating device 10 can execute the functions of the vehicle information acquiring unit 11, the storage unit 12, the gaze point information acquiring unit 13, the image generating unit 14, and the display control unit 15 by executing the program stored in the ROM. Each function of the image generating device 10 will be described.

[0029] The vehicle information acquisition unit 11 acquires vehicle information including information on the behavior of the vehicle 1 from a device provided in the vehicle 1. Specifically, the vehicle information acquisition unit 11 acquires, via the in-vehicle communication device 23, an image of the surroundings of the vehicle 1 captured by the in-vehicle camera 21, information on the vibration prevention function of the in-vehicle camera 21, and the acceleration and angular acceleration of the vehicle 1 detected by the in-vehicle sensor 22. For example, the vehicle information acquisition unit 11 acquires vehicle information from the vehicle 1 at predetermined intervals.

[0030] The storage unit 12 stores an interior image showing the interior of the vehicle 1. The interior image is used by the image generation unit 14 to generate a virtual space that reproduces the interior space 20 of the vehicle 1. The storage unit 12 stores a plurality of types of interior images, and stores an interior image for each model and grade of the vehicle 1, for example. Examples of the interior image include an image of an instrument panel including a steering wheel, a shift lever, a navigation operation unit, an air conditioner operation unit, a dashboard, a speedometer, and the like. Note that the interior image is not limited to an image of an instrument panel, and may be, for example, an image of a driver's seat, an image of a passenger seat, an image of a back seat, an image showing the inside of a door, an image of a ceiling or a floor, and the like.

[0031] The gaze point information acquisition unit 13 acquires information related to the gaze point of the remote user A viewing the display device 32. The gaze point information acquisition unit 13 acquires the gaze point information of the remote user A from the remote controller 34 via the remote communication device 31. The gaze point information of the remote user A includes at least an object that the remote user A is gazing at in the display image of the display device 32, and the position of the object in the display image of the display device 32.

[0032] The image generating unit 14 generates a virtual space that reproduces the vehicle interior space 20, and generates an image viewed from a predetermined virtual viewpoint. The image generating unit 14 also generates an image in which a subject sways in accordance with the behavior of the vehicle 1. In this embodiment, an image viewed in a predetermined direction from a virtual viewpoint in the vehicle cabin of the vehicle 1, in which the subject sways due to the behavior of the vehicle 1, will be described as a viewpoint image.

[0033] In order to generate a viewpoint image, the image generating unit 14 sets a virtual viewpoint at a predetermined position in the cabin of the vehicle 1. There is no particular limitation on where in the cabin the virtual viewpoint is set, but it is preferable that the virtual viewpoint is set at a position where, when an image is displayed on the display device 32, the remote user A viewing the image can feel that he or she is riding in the vehicle 1. For example, the image generating unit 14 sets the virtual viewpoint at a position corresponding to the head of an occupant of the vehicle 1. Note that position information of the head of the occupant of the vehicle 1 is stored in advance in the storage unit 12.

[0034] Next, an example of a method for generating a viewpoint image will be described. When the image generating unit 14 sets a virtual viewpoint, it generates an image showing the surroundings of the vehicle 1 based on the captured image of the surroundings of the vehicle 1. Then, the image generating unit 14 reads out an interior image of the vehicle 1 seen from the virtual viewpoint from the storage unit 12, and generates a virtual space that reproduces the interior space 20 of the vehicle 1 by superimposing the interior image on the image showing the surroundings of the vehicle 1. Furthermore, the image generating unit 14 generates a viewpoint image by cutting out an image of a part of the virtual space that is included in the range of the virtual field of view based on the virtual viewpoint, based on the range of the virtual field of view set in advance.

[0035] FIG. 2 is an explanatory diagram for explaining a viewpoint image. FIG. 2(A) is an example of a virtual space showing the interior of the vehicle 1 generated by the image generation unit 14. In FIG. 2(A), A is an interior image of the vehicle 1 including a car navigation display a and a steering wheel b, and B is a captured image of the surroundings of the vehicle 1. The image generation unit 14 generates a virtual space showing the interior of the vehicle 1 by superimposing and displaying the interior image A on the captured image of the surroundings B. Also, in FIG. 2(A), C indicates a portion included in the range of the virtual field of view when looking ahead of the vehicle 1 from the virtual viewpoint.

[0036] Fig. 2(B) shows the viewpoint image C displayed on the display device 32. The display device 32 displays the viewpoint image C as an image seen from a virtual viewpoint ahead of the vehicle 1. In the example of Fig. 2, the viewpoint image C includes an interior image A showing the state inside the cabin of the vehicle 1 seen from the virtual viewpoint, and a surrounding captured image B of the vehicle 1 showing the surroundings of the vehicle 1 seen from the virtual viewpoint.

[0037] Next, the movement of the subject in the viewpoint image will be described. The image generating unit 14 generates the viewpoint image such that, among the subjects appearing in the viewpoint image, at least the subject other than the subject gazed upon by the remote user A sways due to the behavior of the vehicle 1. The image generating unit 14 sets the object gazed upon by the remote user A among the subjects appearing in the viewpoint image as the target object. In this embodiment, the image generating unit 14 sets, as the target object, either one of the interior image and the peripheral captured image of the vehicle 1 that constitute the viewpoint image, based on the gaze point of the remote user A.

[0038] For example, when the object gazed upon by the remote user A is a subject that appears in the interior image, the image generating unit 14 sets the interior image as the target object. In the example of the viewpoint image shown in FIG. 2B, it is assumed that the remote user A gazes at the display a. When the image generating unit 14 identifies the gaze point of the remote user A as the position of the display a from the gaze point information of the remote user A acquired by the gaze point information acquiring unit 13, it sets the interior image A including the display a as the target object. On the other hand, for example, when the object gazed upon by the remote user A is a subject that appears in the peripheral captured image of the vehicle 1, the image generating unit 14 sets the peripheral captured image of the vehicle 1 as the target object. In the example of the viewpoint image shown in FIG. 2B, it is assumed that the remote user A gazes at other vehicles and traffic lights other than the vehicle 1 that appear in the peripheral captured image B. When the image generation unit 14 identifies the gaze point of remote user A as the position of another vehicle or a traffic light from the gaze point information of remote user A acquired by the gaze point information acquisition unit 13, it sets the surrounding captured image B including the other vehicles and traffic lights as the target object.

[0039] When the image generating unit 14 sets a target object in the interior image or the surrounding captured image of the vehicle 1, the image generating unit 14 generates a viewpoint image in which the rocking level of the target object is lower than the rocking level of objects other than the target object. The rocking level is the rocking width or speed at which the subject rocks in response to the behavior of the vehicle 1. For example, when the object that the remote user A is gazing at is included in the interior image and the target object is set in the interior image, the image generating unit 14 generates a viewpoint image in which the rocking level at which the interior image rocks in response to the behavior of the vehicle 1 is lower than the rocking level at which the surrounding captured image of the vehicle 1 rocks in response to the behavior of the vehicle 1. On the other hand, when the object that the remote user A is gazing at is included in the surrounding captured image of the vehicle 1 and the target object is set in the surrounding captured image of the vehicle 1, the image generating unit 14 generates a viewpoint image in which the rocking level at which the surrounding captured image of the vehicle 1 rocks in response to the behavior of the vehicle 1 is lower than the rocking level at which the interior image rocks in response to the behavior of the vehicle 1. It should be noted that as long as the oscillation level of the target object is lower than the oscillation levels of objects other than the target object, there is no particular limitation as to whether the target object sways in response to the behavior of the vehicle 1 or not.

[0040] In this embodiment, the image generating unit 14 changes the display mode of the object other than the target object in accordance with the behavior of the vehicle 1 in order to make the rocking level of the target object lower than the rocking level of the object other than the target object. The change in the display mode of the object other than the target object includes at least one of a change in the size of the image, a change in the display position of the image, and a change in the tilt angle of the image. The image generating unit 14 sets the rate at which the display mode of the object other than the target object is changed to the same rate as the displacement amount of the virtual viewpoint according to the behavior of the vehicle 1. In other words, the image generating unit 14 changes the display mode of the object other than the target object in a directly proportional relationship to the displacement amount of the virtual viewpoint. The change in the size of the image, the change in the display position of the image, and the change in the tilt angle of the image will be described with examples.

[0041] An example of changing the size of an image will be described. For example, when the vehicle 1 decelerates and a backward acceleration occurs in the vehicle 1, a forward acceleration occurs in the occupant of the vehicle 1. In this case, the image generating unit 14 enlarges the object other than the target object by a predetermined magnification. As a result, a virtual space in which the object other than the target object is enlarged is generated. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses a state in which the distance between the virtual viewpoint and the object other than the target object has decreased. On the other hand, for example, when the vehicle 1 accelerates and a forward acceleration occurs in the vehicle 1, a backward acceleration occurs in the occupant of the vehicle 1. In this case, the image generating unit 14 reduces the object other than the target object by a predetermined magnification. As a result, a virtual space in which the object other than the target object is reduced is generated. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses a state in which the distance between the virtual viewpoint and the object other than the target object has increased.

[0042] An example of changing the display position of an image will be described. For example, when the vehicle 1 accelerates and the vehicle 1 sinks backward, the viewpoint of the occupant of the vehicle 1 is moved upward. In this case, the image generating unit 14 moves objects other than the target object downward by a predetermined distance. As a result, a virtual space is generated in which the display positions of objects other than the target object have moved downward. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses the state in which objects other than the target object viewed from the virtual viewpoint have moved downward. On the other hand, for example, when the vehicle 1 decelerates and the vehicle 1 sinks forward, the viewpoint of the occupant of the vehicle 1 is moved downward. In this case, the image generating unit 14 moves objects other than the target object upward by a predetermined distance. As a result, a virtual space is generated in which the display positions of objects other than the target object have moved upward. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses the state in which objects other than the target object viewed from the virtual viewpoint have moved upward.

[0043] An example of changing the tilt angle of an image will be described. For example, when the vehicle 1 is traveling around a curve in a left direction, if the vehicle 1 is tilted to the left, the vehicle occupant's posture is also tilted to the left. In this case, the image generating unit 14 tilts everything except the target object to the left (counterclockwise direction) by a predetermined angle. This generates a virtual space in which everything except the target object is tilted to the left. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses the state in which everything except the target object is tilted to the left as seen from the virtual viewpoint. On the other hand, for example, when the vehicle 1 is traveling around a curve in a right direction, if the vehicle 1 is tilted to the right, the vehicle occupant's posture is also tilted to the right. In this case, the image generating unit 14 tilts everything except the target object to the right (clockwise direction) by a predetermined angle. This generates a virtual space in which everything except the target object is tilted to the right. Then, the image generating unit 14 generates a viewpoint image from the generated virtual space, which expresses the state in which everything except the target object is tilted to the right as seen from the virtual viewpoint.

[0044] In this way, the image generating unit 14 changes the size, display position, and tilt angle of objects other than the target object in accordance with the behavior of the vehicle 1, thereby generating a viewpoint image in which objects other than the target object sway in accordance with the behavior of the vehicle 1. In other words, the image generating unit 14 executes at least one of a process of changing the size of objects other than the target object, a process of changing the display position of objects other than the target object, a process of changing the tilt angle of objects other than the target object, and a combination of these processes, as a process of shaking objects other than the target object in accordance with the behavior of the vehicle 1 (hereinafter, also referred to as a shaking process).

[0045] Here, the relationship between the sway level and the display mode of objects other than the target object will be described. The image generating unit 14 sets the sway level of objects other than the target object based on the acceleration and angular acceleration of the vehicle 1 among the vehicle information of the vehicle 1. For example, the image generating unit 14 sets the sway level to be larger as the acceleration of the vehicle 1 or the angular acceleration of the vehicle 1 is faster, and executes a sway process in which objects other than the target object sway more. The image generating unit 14 generates at least one of a viewpoint image in which the sway width of the target object is smaller than the sway width of objects other than the target object, and a viewpoint image in which the speed of the target object is slower than the speed of objects other than the target object.

[0046] In the process of changing the size of objects other than the target object, the image generating unit 14 sets a larger enlargement rate of objects other than the target object or a smaller reduction rate of objects other than the target object as the shaking level increases. Also, the image generating unit 14 sets a shorter time required for objects other than the target object to change to the set enlargement rate or reduction rate as the shaking level increases. As a result, the speed at which the size of objects other than the target object changes becomes faster and the rate at which the size changes becomes greater as the shaking level of objects other than the target object increases, i.e., the greater the degree to which the vehicle 1 is shaking.

[0047] Furthermore, in the process of changing the display position of the object other than the target object, the image generating unit 14 sets a longer distance for the object other than the target object to move as the shaking level increases. Furthermore, the image generating unit 14 sets a shorter time required for the object other than the target object to move to the changed display position as the shaking level increases. As a result, the speed at which the object other than the target object moves becomes faster and the distance it moves becomes longer as the shaking level of the object other than the target object increases, i.e., the degree to which the vehicle 1 is shaking increases.

[0048] Furthermore, in the process of changing the tilt angle of objects other than the target object, the image generating unit 14 sets a larger tilt angle of objects other than the target object as the sway level increases. Furthermore, the image generating unit 14 sets a shorter time required for objects other than the target object to tilt as the sway level increases. As a result, the speed at which objects other than the target object tilt becomes faster and the tilt angle becomes larger as the sway level of objects other than the target object increases, i.e., the greater the degree to which the vehicle 1 is swaying.

[0049] The display control unit 15 outputs a control signal for displaying the viewpoint image generated by the image generation unit 14 on the display device 32 provided in the remote space 30. For example, the display control unit 15 converts image data of the viewpoint image into a control signal that can be displayed by the display device 32. The display control unit 15 transmits the converted control signal to the remote communication device 31 via a communication device (not shown) provided in the server.

[0050] Next, an image display control flow executed by the image generating device 10 will be described with reference to the flowchart shown in Fig. 3. Fig. 3 is a flowchart showing the image display control executed by the image generating device 10. The image generating device 10 executes the process of the flowchart shown in Fig. 3 at predetermined intervals.

[0051] In step S101, the image generating device 10 acquires vehicle information from a device provided in the vehicle 1. The vehicle information includes a captured image of the surroundings of the vehicle 1 captured by the on-board camera 21, information on the anti-shake function of the on-board camera 21, acceleration in three axes of the vehicle 1 measured by the acceleration sensor 22a, and angular acceleration in three directions of the vehicle 1 measured by the angular acceleration sensor 22b. In step S101, the image generating device 10 acquires the vehicle information at predetermined intervals, and ends the process of step S101 after a predetermined time has elapsed.

[0052] In step S102, the image generating device 10 acquires gaze point information of the remote user A from the remote controller 34 via the remote communication device 31. The gaze point information of the remote user A includes an object that the remote user A is viewing in the display image of the display device 32 and a position of the object in the display image of the display device 32.

[0053] In step S103, the image generating device 10 determines whether the subject gazed at by the remote user A is included in either the interior image or the captured image of the surroundings of the vehicle 1, based on the gaze point information of the remote user A acquired in step S102. When the image generating device 10 determines, based on the gaze point information of the remote user A, that the subject gazed at by the remote user A is an object appearing in the interior image, the image generating device 10 sets the interior image as the target object. On the other hand, when the image generating device 10 determines, based on the gaze point information of the remote user A, that the subject gazed at by the remote user A is an object appearing in the captured image of the surroundings of the vehicle 1, the image generating device 10 sets the captured image of the surroundings of the vehicle 1 as the target object. Examples of methods for identifying the remote user A from the gaze point information of the remote user A include a method of identifying the remote user A from the type of subject gazed at by the remote user A, and a method of identifying the remote user A from the position of the gaze point of the remote user A in the display image of the display device 32. If the target object is set to the interior image, the process proceeds to step S104, and if the target object is set to the surrounding captured image of the vehicle 1, the process proceeds to step S108.

[0054] If the target object is set to an interior image in step S103, the process proceeds to step S104. In step S104, the image generating device 10 determines whether or not the shake prevention function of the vehicle-mounted camera 21 is enabled. For example, the image generating device 10 determines whether or not the shake prevention function of the vehicle-mounted camera 21 is enabled based on information related to the shake prevention function of the vehicle-mounted camera 21 among the vehicle information acquired in step S101. If it is determined that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S105, and if it is determined that the shake prevention function of the vehicle-mounted camera 21 is disabled, the process proceeds to step S106.

[0055] If it is determined in step S104 that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S105. In step S105, since the interior image is set as the target object in step S103, the image generating device 10 performs a shaking process on the peripheral captured image of the vehicle 1, which is other than the target object. Since it is determined in step S103 that the shake prevention function of the vehicle-mounted camera 21 is enabled, the peripheral captured image of the vehicle 1 does not shake or is less likely to shake in response to the shaking of the vehicle 1. Therefore, in this step, the image generating device 10 performs a shaking process on the peripheral captured image of the vehicle 1, and processes the peripheral captured image of the vehicle 1 so that it shakes in response to the shaking of the vehicle 1. The shaking process includes at least one of a process of changing the size of the peripheral captured image of the vehicle 1, a process of changing the display position of the peripheral captured image of the vehicle 1, and a process of changing the tilt angle of the peripheral captured image of the vehicle 1.

[0056] In step S106, the image generating device 10 sets a virtual viewpoint at a position corresponding to the head of an occupant of the vehicle 1, and generates a virtual space that reproduces the interior space of the vehicle 1 by superimposing an interior image on a captured image of the surroundings of the vehicle 1 (see FIG. 2(A)). Then, the image generating device 10 generates a viewpoint image, which is an image seen in a predetermined direction from the virtual viewpoint in the cabin of the vehicle 1, by cutting out an image of a portion of the virtual space that is included in a virtual field of view based on the virtual viewpoint (see FIG. 2(B)).

[0057] When the process proceeds from step S105 to step S106, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 that has been processed in step S105 to shake in accordance with the shaking of the vehicle 1. On the other hand, when the process proceeds from step S104 to step S106, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 captured by the in-vehicle camera 21 itself. This is because the vibration prevention function of the in-vehicle camera 21 is determined to be disabled in step S104, and therefore the captured image of the surroundings of the vehicle 1 captured by the in-vehicle camera 21 shakes in accordance with the shaking of the vehicle 1.

[0058] In step S107, the image generating device 10 outputs the viewpoint image generated in step S108 to the display device 32 via the communication device provided in the server and the remote communication device 31. When the process in step S108 ends, the image generating device 10 ends the process of the flowchart shown in FIG.

[0059] If the target object is set in the peripheral captured image of the vehicle 1 in step S103, the process proceeds to step S108. In step S108, the image generating device 10 determines whether or not the shake prevention function of the vehicle-mounted camera 21 is enabled. Since step S108 corresponds to step S104, the explanation of step S104 is cited. If it is determined that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S109, and if it is determined that the shake prevention function of the vehicle-mounted camera 21 is disabled, the process proceeds to step S111.

[0060] If it is determined in step S108 that the shake prevention function of the in-vehicle camera 21 is enabled, the process proceeds to step S109. In step S109, the image generating device 10 executes a shaking process on the interior image, which is other than the target object, because the peripheral captured image of the vehicle 1 is set as the target object in step S103. The shaking process includes at least one of a process of changing the size of the interior image of the vehicle 1, a process of changing the display position of the interior image of the vehicle 1, and a process of changing the tilt angle of the interior image of the vehicle 1.

[0061] In step S110, the image generating device 10 generates a viewpoint image, which is an image seen in a predetermined direction from a virtual viewpoint in the cabin of the vehicle 1. Since step S110 corresponds to step S106, the explanation of step S106 is cited. When the processing of step S110 ends, the process proceeds to step S107. The explanation of step S107 is the same as that given above.

[0062] If it is determined in step S108 that the shake prevention function of the vehicle-mounted camera 21 is disabled, the process proceeds to step S111. In step S111, the image generating device 10 reduces the shaking of the peripheral captured image of the vehicle 1 because the peripheral captured image of the vehicle 1 shakes in response to the shaking of the vehicle 1. The image generating device 10 executes a shake reduction process so that the shaking of the peripheral captured image of the vehicle 1 is reduced even if the vehicle 1 shakes. As a result, even if the vehicle 1 shakes, the object that the remote user A is gazing at is less likely to shake, so that the possibility of the remote user A becoming sick can be reduced. When the process in step S108 ends, the process proceeds to step S109. The explanations given above are used for the explanations after step S109. It should be noted that the shake reduction process known at the time of filing of the present application can be applied to the image generating device 10.

[0063] As described above, the image generating device 10 according to the present embodiment includes the vehicle information acquiring unit 11 that acquires vehicle information including information on the behavior of the vehicle 1 from the on-board camera 21 and the on-board sensor 22 provided in the vehicle 1, the image generating unit 14 that generates a viewpoint image, which is an image viewed in a predetermined direction from a virtual viewpoint in the cabin of the vehicle 1, in which a subject sways due to the behavior of the vehicle 1, the display control unit 15 that outputs a control signal for displaying the image generated by the image generating unit 14 on the display device 32 provided in the remote space 30, and the gaze point information acquiring unit 13 that acquires information on the gaze point of the remote user A viewing the display device 32. In the present embodiment, the image generating unit 14 generates a viewpoint image in which the sway level of a target object corresponding to the gaze point of the remote user A among the subjects appearing in the viewpoint image is lower than the sway level of objects other than the target object. As a result, the remote user A who is not riding in the vehicle 1 can easily grasp the behavior of the vehicle 1 through vision without impeding the sense of immersion by viewing the viewpoint image in which objects other than the object he or she is gazing at sway in response to the behavior of the vehicle 1. In addition, the target that the remote user A is gazing at is less likely to be shaken in response to the behavior of the vehicle 1 than other targets, so that the remote user A can be prevented from becoming motionless.

[0064] In this embodiment, the rocking level is a rocking width or a rocking speed at which the interior image or the captured image of the vehicle 1 rocks in response to the behavior of the vehicle 1. This allows the remote user A to intuitively grasp how the vehicle 1 is behaving from the object captured in his / her peripheral vision.

[0065] Furthermore, in this embodiment, the image generating unit 14 generates at least one of a viewpoint image in which the swing width of the target object is smaller than that of objects other than the target object and a viewpoint image in which the speed of the target object is slower than that of objects other than the target object. This makes it possible to prevent the remote user A from getting sick, since objects included in the central visual field of the remote user A are less likely to swing and objects included in the peripheral visual field of the remote user A are more likely to swing.

[0066] In addition, in this embodiment, the vehicle information includes a peripheral image of the vehicle 1 captured by the on-board camera 21 installed in the vehicle, and the viewpoint image includes an interior image showing the interior of the vehicle as seen from a virtual viewpoint, and a peripheral image of the vehicle 1. The image generating unit 14 sets one of the interior image and the peripheral image of the vehicle 1 as a target object based on the gaze point of the remote user A. As a result, when the remote user A is gazing at the periphery of the vehicle 1, a viewpoint image in which the vibration level of the peripheral image of the vehicle 1 is lower than the vibration level of the interior image can be generated. Also, when the remote user A is in the interior space of the vehicle 1, a viewpoint image in which the vibration level of the interior image is lower than the vibration level of the peripheral image of the vehicle 1 can be generated.

[0067] In addition, in this embodiment, when the image generating unit 14 sets the peripheral captured image of the vehicle 1 as the target object, the image generating unit 14 generates a viewpoint image in which the shaking level of the peripheral captured image of the vehicle 1 is lower than the shaking level of the interior image. As a result, when the remote user A is gazing at the periphery of the vehicle 1, the remote user A can intuitively grasp the behavior of the vehicle 1 from the shaking of the interior image while suppressing the remote user A from getting sick.

[0068] Furthermore, in this embodiment, when the image generating unit 14 sets an interior image as the target object, it generates a viewpoint image in which the shaking level of the interior image is lower than the shaking level of the captured image of the surroundings of the vehicle 1. As a result, when the remote user A is gazing at the interior space of the vehicle 1, the remote user A can intuitively grasp the behavior of the vehicle 1 from the shaking of the captured image of the surroundings of the vehicle 1 while suppressing the remote user A from getting sick.

[0069] Additionally, in this embodiment, the vehicle information of the vehicle 1 includes at least one of the acceleration and angular acceleration of the vehicle 1. This makes it possible to generate viewpoint images in which the subject oscillates in accordance with the behavior of the vehicle 1.

[0070] The above-described embodiments are described for the purpose of facilitating understanding of the present invention, and are not described for the purpose of limiting the present invention. Therefore, the elements disclosed in the above embodiments are intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

[0071] For example, in the above-described embodiment, an interior image or a captured image of the periphery of the vehicle 1 is described as an example of the target object, but the method is not limited to alternatively selecting an interior image or a captured image of the periphery of the vehicle 1 to set the target object, and a predetermined range including the gaze point of the remote user A may be set as the target object. In this case, the image generating device 10 generates a viewpoint image in which the rocking level of the predetermined range set as the target object is lower than the rocking level outside the predetermined range. An image generating device 10 according to a modified example in which the image generating unit 14 is different from the above-described embodiment will be described.

[0072] An image display control flow executed by image generating device 10 according to a modified example will be described with reference to Fig. 4. Fig. 4 is a flowchart showing image display control executed by image generating device 10 according to a modified example. Compared to the flowchart in Fig. 3, the flowchart in Fig. 4 replaces steps S103 to S106 and steps S108 to S110 with steps S201 to S204. Therefore, for the steps described using the flowchart in Fig. 3, the above-mentioned explanation will be used and steps S201 to S204 will be described.

[0073] In step S201, the image generating device 10 judges whether or not the shake prevention function of the vehicle-mounted camera 21 is enabled. Since step S201 corresponds to step S104 and step S108 in Fig. 3, the explanation of step S104 and step S108 is cited. If it is judged that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S202, and if it is judged that the shake prevention function of the vehicle-mounted camera 21 is disabled, the process proceeds to step S204.

[0074] If it is determined in step S201 that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S202. In step S202, the image generating device 10 performs a shake process on the outside of a predetermined range including the gaze point of the remote user A. For example, the image generating device 10 identifies the position of the subject that the remote user A is gazing at in the display image of the display device 32 from the gaze point information of the remote user A acquired in step S102. The image generating device 10 sets a predetermined range including the subject as the target object. The predetermined range may be a range including the subject that the remote user A is gazing at, and its shape, size, etc. are not particularly limited. For example, the predetermined range is stored in advance in the storage unit 12 as a range indicating the central visual field of the remote user A.

[0075] When the image generating device 10 sets a predetermined range including the gaze point of the remote user A as the target object, the image generating device 10 executes a rocking process on the outside of the target object. For example, when the range of the central visual field of the remote user A is set as the target object, the image generating device 10 executes a rocking process on the object included in the peripheral visual field of the remote user A. In a modified example, the object of the rocking process is either an interior image, a captured image of the periphery of the vehicle 1, or both an interior image and a captured image of the periphery of the vehicle 1. The rocking process in step S201 is similar to the rocking process in steps S105 and S109 shown in FIG. 3. Therefore, the description of the rocking process will be made by referring to the description of these steps.

[0076] In step S203, the image generating device 10 sets a virtual viewpoint at a position corresponding to the head of an occupant of the vehicle 1, and generates a virtual space that reproduces the interior space of the vehicle 1 by superimposing the interior image on the peripheral captured image of the vehicle 1. Then, the image generating device 10 generates a viewpoint image, which is an image seen in a predetermined direction from the virtual viewpoint in the cabin of the vehicle 1, from the virtual space.

[0077] When the process proceeds from step S201 to step S204 and then to step S203, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 that has been processed in step S204 to shake in response to the shaking of the vehicle 1. On the other hand, when the process proceeds from step S201 to step S202 and then to step S203, the image generating device 10 uses the captured image of the surroundings of the vehicle 1 itself captured by the in-vehicle camera 21. This is because the vibration prevention function of the in-vehicle camera 21 is determined to be disabled in step S201, and therefore the captured image of the surroundings of the vehicle 1 captured by the in-vehicle camera 21 shakes in response to the shaking of the vehicle 1.

[0078] When the process of step S203 is completed, the process proceeds to step S107, and the viewpoint image generated in step S203 is output to the display device 32.

[0079] If it is determined in step S201 that the shake prevention function of the vehicle-mounted camera 21 is enabled, the process proceeds to step S204. In step S204, the image generating device 10 reduces the shaking of the captured image of the periphery of the vehicle 1 because the captured image of the periphery of the vehicle 1 shakes in response to the shaking of the vehicle 1. Since step S204 corresponds to step S111 in FIG. 3, the explanation of step S111 is referred to. When the process in step S204 ends, the process proceeds to step S202. The explanations given above are referred to for the explanations of steps S202 and onward.

[0080] As described above, the image generating unit 14 included in the image generating device 10 according to the modified example sets a predetermined range including the gaze point of the user as a target object in the viewpoint image. This makes it possible to generate a viewpoint image in which the shaking level of the subject included in the central visual field of the remote user A is lower than the shaking level of the subject included in the peripheral visual field of the remote user A. Since a viewpoint image that matches the actual central visual field and peripheral visual field of the remote user A can be generated, the remote user A can easily grasp the behavior of the vehicle 1 through vision without being hindered in the sense of immersion. In addition, the object included in the central visual field of the remote user A is less likely to shake in response to the behavior of the vehicle 1 compared to the other objects, and it is possible to suppress the remote user A from getting sick.

[0081] In addition, the rocking process performed by the image generating device 10 according to the modified example may be such that the rocking level of objects other than the target object is not uniform, but the rocking level is increased as the distance from the gaze point of the remote user A increases. This makes it easier for the remote user A to visually recognize the object included in the central visual field, since the object closer to the central visual field is less likely to rock in response to the behavior of the vehicle 1 in the peripheral visual field of the remote user A. Also, the object farther from the central visual field is more likely to rock in response to the behavior of the vehicle 1 in the peripheral visual field of the remote user A, making it easier for the remote user A to intuitively grasp the behavior of the vehicle 1 in the peripheral visual field.

[0082] The rocking process executed by the image generating device 10 according to the modified example may be, for example, a process of gradually changing the rocking level over time by a morphing process so that no inflection point occurs in the difference in the rocking level near the boundary between a predetermined range set as the target object and the range. Since the rocking level changes continuously at the boundary between the central visual field and the peripheral visual field of the remote user A, for example, even if the vehicle 1 suddenly rocks significantly, it is possible to suppress the remote user A from feeling uncomfortable.

[0083] In the above embodiment, the display screen of a head-mounted display worn by the remote user A is used as the display device 32. In this case, the virtual viewpoint may be the viewpoint of the remote user A, not a preset viewpoint, and may be a viewpoint that moves in accordance with the movement of the head of the remote user A. The image generating device 10 generates an interior image that matches the movement of the head of the remote user A in addition to the behavior of the vehicle 1, so that the remote user A can easily grasp the behavior of the vehicle 1 even on a device that can be worn by the remote user A. Note that FIG. 5 shows an example of a virtual space when the display device 32 is incorporated in a head-mounted display.

[0084] In the above-described embodiment, an image viewed from a virtual viewpoint toward the front of vehicle 1 is described as a viewpoint image, but the direction viewed from the virtual viewpoint is not limited to the front of vehicle 1; for example, an image viewed from the virtual viewpoint toward the left or right side of vehicle 1 may be used as a viewpoint image.

[0085] In addition, in the above-described embodiment, an example was given of a configuration in which an interior image of vehicle 1 is stored in memory unit 12, but the interior image of vehicle 1 may also be an image captured by a camera mounted on vehicle 1 that captures images of the interior of vehicle 1.

[0086] In addition, in the above-described embodiment, the processes of steps S104 and S108 in FIG. 3 do not necessarily need to be executed. For example, the image generating device 10 may execute these processes only when outputting a viewpoint image to the display device 32 for the first time to the remote user A.

[0087] In the above embodiment, the remote controller 34 executes the process of identifying the gaze point of the remote user A. However, the present invention is not limited to this, and the image generating device 10 may include a gaze point identifying unit for identifying the gaze point of the remote user A instead of the remote controller 34. In this case, the image generating device 10 acquires an image captured by the remote camera 33, and identifies the gaze point of the remote user A based on the captured image. [Explanation of symbols]

[0088] 100...Image display system 1. Vehicle 21…In-vehicle camera 22…In-vehicle sensor 22a...Acceleration sensor 22b...Angular acceleration sensor 23...In-vehicle communication device 10...Image generating device 11…Vehicle information acquisition unit 12...Storage section 13...Gaze point information acquisition unit 14...Image generation unit 15...Display control unit 31...Remote communication equipment 32...Display device 33...Remote camera 34…Remote controller

Claims

1. A vehicle information acquisition unit that acquires vehicle information including information on a behavior of the vehicle from a device provided in the vehicle; an image generating unit that generates, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in a vehicle interior of the vehicle, in which a subject sways due to the behavior of the subject; a display control unit that outputs a control signal for displaying the image generated by the image generation unit on a display device provided in a space different from the vehicle interior; a gaze point information acquisition unit that acquires information regarding a gaze point of a user viewing the display device, The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle, the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image, The image generating unit includes: When the vehicle surroundings image is set to a target object among the subjects that corresponds to the user's gaze point based on the user's gaze point, an image generating device generates a viewpoint image in which the vibration level of the vehicle surroundings image is lower than the vibration level of the interior image.

2. A vehicle information acquisition unit that acquires vehicle information including information on a behavior of the vehicle from a device provided in the vehicle; an image generating unit that generates, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in a vehicle interior of the vehicle, in which a subject sways due to the behavior of the subject; a display control unit that outputs a control signal for displaying the image generated by the image generation unit on a display device provided in a space different from the vehicle interior; a gaze point information acquisition unit that acquires information regarding a gaze point of a user viewing the display device, The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle, the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image, The image generating unit includes: When the interior image is set to a target object among the subjects that corresponds to the user's gaze point based on the user's gaze point, an image generating device generates a viewpoint image in which the sway level of the interior image is lower than the sway level of the vehicle surroundings image.

3. 3. The image generating device according to claim 1, The sway level is a sway width or speed at which the subject sways in response to the behavior of the image generating device.

4. 4. The image generating device according to claim 3, The image generation unit is an image generating device that generates at least one of the viewpoint images in which the oscillation range of the target object is smaller than the oscillation range of objects other than the target object, and the viewpoint image in which the speed of the target object is slower than the speed of objects other than the target object.

5. The image generating device according to any one of claims 1 to 4, The image generating unit is an image generating device that sets a predetermined range of the viewpoint image that includes the user's gaze point as the target object.

6. The image generating device according to any one of claims 1 to 5, The vehicle information includes at least one of an acceleration and an angular acceleration of the vehicle.

7. 1. An image generating method executed by a processor to generate an image to be displayed on a display device provided in a space other than a vehicle interior, comprising: Acquiring vehicle information including information on a behavior of the vehicle from a device provided in the vehicle; generating, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in the interior of the vehicle, in which the subject sways due to the behavior of the subject; Acquire information regarding a gaze point of a user viewing the display device; a viewpoint image generating step of generating the viewpoint image in which a rocking level of a target object corresponding to a gaze point of the user among the subjects is made lower than the rocking level of an object other than the target object, The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle, the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image, The viewpoint image generating step includes: An image generating method including a step of lowering a vibration level of the vehicle surroundings image compared to the vibration level of the interior image when the vehicle surroundings image is set as the target object based on the user's gaze point.

8. An image generation method executed by a processor to generate an image to be displayed on a display device provided in a space other than a vehicle interior, comprising: Acquiring vehicle information including information on a behavior of the vehicle from a device provided in the vehicle; generating, as a viewpoint image, an image viewed in a predetermined direction from a virtual viewpoint in the interior of the vehicle, in which the subject sways due to the behavior of the subject; Acquire information regarding a gaze point of a user viewing the display device; a viewpoint image generating step of generating the viewpoint image in which a rocking level of a target object corresponding to a gaze point of the user among the subjects is made lower than the rocking level of an object other than the target object, The vehicle information includes a vehicle surroundings image captured by an imaging device provided in the vehicle, the viewpoint image includes an interior image showing an interior of the vehicle as seen from the virtual viewpoint, and the vehicle surroundings image, The viewpoint image generating step includes: An image generating method including a step of lowering the sway level of the interior image compared to the sway level of the vehicle surroundings image when the interior image is set as the target object based on the user's gaze point.

Images