Proposal method and proposal device

Abstract

To reduce influence of visual information simultaneously including a video image displayed by a display device in a vehicle and a scene outside a vehicle on vehicle sickness of an occupant.SOLUTION: A controller 18 performs processing of: acquiring a vehicle outside image from a first camera that photographs the outside of a vehicle 1; acquiring a display video image that is a video image displayed on a display unit in the vehicle 1; and presenting guidance for urging an occupant to rotate at least one of a seat and a head of the occupant so that a direction 3f in which the display video image viewed by the occupant 2 approaches a direction 5f in which scenery 5 outside the vehicle viewed by the occupant flows based on an acquired video image outside the vehicle and the display video image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a suggestion method and a suggestion device for suggesting actions to reduce car sickness to vehicle occupants. [Background technology]

[0002] The image display device of Patent Document 1 synthesizes a background image corresponding to the vehicle behavior onto the image being watched by the occupants on a television or other device, thereby matching visual information with vestibular information and somatosensory information generated by the movement of the vehicle, thereby reducing car sickness in the occupants. Non-Patent Document 1 describes a technique for estimating the occurrence of car sickness in a passenger based on visual and vestibular information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-149154 A [Non-patent literature]

[0004] [Non-Patent Document 1] Takahiro Wada et al., "A Computational Model of Motion Sickness Considering Visual and Vestibular Information", 2020 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1758-1763 Summary of the Invention [Problem to be solved by the invention]

[0005] The image display device of Patent Document 1 reduces car sickness in occupants by presenting images corresponding to vehicle behavior from the display device. Therefore, it cannot reduce car sickness caused by optical flow that occurs outside the range of the occupant's field of vision that is occupied by the display device. The present invention aims to reduce the effect that visual information that simultaneously includes an image displayed on a display device inside the vehicle and the scenery outside the vehicle has on occupants car sickness. [Means for solving the problem]

[0006] In a proposed method according to one aspect of the present invention, a controller performs the following processes: acquiring an outside-vehicle image from a first camera that captures the outside of the vehicle; acquiring a display image that is displayed on a display device inside the vehicle; and presenting guidance to the occupant to rotate at least one of the seat or the occupant's head based on the acquired outside-vehicle image and the display image so that the direction in which the display image seen by the occupant moves approaches the direction in which the scenery outside the vehicle flows as seen by the occupant. Effect of the Invention

[0007] According to the present invention, the effect of visual information simultaneously including an image displayed on a display device inside the vehicle and the scenery outside the vehicle on vehicle sickness in occupants can be reduced. [Brief description of the drawings]

[0008] [Figure 1] 1 is a schematic diagram illustrating an example of a car sickness reduction proposal device according to an embodiment. [Diagram 2] 1A and 1B are schematic diagrams illustrating a proposed method for reducing car sickness according to an embodiment. [Diagram 3] 1 is a flowchart (part 1) of an example of a proposed method for reducing car sickness according to an embodiment. [Figure 4] 1 is a schematic diagram showing an example of a tilt angle of an occupant's head. FIG. [Diagram 5] 2 is a flowchart (part 2) of an example of a method for proposing reduction of car sickness according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that each drawing is a schematic view and may differ from the actual one. In addition, the embodiment of the present invention shown below is an example of an apparatus and a method for embodying the technical idea of ​​the present invention, and the technical idea of ​​the present invention does not specify the structure, arrangement, etc. of the components as described below. The technical idea of ​​the present invention can be modified in various ways within the technical scope defined by the claims described in the claims.

[0010] (composition) 1 is a schematic diagram of an example of a car sickness reduction suggestion device according to an embodiment. The car sickness reduction suggestion device 10 is provided in a vehicle 1 and presents information to an occupant 2 of the vehicle 1 to encourage the occupant to take action to reduce car sickness. The car sickness reduction proposal device 10 includes an in-vehicle camera 11, an outside-vehicle camera 12, a communication device 13, a human-machine interface (HMI) 14, and a controller 18.

[0011] The in-vehicle camera 11 is a camera installed inside the vehicle, and may be a camera equipped with an imaging element such as a CCD or a CMOS, an infrared camera, or the like. The in-vehicle camera 11 is installed at a position where the posture of the head of the occupant 2 and the movement state of the head of the occupant 2 can be detected from the image acquired by the in-vehicle camera 11. For example, the in-vehicle camera 11 is installed at a position where the head (face) of the occupant 2 and the mobile terminal device 3 carried by the occupant 2 can be photographed. The object photographed by the in-vehicle camera 11 is an object present inside the vehicle, and may be, for example, the body of the occupant 2 (e.g., head, arms, and legs), the mobile terminal device 3 carried by the occupant 2, the seats and windows of the vehicle, in-vehicle devices such as displays, and switches of the in-vehicle devices. The in-vehicle camera 11 is an example of a "second camera" described in the claims.

[0012] The exterior camera 12 is a camera installed outside or inside the vehicle, and captures the scenery outside the vehicle that can be seen from inside the vehicle (hereinafter referred to as "scenery outside the vehicle"). The exterior camera 12 may be a camera similar to the interior camera 11. The exterior camera 12 is an example of the "first camera" described in the claims. The communication device 13 transmits and receives information to and from the portable terminal device 3 carried by the occupant 2. For example, the communication device 13 transmits and receives information to and from the portable terminal device 3 using a short-range wireless communication function. The short-range wireless communication standard between the communication device 13 and the portable terminal device 3 may be, for example, Wifi (Wireless Fidelity) (registered trademark) or Bluetooth (registered trademark).

[0013] The mobile terminal device 3 is a portable information communication terminal. Examples of the mobile terminal device 3 include a smartphone, a notebook computer, and a tablet terminal. The HMI 14 is an interface device that transmits and receives information between the car sickness reduction proposal device 10 and an occupant 2 (e.g., the driver) of the vehicle 1. For example, the HMI 14 may include a display device that can be seen by the occupant 2, and a speaker or buzzer that outputs an alarm sound, a notification sound, and audio information.

[0014] The controller 18 is an electronic control unit that controls the operation of the car sickness reduction proposal device 10. The controller 18 includes a processor 18a and peripheral components such as a memory device 18b. The processor 18a may be, for example, a CPU or an MPU. The memory device 18b may include a semiconductor memory device, a magnetic memory device, an optical memory device, etc. The memory device 18b may include a register, a cache memory, and a memory such as a ROM and a RAM used as a main memory device. The functions of the controller 18 described below are realized, for example, by the processor 18a executing a computer program stored in the storage device 18b. The controller 18 may be formed of dedicated hardware for executing each information processing described below. For example, the controller 18 may include a functional logic circuit set in a general-purpose semiconductor integrated circuit. For example, the controller 18 may include a PLD such as an FPGA.

[0015] Next, a description will be given of a method for proposing reduction of car sickness according to an embodiment, which is implemented by the device 10 for proposing reduction of car sickness. Figures 2(a) and 2(b) are schematic diagrams of the method for proposing reduction of car sickness according to an embodiment. Reference numeral 5 in Figures 2(a) and 2(b) indicates an example of a scene outside the vehicle that the occupant 2 can see from inside the vehicle 1. For example, Figure 2(a) shows an example of a scene 5 that can be seen in front of the vehicle 1 when the vehicle 1 is moving forward. In the scene 5 in Figure 2(a), the scene flows from the vanishing point toward the periphery of the field of view as indicated by an arrow 5f. Therefore, the scene 5 in Figure 2(a) produces an optical flow 5f in the vision of the occupant 2 that flows from the vanishing point toward the periphery of the field of view.

[0016] On the other hand, Fig. 2(b) shows an example of a scene 5 seen to the right of the vehicle 1 when the vehicle 1 is moving forward. In the scene 5 in Fig. 2(b), the scene flows to the right of the field of view as indicated by an arrow 5f. Therefore, the scene 5 in Fig. 2(a) produces an optical flow 5f in the visual field of the occupant 2, which flows to the right of the field of view.

[0017] Here, if occupant 2 is working inside vehicle 1 (for example, using a portable terminal device 3, reading, or viewing a document), there is a risk that occupant 2 will become car sick because the direction of the optical flow that occurs in the vision of occupant 2 within the range of his or her field of vision where occupant 2 is gazing at the display screen of the portable terminal device 3, a book, or a document (hereinafter referred to as the "gazing range") differs from the direction of the optical flow that occurs in the vision of occupant 2 outside the gaze range where the scenery 5 occurs. For example, when occupant 2 is looking at an image displayed on a display device inside vehicle 1, the optical flow produced in occupant 2's vision by the displayed image differs in direction from the optical flow produced in occupant 2's vision by a landscape 5 outside the viewing range, which may affect occupant 2's car sickness.

[0018] 2(a), it is assumed that the occupant 2 is using the mobile terminal device 3 while facing the front of the vehicle 1, and the mobile terminal device 3 is displaying an image that flows to the right as indicated by an arrow 3f. In the following description, the image displayed by the mobile terminal device 3 may be referred to as a "mobile terminal image." In this case, the direction of the optical flow 3f through which the mobile device image is presented to the occupant 2's vision within the gaze range differs from the direction of the optical flow 5f through which the scenery 5 is presented to the occupant 2's vision outside the gaze range, and the difference in the direction of the optical flows 3f and 5f may affect the occupant 2's car sickness.

[0019] Therefore, the controller 18 acquires an outside image obtained by photographing the scenery outside the vehicle 1 using the exterior camera 12. The controller 18 also acquires, via the communication device 13, a mobile terminal image which is an image to be displayed on the display device of the mobile terminal device 3. Then, based on the acquired vehicle exterior image and mobile device image, the HMI 14 presents guidance to urge the occupant 2 to rotate at least one of the seat in which the occupant 2 sits or the head of the occupant 2 so that the direction 3f in which the mobile device image seen by the occupant 2 moves approaches the direction 5f in which the vehicle exterior scenery 5 seen by the occupant 2 flows. For example, the HMI 14 presents guidance to urge the occupant 2 to rotate at least one of the seat or the head of the occupant 2 so that the direction 3f in which the mobile device image seen by the occupant 2 moves matches the direction 5f in which the vehicle exterior scenery 5 seen by the occupant 2 flows.

[0020] For example, the controller 18 detects the position of the head of the occupant 2 from the in-vehicle image generated by the in-vehicle camera 11 capturing the inside of the vehicle. Based on the optical conditions (viewpoint, optical axis direction, angle of view, etc.) of the outside camera 12 and the position of the head of the occupant 2, the controller 18 converts the outside-vehicle image of the outside-vehicle camera 12 into a virtual image of the outside-vehicle scenery 5 seen by the occupant 2 when the occupant 2 faces various directions. Based on the optical flow of the virtual image, the controller 18 estimates the optical flow of the outside-vehicle scenery 5 in the occupant's vision. The controller 18 may estimate the seat in which the occupant 2 sits or the rotation direction and rotation position of the head of the occupant 2 by estimating the direction of the head of the occupant 2 that can match the direction of the optical flow of the mobile terminal image in the occupant's vision with the direction of the optical flow of the outside-vehicle scenery 5.

[0021] 2(b), the occupant 2 is prompted to rotate the seat (i.e., rotate the seat clockwise) so that the front direction of the seat of the occupant 2 faces the right direction in the vehicle width direction of the vehicle 1 or so that the front direction of the seat is tilted to the right. Alternatively, the occupant 2 is prompted to rotate the head (i.e., rotate the head clockwise) so that the front direction of the head of the occupant 2 faces the right direction in the vehicle width direction of the vehicle 1 or so that the front direction of the head is tilted to the right. As a result, the optical flow 5f through which the scenery 5 appears to the occupant 2 is a flow that flows to the right, similar to the optical flow 3f of the mobile device image. As a result, the effect of the difference in direction between the optical flows 3f and 5f on the car sickness of the occupant 2 can be reduced. In the above description, a mobile terminal image that is an image displayed on the display device of the mobile terminal device 3 carried by the occupant 2 is given as an example of a display image displayed on the display device inside the vehicle 1, but the present invention is not limited to this. For example, the display image displayed on the display device inside the vehicle 1 may be a display image displayed on the display screen of the HMI 14 described above, or may be a display image displayed on the display screen of a navigation device or an In-Vehicle Infotainment (IVI) system.

[0022] FIG. 3 is a flowchart (part 1) of an example of a proposed method for reducing car sickness according to the embodiment. In step S1, controller 18 acquires an in-vehicle image generated by in-vehicle camera 11 capturing an image of the interior of the vehicle from in-vehicle camera 11. Controller 18 acquires a tilt angle θ of the head of occupant 2 based on the in-vehicle image acquired from in-vehicle camera 11. FIG. 4 is a schematic diagram of an example of a head tilt angle θ, which is the tilt angle of the head 2h of the occupant 2. As shown in FIG.

[0023] For example, the controller 18 may obtain the angle between a vertical direction perpendicular to the vehicle horizontal plane of the vehicle 1 and the up-down direction of the head 2h of the occupant 2 as the head tilt angle θ. Here, the "vehicle horizontal plane" means a plane that includes a straight line parallel to the front-rear direction of the vehicle 1 and a straight line parallel to the vehicle width direction. For example, the controller 18 may obtain the head tilt angle θ at which the head 2h of the occupant 2 tilts in a forward direction of the torso 2b. In addition, for example, the controller 18 may detect belongings (mobile terminal device 3, books, documents, etc.) held by the occupant 2 from an in-vehicle image by image recognition, and obtain the head tilt angle θ at which the head 2h of the occupant 2 tilts in a direction toward the position of the belongings.

[0024] Refer to Fig. 3. In step S2, the controller 18 determines whether the head tilt angle θ is greater than a predetermined threshold value Ta. If the head tilt angle θ is greater than the predetermined threshold value Ta (step S2: Y), the process proceeds to step S4. In step S4, the controller 18 determines whether the mobile terminal device 3 is communicating with the vehicle 1 via the communication device 13. In this way, the controller 18 estimates the behavior of the occupant 2 based on the head tilt angle θ acquired based on the in-vehicle image and the communication status between the communication device 13 and the mobile terminal device 3.

[0025] If the head tilt angle θ is equal to or smaller than the predetermined threshold value Ta (step S2: N), the controller 18 presumes that the occupant 2 is looking at the scenery outside the vehicle 1, and the process proceeds to step S3. When the head tilt angle θ is greater than the predetermined threshold value Ta (step S2: Y), the controller 18 estimates that the occupant 2 is not looking at the scenery outside the vehicle, but is looking down and performing some kind of work.

[0026] If the portable terminal device 3 is communicating with the vehicle 1 via the communication device 13 (step S4: Y), the controller 18 presumes that the occupant 2 is using the portable terminal device 3, and the process proceeds to step S5. If the portable terminal device 3 is not communicating with the vehicle 1 via the communication device 13 (step S4: N), the controller 18 presumes that the occupant 2 is performing an operation other than using the portable terminal device 3 (e.g., reading a book, viewing a document, etc.), and the process proceeds to step S6.

[0027] The portable terminal device 3 is configured to communicate with the vehicle 1 via the communication device 13 so that the controller 18 can determine whether or not the occupant 2 is using the portable terminal device 3. For example, the mobile terminal device 3 may have a dedicated application program (hereinafter sometimes referred to as a "dedicated program") installed that periodically or continuously transmits signals to the vehicle 1 while the mobile terminal device 3 is being used by the occupant 2.

[0028] For example, the dedicated program may periodically transmit a signal to the vehicle 1 of a screenshot of the mobile terminal image displayed on the display screen of the mobile terminal device 3 while the mobile terminal device 3 is being used by the occupant 2. Alternatively, the dedicated program may continuously transmit a signal of the mobile terminal image displayed on the display screen of the mobile terminal device 3 to the vehicle 1 while the mobile terminal device 3 is being used by the occupant 2. Here, the use of the portable terminal device 3 by the occupant 2 may include the occupant 2 operating the portable terminal device 3 and the occupant 2 viewing the portable terminal image displayed on the display screen of the portable terminal device 3.

[0029] In step S3, the controller 18 acquires an outside image from the outside camera 12. After that, the process proceeds to step S7. In step S5, the controller 18 acquires the mobile terminal image from the mobile terminal device 3, and then the process proceeds to step S7. In step S6, the controller 18 acquires a dummy image that does not include optical flow. For example, the controller 18 acquires a still image (e.g., a blank image filled with black or other pixel values) as the dummy image. Then, the process proceeds to step S7.

[0030] In step S7, the controller 18 calculates the optical flow occurring in the visual field of the occupant 2 from the image acquired in any one of steps S3, S5, or S6. For example, the controller 18 estimates the position of the viewpoint and the direction of the line of sight of the occupant 2 based on an image of the head 2h of the occupant 2 captured by the in-vehicle camera 11. When an outside-vehicle image is acquired in step S3, for example, the controller 18 converts the outside-vehicle image of the outside-vehicle camera 12 into a virtual image of the outside-vehicle scenery 5 seen by the occupant 2 based on the optical conditions of the outside-vehicle camera 12 and the viewpoint position and line-of-sight direction of the occupant 2. The controller 18 may extract the optical flow on the virtual image as the optical flow of the outside-vehicle scenery in the occupant's vision.

[0031] Furthermore, when the mobile terminal image is acquired in step S5, for example, the controller 18 converts the mobile terminal image displayed on the display screen of the mobile terminal device 3 into a virtual image of the mobile terminal image seen by the occupant 2, based on the position of the mobile terminal device 3 recognized from the in-vehicle image of the in-vehicle camera 11 and the position of the viewpoint and the direction of the line of sight of the occupant 2. The optical flow on the virtual image may be extracted as the optical flow of the mobile terminal image in the occupant's vision. Furthermore, in the case where a dummy image is acquired in step S6, for example, the controller 18 may extract the optical flow in the visual sense of the occupant from the dummy image.

[0032] In step S8, the controller 18 detects head movement information related to the movement of the head 2h of the occupant 2 from the in-vehicle image captured by the in-vehicle camera 11. For example, the controller 18 may detect the head movement information based on changes in the position and tilt (posture) of the head 2h of the occupant 2 in different frames of the in-vehicle image and the shooting period between each frame. In step S9, the controller 18 estimates the degree of car sickness of the occupant 2 based on the optical flow calculated in step S7 and the head movement information detected in step S8. For example, the controller 18 may estimate the degree of car sickness of the occupant 2 using the method proposed in the above-mentioned Non-Patent Document 1. Thereafter, the process proceeds to step S10 in FIG.

[0033] FIG. 5 is a flowchart (part 2) of an example of a proposed method for reducing car sickness according to the embodiment. In the proposed method for reducing car sickness shown in Figure 5, the controller 18 presents information encouraging the occupant 2 to take action to reduce car sickness, based on the behavior of the occupant 2 estimated based on the communication status of the mobile terminal device 3 and the in-car image, and the degree of car sickness estimated in step S9. For example, guidance may be presented to encourage the occupant to rotate at least one of the seat or head, to stop the occupant's current work, or to take a break.

[0034] In step S10, the controller 18 determines whether the estimated value of the degree of car sickness of the occupant 2 is greater than a predetermined threshold Te1. If the estimated value of the degree of car sickness is greater than the predetermined threshold Te1 (step S10: Y), the process proceeds to step S11. On the other hand, if the estimated value of the degree of car sickness is equal to or less than the predetermined threshold Te1 (step S10: N), the process proceeds to step S12. In step S12, the controller 18 determines whether the estimated value of the degree of car sickness of the occupant 2 is also greater than a predetermined threshold Te2 that is smaller than the above-mentioned threshold Te1. If the estimated value of the degree of car sickness is equal to or smaller than the predetermined threshold Te2, the process returns to step S1 in Fig. 3. If the estimated value of the degree of car sickness is greater than the predetermined threshold Te2 (step S12: Y), the process proceeds to step S13.

[0035] In step S13, the controller 18 determines whether the head tilt angle θ is greater than a predetermined threshold value Ta. If the head tilt angle θ is equal to or less than the predetermined threshold value Ta (step S13: N), the process returns to step S1 in Fig. 3. If the head tilt angle θ is greater than the predetermined threshold value Ta (step S13: Y), the process proceeds to step S14. In step S14, the controller 18 determines whether the mobile terminal device 3 is communicating with the vehicle 1 via the communication device 13.

[0036] If the portable terminal device 3 is not communicating with the vehicle 1 via the communication device 13 (step S14: N), the controller 18 assumes that the portable terminal device 3 is engaged in an operation other than using the portable terminal device 3 (e.g., reading, viewing a document, etc.), and the process proceeds to step S16. If the mobile terminal device 3 is communicating with the vehicle 1 via the communication device 13 (step S14: Y), the controller 18 assumes that the occupant 2 is using the mobile terminal device 3, and the process proceeds to step S15.

[0037] In step S15, the controller 18 displays, via the HMI 14, guidance to encourage the occupant 2 to rotate at least one of the seat of the occupant 2 or the head 2h of the occupant 2 so that the direction in which the mobile device image viewed by the occupant 2 moves approaches the direction in which the scenery outside the vehicle flows as viewed by the occupant 2. For example, guidance may be displayed to encourage the occupant 2 to rotate at least one of the seat of the occupant 2 or the head 2h of the occupant 2 so that the direction in which the mobile device image viewed by the occupant 2 moves matches the direction in which the scenery outside the vehicle flows as viewed by the occupant 2.

[0038] For example, the guidance presented by the HMI 14 may be voice guidance using a voice signal. The controller 18 may also display, as visual guidance, guidance encouraging the occupant 2 to rotate at least one of the seat of the occupant 2 or the head 2h of the occupant 2 on a display screen of the mobile terminal device 3 or a display screen in the vehicle 1 (for example, a display screen of a navigation device or a display screen of an in-vehicle infotainment system).

[0039] For example, when a mobile terminal image flowing to the right is displayed on the mobile terminal device 3, guidance may be presented to urge the occupant 2 to rotate the seat so that the front direction of the seat of the occupant 2 faces the right direction in the vehicle width direction of the vehicle 1 or so that the front direction of the seat is tilted to the right. Alternatively, guidance may be presented to urge the occupant 2 to rotate his / her head so that the front direction of the head faces the right direction in the vehicle width direction of the vehicle 1 or so that the front direction of the head is tilted to the right.

[0040] Also, for example, when a mobile terminal image flowing to the left is displayed on the mobile terminal device 3, guidance may be presented to urge the occupant 2 to rotate the seat so that the front direction of the seat of the occupant 2 faces the left direction in the vehicle width direction of the vehicle 1 or so that the front direction of the seat is tilted to the left. Alternatively, guidance may be presented to urge the occupant 2 to rotate his / her head so that the front direction of the head faces the left direction in the vehicle width direction of the vehicle 1 or so that the front direction of the head is tilted to the left.

[0041] Also, for example, when a mobile terminal image flowing downward is displayed on the mobile terminal device 3, guidance may be presented to occupant 2 urging him / her to rotate their head so that the front of their head faces downward. After the controller 18 presents the guidance, the process returns to step S1 in FIG.

[0042] In step S16, the controller 18 displays a guidance to encourage the occupant 2 to stop the current task (e.g., reading, viewing a document, etc.) through the HMI 14. The guidance to encourage the occupant 2 to stop the task may be audio guidance or visual guidance. After the controller 18 displays the guidance, the process returns to step S1 in FIG. 3. On the other hand, if the estimated value of the degree of car sickness is greater than the predetermined threshold Te1 (step S10: Y), in step S11, the controller 18 displays guidance encouraging the driver to stop the vehicle 1 and take a rest through the HMI 14. The guidance encouraging the driver to take a rest may be audio guidance or visual guidance. After the controller 18 displays the guidance, the process ends.

[0043] (Effects of the embodiment) (1) The controller 18 executes a process of acquiring an outside-vehicle image from a first camera that captures the outside of the vehicle 1, a process of acquiring a display image which is an image to be displayed on a display device inside the vehicle 1, and a process of presenting guidance to the occupant 2 to rotate at least one of the seat or the head of the occupant 2 based on the acquired outside-vehicle image and the display image so that the direction in which the display image seen by the occupant 2 moves approaches the direction in which the scenery outside the vehicle flows as seen by the occupant 2. This allows the direction in which the image displayed by the display device flows to be closer to the direction in which the scenery outside the vehicle flows as seen by the occupant 2, thereby reducing the impact that visual information that simultaneously includes the image displayed by the display device and the scenery outside the vehicle has on the occupant's car sickness.

[0044] (2) The controller 18 acquires in-vehicle images from a second camera that captures the interior of the vehicle 1, detects head movement information, which is information regarding the movement of the head of the occupant 2, from the in-vehicle images, calculates the optical flow in the vision of the occupant 2, and estimates the degree of car sickness of the occupant 2 based on the head movement information and the optical flow.

[0045] This makes it possible to estimate the degree of car sickness of occupant 2 by taking into account the optical flow in the occupant's vision. (3) The controller 18 may estimate the behavior of the occupant 2 based on the communication status between the communication device 13 and the mobile terminal device 3 and the in-vehicle image, and may calculate the optical flow from any of the images selected according to the estimated result of the behavior of the occupant 2, from among the image outside the vehicle, the mobile terminal image which is the display image displayed on the display device of the mobile terminal device 3, or the image not including optical flow. This allows the optical flow in the vision of the occupant 2 to be calculated accurately.

[0046] (4) The controller 18 may estimate the behavior of the occupant 2 based on the communication status between the communication device 13 and the mobile terminal device 3 and the in-vehicle image, and based on the behavior of the occupant 2 and the degree of car sickness, select whether to present guidance encouraging the occupant 2 to rotate at least one of the seat or his / her head, to interrupt his / her current work, or to take a rest. This allows the occupant 2 to be appropriately guided in taking actions to reduce car sickness. [Explanation of symbols]

[0047] Reference Signs List 1: vehicle, 2: occupant, 3: mobile terminal device, 11: in-vehicle camera, 12: exterior camera, 13: communication device, 14: human machine interface (HMI), 18: controller, 18a: processor, 18b: storage device

Claims

1. A process of acquiring an outside image of the vehicle from a first camera that captures an image of the outside of the vehicle; A process of acquiring a display image, which is an image displayed on a display device inside the vehicle; A process of presenting a guidance to the occupant to rotate at least one of the seat or the head of the occupant so that a direction in which the display image seen by the occupant moves approaches a direction in which the scenery outside the vehicle flows as seen by the occupant, based on the acquired vehicle exterior image and the display image; The proposed method is characterized by having a controller execute the following:

2. The controller: Acquire an interior image from a second camera that captures an image of the interior of the vehicle; Detecting head movement information, which is information regarding the movement of the head of the occupant, from the in-vehicle image; Calculating an optical flow in the visual field of the occupant; estimating the degree of car sickness of the occupant based on the head movement information and the optical flow; The method of claim 1 .

3. The controller: Estimating the behavior of the occupant based on a communication state between the communication device and the mobile terminal device and the in-vehicle image; Calculating the optical flow from any one of the outside image, the mobile terminal image which is the display image displayed on the display device of the mobile terminal device, and an image not including an optical flow, selected according to the estimation result of the behavior of the occupant. The method according to claim 2 .

4. The controller: estimating the behavior of the occupant based on a communication state between the communication device and the mobile terminal device and the in-vehicle image; Select whether to present a guidance to encourage the occupant to rotate at least one of the seat or the head, to present a guidance to encourage the occupant to stop the current work, or to present a guidance to encourage the occupant to take a rest, based on the behavior of the occupant and the degree of the car sickness. The method according to claim 2 .

5. A first camera for capturing an image of the outside of the vehicle; a controller that executes a process of acquiring an outside image from the first camera, a process of acquiring a display image which is an image to be displayed on a display device inside the vehicle, and a process of presenting a guidance to an occupant to rotate at least one of the seat or the head of the occupant based on the acquired outside image and the display image so that a direction in which the display image seen by the occupant moves approaches a direction in which the scenery outside the vehicle flows as seen by the occupant; A proposed device comprising:

Images