In-vehicle systems, vehicles, information processing devices, information processing methods, and programs

Abstract

The notification will be displayed in a location easily visible to the driver. [Solution] The system comprises a sensor, a plurality of display devices, and a control unit that estimates a first direction, which is the direction the driver is looking, based on first data, which is data acquired by the sensor, and outputs a predetermined notification to a first display, which is a display device selected from the plurality of display devices based on the first direction, when a predetermined notification is directed to the driver.

Description

Technical Field

[0001] The present disclosure relates to vehicle technology.

Background Art

[0002] There is a technology for providing information to reduce the burden on a driver who drives an automobile. In relation to this, for example, Patent Document 1 discloses an in-vehicle system in which a display for information provision is arranged near a steering wheel.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to output a notification at a location where it is easy to enter the driver's field of vision.

Means for Solving the Problems

[0005] One aspect of the present disclosure is a vehicle-mounted system including a sensor, a plurality of display devices, estimating a first direction, which is a direction in which a driver is gazing, based on first data that is data acquired by the sensor, and outputting the predetermined notification to a first display, which is a display device selected based on the first direction among the plurality of display devices, when the predetermined notification directed to the driver occurs. a vehicle-mounted system.

[0006] One aspect of the present disclosure is An information processing device that outputs information to a plurality of display devices located in front of the driver's seat inside a vehicle, comprising a control unit that performs the following: first data, which is data acquired by a sensor, to estimate a first direction, which is the direction in which the driver of a vehicle equipped with the plurality of display devices is looking; and when a predetermined notification is generated from the vehicle toward the driver, to output the predetermined notification to a first display, which is a display device selected from the plurality of display devices based on the first direction. It is an information processing device.

[0007] One aspect of this disclosure is, An information processing method performed by an in-vehicle system equipped with an information processing device that outputs information via multiple display devices, comprising: a step of estimating a first direction, which is the direction in which the driver of the vehicle equipped with the in-vehicle system is looking, based on first data, which is data acquired by a sensor; and a step of outputting a predetermined notification to a first display, which is a display device selected from the multiple display devices based on the first direction, when a predetermined notification is generated from the in-vehicle system toward the driver. It is an information processing method.

[0008] Other embodiments include a vehicle equipped with the above-mentioned in-vehicle system, a program for causing a computer to execute the above-mentioned information processing method, or a computer-readable storage medium that non-temporarily stores the program. [Effects of the Invention]

[0009] According to this disclosure, notifications can be displayed in a location that is easily visible to the driver. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing the interior of vehicle 1 according to this embodiment. [Figure 2] Enlarged view of the steering wheel 4 and its surroundings. [Figure 3]Figure showing an example of a component of vehicle 1. [Figure 4] Figure showing an example of a component of vehicle ECU 10. [Figure 5] Figure showing a list of screens output to the outer display. [Figure 6] Figure showing an example of a screen output to the outer display. [Figure 7] Figure showing an example of a screen output to the outer display. [Figure 8] Figure showing a list of screens output to the inner display. [Figure 9] Figure showing an example of a screen output to the inner display. [Figure 10] Figure showing an example of a screen output to the inner display. [Figure 11] Flowchart of the process executed by the HMI control unit 1011. [Figure 12] Flowchart of the process of outputting a notification to the display device where an operation is being performed based on data for determining the display device being operated by the driver. [Figure 13] Figure showing an example of a display device according to a modification.

Mode for Carrying Out the Invention

[0011] In recent years, with the advancement of automobiles, displays for providing information are becoming larger or the number of displays is increasing.

[0012] For example, a method has been proposed in which a large touch panel display is arranged between the driver's seat and the passenger seat in the front of the vehicle interior to perform operations and information provision related to the vehicle in a lump. However, in such a method, when operating components of the vehicle or checking the output information, the driver's line of sight moves significantly. For the driving safety of the vehicle, it is preferable to minimize the movement of the driver's line of sight.

[0013] Generally, in the portion in front of the driver's seat where the instrument panel was conventionally located, a display device such as a liquid crystal display is often arranged, and by outputting information to the display device, the movement of the line of sight can be minimized. However, there is a limit to the size of the display device arranged in the front, and it is not always possible to display all the information required by the driver. Also, if these pieces of information are output simultaneously in the same area, for example, the information necessary for the vehicle's travel and auxiliary information such as entertainment will be mixed, which will hinder intuitive information transmission.

[0014] There are also problems with the arrangement of the input interface for operation. For example, a steering wheel having cursor keys and push button switches and enabling input operations while driving is known. However, with such an input interface, since the layout of buttons etc. is fixed, it is difficult to adapt to a plurality of different targets. For example, when raising or lowering the set temperature of the air conditioner, it can be done with two physical keys, but with physical keys, it is not possible to directly specify an arbitrary point on the map. A form of operation by a touch panel is also conceivable, but as described above, the touch panel display arranged in the front of the vehicle interior is located at a position far from the steering wheel, so in addition to the movement of the line of sight, the driver's hand also moves significantly.

[0015] In order to solve these problems, it is preferable to arrange a display device capable of input operations by touch near the steering wheel.

[0016] An in-vehicle system according to one aspect of the present disclosure is The system comprises a sensor, a plurality of display devices, and a control unit that estimates a first direction, which is the direction the driver is looking, based on first data acquired by the sensor, and outputs a predetermined notification to a first display, which is a display device selected from the plurality of display devices based on the first direction, when a predetermined notification is directed to the driver.

[0017] The display device has a pair of display units positioned on either side of the steering wheel. Furthermore, the pair of display units are configured to allow touch operation via a touch panel.

[0018] The first data refers to data acquired by sensors in the in-vehicle system. For example, the first data may include image data acquired by an in-vehicle camera, temperature map data acquired by an infrared sensor, detection data acquired by a touch sensor, etc.

[0019] The first direction refers to the direction in which the driver of a vehicle equipped with an in-vehicle system according to one aspect of this disclosure is looking. The first direction is determined based on the first data.

[0020] The first display refers to a display located within a predetermined range (within a predetermined angle) from a virtual axis indicating a first direction.

[0021] The control unit determines the first direction based on the first data and, if there is a predetermined notification for the driver, outputs the predetermined notification to the first display, which is a display device selected based on the first direction.

[0022] This configuration allows notifications to be displayed in a location easily visible to the driver.

[0023] The control unit may output the predetermined notification to the display device closest to the axis indicating the first direction.

[0024] As a result, an in-vehicle system according to one aspect of this disclosure can output a notification to a display device that is most likely to be in the driver's line of sight.

[0025] Furthermore, the control unit may determine a display device among the plurality of display devices that is located within a predetermined range from the axis indicating the first direction as the first display, and output the predetermined notification to the first display.

[0026] As a result, an in-vehicle system according to one aspect of this disclosure can output a notification to a display device located within the driver's field of vision.

[0027] Furthermore, if multiple display devices exist within a predetermined range from the axis indicating the first direction, the control unit may determine the display device with the highest pre-assigned priority as the first display and output the predetermined notification to the first display.

[0028] For example, an in-vehicle system according to one aspect of this disclosure may assign priority to display devices that output a predetermined notification based on the interaction with the display of other information. An in-vehicle system according to one aspect of this disclosure may output a predetermined notification to a display device with higher priority. This allows the notification to be displayed in an appropriate location within the vehicle.

[0029] Furthermore, if no display device exists within a predetermined range from the axis indicating the first direction, the control unit may determine the first display based on the type of notification or the source of the notification, and output the predetermined notification to the first display.

[0030] As a result, an in-vehicle system according to one aspect of this disclosure can output notifications to a reasonably determined display device even when there is no display device that is close to the driver's line of sight.

[0031] Furthermore, after outputting the predetermined notification to the first display, the control unit may determine whether the driver is looking at the first display based on second data, which is data acquired by the sensor and is different from the first data. If it determines that the driver is not looking at the first display, it may estimate a second direction, which is the direction the driver is looking and is different from the first direction, and output the predetermined notification to the second display, which is a display device selected based on the second direction.

[0032] As a result, an in-vehicle system according to one aspect of this disclosure can output notifications to a more appropriate location in accordance with the driver's eye movements.

[0033] Furthermore, the first data includes data for determining which of the plurality of display devices the driver is operating, and the control unit may output the predetermined notification to the third display if it determines, based on the first data, that the driver is operating the third display.

[0034] For example, when a driver is operating a display while driving, it is often assumed that they are intently watching the display they are operating. Therefore, an in-vehicle system according to one aspect of this disclosure may output a predetermined notification to the display that the driver is operating.

[0035] As a result, an in-vehicle system according to one aspect of this disclosure can output notifications to a display that is likely to be viewed by the driver, based on the driver's actions. Therefore, an in-vehicle system according to one aspect of this disclosure can output notifications to a display that is likely to be within the driver's field of vision.

[0036] Furthermore, the sensor may include an in-vehicle camera, and the first data may include image data captured by the in-vehicle camera.

[0037] Furthermore, the sensor may include an in-vehicle camera, and the second data may include image data captured by the in-vehicle camera.

[0038] As a result, an in-vehicle system according to one aspect of this disclosure can estimate the direction of the driver's gaze by analyzing the driver's image. Therefore, an in-vehicle system according to one aspect of this disclosure can accurately estimate the direction of the driver's gaze.

[0039] Embodiments of this disclosure will be described below with reference to the drawings. The configurations of the following embodiments are illustrative, and this disclosure is not limited to the configurations of these embodiments.

[0040] (First embodiment) An overview of the vehicle system according to the first embodiment will be described. The stem consists of vehicle 1.

[0041] Vehicle 1 will be described with reference to Figure 1. Figure 1 shows the interior of vehicle 1. As shown in the figure, the interior of vehicle 1 is equipped with a driver's seat 2 and a passenger seat 3. In this embodiment, vehicle 1 is a right-hand drive vehicle, and the driver's seat 2 is located on the right side of the vehicle.

[0042] A steering wheel 4 is positioned in front of the driver's seat 2. The steering wheel may be circular, or it may be U-shaped (a so-called irregularly shaped steering wheel) as shown in the illustration.

[0043] The steering wheel 4 is connected to the steering system of the vehicle 1 via the steering column. The steering column is covered by a column cover.

[0044] The passenger seat 3 is located to the left of the driver's seat 2, and a center display 24 is positioned midway between the driver's seat 2 and the passenger seat 3, towards the front of the vehicle. The center display 24 displays, for example, supplementary information such as car navigation, or entertainment content such as audio and television.

[0045] A meter display 21 is provided on the front side of the steering wheel 4. The meter display 21 is a full-screen display located in front of the driver's seat 2 and provides various information to the driver of the vehicle 1. The meter display 21 displays information necessary for driving the vehicle, such as vehicle speed, engine speed (if the vehicle 1 has an engine), charge / discharge status (if the vehicle 1 has a drive battery), and mileage. The meter display 21 is an example of an "instrument unit" in this disclosure.

[0046] A pair of display devices 27 are provided in front of the driver's seat 2 inside the vehicle. The pair of display devices 27 include touch panel displays positioned on both sides of the steering wheel 4. In this embodiment, the display positioned closer to the inside of the vehicle is referred to as the inner display, and the display positioned closer to the outside of the vehicle is referred to as the outer display. The inner display 22 and the outer display 23 are connected to the instrument panel of the vehicle 1.

[0047] The detailed structures of the inner display 22 and the outer display 23 will be described with reference to Figure 2. The inner display 22 and outer display 23 are vertically oriented displays and are connected to a support portion 5, which is a member that supports each display. The support portion 5 has a shape that protrudes in the left-right direction from the steering wheel. The outer display 23 is connected to the right end of the support portion 5 when facing forward of the vehicle, and the inner display 22 is connected to the left end when facing forward of the vehicle. This allows the display surface to be positioned near the grip portion of the steering wheel 4.

[0048] The above explanation is an example where vehicle 1 is a right-hand drive vehicle; if vehicle 1 is a left-hand drive vehicle, the left and right sides will be reversed. That is, the outer display 23 will be connected to the left end of the support part 5 facing the front of the vehicle, and the inner display 22 will be connected to the right end facing the front of the vehicle.

[0049] The support portion 5 may also serve as a column cover that covers the steering column. The support portion 5 extends from the portion through which the steering column passes in in the left-right direction (vehicle width direction), and each of these extends toward the rear of the vehicle, and the angle of the steering wheel with respect to the left-right direction along the way It has a shape that allows it to bend and become smaller. Furthermore, the inner display 22 and the outer display 23 are angled with respect to the left-right direction of the steering wheel, and are positioned so as to tilt toward the steering wheel. That is, the inner display 22 protrudes further to the rear of the vehicle the closer it is to the inside of the vehicle, and the outer display 23 protrudes further to the rear of the vehicle the closer it is to the outside of the vehicle. In this way, the inner display 22 and the outer display 23 are positioned so as to face the driver. The area 4R shown by the dashed line in Figure 2 is the part (grip area) where the steering wheel 4 is gripped by the right hand, and the area 4L shown by the dashed line in Figure 2 is the part (grip area) where the steering wheel 4 is gripped by the left hand. By making the support part 5 the shape described above, the positions of the inner display 22 and the outer display 23 in the longitudinal direction of the vehicle can be brought closer to the grip area of ​​the steering wheel 4. Therefore, the amount of hand movement when the driver performs a touch operation on the inner display 22 or the outer display 23 while driving can be reduced.

[0050] [System Configuration] Next, we will describe the configuration of each device that makes up the vehicle system of vehicle 1. First, let's describe the components of Vehicle 1. Figure 3 is a schematic diagram showing an example of the configuration of Vehicle 1. Vehicle 1 consists of a vehicle system, an electrical system, and an input / output system.

[0051] A vehicle system is a system that controls the motion of a vehicle. A vehicle system includes multiple components related to the vehicle's movement. An input / output system is a system that provides information to the occupants of a vehicle and receives input from them. An input / output system includes multiple components that perform information input and output. An electrical system is a system that controls the electrical components of a vehicle's body. It includes multiple components related to the electrical components of the vehicle's body. The components included in each system may include multiple electrical components and an ECU that controls those electrical components.

[0052] The vehicle ECU 10 and the body ECU 30 can be configured as computers having a processor (CPU, GPU, etc.), main memory (RAM, ROM, etc.), and auxiliary memory (EPROM, hard disk drive, removable media, etc.). The auxiliary memory stores an operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, various functions (software modules) that match a predetermined purpose, as described later, can be realized. However, some or all of the functions may be realized as hardware modules by hardware circuits such as ASICs and FPGAs.

[0053] The vehicle ECU10 is connected to several components related to the vehicle's operation.

[0054] The GPS unit 11 is a unit for acquiring the location information of vehicle 1. The GPS unit 11 includes a GPS antenna and a positioning module for determining the location information. The GPS antenna is an antenna that receives positioning signals transmitted from positioning satellites (also called GNSS satellites). The positioning module is a module that calculates location information based on the signals received by the GPS antenna.

[0055] The ETC (Electronic Toll Collection System) unit 12 collects tolls for toll roads. This is a unit for receiving data wirelessly. The ETC unit 12 consists of an antenna and a communication module.

[0056] The wireless communication unit 13 includes an antenna and a communication module for wireless communication. The antenna is an antenna element that inputs and outputs wireless signals. In this embodiment, the antenna is suitable for mobile communication (e.g., 3G, 4G, 5G, etc.). The antenna may be configured to include multiple physical antennas. The communication module is a module for performing mobile communication. Furthermore, the wireless communication unit 13 may include communication modules other than those for mobile communications. For example, the wireless communication unit 13 may include a Bluetooth® communication module. This makes it possible to establish a wireless connection with a user terminal located inside the vehicle, enabling the provision of services in cooperation with the user terminal (e.g., hands-free calling).

[0057] Sensor group 14 is a collection of multiple sensors for acquiring sensor data used during vehicle operation. Sensors may acquire physical quantities or image data, etc. In this embodiment, examples of sensors include a sensor for detecting vehicle speed (vehicle speed sensor 141), an image sensor for acquiring images of the area in front of the vehicle (onboard camera 142), and an image sensor for acquiring images of the interior of the vehicle (in-cabin camera 143). Furthermore, the vehicle ECU 10 may also be connected to components belonging to the drivetrain system (e.g., engine, motor, battery, transmission, etc.).

[0058] The body ECU30 is connected to multiple electrical components of the vehicle. In this embodiment, examples of electrical components in a vehicle include a car air conditioner, wipers, headlights, side mirrors, and power seats.

[0059] The air conditioning unit 31 is the car air conditioning system for vehicle 1. The air conditioning unit 31 includes, for example, a cooling unit, a heating unit, a compressor, a condenser, etc. The air conditioning unit 31 may also include a control unit for controlling these components.

[0060] The wiper unit 32 includes a front wiper, a rear wiper, and a motor for driving them.

[0061] The headlight unit 33 includes the headlights of the vehicle 1 and a unit for driving the headlights. The headlights are configured to switch between high beams and low beams, for example. If the headlights of the vehicle 1 are adaptive headlights, the illumination of the light can also be controlled based on commands from the body ECU. The headlight unit 33 may further include taillights, brake lights, turn signals, and / or units for driving these.

[0062] The mirror actuator 34 includes side mirrors positioned on the left and right sides of the vehicle 1, and a drive unit for driving the mirrors. The mirror actuator 34 allows the angle of the side mirrors to be adjusted up, down, left, and right.

[0063] The seat actuator 35 includes a drive unit for adjusting the position of a seat provided in the vehicle 1. The seat actuator 35 allows adjustment of the position, height, angle, etc., of the seat surface and backrest.

[0064] Next, we will describe the devices used for inputting and outputting information. In this embodiment, the vehicle 1 has a meter display 21, an inner display 22, an outer display 23, and a center display as displays for outputting information. It has four displays, each with 24 pixels.

[0065] The meter display 21 is a display positioned in front of the driver and outputs information about the vehicle 1. The meter display 21 mainly outputs information about the vehicle's status, such as vehicle speed, engine speed, fuel level, battery level, charge / discharge status, and mileage, but other information may also be output.

[0066] The inner display 22 and outer display 23 are touch-panel displays located on either side of the steering wheel. As mentioned above, the inner display 22 is located towards the center of the vehicle, and the outer display 23 is located on the outside of the vehicle. If vehicle 1 is a right-hand drive vehicle, the inner display 22 is positioned on the left side in the direction of travel, and the outer display 23 is positioned on the right side in the direction of travel. If vehicle 1 is a left-hand drive vehicle, the inner display 22 is positioned on the right side in the direction of travel, and the outer display 23 is positioned on the left side in the direction of travel.

[0067] The center display 24 is a display located in the center of the vehicle. The center display 24 is located in the same place where the car navigation system's display was previously located. The center display 24 can display, for example, supplementary information such as car navigation, or entertainment content such as audio or television.

[0068] The touch panel 25 is a unit for detecting touch operations performed on the aforementioned displays. In this embodiment, the inner display 22, outer display 23, and center display 24 each have a touch panel 25, allowing for touch operation. The touch panel 25 detects touch operations performed on these displays and transmits the detected information to the vehicle ECU 10.

[0069] The audio unit 26 is a unit that performs input and output of audio information. The audio unit 26 may include, for example, a speaker and a microphone.

[0070] Next, we will explain the detailed configuration of the vehicle ECU10. The vehicle ECU 10 is connected to the vehicle system and the drivetrain system, thereby enabling control of vehicle 1. Furthermore, the vehicle ECU 10 has the function of inputting and outputting information through components included in the input / output system, and the function of controlling the electrical system via the body ECU 30, which is responsible for the electrical system. Figure 4 is a diagram illustrating the detailed configuration of the vehicle ECU 10.

[0071] The vehicle ECU 10 is comprised of a control device 101, a storage device 102, and a communication module 103.

[0072] The control device 101 is a calculation unit that realizes various functions of the vehicle ECU 10 by executing a predetermined program. The control device 101 can be realized by a hardware processor such as a CPU. The control device 101 also has RAM, ROM (Read It may be configured to include (Only Memory), cache memory, etc.

[0073] In this embodiment, the control device 101 of the vehicle ECU 10 is configured with six software modules: HMI control unit 1011, vehicle control unit 1012, ADAS control unit 1013, entertainment control unit 1014, navigation unit 1015, and assistant unit 1016. Each software module stores a program in the storage device 102, which will be described later. This may be implemented by having the RAM executed by the control unit 101 (CPU, etc.). The information processing performed by the software module is synonymous with the information processing performed by the control unit 101 (CPU, etc.).

[0074] The HMI control unit 1011 generates a user interface for controlling various components of the vehicle and provides it to the vehicle occupants via the input / output system. The HMI control unit 1011 also passes information input via the input / output system (for example, commands to implement user-specified controls) to the corresponding software module. This makes it possible to control any component of the vehicle through a user interface.

[0075] In the following explanation, commands generated by the user to control the components of Vehicle 1 will be referred to as "user commands." For example, commands to change the air conditioner temperature, commands to change the drive mode, commands to make a phone call, and commands to play music are examples of "user commands."

[0076] Furthermore, the components of a vehicle may include electrical components such as headlights and wipers. Also, the components of a vehicle may include an on-board computer, or software modules executed by the on-board computer (e.g., autonomous driving software, navigation software, etc.).

[0077] The HMI control unit 1011 selects the output destination of the user interface from among several displays (meter display 21, inner display 22, outer display 23, center display 24) depending on the type of component being controlled. The specific configuration will be described later.

[0078] The vehicle control unit 1012 controls components of the vehicle system or components of the drive system. The vehicle control unit 1012 may control a target component (e.g., a drive motor) based on a command (e.g., an acceleration command) received from another component (e.g., an ADAS control unit 1013, described later), or it may control a target component (e.g., a headlight unit) based on a user command (e.g., a headlight activation command) obtained via the HMI control unit 1011.

[0079] The ADAS control unit 1013 controls the Advanced Driver-Assistance Systems. For example, the ADAS control unit 1013 generates vehicle control commands based on sensor information acquired from the on-board sensors included in the sensor group 14, thereby realizing driver assistance such as cruise control, pre-collision safety, and lane keeping assist. For example, the ADAS control unit 1013 controls the vehicle system and the drive system by transmitting control commands to the vehicle control unit 1012.

[0080] The entertainment control unit 1014 controls the entertainment provided to the vehicle's occupants. Examples of entertainment functions include audio functions and television / radio functions.

[0081] The navigation unit 1015 provides a car navigation function. Based on location information obtained from the GPS unit 11 and traffic information obtained from the wireless communication unit 13, the navigation unit 1015 can generate and provide vehicle route guidance. The navigation unit 1015 may, for example, accept the vehicle's destination and waypoints via the HMI control unit 1011, and output information regarding the generated route.

[0082] Assistant unit 1016 provides information to the vehicle occupants by executing a language model capable of natural language interaction. The language model is, for example, LLM (Large Language Model). This could also be a Model. For example, by running an LLM that has information about the vehicle's occupants, it becomes possible to provide information in natural language. If there are multiple occupants in the vehicle, the assistant unit 1016 may obtain a language model corresponding to each person via the network and run any of the language models.

[0083] The storage device 102 is a means for storing information and is composed of storage media such as RAM, magnetic disks, and flash memory. The storage device 102 stores programs executed by the control device 101, data used by those programs, and so on.

[0084] The communication module 103 is a communication interface for connecting the vehicle ECU 10 to the vehicle network. The communication module 103 is configured to communicate with components of the vehicle 1 via a network such as CAN (Controller Area Network). ru.

[0085] The specific configuration of the vehicle ECU 10 can be appropriately modified, with components omitted, replaced, and added depending on the embodiment. For example, the control device 101 may include multiple hardware processors. The hardware processors may consist of microprocessors, FPGAs, GPUs, etc. Input / output devices other than those exemplified (e.g., optical drives, etc.) may also be added. Furthermore, the vehicle ECU 10 may be composed of multiple computers. In this case, the hardware configurations of each computer may or may not be identical.

[0086] [Interface screen provided by the HMI control unit] Next, the user interface provided by the HMI control unit 1011 will be described.

[0087] The HMI control unit 1011 can generate multiple types of graphical user interface (GUI) screens and output them via the multiple displays mentioned above. In this embodiment, the HMI control unit 1011 aggregates information related to vehicle control on the outer display 23 and aggregates and outputs information related to occupant convenience and comfort on the inner display 22. This allows occupants of the vehicle 1 to intuitively recognize whether the desired information is output to the left or right side. Furthermore, the touch panel operating surfaces of the inner display 22 and outer display 23 are positioned near the steering wheel. This minimizes the amount of eye and hand movement required by the vehicle driver.

[0088] The HMI control unit 1011 can switch between multiple types of screens that are output to the outer display 23. Figure 5 illustrates the transitions between multiple types of screens output to the outer display 23. As shown, the HMI control unit 1011 can output multiple screens, centered around the main screen 501, to the outer display 23.

[0089] (1) Main screen 501 The main screen 501 is the screen that is normally displayed on the outer display while driving. Figure 6(A) shows an example of the main screen 501. The content displayed on the main screen can be customized by the user.

[0090] (2) Headlight control screen 502 This screen is used to set the illumination status of the headlights of vehicle 1. Figure 7(A) shows an example of the headlight control screen 502. The headlight control screen 502 allows the user to select the headlight lighting status (automatic, on, off, small lights, daytime running lights, etc.). The vehicle control unit 1012 generates a user command based on the touch operation acquired via the HMI control unit 1011 and controls the headlight unit 33 by transmitting the user command to the body ECU 30. Furthermore, the vehicle control unit 1012 may acquire data indicating the headlight illumination status from the body ECU 30 and reflect the illumination status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 may reflect the current headlight illumination status in the vehicle graphic drawn on the screen.

[0091] (3) Wiper control screen 503 This screen is used to set the operating status of the wipers (front and rear) of vehicle 1. Figure 7(B) shows an example of the wiper control screen 503. On the wiper control screen 503, the operating state of the wipers (stop, automatic operation, intermittent operation, continuous operation, etc.) can be selected. The vehicle control unit 1012 generates a user command based on the touch operation acquired via the HMI control unit 1011 and controls the wiper unit 32 by transmitting the user command to the body ECU 30. Furthermore, the vehicle control unit 1012 may acquire data indicating the wiper's operating status from the body ECU 30 and reflect the operating status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 may reflect the current wiper's operating status in the vehicle graphic drawn on the screen.

[0092] (4) Mirror control screen 504 This screen is used to adjust the deployment state and angle of the side mirrors of vehicle 1. Figure 7(C) shows an example of the mirror control screen 504. On the mirror control screen 504, the status of the side mirror (folded / unfolded state and angle) can be selected. The vehicle control unit 1012 generates a user command based on the touch operation acquired via the HMI control unit 1011 and controls the mirror actuator 34 by transmitting the user command to the body ECU 30. Furthermore, the vehicle control unit 1012 may acquire data indicating the status of the side mirrors from the body ECU 30 and reflect the operating status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 may reflect the current extended / retracted state of the side mirrors in the vehicle graphic drawn on the screen.

[0093] (5) Driving position adjustment screen 505 This screen allows you to adjust the position and angle of the seats (seat cushion and backrest) and the steering wheel of vehicle 1. The driving position adjustment screen 505 allows users to set the position and angle of the seat and steering wheel. The vehicle control unit 1012 receives touch operations via the HMI control unit 1011, generates user commands based on these operations, and transmits these user commands to the body ECU 30 to control the seat actuator 35 and other components.

[0094] (6) ADAS settings screen 506 This screen displays the operating status of the advanced driver assistance system in vehicle 1 and allows for setting adjustments. For example, if the advanced driver assistance system of vehicle 1 includes an auto cruise function, the user of vehicle 1 can set the cruising speed, distance between vehicles, whether or not to follow the vehicle in front, whether or not to use lane tracing, etc., via the ADAS setting screen 506. The information included in the ADAS settings screen 506 can be generated by the ADAS control unit 1013.

[0095] (7) Drive mode setting screen 507 This is the screen for setting the drive mode of vehicle 1. Figure 7(D) shows an example of the drive mode setting screen 507. For example, the user of vehicle 1 can select a mode from several modes, such as normal mode, sport mode, eco mode, and snow mode, via the drive mode setting screen 507. The vehicle control unit 1012 changes the vehicle's drive mode based on this selection.

[0096] (8) Vehicle function setting screen 508 This screen is for using other functions of Vehicle 1. Examples of other functions of Vehicle 1 include opening and closing the electric tailgate and adjusting the suspension.

[0097] (9) Display mode setting screen 509 This screen is for setting the types of information displayed on the meter display 21. For example, if the meter display 21 can display multiple pieces of information other than speed, the user of the vehicle 1 can customize the information displayed on the meter display 21 via the display mode setting screen 509.

[0098] The multiple screens exemplified above can be switched by performing a predetermined operation, such as swiping left or right, on the touch panel provided on the outer display 23. Furthermore, the HMI control unit 1011 can display a shortcut screen 510 on the outer display 23 for jumping to each screen. Figure 6(B) shows an example of the shortcut screen 510. The shortcut screen can be accessed, for example, by performing a predetermined operation (for example, swiping from the bottom of the screen upwards) on each screen, including the main screen 501. The shortcut screen 510 has multiple interfaces (for example, buttons) for transitioning to each screen, and by operating these interfaces, the user can move to the desired screen.

[0099] In this way, the HMI control unit 1011 aggregates and outputs information related to vehicle control to the outer display 23. Information related to vehicle control typically refers to information related to the vehicle's driving control. Examples of information related to vehicle driving control include information for controlling components belonging to the vehicle system and information for controlling components belonging to the drive system. Furthermore, information for controlling components belonging to the electrical system, such as lights and wipers, which are necessary for the vehicle to drive properly, can also be considered information related to vehicle driving control. Furthermore, information related to vehicle control, even if not directly related to vehicle driving control, can be output to the outer display 23. For example, information for controlling the vehicle's electric doors may be output to the outer display 23.

[0100] Similarly, the HMI control unit 1011 can switch the screen output to the inner display 22 to multiple types of screens. Figure 8 illustrates the transitions between multiple types of screens output to the inner display 22. As shown, the HMI control unit 1011 can output multiple screens, centered around the main screen 801, to the inner display 22.

[0101] (1) Main screen 801 The main screen 801 is the screen that is normally displayed on the inner display while driving. Figure 9(A) shows an example of the main screen 801. The content displayed on the main screen can be customized by the user.

[0102] (2) Navigation screen 802 This is a screen related to car navigation. Figure 10(A) shows an example of the navigation screen 802. The information contained in the navigation screen 802 is generated by the navigation unit 1015. Furthermore, when map information from the car navigation system is displayed on the center display 24, the navigation screen 802 may output supplementary information regarding the vehicle 1's route. Examples of supplementary route information include guidance on the next turn, estimated time of arrival, and remaining distance.

[0103] Furthermore, the navigation screen 802 may include some of the information and user interface displayed on the center display 24. For example, frequently used user interfaces such as a button to start directions to home, a button to access bookmarks, and a button to start directions to a charging station can be placed on the navigation screen 802.

[0104] (3) Audio screen 803 This is a screen related to the audio functions of vehicle 1. Figure 10(B) shows an example of the audio screen 803. The information contained in the audio screen 803 is generated by the entertainment control unit 1014. The entertainment control unit 1014 can acquire and play music via a network or media. The audio screen 803 can include information about the music currently being played and an interface for selecting music.

[0105] (4) Air conditioner settings screen 804 This is a screen for controlling the car air conditioner of vehicle 1. Figure 10(C) shows an example of the air conditioner setting screen 804. The air conditioning settings screen 804 allows users to set the temperature and airflow of the car's air conditioning system, as well as the air conditioning devices to be operated (steering wheel heater, seat heater, defroster, etc.). The vehicle control unit 1012 generates user commands based on the operations performed on the screen and transmits these user commands to the body ECU 30 to control the air conditioning unit 31 and other components. Although the car air conditioner is a vehicle 1 piece of equipment controlled by external commands, it relates to the comfort of the occupants, so the air conditioner setting screen 804 is displayed on the inner display 22, not the outer display 23.

[0106] (5) Phone screen 805 This screen is for wirelessly connecting with the user terminal and making hands-free calls. If the wireless communication unit 13 has a module for wireless communication with the user terminal, it can make (or receive) calls from the user terminal via this module.

[0107] (6) AI Partner Screen 806 This is a screen for interacting with a virtual agent (AI partner) capable of natural language dialogue. Figure 10(D) shows an example of the AI ​​partner screen 806. This screen includes a virtual agent character, and the vehicle occupant can obtain information by interacting with this character. The natural language dialogue service is provided by the assistant unit 1016. The AI ​​partner screen 806 may include an interface for giving instructions to the agent. If there are multiple occupants in the vehicle, the assistant unit 1016 may be configured to run any agent corresponding to each person. The AI ​​partner screen 806 may also include interfaces for starting and ending interactions with the agent, and interfaces for switching agents.

[0108] The multiple screens exemplified above can be switched by performing a predetermined operation, such as swiping left or right, on the touch panel provided on the inner display 22. Furthermore, the HMI control unit 1011 can display a shortcut screen 810 on the inner display 22 for jumping to each screen. Figure 9(B) shows an example of the shortcut screen 810. The shortcut screen can be accessed, for example, by performing a predetermined operation (for example, swiping from the bottom of the screen upwards) on each screen, including the main screen 801. The shortcut screen 810 has multiple interfaces (for example, buttons) for transitioning to each screen, and by operating these interfaces, the user can move to the desired screen.

[0109] In this way, the HMI control unit 1011 aggregates and outputs information regarding the convenience and comfort of the occupants to the inner display 22.

[0110] Furthermore, information about components that are part of the vehicle, such as car air conditioners, but relate to the convenience or comfort of the occupants, may be output to the inner display 22 without being treated as information related to vehicle control. Thus, if the information in question falls under both vehicle control information and information regarding occupant convenience (or comfort), the information may be selectively output to either the inner display 22 or the outer display 23 according to predetermined criteria.

[0111] [Interruption display on interface screen] Next, we will explain interrupt display. While the vehicle is in motion, various components may generate notifications for the vehicle's occupants. For example, if the vehicle is connected to a user terminal, it may be necessary to notify the occupants of incoming phone calls as needed. Also, if an event occurs while using ADAS functions (for example, a request to hand over from autonomous driving to manual driving), it may be necessary to notify the occupants of this event. When the HMI control unit 1011 receives a request for such a notification (notification request) from a component of the vehicle 1, it displays an interruption via the inner display 22 or outer display 23.

[0112] Figure 11 is a flowchart of the process performed by the HMI control unit 1011. This process is performed periodically while the vehicle's main switch is turned on.

[0113] First, in step S10, the HMI control unit 1011 acquires data (referred to here as the first data) acquired by one of the sensors in the vehicle system sensor group 14. For example, in step S10, the HMI control unit 1011 may acquire image data acquired by the in-vehicle camera 143. The process then proceeds to step S11. The in-vehicle camera 143 is a camera capable of imaging the interior of a vehicle and captures an image of the driver's face. The in-vehicle camera 143 may be a visible light camera or an infrared camera. The first data may be a visible light image or an infrared image.

[0114] In step S11, the HMI control unit 1011 performs the following based on the first data acquired by the sensor. Then, the first direction, which is the direction in which the driver of vehicle 1 is looking, is estimated. For example, the HMI control unit 1011 may analyze the image data acquired by the in-vehicle camera 143 to determine the direction of the driver's face and estimate the first direction. Subsequently, the process transitions to step S12.

[0115] In step S12, the HMI control unit 1011 determines whether or not a notification has been issued to the driver from the vehicle ECU 10 or body ECU 30 installed in the vehicle 1. If the HMI control unit 1011 determines that the notification has occurred, the process proceeds to step S13. If the HMI control unit 1011 determines that the notification has not occurred, the process terminates.

[0116] In step S13, the HMI control unit 1011 outputs a notification from the HMI control unit 1011 to the driver of vehicle 1 to the display device selected based on the first direction (hereinafter referred to as the first display). Here, the display device selected based on the first direction may be the display device closest to the virtual axis representing the first direction.

[0117] In step S14, after a predetermined time has elapsed since outputting a predetermined notification to the first display, the direction in which the vehicle driver is looking is estimated based on data newly acquired by one of the sensors included in the sensor group 14 (referred to here as the second data). For example, the HMI control unit 1011 may analyze the image data acquired by the in-cabin camera 143 to determine the orientation of the driver's face and estimate the direction in which the vehicle driver is looking. Here, the predetermined time is sufficient time to determine whether or not the driver has seen the notification output to the first display.

[0118] In step S15, the HMI control unit 1011 determines, based on the second data, whether the driver is looking at the first display. If the HMI control unit 1011 determines that the driver is not looking at the first display, the process proceeds to step S16. If the HMI control unit 1011 determines that the driver is looking at the first display, the process ends.

[0119] In step S16, the HMI control unit 1011 outputs a predetermined notification from the HMI control unit 1011 to the driver of vehicle 1 to a display device selected based on the second direction the vehicle driver is looking at (hereinafter referred to as the second display). Here, the display device selected based on the second direction may be the display device closest to the virtual axis representing the second direction. For example, if the HMI control unit 1011 determines that the driver is not looking at the first display on which the notification was output, the HMI control unit 1011 outputs the notification again to the second display located in the direction the driver is looking at.

[0120] Furthermore, the HMI control unit 1011 may clear the displayed notifications after a predetermined time has elapsed since outputting them to the first and second displays. Also, for example, if the user taps either the first or second display after the HMI control unit 1011 has outputted a notification to the first or second display, the HMI control unit 1011 may clear the notification displayed on the display device tapped by the user. Alternatively, the HMI control unit 1011 may clear the notification displayed on the first or second display when the task related to the notification is completed. For example, if the task related to the notification was an incoming phone call, the notification displayed on the first or second display may be cleared when the phone call is completed.

[0121] As described above, in this embodiment, the HMI control unit 1011 of the vehicle ECU estimates the direction the driver is looking based on the data acquired by the sensor, and outputs the notification to a display located in the direction the driver is looking when a notification is issued to the driver from the vehicle ECU 10 or the body ECU 30.

[0122] According to such a configuration, the in-vehicle system according to one aspect of this disclosure will be able to output notifications in a location that is easily visible to the driver.

[0123] (Second Embodiment) In the first embodiment described above, the HMI control unit 1011 estimated the first direction, which is the direction the driver is looking, based on the first data acquired by the sensor. In the first embodiment, for example, an in-vehicle camera 143 is assumed to be the sensor, and the first direction is estimated by analyzing the first data, which is image data acquired by the in-vehicle camera. In contrast, in the second embodiment, when the HMI control unit 1011 detects that the driver is operating a display, it simply outputs a notification to the display without re-detecting the direction of the driver's gaze.

[0124] Figure 12 is a flowchart of the process that outputs a notification to the display device being operated by the driver, based on data used to determine which display device is being operated. This process is executed periodically while the vehicle's main switch is turned on.

[0125] In step S20, the HMI control unit 1011 acquires data detected by various sensors installed in the vehicle 1. For example, the HMI control unit 1011 may acquire data detected by touch sensors on the displays of multiple touch panels 25. Here, the touch panels are included in multiple display devices mounted in the vehicle 1. Note that the sensor data acquired by the HMI control unit 1011 is not limited to data acquired from the touch panels 25. The HMI control unit 1011 may also acquire data from other types of display devices indicating that a driver or other person is operating the display device.

[0126] In step S21, the HMI control unit 1011 determines whether the data acquired by the sensor indicates that the driver is operating on one of the display devices. For example, the sensor in step S21 is a touch sensor on the display of the display device to be detected (e.g., touch panel 25). If the HMI control unit 1011 determines that the data acquired by the sensor indicates that the driver is operating any of the display devices, the process proceeds to step S22. In this embodiment, the display device that the driver is operating is referred to as the third display. If the HMI control unit 1011 determines that the data acquired by the sensor indicates that the driver is not operating any of the display devices, the process proceeds to step S23.

[0127] In step S22, the HMI control unit 1011 determines which display device (third display) is being operated by the driver and outputs a notification to the driver from the HMI control unit 1011 to the third display.

[0128] In other words, when the HMI control unit 1011 detects that the driver is operating any of the display devices, it outputs a notification to the display being operated without estimating the first direction.

[0129] In step S23, the HMI control unit 1011 outputs a notification to the driver, originating from the vehicle ECU 10 or the body ECU 30, to a display determined based on the direction of the driver's gaze.

[0130] Specifically, if the process proceeds to step S23, the HMI control unit 1011 executes the processes described in steps S11 to S16 in the first embodiment.

[0131] With the above configuration, an in-vehicle system according to one aspect of this disclosure can reliably output a notification to a display device located in the direction the driver is looking, by detecting that the driver is operating a display device. (modified version)

[0132] In this embodiment, the HMI control unit 1011 outputs a notification to the driver of vehicle 1 from the HMI control unit 1011 to the display device closest to the virtual axis representing the first direction.

[0133] However, the HMI control unit 1011 does not necessarily have to output notifications only to the display device closest to the virtual axis representing the first direction. The HMI control unit 1011 may determine one or more display devices located within a predetermined range from the virtual axis indicating the first direction as the first display and output notifications to that first display.

[0134] Furthermore, for example, if multiple display devices exist within a predetermined range from a virtual axis representing the first direction, the HMI control unit 1011 may determine the display device with the highest pre-assigned priority among the multiple display devices as the first display and output a notification to the first display. Here, "existing within a predetermined range from the virtual axis representing the first direction" may mean existing within a predetermined angle from a point on the virtual axis representing the first direction, with the axis as the base axis. Alternatively, "existing within a predetermined range from the virtual axis representing the first direction" may mean existing within a predetermined distance from the virtual axis representing the first direction.

[0135] The HMI control unit 1011 may determine both the display device closest to the virtual axis representing the first direction and the display device with the highest pre-assigned priority as the first display, and output a predetermined notification to the first display.

[0136] Alternatively, the HMI control unit 1011 may determine all of the multiple display devices located within a predetermined range from a virtual axis representing a first direction as the first display, and output a notification to the first display.

[0137] Alternatively, the HMI control unit 1011 may determine one of several display devices located within a predetermined range from a virtual axis representing a first direction as the first display, output a notification to the first display, and output a message to the other display devices indicating that "a notification has been output to the said display device."

[0138] Furthermore, if no display device exists within a predetermined range from a virtual axis representing the first direction, the HMI control unit 1011 may determine which display device (first display) will output the notification, depending on the type and source of the notification.

[0139] In this embodiment, the HMI control unit 1011 outputs a notification from the HMI control unit 1011 to the driver of vehicle 1 to the first display, then determines the second direction the driver is looking, and outputs the notification from the HMI control unit 1011 to the display device closest to the virtual axis representing the second direction.

[0140] However, the HMI control unit 1011 does not necessarily have to output notifications only to the display device closest to the virtual axis representing the second direction. The HMI control unit 1011 may determine one or more display devices located within a predetermined range from the virtual axis indicating the second direction as the second display and output notifications to that second display.

[0141] Furthermore, for example, if multiple display devices exist within a predetermined range from a virtual axis representing the second direction, the HMI control unit 1011 may determine the display device with the highest pre-assigned priority among the multiple display devices as the second display and output a notification to the second display. Here, "existing within a predetermined range from the virtual axis representing the second direction" may mean existing within a predetermined angle from a point on the virtual axis representing the second direction, with the axis as the base axis. Alternatively, "existing within a predetermined range from the virtual axis representing the second direction" may mean existing within a predetermined distance from the virtual axis representing the second direction.

[0142] The HMI control unit 1011 may determine both the display device closest to the virtual axis representing the second direction and the display device with the highest pre-assigned priority as the second display, and output a predetermined notification to the second display.

[0143] Alternatively, the HMI control unit 1011 may determine all of the multiple display devices located within a predetermined range from a virtual axis representing the second direction as the second display, and output a notification to the second display.

[0144] Alternatively, the HMI control unit 1011 may determine one of several display devices located within a predetermined range from a virtual axis representing the second direction as the second display, output a notification to the second display, and output a message to the other display devices indicating that "a notification has been output to the said display device."

[0145] Furthermore, if no display device exists within a predetermined range from the virtual axis representing the second direction, the HMI control unit 1011 may determine which display device (second display) outputs the notification, depending on the type and source of the notification.

[0146] (Other variations) The embodiments described above are merely examples, and this disclosure may be modified as appropriate without departing from its essence. For example, the processes and means described in this disclosure can be freely combined and implemented, as long as no technical inconsistencies arise.

[0147] Furthermore, although the embodiment illustrates a configuration in which the meter display 21, inner display 22, and outer display 23 are independent of each other, each display does not necessarily have to be physically independent.

[0148] For example, as shown in Figure 13, the meter display 21, inner display 22, and outer display 23 may be realized by a single display 1301. In the example in Figure 13, the display 1301 has three display areas (display sections) indicated by reference numerals 1302, 1303, and 1304. Each of the three display areas is planar, and the space between these display areas is curved. In the illustrated example, display area 1302 functions as the inner display 22, and display area 1303 functions as the outer display 23 (for right-hand drive vehicles). Additionally, display area 1304 functions as the meter display 21.

[0149] The portion connecting display areas 1302 and 1304 (reference numeral 1305), and the portion connecting display areas 1303 and 1304 (reference numeral 1306), can be used for display purposes, but are physically curved and distorted, so are not used as display units in normal configurations.

[0150] However, if the driver's gaze is directed towards display area 1305 or display area 1306, the HMI control unit 1011 may output a notification to display area 1305 or display area 1306. Alternatively, if the driver's gaze is directed towards display area 1305 or display area 1306 and a notification is output to display area 1302 or display area 1303, the HMI control unit 1011 may output information to display area 1305 or display area 1306 indicating that a notification is displayed in another display area (for example, display area 1302 or display area 1303).

[0151] Furthermore, in this embodiment, three types of data were exemplified as the first data: image data, infrared data, and touch sensor detection data. However, the first data may be a combination of these, or it may include other information.

[0152] Furthermore, although the embodiment illustrates the first display as a single display, the first display may be a plurality of displays located within a predetermined range from a virtual axis representing a first direction.

[0153] Furthermore, a process described as being performed by a single device may be divided and executed by multiple devices. Conversely, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is implemented can be flexibly changed.

[0154] The present disclosure can also be realized by supplying a computer program implementing the functions described in the embodiments above to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network. Non-temporary computer-readable storage mediums include, for example, any type of disk such as magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions. [Explanation of Symbols]

[0155] 1. Vehicle 10. Vehicle ECU 101... Control device 102...Storage device 103...Communication module 21... Meter display 22. Inner display 23. Outer display 24... Center display 25...Touch panel 26..Audio Unit 30...Body ECU

Claims

1. Sensors and, Multiple display devices, Based on the first data acquired by the aforementioned sensor, the first direction, which is the direction the driver is looking, is estimated. When a predetermined notification is issued to the driver, the predetermined notification is output to the first display, which is a display device selected from the plurality of display devices based on the first direction. It comprises a control unit that performs the following: In-vehicle systems.

2. The control unit, The predetermined notification is output to the display device closest to the axis indicating the first direction. The in-vehicle system according to claim 1.

3. The control unit, Among the plurality of display devices, a display device located within a predetermined range from the axis indicating the first direction is determined to be the first display, and the predetermined notification is output to the first display. The in-vehicle system according to claim 1.

4. The control unit, If multiple display devices exist within a predetermined range from the axis indicating the first direction, the display device with the highest pre-assigned priority is determined to be the first display, and the predetermined notification is output to the first display. The in-vehicle system according to claim 3.

5. The control unit, If no display device exists within a predetermined range from the axis indicating the first direction, the first display is determined based on the type of notification or the source of the notification, and the predetermined notification is output to the first display. The in-vehicle system according to claim 1.

6. The control unit, After outputting the predetermined notification to the first display, it is determined whether the driver is looking at the first display based on second data, which is data acquired by the sensor and is different from the first data. If it is determined that the driver is not looking at the first display, a second direction, which is different from the first direction and is the direction the driver is looking, is estimated. The predetermined notification is output to a second display, which is a display device selected based on the second direction. The in-vehicle system according to claim 1.

7. The first data includes data for determining which of the plurality of display devices the driver is operating, If the control unit determines, based on the first data, that the driver is performing an operation on the third display, it outputs the predetermined notification to the third display. The in-vehicle system according to claim 1.

8. The aforementioned sensor includes an in-vehicle camera, The first data includes image data captured by the in-vehicle camera, The in-vehicle system according to claim 1.

9. The aforementioned sensor includes an in-vehicle camera, The second data includes image data captured by the in-vehicle camera, The in-vehicle system according to claim 6.

10. A vehicle equipped with the in-vehicle system according to any one of claims 1 to 9.

11. An information processing device that outputs information to multiple display devices located in front of the driver's seat inside the vehicle, Based on the first data, which is data acquired by the sensor, the first direction, which is the direction in which the driver of the vehicle equipped with the multiple display devices is looking, is estimated. When a predetermined notification is generated from the vehicle toward the driver, the predetermined notification is output to the first display, which is a display device selected from among the plurality of display devices based on the first direction. It comprises a control unit that performs the following: Information processing device.

12. The control unit, The predetermined notification is output to the display device closest to the axis indicating the first direction. The information processing apparatus according to claim 11.

13. The control unit, Among the plurality of display devices, a display device located within a predetermined range from the axis indicating the first direction is determined to be the first display, and the predetermined notification is output to the first display. The information processing apparatus according to claim 11.

14. The control unit, If multiple display devices exist within a predetermined range from the axis indicating the first direction, the display device with the highest pre-assigned priority is determined to be the first display, and the predetermined notification is output to the first display. The information processing apparatus according to claim 13.

15. The control unit, If no display device exists within a predetermined range from the axis indicating the first direction, the first display is determined based on the type of notification or the source of the notification, and the predetermined notification is output to the first display. The information processing apparatus according to claim 11.

16. The control unit, After outputting the predetermined notification to the first display, it is determined whether the driver is looking at the first display based on second data, which is data acquired by the sensor and is different from the first data. If it is determined that the driver is not looking at the first display, a second direction, which is different from the first direction and is the direction the driver is looking, is estimated. The predetermined notification is output to a second display, which is a display device selected based on the second direction. The information processing apparatus according to claim 11.

17. The first data includes data for determining which of the plurality of display devices the driver is operating, If the control unit determines, based on the first data, that the driver is performing an operation on the third display, it outputs the predetermined notification to the third display. The information processing apparatus according to claim 11.

18. The aforementioned sensor includes an in-vehicle camera, The first data includes image data captured by the in-vehicle camera, The information processing apparatus according to claim 11.

19. An information processing method performed by an in-vehicle system equipped with an information processing device that outputs information via multiple display devices, A step of estimating a first direction, which is the direction in which the driver of the vehicle equipped with the in-vehicle system is looking, based on first data acquired by the sensor, The system includes the step of outputting a predetermined notification to the driver from the in-vehicle system to a first display, which is a display device selected from among the plurality of display devices based on the first direction, when a predetermined notification is generated from the in-vehicle system to the driver. Information processing methods.

20. A program for causing a computer to execute the information processing method described in claim 19.

Images