In-vehicle system, vehicle, information processing device, information processing method, and storage medium

Abstract

The present application relates to a vehicle-mounted system, a vehicle, an information processing device, an information processing method, and a storage medium, which output a notification to a place easy to enter the driver's field of view. It is provided with: a sensor; a plurality of display devices; and a control section that performs the following processing: infers a direction in which the driver is looking, i.e., a first direction, based on data obtained by the above sensor, i.e., first data; and in a case where a prescribed notification to the above driver is generated, outputs the above prescribed notification to a display device selected from among the above plurality of display devices based on the above first direction, i.e., a first display.

Description

Technical Field

[0001] This disclosure relates to vehicle technology. Background Technology

[0002] There is a technology for providing information to reduce the burden on drivers of automobiles. Relatedly, for example, Patent Document 1 discloses an in-vehicle system in which an information-providing display is positioned near the steering wheel.

[0003] Patent Document 1: Japanese Patent No. 7306210

[0004] Patent Document 2: Japanese Patent Application Publication No. 2017-097687 Summary of the Invention

[0005] The purpose of this disclosure is to deliver notifications to locations that are easily within the driver's field of vision.

[0006] One aspect of this disclosure provides an in-vehicle system comprising: a sensor; a plurality of display devices; and a control unit that performs the following processing: inferring the direction the driver is looking at, i.e., a first direction, based on data obtained from the sensor, i.e., first data; and, upon generating a predetermined notification toward the driver, outputting the predetermined notification to a display device selected from the plurality of display devices based on the first direction, i.e., a first display.

[0007] One aspect of this disclosure provides an information processing apparatus that outputs information to a plurality of display devices disposed in front of the driver's seat in a vehicle compartment. The information processing apparatus includes a control unit that performs the following processing: infers the direction in which the driver of the vehicle equipped with the plurality of display devices is looking, i.e., a first direction, based on data obtained by sensors, i.e., first data; and outputs the predetermined notification to a display device selected from the plurality of display devices based on the first direction, i.e., a first display, when a predetermined notification is generated from the vehicle toward the driver.

[0008] One aspect of this disclosure provides an information processing method performed by an in-vehicle system having an information processing device for outputting information via multiple display devices, wherein the information processing method includes: inferring the direction in which the driver of a vehicle equipped with the in-vehicle system is looking, i.e., a first direction, based on data obtained from sensors, i.e., first data; and, upon the generation of a predetermined notification from the in-vehicle system toward the driver, outputting the predetermined notification to a display device selected from the multiple display devices based on the first direction, i.e., a first display.

[0009] Alternatively, vehicles equipped with the aforementioned in-vehicle system, programs for causing computers to execute the aforementioned information processing methods, or computer-readable storage media that non-temporarily store the program can be cited as examples.

[0010] According to this disclosure, it is possible to output notifications to locations that are easily within the driver's field of vision. Attached Figure Description

[0011] Figure 1 This is a diagram showing the interior of vehicle 1 according to the embodiment.

[0012] Figure 2 This is an enlarged view of the steering wheel and its surrounding area.

[0013] Figure 3 This is a diagram showing an example of the constituent elements of vehicle 1.

[0014] Figure 4 This is a diagram showing an example of the components of a vehicle ECU10.

[0015] Figure 5 It is a diagram showing an overview of the screen output to an external monitor.

[0016] Figure 6 This is a diagram showing an example of a screen being output to an external display.

[0017] Figure 7 This is a diagram showing an example of a screen being output to an external display.

[0018] Figure 8 It is a diagram showing an overview of the screen being output to the internal display.

[0019] Figure 9 This is a diagram showing an example of a screen being output to an internal display.

[0020] Figure 10 This is a diagram showing an example of a screen being output to an internal display.

[0021] Figure 11 This is a flowchart of the processes performed by the HMI control unit 1011.

[0022] Figure 12 This is a flowchart of a process that outputs a notification to the display device being operated based on data from the display device used to determine what the driver is doing.

[0023] Figure 13 This is a diagram illustrating one example of a display device involved in a variation.

[0024] Explanation of reference numerals in the attached figures

[0025] 1...Vehicle; 10...Vehicle ECU; 101...Control device; 102...Storage device; 103...Communication module; 21...Instrument display; 22...Internal display; 23...External display; 24...Central display; 25...Touch panel; 26...Sound unit; 30...Body ECU. Detailed Implementation

[0026] In recent years, with the increasing sophistication of automobiles, displays used to provide information have become larger or more numerous.

[0027] For example, a method has been proposed to place a large touch panel display between the driver's seat and the front passenger seat in the front of the vehicle to allow for simultaneous vehicle-related operations and information provision.

[0028] However, in such a method, the driver's gaze shifts significantly when operating vehicle components or confirming output information. For vehicle safety, it is preferable to minimize this shift in the driver's gaze.

[0029] Generally speaking, the area in front of the driver's seat that was previously equipped with an instrument panel is often equipped with an LCD or other display device. By outputting information to this display device, the movement of the eyes can be minimized.

[0030] However, the size of the display device positioned on the front is limited, and it may not be able to display all the information the driver needs. In addition, if this information is output in the same area at the same time, such as information needed for driving the vehicle and auxiliary information such as entertainment, it will hinder the intuitive transmission of information.

[0031] In addition, there are also issues to be addressed in configuring the input interfaces used for operation.

[0032] For example, there is a known steering wheel with cursor keys and push-button switches that allows input operations while driving. However, in such an input interface, the layout of the buttons, etc., is fixed, making it difficult to adapt to multiple different objects. For example, setting the temperature of the air conditioner can be done using two physical keys, but it is not possible to directly specify any arbitrary location on the map using physical keys.

[0033] Although operation via a touch panel was also considered, the touch panel display, as described above, is located far from the steering wheel in the front of the cabin, so in addition to the movement of the eyes, the driver's hands also need to move significantly.

[0034] To address these issues, it is preferable to place a display device capable of touch-based input near the steering wheel.

[0035] One aspect of the present disclosure relates to an in-vehicle system comprising: a sensor; a plurality of display devices; and a control unit that performs the following processing: inferring the direction the driver is looking at, i.e., a first direction, based on data obtained from the aforementioned sensor, i.e., first data; and, upon generating a prescribed notification toward the aforementioned driver, outputting the prescribed notification to a display device selected from the plurality of display devices based on the aforementioned first direction, i.e., a first display.

[0036] The display device has a pair of display units positioned on either side of the steering wheel. Furthermore, the pair of display units are configured to be operable via a touch panel.

[0037] First data refers to data acquired by sensors within the vehicle's onboard system. For example, first data could be image data acquired by an onboard camera, temperature mapping data acquired by an infrared sensor, or detection data acquired by a touch sensor.

[0038] The first direction refers to the direction that the driver of a vehicle equipped with an in-vehicle system according to one aspect of this disclosure is looking in. The first direction is derived based on first data.

[0039] The first display refers to a display located within a specified range (within a specified angle) of a virtual axis indicating the first direction.

[0040] The control unit determines a first direction based on the first data, and when there is a prescribed notification for the driver, outputs the prescribed notification to the display device selected based on the first direction, namely the first display.

[0041] Based on this configuration, notifications can be output to locations that are easily within the driver's field of vision.

[0042] The aforementioned control unit can output the aforementioned notification to the display device closest to the axis representing the first direction.

[0043] Thus, the in-vehicle system according to one aspect of this disclosure is able to output notifications to the display device most likely to enter the driver's field of vision.

[0044] In addition, the control unit can designate a display device that exists in a region within a specified distance from the axis representing the first direction as the first display device, and output the specified notification to the first display device.

[0045] Thus, the in-vehicle system according to one aspect of this disclosure is able to output notifications to a display device located within the driver's field of vision.

[0046] In addition, when there are multiple display devices in an area within a specified distance from the axis representing the first direction, the control unit may determine the display device with the highest pre-assigned priority as the first display and output the specified notification to the first display.

[0047] For example, in one aspect of this disclosure, the in-vehicle system sometimes prioritizes display devices that output prescribed notifications based on considerations of displaying other information. By outputting prescribed notifications to higher-priority display devices, the in-vehicle system of this disclosure can output the notification within the vehicle through appropriate configuration.

[0048] Furthermore, when there is no display device in the area within a predetermined distance from the axis representing the first direction, the control unit can determine the first display based on the type of notification or the source of the notification, and output the predetermined notification to the first display.

[0049] Therefore, the vehicle system according to one aspect of this disclosure can output notifications to a reasonably determined display device even in the absence of a display device in the direction of the driver's line of sight.

[0050] In addition, after outputting the aforementioned notification to the aforementioned first display, the control unit may determine whether the driver is looking at the aforementioned first display based on data obtained by the aforementioned sensor that is different from the aforementioned first data, namely second data. If it is determined that the driver is not looking at the aforementioned first display, the control unit may infer the direction that the driver is looking at that is different from the aforementioned first direction, namely the second direction, and output the aforementioned notification to the display device selected based on the aforementioned second direction, namely the second display.

[0051] Therefore, the vehicle system involved in one aspect of this disclosure can output notifications to more appropriate locations in accordance with the movement of the driver's gaze.

[0052] In addition, the first data may include data for determining which of the plurality of display devices is being operated by the driver. When the control unit determines, based on the first data, that the driver is operating the third display, it outputs the aforementioned notification to the third display.

[0053] For example, it is generally conceivable that a driver would be looking at a display while operating it while driving. Therefore, an in-vehicle system according to one aspect of this disclosure can output a prescribed notification to the display that the driver is operating.

[0054] Therefore, the in-vehicle system according to one aspect of this disclosure can output notifications to displays that are highly likely to be viewed by the driver based on the driver's actions. Thus, the in-vehicle system according to one aspect of this disclosure can output notifications to displays that are highly likely to enter the driver's field of vision.

[0055] In addition, the aforementioned sensor may include an onboard camera, and the aforementioned first data includes image data captured by the onboard camera.

[0056] In addition, the aforementioned sensor may include an onboard camera, and the aforementioned second data includes image data captured by the onboard camera.

[0057] Therefore, the vehicle-mounted system according to one aspect of this disclosure can infer the direction of the driver's gaze by analyzing the driver's image. Consequently, the vehicle-mounted system according to one aspect of this disclosure can infer the direction of the driver's gaze with good accuracy.

[0058] Hereinafter, embodiments of the present disclosure will be described based on the accompanying drawings. The following embodiments are illustrative examples, and the present disclosure is not limited to the configuration of the embodiments.

[0059] (First Implementation)

[0060] An overview of the vehicle system according to the first embodiment will be described. The vehicle system according to this embodiment is configured to include a vehicle 1.

[0061] Reference Figure 1 The following describes vehicle 1.

[0062] Figure 1 This diagram shows the interior of vehicle 1. As shown, a driver's seat 2 and a passenger seat 3 are provided inside the passenger compartment of vehicle 1. In this embodiment, vehicle 1 is a right-hand drive vehicle, and the driver's seat 2 is located on the right side of the vehicle.

[0063] A steering wheel 4 is located on the front side of the driver's seat 2. The steering wheel can be either round or U-shaped as shown in the figure (a so-called irregular steering wheel).

[0064] The steering wheel 4 is connected to the steering control device of the vehicle 1 via the steering column. The steering column is covered by a steering column cover.

[0065] The passenger seat 3 is located to the left of the driver's seat 2, and a central display screen 24 is located at the front of the vehicle, between the driver's seat 2 and the passenger seat 3. The central display screen 24 displays auxiliary information such as car navigation, or entertainment-related content such as audio and television.

[0066] An instrument display 21 is provided on the side in front of the steering wheel 4. Located in front of the driver's seat 2, the instrument display 21 is a full-screen display that provides various information to the driver of the vehicle 1. The instrument display 21 displays, for example, information necessary for vehicle operation such as vehicle speed, engine speed (if the vehicle 1 has an engine), charging / discharging status (if the vehicle 1 has a drive battery), and distance traveled. The instrument display 21 is an example of the "instrument section" in this disclosure.

[0067] A pair of display devices 27 are provided in front of the driver's seat 2 inside the vehicle compartment. The pair of display devices 27 includes touch panel displays respectively located on both sides of the steering wheel 4. In this embodiment, the display located closer to the inside of the vehicle is called the inner display, and the display located closer to the outside of the vehicle is called the outer display. The inner display 22 and the outer display 23 are connected to the instrument panel of the vehicle 1.

[0068] Reference Figure 2 The detailed structure of the internal display 22 and the external display 23 will be described.

[0069] The inner display 22 and the outer display 23 are elongated displays, connected to the support portion 5, which serves as a component supporting each display. The support portion 5 has a shape that protrudes to the left and right of the steering wheel. The outer display 23 is connected to the right end of the support portion 5 facing forward of the vehicle, and the inner display 22 is connected to the left end facing forward of the vehicle. Thus, the display surface can be arranged near the grip portion of the steering wheel 4.

[0070] The above description is an example of the case where vehicle 1 is a right-hand drive vehicle. In the case where vehicle 1 is a left-hand drive vehicle, the left and right sides are reversed. That is, the external display 23 is connected to the left end of the support 5 facing the front of the vehicle, and the internal display 22 is connected to the right end facing the front of the vehicle.

[0071] The support portion 5 can also serve as a pillar cover for the steering column. The support portion 5 has a shape that extends from the portion through the steering column in the left and right directions (vehicle width direction) and extends towards the rear of the vehicle, and bends at a smaller angle relative to the left and right directions of the steering wheel midway.

[0072] Furthermore, the inner display 22 and the outer display 23 are angled relative to the left and right directions of the steering wheel, and are respectively configured to tilt towards the steering wheel side. That is, the inner display 22 is configured to protrude further rearward of the vehicle the closer it is to the inside of the vehicle, and the outer display 23 is configured to protrude further rearward of the vehicle the closer it is to the outside of the vehicle. In this way, the inner display 22 and the outer display 23 are respectively configured to face the driver.

[0073] Figure 2 The area 4R, indicated by the dashed line, is the part where the right hand holds the steering wheel 4 (the grip area). Figure 2 The area 4L, indicated by the dashed line, is the part where the left hand holds the steering wheel 4 (the grip area). By making the support part 5 into the shape described above, the positions of the inner display 22 and the outer display 23 in the vehicle's longitudinal direction can be made closer to the grip area of ​​the steering wheel 4. Therefore, the amount of hand movement required when the driver touches the inner display 22 or the outer display 23 while driving can be further reduced.

[0074] [System Composition]

[0075] Next, the configuration of each device in the vehicle system of vehicle 1 will be explained.

[0076] First, the constituent elements of vehicle 1 will be explained. Figure 3 This is a diagram schematically illustrating an example of the configuration of vehicle 1. Vehicle 1 is configured to include a vehicle system, an electrical system, and an input / output system.

[0077] The vehicle system is the system that controls the movement of vehicle 1. The vehicle system contains multiple components related to the movement of the vehicle.

[0078] An input / output system is a system that provides information to the occupants of vehicle 1 and receives input from them. An input / output system comprises multiple components for inputting and outputting information.

[0079] The electrical system is a system that controls the electrical components present in the vehicle's body. It includes multiple components related to the electrical components present in the vehicle's body.

[0080] Each system may contain multiple electrical components and an ECU that controls those electrical components.

[0081] The vehicle ECU10 and body ECU30 can be configured as a computer with a processor (CPU, GPU, etc.), main storage (RAM, ROM, etc.), and auxiliary storage (EPROM, hard disk drive, removable media, etc.). The auxiliary storage stores the operating system (OS), various programs, various tables, etc. By executing the programs stored therein, various functions (software modules) that meet the specified purpose can be achieved, as described later. However, some or all of the functions can also be implemented as hardware modules using hardware circuits such as ASICs and FPGAs.

[0082] The vehicle ECU10 is connected to several components related to the vehicle's operation.

[0083] GPS unit 11 is a unit used to obtain the location information of vehicle 1. GPS unit 11 includes a GPS antenna and a positioning module for locating the location information. The GPS antenna is an antenna that receives positioning signals transmitted from positioning satellites (also known as GNSS satellites). The positioning module is a module that calculates the location information based on the signals received by the GPS antenna.

[0084] The ETC (Electronic Toll Collection System) unit 12 is a unit used to collect tolls for toll roads via wireless communication. The ETC unit 12 is configured to include an antenna and a communication module.

[0085] The wireless communication unit 13 includes an antenna and a communication module for performing wireless communication. The antenna is an antenna element that inputs and outputs wireless signals. In this embodiment, the antenna is suitable for mobile communication (e.g., 3G, 4G, 5G, etc.). The antenna may be configured to include multiple physical antennas. The communication module is a module for performing mobile communication.

[0086] Alternatively, the wireless communication unit 13 may include a communication module other than one for communication with mobile devices. For example, the wireless communication unit 13 may include a Bluetooth-based communication module. This enables wireless connection with a user terminal located inside the vehicle, allowing for the provision of services in cooperation with the user terminal (e.g., hands-free calling).

[0087] Sensor group 14 is a collection of multiple sensors used to acquire sensor data during vehicle operation. These sensors can be sensors that acquire physical quantities or sensors that acquire image data, etc. In this embodiment, sensor group 14 is exemplified by a sensor that detects vehicle speed (vehicle speed sensor 141), an image sensor that acquires an image of the front of the vehicle (vehicle-mounted camera 142), and an image sensor that acquires an image of the interior of the vehicle (interior camera 143).

[0088] In addition, components belonging to the drive system (such as engine, motor, battery, transmission, etc.) can be further connected to the vehicle ECU10.

[0089] The vehicle body ECU30 is connected to multiple electrical components of the vehicle 1.

[0090] In this embodiment, an automotive air conditioner, windshield wipers, headlights, side mirrors, and power seats are examples of electrical devices included in vehicle 1.

[0091] The air conditioning unit 31 is the automotive air conditioning system of vehicle 1. The air conditioning unit 31 includes, for example, a cooling unit, a heating unit, a compressor, a condenser, etc. In addition, the air conditioning unit 31 may also include a control unit for controlling them.

[0092] The wiper unit 32 includes a front wiper, a rear wiper, and a motor for driving them.

[0093] The headlight unit 33 includes the headlights of the vehicle 1 and a unit for driving the headlights. The headlights are configured to switch between high beams and low beams, for example. When the headlights of the vehicle 1 are adaptive headlights, the illumination can also be controlled based on commands from the vehicle's ECU. The headlight unit 33 may further include taillights, brake lights, turn signals, and / or units for driving them.

[0094] The rearview mirror actuator 34 includes left and right side rearview mirrors disposed on the vehicle 1 and a drive unit for driving the rearview mirrors. The rearview mirror actuator 34 can adjust the angle of the side rearview mirrors up, down, left, and right.

[0095] The seat actuator 35 includes a drive unit for adjusting the position of the seat installed in the vehicle 1. The seat actuator 35 allows adjustment of the position, height, angle, etc. of the seat surface and backrest.

[0096] Next, the device used for inputting and outputting information will be described.

[0097] In this embodiment, vehicle 1 has four displays, namely instrument display 21, internal display 22, external display 23 and central display 24, which are used for outputting information.

[0098] The instrument display 21 is a display positioned in front of the driver, outputting information related to the vehicle 1. The instrument display 21 primarily outputs vehicle status-related information such as vehicle speed, engine speed, fuel level, battery level, charging / discharging status, and distance traveled, but it can also output other information.

[0099] The inner display 22 and the outer display 23 are touch-screen displays located on either side of the steering wheel. As described above, the inner display 22 is located on the center side of the vehicle, and the outer display 23 is located on the outer side of the vehicle.

[0100] When vehicle 1 is a right-hand drive vehicle, the internal display 22 is positioned on the left side of the vehicle's direction of travel, and the external display 23 is positioned on the right side of the vehicle's direction of travel. When vehicle 1 is a left-hand drive vehicle, the internal display 22 is positioned on the right side of the vehicle's direction of travel, and the external display 23 is positioned on the left side of the vehicle's direction of travel.

[0101] The central display 24 is a display located in the center of the vehicle. The central display 24 is positioned where a display previously equipped with a vehicle navigation system was installed. The central display 24 displays, for example, auxiliary information such as car navigation, or entertainment-related content such as audio and television.

[0102] The touch panel 25 is a unit for detecting touch operations performed on the aforementioned displays. In this embodiment, the inner display 22, the outer display 23, and the central display 24 each have a touch panel 25, enabling touch-based operations. The touch panel 25 detects touch operations performed on these displays and sends the detected content to the vehicle ECU 10.

[0103] The sound unit 26 is a unit that inputs and outputs information based on sound. The sound unit 26 may include, for example, a speaker and a microphone.

[0104] Next, the detailed composition of the vehicle ECU10 will be explained.

[0105] The vehicle ECU 10 is connected to the vehicle system and the drive system (powertrain system), thereby enabling control of the vehicle 1. In addition, the vehicle ECU 10 has the function of inputting and outputting information through components included in the input / output system and the function of controlling the electrical system via the body ECU 30 that controls the electrical system.

[0106] Figure 4 This is a diagram illustrating the detailed structure of the vehicle's ECU10.

[0107] The vehicle ECU 10 is configured to have a control device 101, a storage device 102 and a communication module 103.

[0108] The control device 101 is a computing unit that executes a prescribed program to implement various functions of the vehicle ECU 10. The control device 101 can be implemented by a hardware processor such as a CPU. Furthermore, the control device 101 can be configured to include RAM, ROM (Read Only Memory), cache memory, etc.

[0109] In this embodiment, the control unit 101 of the vehicle ECU 10 is configured with six software modules: an HMI control unit 1011, a vehicle control unit 1012, an ADAS control unit 1013, an entertainment control unit 1014, a navigation unit 1015, and an auxiliary unit 1016. Each software module can be implemented by the control unit 101 (CPU, etc.) executing a program stored in the storage device 102 (described later). Information processing performed by the software modules is synonymous with information processing performed by the control unit 101 (CPU, etc.).

[0110] The HMI control unit 1011 generates a user interface for controlling various components of the vehicle and provides it to the vehicle occupants via the input / output system. Additionally, the HMI control unit 1011 passes information input via the input / output system (e.g., instructions for implementing user-specified controls) to the corresponding software modules.

[0111] Therefore, any component of the vehicle can be controlled via a user interface.

[0112] In the following description, user-generated instructions for controlling the components of vehicle 1 will be referred to as "user instructions". For example, instructions to change the air conditioning temperature, change the driving mode, make a phone call, or play music are examples of "user instructions".

[0113] The components of a vehicle can be electrical devices such as headlights and windshield wipers. Additionally, the components can be an onboard computer or software modules executed by that computer (e.g., autonomous driving software, navigation software, etc.).

[0114] The HMI control unit 1011 selects the user interface output destination from multiple displays (instrument display 21, internal display 22, external display 23, and central display 24) according to the type of component being controlled. The specific configuration will be described later.

[0115] The vehicle control unit 1012 controls the components of the vehicle system or the drive system.

[0116] The vehicle control unit 1012 can control the target component (e.g., the drive motor) based on instructions (e.g., acceleration instructions) received from other components (e.g., the ADAS control unit 1013 described later), or based on user instructions (e.g., headlight illumination instructions) obtained via the HMI control unit 1011.

[0117] The ADAS control unit 1013 controls the Advanced Driver-Assistance Systems. For example, the ADAS control unit 1013 generates vehicle control commands based on sensor information obtained from the onboard sensors included in the sensor group 14, enabling driver assistance functions such as cruise control, pre-collision safety, and lane keeping assist. The ADAS control unit 1013 controls the vehicle system and drive system, for example, by sending control commands to the vehicle control unit 1012.

[0118] The entertainment control unit 1014 performs controls related to the entertainment provided to the occupants of the vehicle. Examples of entertainment-related functions include audio functions and television / radio functions.

[0119] The navigation unit 1015 provides car navigation functionality. The navigation unit 1015 can generate and provide route guidance for the vehicle based on location information obtained from the GPS unit 11 and traffic information obtained from the wireless communication unit 13. For example, the navigation unit 1015 can also receive the vehicle's destination and route designation via the HMI control unit 1011 and output information related to the generated route.

[0120] The auxiliary unit 1016 provides information to vehicle occupants by executing a language model capable of dialogue based on natural language. The language model can be, for example, an LLM (Large Language Model). For instance, by executing an LLM containing information related to the vehicle occupants, information can be provided based on natural language. Where there are multiple occupants, the auxiliary unit 1016 can obtain the language model corresponding to each person via a network and execute any given language model.

[0121] Storage device 102 is a mechanism for storing information, and is composed of storage media such as RAM, disk, and flash memory. Storage device 102 stores programs executed by control device 101, data used by those programs, etc.

[0122] The communication module 103 is a communication interface for connecting the vehicle ECU 10 to the vehicle network. The communication module 103 is configured, for example, to communicate with components of the vehicle 1 via a network such as CAN (Controller Area Network).

[0123] The specific configuration of the vehicle ECU 10 can be appropriately modified by omitting, substituting, or adding components depending on the implementation method. For example, the control device 101 may include multiple hardware processors. These hardware processors may be microprocessors, FPGAs, GPUs, etc. Additionally, input / output devices other than those shown (e.g., optical drivers) may be added. Furthermore, the vehicle ECU 10 may be composed of multiple computers. In this case, the hardware configuration of each computer may be identical or different.

[0124] [Interface screen provided by HMI control unit]

[0125] Next, the user interface provided by the HMI control unit 1011 will be described.

[0126] The HMI control unit 1011 can generate various types of graphical user interface (GUI) screens and output them via the aforementioned multiple displays.

[0127] In this embodiment, the HMI control unit 1011 centrally outputs vehicle control-related information to the external display 23 and centrally outputs information related to occupant convenience and comfort to the internal display 22. Thus, the occupants of the vehicle 1 can intuitively understand which side (left or right) the desired information is output to. Furthermore, the touch panel operation surfaces of both the internal display 22 and the external display 23 are positioned near the steering wheel. This minimizes the driver's line of sight and hand movement.

[0128] The HMI control unit 1011 can switch the screen output to the external display 23 to multiple types of screens.

[0129] Figure 5 This diagram illustrates the transfer of multiple types of screens output to the external display 23. As shown, the HMI control unit 1011 can output multiple screens centered on the main screen 501 to the external display 23.

[0130] (1) Main screen 501

[0131] The main screen 501 is the screen that is usually displayed on the external display while driving. Figure 6 (A) is a diagram representing an example of the main screen 501. The content displayed on the main screen can be customized by the user.

[0132] (2) Headlight control screen 502

[0133] This is a screen showing the illuminated state of the headlights on vehicle 1. Figure 7 (A) is a diagram representing an example of a headlight control screen 502.

[0134] In the headlight control screen 502, the illumination status of the headlights can be selected (automatic, on, off, parking lights, daytime running lights, etc.). The vehicle control unit 1012 generates user commands based on the touch operation obtained via the HMI control unit 1011, and controls the headlight unit 33 by sending the user commands to the vehicle ECU 30.

[0135] Additionally, the vehicle control unit 1012 can also obtain data indicating the headlight illumination status from the vehicle ECU 30 and display the illumination status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 can display the current headlight illumination status in the vehicle graphic shown on the screen.

[0136] (3) Wiper control screen 503

[0137] This is a screen that sets the operating status of the windshield wipers (front and rear) of vehicle 1. Figure 7 (B) is a diagram representing an example of a windshield wiper control screen 503.

[0138] In the wiper control screen 503, the operating state of the wipers can be selected (stop, automatic operation, intermittent operation, continuous operation, etc.). The vehicle control unit 1012 generates user commands based on the touch operation obtained via the HMI control unit 1011, and controls the wiper unit 32 by sending the user commands to the vehicle ECU 30.

[0139] Additionally, the vehicle control unit 1012 can also obtain data indicating the operating status of the windshield wipers from the vehicle ECU 30 and display the operating status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 can display the current operating status of the windshield wipers in the vehicle graphic shown on the screen.

[0140] (4) Rearview mirror control screen 504

[0141] It is a screen showing the unfolding state and angle of the side rearview mirrors on vehicle 1. Figure 7 (C) is a diagram representing an example of a rearview mirror control screen 504.

[0142] In the rearview mirror control screen 504, the state of the side rearview mirror (folded / unfolded state and angle) can be selected. The vehicle control unit 1012 generates user commands based on the touch operation obtained via the HMI control unit 1011, and controls the rearview mirror actuator 34 by sending the user commands to the vehicle ECU 30.

[0143] In addition, the vehicle control unit 1012 can also obtain data indicating the status of the rearview mirrors from the vehicle ECU 30 and reflect the operation status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 can make the graphic of the vehicle shown on the screen reflect the current unfolded / folded state of the side rearview mirrors.

[0144] (5) Driving posture adjustment screen 505

[0145] This is a screen showing the adjustment of the position / angle of the seats (seat surface and backrest) and the position / angle of the steering wheel in vehicle 1.

[0146] In the driving posture adjustment screen 505, the position and angle of the seat and steering wheel can be set. The vehicle control unit 1012 obtains the touch operation obtained by the HMI control unit 1011 via the HMI control unit 1011, generates a user command based on the touch operation, and controls the seat actuator 35 and the like by sending the user command to the vehicle ECU 30.

[0147] (6) ADAS setting screen 506

[0148] This screen is used to notify the vehicle 1 of the operation status of its advanced driver assistance system and to configure it.

[0149] For example, if the advanced driver assistance system of vehicle 1 includes an automatic cruise function, the user of vehicle 1 can set the cruise speed, inter-vehicle distance, whether or not to follow the vehicle in front, and whether or not to follow the lane via the ADAS setting screen 506.

[0150] The ADAS control unit 1013 can generate the information contained in the ADAS setting screen 506.

[0151] (7) Driving mode setting screen 507

[0152] This is the screen for setting the driving mode for vehicle 1. Figure 7 (D) is a diagram representing an example of the driving mode setting screen 507.

[0153] For example, the user of vehicle 1 can select a mode from multiple modes such as standard mode, sport mode, eco mode, and snow mode via the driving mode setting screen 507. The vehicle control unit 1012 changes the vehicle's driving mode based on this selection.

[0154] (8) Vehicle function setting screen 508

[0155] This is a screen used to utilize other functions of vehicle 1. Examples of other functions of vehicle 1 include the opening and closing of the power tailgate and suspension adjustment.

[0156] (9) Display mode setting screen 509

[0157] This is a screen for setting the types of information to be displayed on the instrument display 21. For example, if the instrument display 21 can display multiple pieces of information other than speed, the user of the vehicle 1 can customize the information displayed on the instrument display 21 via the display mode setting screen 509.

[0158] For example, multiple screens, as illustrated above, can be switched by performing a prescribed operation, such as swiping left or right, on the touch panel provided on the external display 23.

[0159] Furthermore, the HMI control unit 1011 can enable the external display 23 to display a shortcut screen 510 for jumping to each screen. Figure 6Figure (B) is an example of a shortcut screen 510. For example, the shortcut screen can be brought up by performing a prescribed operation (e.g., swiping from bottom to top) on each screen containing the main screen 501. The shortcut screen 510 is provided with multiple interfaces (e.g., buttons) for moving to each screen, and by operating these interfaces, one can move to the target screen.

[0160] In this way, the HMI control unit 1011 will output at least the information related to vehicle control to the external display 23.

[0161] Information related to vehicle control typically refers to information related to vehicle driving control. Examples of driving control related information include information used to control components belonging to the vehicle system, information used to control components belonging to the drive system, etc. Additionally, information about components in the electrical system, such as lights and windshield wipers, necessary for proper vehicle driving, can also be considered driving control related information.

[0162] Furthermore, any information related to vehicle control can be output to the external display 23, and information not directly related to vehicle driving control can also be output to the external display 23. For example, information used to control the vehicle's electric doors can be output to the external display 23.

[0163] Similarly, the HMI control unit 1011 can switch the image output to the internal display 22 to multiple types of images.

[0164] Figure 8 This diagram illustrates the transfer of multiple types of screens output to the internal display 22. As shown, the HMI control unit 1011 can output multiple screens centered on the main screen 801 to the internal display 22.

[0165] (1) Main screen 801

[0166] The main screen 801 is the screen that is usually displayed on the internal display while driving. Figure 9 (A) is a diagram representing an example of the main screen 801. The content displayed on the main screen can be customized by the user.

[0167] (2) Navigation screen 802

[0168] It's a screen related to car navigation. Figure 10 Figure (A) is an example of a navigation screen 802. The information contained in the navigation screen 802 is generated by the navigation unit 1015.

[0169] When the map information in the car navigation is displayed on the central display screen 24, auxiliary information related to the route of vehicle 1 can also be output on the navigation screen 802. This auxiliary information may include, for example, guidance on locations for upcoming left or right turns, estimated arrival time, and remaining distance.

[0170] Alternatively, the navigation screen 802 may include information displayed on the central display 24, or a portion of the user interface.

[0171] For example, frequently used user interfaces such as buttons for starting route guidance to home, buttons for bringing up bookmarks, and buttons for starting route guidance to charging points can also be configured on the navigation screen 802.

[0172] (3) Audio and video 803

[0173] The image is related to the audio function of vehicle 1. Figure 10 Figure (B) is a diagram representing an example of audio display 803. The information contained in the audio display 803 is generated by the entertainment control unit 1014. The entertainment control unit 1014 can obtain and play music via a network or media. The audio display 803 can contain information related to the music being played and an interface for selecting music.

[0174] (4) Air Conditioner Setting Screen 804

[0175] This is a screen used to control the car's air conditioning system in vehicle 1. Figure 10 (C) is a diagram representing an example of an air conditioner setting screen 804.

[0176] In the air conditioning setting screen 804, the temperature, airflow, and operating air conditioning devices (steering wheel heater, seat heater, defroster, etc.) of the car's air conditioning can be set. The vehicle control unit 1012 generates user commands based on the operations performed on the screen and controls the air conditioning unit 31, etc., by sending the user commands to the vehicle ECU 30.

[0177] Although the car air conditioner is an equipment of vehicle 1 controlled by external commands, the air conditioner setting screen 804 is not displayed on the external display 23 but on the internal display 22 because it is related to the comfort of the occupants.

[0178] (5) Telephone screen 805

[0179] This is a screen used for hands-free calling via wireless connection with a user terminal. When the wireless communication unit 13 has a module for wireless communication with a user terminal, calls (or answers) from the user terminal, such as telephone calls, can be made via this module.

[0180] (6) AI Partner Screen 806

[0181] It is a screen used to converse with a virtual agent (AI partner) that can communicate based on natural language. Figure 10 (D) is a diagram representing an example of an AI partner screen 806. This screen contains a virtual agent role that vehicle occupants can interact with to obtain information. Natural language-based dialogue services are provided by the auxiliary unit 1016.

[0182] The AI ​​partner screen 806 may also include an interface for instructing agents. In cases where there are multiple occupants in the vehicle, the auxiliary unit 1016 can be configured to execute any agent corresponding to each person. The AI ​​partner screen 806 may include interfaces for starting and ending dialogue with agents, interfaces for switching agents, etc.

[0183] For example, multiple screens, as illustrated above, can be switched by performing a prescribed operation, such as swiping left or right, on the touch panel provided on the internal display 22.

[0184] Furthermore, the HMI control unit 1011 can enable the internal display 22 to display a shortcut screen 810 for jumping to each screen. Figure 9 Figure (B) is an example of a shortcut screen 810. For example, the shortcut screen can be brought up by performing a specified operation (e.g., swiping from bottom to top) on each screen containing the main screen 801. The shortcut screen 810 is provided with multiple interfaces (e.g., buttons) for moving to each screen, and by operating these interfaces, one can move to the target screen.

[0185] In this way, the HMI control unit 1011 centrally outputs information related to the convenience and comfort of the occupants to the internal display 22.

[0186] While components such as car air conditioning are part of the vehicle, information about components related to the convenience or comfort of occupants may be output to the internal display 22 without being processed as information related to vehicle control.

[0187] Thus, when the information of the object conforms to both information related to vehicle control and information related to occupant convenience (or comfort), the information can be selectively output to either the internal display 22 or the external display 23 according to a prescribed reference.

[0188] [Interruption display on the interface screen]

[0189] Next, we will explain the interruption display.

[0190] Sometimes, notifications to vehicle occupants are generated from various components while the vehicle is in motion. For example, when the vehicle is connected to a user terminal, it is necessary to notify the occupants of incoming telephone calls as needed. Additionally, when an event occurs during the use of ADAS functions (e.g., a request to switch from autonomous driving to manual driving), it is necessary to notify the occupants of that event. When the HMI control unit 1011 generates a request (called a notification request) for such a notification from a component of the vehicle 1, it interrupts the display via the internal display 22 or the external display 23.

[0191] Figure 11 This is a flowchart of the process executed by the HMI control unit 1011. This process is executed periodically during the period when the vehicle's main switch is turned on.

[0192] First, in step S10, the HMI control unit 1011 acquires data (here referred to as first data) obtained by any one of the sensors in the sensor group 14 of the vehicle system. For example, in step S10, the HMI control unit 1011 may acquire image data obtained by the in-cabin camera 143. Next, the process proceeds to step S11. The in-cabin camera 143 is a camera capable of capturing images inside the vehicle, including an image of the driver's face. The in-cabin camera 143 can be a visible light camera or an infrared camera. The first data can be a visible light image or an infrared image.

[0193] In step S11, the HMI control unit 1011 infers the direction in which the driver of vehicle 1 is looking, i.e., the first direction, based on the first data obtained from the sensor. For example, the HMI control unit 1011 can analyze the image data obtained from the in-cabin camera 143 to determine the orientation of the driver's face and infer the first direction. Then, the process moves to step S12.

[0194] In step S12, the HMI control unit 1011 determines whether a notification to the driver has been generated from the vehicle ECU 10 or the body ECU 30 installed in the vehicle 1.

[0195] If the HMI control unit 1011 determines that the notification has been generated, the process proceeds to step S13. If the HMI control unit 1011 determines that the notification has not been generated, the process ends.

[0196] In step S13, the HMI control unit 1011 outputs a notification from the HMI control unit 1011 to the driver of the vehicle 1 to a display device selected based on the first direction (hereinafter referred to as the first display). Here, the display device selected based on the first direction may be the display device closest to the virtual axis representing the first direction.

[0197] In step S14, after a predetermined time has elapsed since the specified notification was output to the first display, the direction of the driver's gaze is inferred based on newly acquired data (here referred to as second data) from any of the sensors included in the sensor group 14. For example, the HMI control unit 1011 can analyze the image data acquired by the in-cabin camera 143 to determine the orientation of the driver's face and infer the direction the driver is looking. Here, the predetermined time refers to a time sufficient to determine whether the driver has visually acknowledged the notification output to the first display.

[0198] In step S15, the HMI control unit 1011 determines whether the driver is looking at the first display based on the second data. If the HMI control unit 1011 determines that the driver is not looking at the first display, the process proceeds to step S16. If the HMI control unit 1011 determines that the driver is looking at the first display, the process ends.

[0199] In step S16, the HMI control unit 1011 outputs a prescribed notification from the HMI control unit 1011 to a display device (hereinafter referred to as the second display) selected based on the second direction in which the driver of the vehicle is looking. Here, the display device selected based on the second direction may be the display device closest to a virtual axis representing the second direction. For example, if the HMI control unit 1011 determines that the driver has not observed the first display that output the notification, the HMI control unit 1011 outputs the notification again to the second display that is located in the direction in which the driver is looking.

[0200] The HMI control unit 1011 can eliminate the output notification after a predetermined time has elapsed after outputting a notification to the first display and the second display. Alternatively, the HMI control unit 1011 can also eliminate the notification output to the display device clicked by the user after outputting a notification to the first display or the second display, provided the user clicks on either display. Or, the HMI control unit 1011 can eliminate the notification output to the first display or the second display when the task related to the notification has ended. For example, if the task related to the notification is an incoming telephone call, the notification output to the first display or the second display can be eliminated when the telephone call ends.

[0201] As explained above, in this embodiment, the HMI control unit 1011 of the vehicle ECU infers the direction of the driver's gaze based on data obtained from sensors, and outputs a notification to a display located in the direction of the driver's gaze when a notification is generated from the vehicle ECU 10 or the body ECU 30.

[0202] According to this configuration, one aspect of the vehicle system disclosed herein is capable of outputting notifications to locations easily accessible to the driver.

[0203] (Second Implementation)

[0204] In the first embodiment described above, the HMI control unit 1011 infers the direction of the driver's gaze, i.e., the first direction, based on the first data obtained from the sensor. In the first embodiment, for example, it is envisioned that the in-cabin camera 143 is used as a sensor to analyze the image data obtained from the vehicle-mounted camera, i.e., the first data, to infer the first direction.

[0205] In contrast, in the second embodiment, when the HMI control unit 1011 detects that the driver is operating a certain display, it does not re-detect the direction of the driver's gaze, but only outputs a notification to the display.

[0206] Figure 12 This is a flowchart of a process that outputs a notification to the display device being operated based on data from the display device used to determine the driver's current operation. This process is executed periodically during the period when the vehicle's main power switch is on.

[0207] In step S20, the HMI control unit 1011 acquires data detected by various sensors present in the vehicle 1. For example, the HMI control unit 1011 can acquire data detected by touch sensors on the displays of multiple touch panels 25. Here, the touch panel is a touch panel included in multiple display devices mounted in the vehicle 1. The sensor data acquired by the HMI control unit 1011 is not limited to data acquired from the touch panels 25. The HMI control unit 1011 can also acquire data indicating that the driver or others are operating the display device from other types of display devices.

[0208] In step S21, the HMI control unit 1011 determines whether the data obtained by the sensor indicates that the driver is operating on any of the display devices.

[0209] For example, the sensor in step S21 is a touch sensor on the display of the display device (e.g., touch panel 25) that is being detected. If the HMI control unit 1011 determines that the data obtained by the sensor indicates that the driver is operating any display device, the process proceeds to step S22. In this embodiment, the display device being operated by the driver is referred to as the third display. If the HMI control unit 1011 determines that the data obtained by the sensor indicates that the driver is not operating any display device, the process proceeds to step S23.

[0210] In step S22, the HMI control unit 1011 determines the display device (third display) being operated by the driver and outputs a notification from the HMI control unit 1011 to the third display.

[0211] That is, when the HMI control unit 1011 detects that the driver is operating any display device, it outputs a notification to the display being operated without inferring the first direction.

[0212] In step S23, the HMI control unit 1011 outputs a notification for the driver generated from the vehicle ECU 10 or the body ECU 30 to a display determined based on the driver's line of sight.

[0213] Specifically, when the process moves to step S23, the HMI control unit 1011 performs the processing described in steps S11 to S16 in the first embodiment.

[0214] With the above configuration, the vehicle system according to one aspect of this disclosure can reliably output a notification to the display device located in the direction the driver is looking at by detecting that the driver is operating the display device.

[0215] (Variation example)

[0216] In one embodiment, the HMI control unit 1011 outputs a notification from the HMI control unit 1011 to the driver of the vehicle 1 to the display device closest to the virtual axis representing the first direction.

[0217] However, the HMI control unit 1011 may not only output notifications to the display device closest to the virtual axis representing the first direction. The HMI control unit 1011 may designate one or more display devices that exist within a predetermined distance from the virtual axis representing the first direction as the first display and output notifications to the first display.

[0218] Furthermore, for example, when multiple display devices exist within a region that is within a predetermined distance from the virtual axis representing the first direction, the HMI control unit 1011 can determine the display device with the highest pre-assigned priority among the multiple display devices as the first display and output a notification to the first display. Here, "within a predetermined distance from the virtual axis representing the first direction" can mean a range within a predetermined angle with respect to a point on the virtual axis representing the first direction. Alternatively, "within a predetermined distance from the virtual axis representing the first direction" can mean a range within a predetermined distance from the virtual axis representing the first direction.

[0219] In addition, the HMI control unit 1011 can also determine the display device closest to the virtual axis representing the first direction and the display device with the highest pre-assigned priority as the first display, and output a prescribed notification to the first display.

[0220] Alternatively, the HMI control unit 1011 may designate all of the multiple display devices existing in a region within a specified distance from the virtual axis representing the first direction as the first display and output a notification to the first display.

[0221] Alternatively, the HMI control unit 1011 may designate any one of a plurality of display devices existing in a region within a specified range from the virtual axis representing the first direction as the first display device, and output a notification to the first display device, and output a display indicating "outputting a notification to the first display device" to the other display devices.

[0222] When there is no display device in the area within a specified distance from the virtual axis representing the first direction, the HMI control unit 1011 can determine the display device (first display) that outputs the notification based on the type of notification and the source of the notification.

[0223] In addition, in the embodiment, after outputting a notification from the HMI control unit 1011 to the driver of the vehicle 1 to the first display, the HMI control unit 1011 determines the second direction the driver is looking at again, and outputs the notification from the HMI control unit 1011 to the driver of the vehicle 1 to the display device closest to the virtual axis representing the second direction.

[0224] However, the HMI control unit 1011 may not only output notifications to the display device closest to the virtual axis representing the second direction. The HMI control unit 1011 may designate one or more display devices that exist within a predetermined range from the virtual axis representing the second direction as the second display and output notifications to the second display.

[0225] Furthermore, for example, when multiple display devices exist within a region that is within a predetermined distance from the virtual axis representing the second direction, the HMI control unit 1011 can determine the display device with the highest pre-assigned priority among the multiple display devices as the second display and output a notification to the second display. Here, "within a predetermined distance from the virtual axis representing the second direction" can mean a range within a predetermined angle with respect to a point on the virtual axis representing the second direction, with that axis as the base axis. Alternatively, "within a predetermined distance from the virtual axis representing the second direction" can mean a range within a predetermined distance from the virtual axis representing the second direction.

[0226] In addition, the HMI control unit 1011 can also determine the display device closest to the virtual axis representing the second direction and the display device with the highest pre-assigned priority as the second display, and output a specified notification to the second display.

[0227] Alternatively, the HMI control unit 1011 may designate all of the multiple display devices existing in a region within a specified distance from the virtual axis representing the second direction as the second display and output a notification to the second display.

[0228] Alternatively, the HMI control unit 1011 may designate any one of a plurality of display devices existing in a region within a specified range from the virtual axis representing the second direction as the second display, and output a notification to the second display, and output a display indicating "outputting a notification to the first display device" to the other display devices.

[0229] When there is no display device in the area within a specified distance from the virtual axis representing the second direction, the HMI control unit 1011 can determine the display device (second display) that outputs the notification based on the type and source of the notification.

[0230] (Other variations)

[0231] The above-described implementation is merely an example, and this disclosure can be appropriately modified without departing from its spirit.

[0232] For example, the processing and methods described in this disclosure can be freely combined and implemented as long as they do not create technical contradictions.

[0233] Furthermore, although the instrument display 21, internal display 22 and external display 23 are shown to be independent in the embodiments, the displays do not necessarily need to be physically independent.

[0234] For example, it can also be like Figure 13As shown, the instrument display 21, internal display 22, and external display 23 are implemented by a single display 1301. Figure 13 In the example, the display 1301 has three display areas (display sections) indicated by reference numerals 1302, 1303, and 1304. The three display areas are all planar, and the structure between the display areas is curved.

[0235] In the illustrated example, display area 1302 functions as an internal display 22, and display area 1303 functions as an external display 23 (in the case of a right-hand drive vehicle). Additionally, display area 1304 functions as an instrument display 21.

[0236] The portion connecting display areas 1302 and 1304 (reference numeral 1305) and the portion connecting display areas 1303 and 1304 (reference numeral 1306) are capable of display, but they are deformed due to physical bending and are therefore not used as display parts in the usual way.

[0237] However, when the driver's line of sight is in the direction of display area 1305 or display area 1306, the HMI control unit 1011 may also output a notification to display area 1305 or display area 1306. Alternatively, when the driver's line of sight is in the direction of display area 1305 or display area 1306 and a notification is output to display area 1302 or display area 1303, the HMI control unit 1011 may also output information to display area 1305 or display area 1306 indicating that a notification is being displayed in another display area (e.g., display area 1302 or display area 1303).

[0238] In addition, in the implementation, three types of first data are exemplified: image data, infrared data, and touch sensor detection data. However, the first data may also be a combination of them, or may include other information.

[0239] In addition, in the embodiments, the first display is exemplified as a single display, but the first display may also be a plurality of displays existing within a predetermined range from the virtual axis representing the first direction.

[0240] Alternatively, processes described as being performed by a single device can be distributed among multiple devices. Or, processes described as being performed by different devices can be performed by a single device. In a computer system, the hardware configuration (server configuration) used to implement various functions can be flexibly changed.

[0241] This disclosure can also be implemented by supplying a computer with a computer program containing the functions described in the above embodiments, and having the program read and executed by one or more processors of the computer. Such a computer program can be provided to the computer either through a non-transitory computer-readable storage medium connectable to the computer's system bus or via a network. Non-transitory computer-readable storage media include, for example, any type of disk media such as disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of media suitable for storing electronic instructions.

Claims

1. A vehicle-mounted system, wherein, have: sensor; Multiple display devices; as well as The control unit performs the following processing: infers the direction the driver is looking at, i.e., the first direction, based on the data obtained from the aforementioned sensors, i.e., the first data; and, when a prescribed notification toward the aforementioned driver is generated, outputs the prescribed notification to the first display device selected from the plurality of display devices based on the aforementioned first direction, i.e., the first display.

2. The vehicle-mounted system according to claim 1, wherein, The aforementioned control unit outputs the aforementioned notification to the display device closest to the axis indicating the aforementioned first direction.

3. The vehicle-mounted system according to claim 1, wherein, The control unit determines the display device that exists in a region within a specified distance from the axis representing the first direction as the first display device, and outputs the specified notification to the first display device.

4. The vehicle-mounted system according to claim 3, wherein, When multiple display devices exist in an area within a specified range from the axis representing the first direction, the control unit will determine the display device with the highest pre-assigned priority as the first display and output the specified notification to the first display.

5. The vehicle-mounted system according to claim 1, wherein, When there is no display device in the area within a specified distance from the axis representing the first direction, the control unit determines the first display based on the type of notification or the source of the notification, and outputs the specified notification to the first display.

6. The vehicle-mounted system according to claim 1, wherein, After outputting the aforementioned notification to the first display, the control unit determines whether the driver is looking at the first display based on second data obtained from the sensor, which is different from the first data. If it is determined that the driver is not looking at the first display, it is inferred that the direction the driver is looking in, which is different from the first direction, is the second direction. The notification specified above is output to the display device selected based on the second direction, namely the second display.

7. The vehicle-mounted system according to claim 1, wherein, The aforementioned first data includes data used to determine which of the plurality of display devices is being operated by the driver. If the control unit determines, based on the first data, that the driver is operating the third display, it outputs the aforementioned notification to the third display.

8. The vehicle-mounted system according to claim 1, wherein, The aforementioned sensors include vehicle-mounted cameras. The aforementioned first data includes image data captured by the aforementioned vehicle-mounted camera.

9. The vehicle-mounted system according to claim 6, wherein, The aforementioned sensors include vehicle-mounted cameras. The second data mentioned above includes image data captured by the aforementioned vehicle-mounted camera.

10. A vehicle, wherein, The vehicle system is equipped with any one of claims 1 to 9.

11. An information processing apparatus that outputs information to a plurality of display devices disposed in front of the driver's seat in a vehicle compartment, wherein, The aforementioned information processing apparatus includes a control unit that performs the following processes: Based on the data obtained by the sensors, i.e., the first data, the direction that the driver of the vehicle equipped with the above-mentioned multiple display devices is looking at is inferred, i.e., the first direction. and When a prescribed notification is generated from the vehicle toward the driver, the prescribed notification is output to the first display device selected based on the first direction among the plurality of display devices.

12. The information processing apparatus according to claim 11, wherein, The aforementioned control unit outputs the aforementioned notification to the display device closest to the axis indicating the aforementioned first direction.

13. The information processing apparatus according to claim 11, wherein, The control unit determines the display device that exists in a region within a specified distance from the axis representing the first direction as the first display device, and outputs the specified notification to the first display device.

14. The information processing apparatus according to claim 13, wherein, When multiple display devices exist in an area within a specified range from the axis representing the first direction, the control unit will determine the display device with the highest pre-assigned priority as the first display and output the specified notification to the first display.

15. The information processing apparatus according to claim 11, wherein, When there is no display device in the area within a specified distance from the axis representing the first direction, the control unit determines the first display based on the type of notification or the source of the notification, and outputs the specified notification to the first display.

16. The information processing apparatus according to claim 11, wherein, After outputting the aforementioned notification to the first display, the control unit determines whether the driver is looking at the first display based on second data obtained from the sensor, which is different from the first data. If it is determined that the driver is not looking at the first display, it is inferred that the direction the driver is looking in, which is different from the first direction, is the second direction. The notification specified above is output to the display device selected based on the second direction, namely the second display.

17. The information processing apparatus according to claim 11, wherein, The aforementioned first data includes data used to determine which of the plurality of display devices is being operated by the driver. If the control unit determines, based on the first data, that the driver is operating the third display, it outputs the aforementioned notification to the third display.

18. The information processing apparatus according to claim 11, wherein, The aforementioned sensors include vehicle-mounted cameras. The aforementioned first data includes image data captured by the aforementioned vehicle-mounted camera.

19. An information processing method, performed by an in-vehicle system having an information processing device for outputting information via multiple display devices, wherein, The above information processing methods include: The step of inferring the direction the driver of the vehicle equipped with the aforementioned in-vehicle system is looking at, i.e., the first direction, based on data obtained from sensors, i.e., the first data; and In the event that a prescribed notification is generated from the aforementioned vehicle system toward the aforementioned driver, the step of outputting the prescribed notification to the first display device selected based on the aforementioned first direction among the plurality of display devices is taken.

20. A non-transitory storage medium, wherein, The system contains a program for causing a computer to perform the information processing method of claim 19.

Images