In-vehicle systems, vehicles, and information processing devices

Abstract

To prevent improper operation of the in-car display. [Solution] An in-vehicle system comprising a display device and an information processing device that outputs information via the display device. The display device is located in front of the driver's seat in the passenger compartment, protruding further rearward than the instrument panel located in front of the steering wheel, and positioned on both sides of the steering wheel, and includes a pair of display units having touch panels. The information processing device has a control unit that acquires touch operations performed on each of the pair of display units, and, if it is determined that a touch operation has been performed on either of the pair of display units, at least partially disables the touch operation on the other display unit.

Description

Technical Field

[0006]

[0001] This disclosure relates to vehicle technology.

Background Art

[0002] There is a technology for providing information to reduce the burden on a driver who drives an automobile. In this regard, for example, Patent Document 1 discloses an in-vehicle system in which a display for information provision is arranged near a steering wheel.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] This disclosure aims to prevent inappropriate operations on an in-vehicle display.

Means for Solving the Problems

[0005] One aspect of this disclosure is an in-vehicle system including a display device and an information processing device that outputs information via the display device, wherein the display device is provided at a front part of a driver's seat in a vehicle interior, is arranged at positions on both sides of the steering wheel while protruding to a rear side of the vehicle more than an instrument part disposed in front of the steering wheel, and includes a pair of display parts having touch panels, and the information processing device has a control part that executes acquiring a touch operation performed on each of the pair of display parts and, when it is determined that a touch operation is being performed on either one of the pair of display parts, at least partially invalidating a touch operation on the other display part.

[0006] One aspect of this disclosure is, An information processing device that provides information via a display device including a pair of display units having touch panels, which are located in the front part of the driver's seat inside the vehicle, protruding further rearward than the instrument panel located in front of the steering wheel, and positioned on both sides of the steering wheel, and which have a control unit that performs the following actions: acquire touch operations performed on each of the pair of display units, and if it is determined that a touch operation has been performed on either of the pair of display units, at least partially disable the touch operation on the other display unit.

[0007] Other embodiments include a vehicle equipped with the above-mentioned in-vehicle system, an information processing method performed by the above-mentioned information processing device, a program for causing a computer to perform the information processing method, or a computer-readable storage medium that non-temporarily stores the program. [Effects of the Invention]

[0008] According to this disclosure, it is possible to prevent improper operation of the in-vehicle display. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing the interior of vehicle 1 according to this embodiment. [Figure 2] Enlarged view of the steering wheel 4 and its surroundings. [Figure 3] A diagram showing an example of the components of vehicle 1. [Figure 4] A diagram showing an example of the components of the vehicle ECU10. [Figure 5] A diagram showing a list of screens displayed on the outer display. [Figure 6] A diagram showing an example of a screen output to an outer display. [Figure 7] A diagram showing an example of a screen output to an outer display. [Figure 8]A diagram showing a list of screens displayed on the inner display. [Figure 9] A diagram showing an example of the screen output to the inner display. [Figure 10] A diagram showing an example of the screen output to the inner display. [Figure 11] A flowchart of the processes executed by the HMI control unit 1011. [Figure 12] A flowchart of the processes executed by the HMI control unit 1011. [Figure 13] A table linking vehicle status with details of the deactivation process. [Figure 14] A diagram showing an example of a display device related to a modified example. [Modes for carrying out the invention]

[0010] In recent years, with the increasing sophistication of automobiles, the displays used to provide information are becoming larger, or rather, the number of displays is increasing.

[0011] For example, one proposed method involves placing a large touchscreen display at the front of the vehicle, between the driver's and passenger's seats, to provide a single platform for vehicle operation and information. However, with this method, the driver's gaze shifts significantly when operating the vehicle's components or checking the outputted information. For the sake of vehicle driving safety, it is preferable to minimize the driver's gaze shift.

[0012] Generally, a display device such as an LCD is often placed in the area in front of the driver's seat where the instrument panel was traditionally located. By outputting information to this display device, the movement of the driver's eyes can be minimized. However, there is a limit to the size of the display device arranged in the front, and it is not always possible to display all the information necessary for the driver. Also, if these pieces of information are output simultaneously in the same area, for example, a situation will occur where information necessary for the vehicle's travel and auxiliary information such as entertainment are mixed, which will prevent intuitive information transmission.

[0013] There are also problems with the arrangement of the input interface for operations. For example, a steering wheel having cursor keys and push button switches and enabling input operations while driving is known. However, with such an input interface, since the layout of buttons and the like is fixed, it is difficult to adapt to a plurality of different targets. For example, when raising or lowering the set temperature of the air conditioner, it can be done with two physical keys, but with physical keys, it is not possible to directly specify an arbitrary point on the map. A form of operation by a touch panel is also conceivable. However, as described above, the touch panel display arranged in the front of the vehicle interior is located at a position far from the steering wheel. Therefore, in addition to the movement of the line of sight, the driver's hand also moves significantly.

[0014] In order to solve these problems, an in-vehicle system has been devised in which a display device capable of input operations by touch is arranged near the steering wheel. For example, by arranging displays having touch panels on the left and right of the steering wheel respectively, intuitive and rapid operations by the driver become possible.

[0015] However, if a touch panel display is arranged near the steering wheel, there is a risk that a passenger may stretch their hand near the steering wheel and perform an inappropriate operation on the display. The in-vehicle system according to the present disclosure solves such problems.

[0016] An in-vehicle system according to one aspect of the present disclosure is an in-vehicle system including a display device and an information processing device that outputs information via the display device, wherein the display device is located in front of the driver's seat in the passenger compartment, protrudes further rearward than an instrument panel located in front of the steering wheel, is positioned on both sides of the steering wheel, and includes a pair of display units having touch panels, and the information processing device has a control unit that performs touch operations on each of the pair of display units, and when it is determined that a touch operation has been performed on either of the pair of display units, performs at least partially disables the touch operation on the other display unit.

[0017] The display device has a pair of display units positioned on either side of the steering wheel. Furthermore, the pair of display units are configured to allow touch operation via a touch panel. The pair of display units are positioned in the vehicle's longitudinal direction, protruding further rearward than the instrument panel (typically the meter panel where the speedometer is located) positioned in front of the steering wheel, i.e., closer to the driver. By positioning the pair of display units in this manner, it becomes possible to place the touch panel near the portion of the steering wheel that is gripped by the driver. In other words, it becomes possible to minimize the time the driver takes their hands off the steering wheel during operation.

[0018] However, placing a touchscreen display near the steering wheel could lead to inappropriate touch operations by passengers.

[0019] To address this, the information processing device according to this disclosure, when it is determined that a touch operation is being performed on one of a pair of display units, at least partially disables the touch operation on the other display unit.

[0020] Disabling touch operation may be performed on either one of the pair of control units, or on both. For example, if a touch operation is performed on the display unit located on the right side of the steering wheel, the touch operation on the display unit located on the left side of the steering wheel can be disabled, and if a touch operation is performed on the display unit located on the left side of the steering wheel, the touch operation on the display unit located on the right side of the steering wheel can be disabled. Furthermore, "disabling touch operation" may mean completely disabling touch operation on the target display unit, or it may mean disabling touch operation on a specific of the multiple operation interfaces output to the target display unit.

[0021] With this configuration, operation of the display unit that is not being operated by the driver can be prohibited, thereby preventing inappropriate operation by passengers.

[0022] Furthermore, if the control unit determines that a touch operation is being performed on the first display unit of the pair, which is located closer to the outside of the vehicle, it may at least partially disable the touch operation on the second display unit of the pair, which is located closer to the inside of the vehicle. This is because the display unit located closer to the inside of the vehicle is particularly prone to being operated by passengers.

[0023] Conversely, if it is determined that a touch operation is being performed on the second display unit, it is not necessary to disable the touch operation on the first display unit. The second display unit (located closer to the inside of the vehicle) is intended for the driver as well as passengers. There is a possibility that the system may be operated by a person, and the first display unit (the display unit located closer to the outside of the vehicle) is operated exclusively by the driver. Therefore, if a touch operation on the second display unit triggers the disabling of the touch operation on the first display unit, it may interfere with the driver's operation.

[0024] Furthermore, the control unit may output a predetermined notification via the second display unit when a predetermined notification to the occupant occurs in the vehicle, and may suppress the output of the predetermined notification via the second display unit when touch operation to the second display unit is disabled.

[0025] In this way, temporarily suppressing the output of notifications can prevent passengers from inappropriately operating the second display unit.

[0026] Furthermore, the control unit may activate touch operation on the other display unit if there is no touch operation on the display unit that has been determined to be performing a touch operation for a certain period of time.

[0027] Furthermore, the control unit may not disable the touch operation for either of the pair of display units when the vehicle's driving conditions meet predetermined conditions. For example, in situations where safety problems are unlikely to occur, such as when the vehicle's speed is below a predetermined value, touch operation by a passenger may be permitted.

[0028] Disabling the touch operation at least partially may include either completely disabling the touch operation on the target display unit, or disabling the touch operation on some of the multiple operation interfaces output to the target display unit. For example, it may be preferable to have access to important functions at all times. Therefore, instead of uniformly prohibiting touch operation, it may be acceptable to prohibit touch operation only for some of the multiple control interfaces.

[0029] Embodiments of this disclosure will be described below with reference to the drawings. The configurations of the following embodiments are illustrative, and this disclosure is not limited to the configurations of these embodiments.

[0030] (First embodiment) An overview of the vehicle system according to the first embodiment will be described. The vehicle system according to this embodiment is configured to include vehicle 1.

[0031] Vehicle 1 will be described with reference to Figure 1. Figure 1 shows the interior of vehicle 1. As shown in the figure, the interior of vehicle 1 is equipped with a driver's seat 2 and a passenger seat 3. In this embodiment, vehicle 1 is a right-hand drive vehicle, and the driver's seat 2 is located on the right side of the vehicle.

[0032] A steering wheel 4 is positioned in front of the driver's seat 2. The steering wheel may be circular, or it may be U-shaped (a so-called irregularly shaped steering wheel) as shown in the illustration.

[0033] The steering wheel 4 is connected to the steering system of the vehicle 1 via the steering column. The steering column is covered by a column cover.

[0034] Passenger seat 3 is located to the left of driver's seat 2, and is in a position midway between driver's seat 2 and passenger seat 3. A center display 24 is located at the front. The center display 24 displays supplementary information such as car navigation, or entertainment content such as audio and television.

[0035] A meter display 21 is provided on the front side of the steering wheel 4. The meter display 21 is a full-screen display located in front of the driver's seat 2 and provides various information to the driver of the vehicle 1. The meter display 21 displays information necessary for driving the vehicle, such as vehicle speed, engine speed (if the vehicle 1 has an engine), charge / discharge status (if the vehicle 1 has a drive battery), and mileage. The meter display 21 is an example of an "instrument unit" in this disclosure.

[0036] A pair of display devices are provided in front of the driver's seat 2 inside the vehicle. The pair of display devices include touch panel displays positioned on both sides of the steering wheel 4. In this embodiment, the display positioned closer to the inside of the vehicle is referred to as the inner display, and the display positioned closer to the outside of the vehicle is referred to as the outer display. The inner display 22 and the outer display 23 are connected to the instrument panel of the vehicle 1. The outer display 23 is an example of the "first display unit" in this disclosure, and the inner display 22 is an example of the "second display unit" in this disclosure.

[0037] The detailed structures of the inner display 22 and the outer display 23 will be described with reference to Figure 2. The inner display 22 and outer display 23 are vertically oriented displays and are connected to a support portion 5, which is a member that supports each display. The support portion 5 has a shape that protrudes in the left-right direction from the steering wheel. The outer display 23 is connected to the right end of the support portion 5 when facing forward of the vehicle, and the inner display 22 is connected to the left end when facing forward of the vehicle. This allows the display surface to be positioned near the grip portion of the steering wheel 4.

[0038] The above explanation is an example where vehicle 1 is a right-hand drive vehicle; if vehicle 1 is a left-hand drive vehicle, the left and right sides will be reversed. That is, the outer display 23 will be connected to the left end of the support part 5 facing the front of the vehicle, and the inner display 22 will be connected to the right end facing the front of the vehicle.

[0039] The support portion 5 may also serve as a column cover that covers the steering column. The support portion 5 extends from the portion through which the steering column passes in both the left-right direction (vehicle width direction), and each of these extends toward the rear of the vehicle, and is shaped to bend along the way so that the angle of the steering wheel with respect to the left-right direction becomes smaller. Furthermore, the inner display 22 and the outer display 23 are angled with respect to the left-right direction of the steering wheel, and are positioned so as to tilt toward the steering wheel. That is, the inner display 22 protrudes further to the rear of the vehicle the closer it is to the inside of the vehicle, and the outer display 23 protrudes further to the rear of the vehicle the closer it is to the outside of the vehicle. In this way, the inner display 22 and the outer display 23 are positioned so as to face the driver. The area 4R shown by the dashed line in Figure 2 is the part (grip area) where the steering wheel 4 is gripped by the right hand, and the area 4L shown by the dashed line in Figure 2 is the part (grip area) where the steering wheel 4 is gripped by the left hand. By making the support part 5 the shape described above, the positions of the inner display 22 and the outer display 23 in the longitudinal direction of the vehicle can be brought closer to the grip area of ​​the steering wheel 4. Therefore, when the driver performs a touch operation on the inner display 22 or the outer display 23 while driving... This allows for a reduction in the amount of hand movement involved.

[0040] [System Configuration] Next, we will describe the configuration of each device that makes up the vehicle system of vehicle 1. First, let's describe the components of Vehicle 1. Figure 3 is a schematic diagram showing an example of the configuration of Vehicle 1. Vehicle 1 consists of a vehicle system, an electrical system, and an input / output system.

[0041] A vehicle system is a system that controls the motion of a vehicle. A vehicle system includes multiple components related to the vehicle's movement. An input / output system is a system that provides information to the occupants of a vehicle and receives input from them. An input / output system includes multiple components that perform information input and output. An electrical system is a system that controls the electrical components of a vehicle's body. It includes multiple components related to the electrical components of the vehicle's body. The components included in each system may include multiple electrical components and an ECU that controls those electrical components.

[0042] The vehicle ECU 10 and the body ECU 30 can be configured as computers having a processor (CPU, GPU, etc.), main memory (RAM, ROM, etc.), and auxiliary memory (EPROM, hard disk drive, removable media, etc.). The auxiliary memory stores an operating system (OS), various programs, various tables, etc., and by executing the programs stored therein, various functions (software modules) that match a predetermined purpose, as described later, can be realized. However, some or all of the functions may be realized as hardware modules by hardware circuits such as ASICs and FPGAs.

[0043] The vehicle ECU10 is connected to several components related to the vehicle's operation.

[0044] The GPS unit 11 is a unit for acquiring the location information of vehicle 1. The GPS unit 11 includes a GPS antenna and a positioning module for determining the location information. The GPS antenna is an antenna that receives positioning signals transmitted from positioning satellites (also called GNSS satellites). The positioning module is a module that calculates location information based on the signals received by the GPS antenna.

[0045] The ETC (Electronic Toll Collection System) unit 12 collects tolls for toll roads. This is a unit for receiving data wirelessly. The ETC unit 12 consists of an antenna and a communication module.

[0046] The wireless communication unit 13 includes an antenna and a communication module for wireless communication. The antenna is an antenna element that inputs and outputs wireless signals. In this embodiment, the antenna is suitable for mobile communication (e.g., 3G, 4G, 5G, etc.). The antenna may be configured to include multiple physical antennas. The communication module is a module for performing mobile communication. Furthermore, the wireless communication unit 13 may include communication modules other than those for mobile communications. For example, the wireless communication unit 13 may include a Bluetooth® communication module. This makes it possible to establish a wireless connection with a user terminal located inside the vehicle, enabling the provision of services in cooperation with the user terminal (e.g., hands-free calling).

[0047] The sensor group 14 is a set of multiple sensors used to acquire sensor data during vehicle operation. This is a collection of sensors. The sensors may acquire physical quantities or image data, etc. In this embodiment, examples of sensors include a sensor that detects vehicle speed (vehicle speed sensor 141) and an image sensor that acquires an image of the area in front of the vehicle (onboard camera 142).

[0048] Furthermore, the vehicle ECU 10 may also be connected to components belonging to the drivetrain system (e.g., engine, motor, battery, transmission, etc.).

[0049] The body ECU 30 is connected to multiple electrical components of the vehicle 1's body. In this embodiment, examples of electrical components on the body of vehicle 1 include a car air conditioner, wipers, headlights, side mirrors, and power seats.

[0050] The air conditioning unit 31 is the car air conditioning system for vehicle 1. The air conditioning unit 31 includes, for example, a cooling unit, a heating unit, a compressor, a condenser, etc. The air conditioning unit 31 may also include a control unit for controlling these components.

[0051] The wiper unit 32 includes a front wiper, a rear wiper, and a motor for driving them.

[0052] The headlight unit 33 includes the headlights of the vehicle 1 and a unit for driving the headlights. The headlights are configured to switch between high beams and low beams, for example. If the headlights of the vehicle 1 are adaptive headlights, the illumination of the light can also be controlled based on commands from the body ECU 30. The headlight unit 33 may further include taillights, brake lights, turn signals, and / or units for driving them.

[0053] The mirror actuator 34 includes a drive unit for driving the side mirrors located on the left and right sides of the vehicle 1. The mirror actuator 34 allows the angle of the side mirrors to be adjusted up, down, left, and right.

[0054] The seat actuator 35 includes a drive unit for adjusting the position of a seat provided in the vehicle 1. The seat actuator 35 allows adjustment of the position, height, angle, etc., of the seat surface and backrest.

[0055] Next, we will describe the devices used for inputting and outputting information. In this embodiment, the vehicle 1 has four displays for outputting information: a meter display 21, an inner display 22, an outer display 23, and a center display 24.

[0056] The meter display 21 is a display positioned in front of the driver and outputs information about the vehicle 1. The meter display 21 mainly outputs information about the vehicle's status, such as vehicle speed, engine speed, fuel level, battery level, charge / discharge status, and mileage, but other information may also be output.

[0057] The inner display 22 and outer display 23 are touch-panel displays located on either side of the steering wheel. As mentioned above, the inner display 22 is located towards the center of the vehicle, and the outer display 23 is located on the outside of the vehicle. If vehicle 1 is a right-hand drive vehicle, the inner display 22 is on the left side in the direction of travel of the vehicle. The outer display 23 is positioned on the right side in the direction of travel of the vehicle. If vehicle 1 is a left-hand drive vehicle, the inner display 22 is positioned on the right side in the direction of travel of the vehicle, and the outer display 23 is positioned on the left side in the direction of travel of the vehicle.

[0058] The center display 24 is a display located in the center of the vehicle. The center display 24 is located in the same place where the car navigation system's display was previously located. The center display 24 can display, for example, supplementary information such as car navigation, or entertainment content such as audio or television.

[0059] The touch panel 25 is a unit for detecting touch operations performed on the aforementioned displays. In this embodiment, the inner display 22, outer display 23, and center display 24 each have a touch panel 25, allowing for touch operation. The touch panel 25 detects touch operations performed on these displays and transmits the detected information to the vehicle ECU 10.

[0060] The audio unit 26 is a unit that performs input and output of audio information. The audio unit 26 may include, for example, a speaker and a microphone.

[0061] Next, we will explain the detailed configuration of the vehicle ECU10. The vehicle ECU 10 is connected to the vehicle system and the drivetrain system, thereby enabling control of vehicle 1. Furthermore, the vehicle ECU 10 has the function of inputting and outputting information through components included in the input / output system, and the function of controlling the electrical system via the body ECU 30, which is responsible for the electrical system. Figure 4 is a diagram illustrating the detailed configuration of the vehicle ECU 10.

[0062] The vehicle ECU 10 is comprised of a control device 101, a storage device 102, and a communication module 103.

[0063] The control device 101 is a calculation unit that realizes various functions of the vehicle ECU 10 by executing a predetermined program. The control device 101 can be realized by a hardware processor such as a CPU. The control device 101 also has RAM, ROM (Read It may be configured to include (Only Memory), cache memory, etc.

[0064] In this embodiment, the control device 101 of the vehicle ECU 10 is configured with six software modules: an HMI control unit 1011, a vehicle control unit 1012, an ADAS control unit 1013, an entertainment control unit 1014, a navigation unit 1015, and an assistant unit 1016. Each software module may be implemented by the control device 101 (CPU, etc.) executing a program stored in a storage device 102, which will be described later. The information processing performed by the software modules is synonymous with the information processing performed by the control device 101 (CPU, etc.).

[0065] The HMI control unit 1011 generates a user interface for controlling various components of the vehicle and provides it to the vehicle occupants via the input / output system. The HMI control unit 1011 also passes information input via the input / output system (for example, commands to implement user-specified controls) to the corresponding software module, etc. This makes it possible to control any component of the vehicle through a user interface.

[0066] In the following explanation, commands generated by the user to control the components of Vehicle 1 will be referred to as "user commands." For example, commands to change the air conditioner temperature, commands to change the drive mode, commands to make a phone call, and commands to play music are examples of "user commands."

[0067] Furthermore, the components of a vehicle may include electrical components such as headlights and wipers. Also, the components of a vehicle may include an on-board computer, or software modules executed by the on-board computer (e.g., autonomous driving software, navigation software, etc.).

[0068] The HMI control unit 1011 selects the output destination of the user interface from among several displays (meter display 21, inner display 22, outer display 23, center display 24) depending on the type of component being controlled. The specific configuration will be described later.

[0069] The vehicle control unit 1012 controls components of the vehicle system or components of the drive system. The vehicle control unit 1012 may control a target component (e.g., a drive motor) based on a command (e.g., an acceleration command) received from another component (e.g., an ADAS control unit 1013, described later), or it may control a target component (e.g., a headlight unit) based on a user command (e.g., a headlight activation command) obtained via the HMI control unit 1011. Furthermore, the vehicle control unit 1012 can also control the electrical components of the vehicle body by communicating with the body ECU 30.

[0070] The ADAS control unit 1013 controls the Advanced Driver-Assistance Systems. For example, the ADAS control unit 1013 generates vehicle control commands based on sensor information acquired from the on-board sensors included in the sensor group 14, thereby realizing driver assistance such as cruise control, pre-collision safety, and lane keeping assist. For example, the ADAS control unit 1013 controls the vehicle system and the drive system by transmitting control commands to the vehicle control unit 1012.

[0071] The entertainment control unit 1014 controls the entertainment provided to the vehicle's occupants. Examples of entertainment functions include audio functions and television / radio functions.

[0072] The navigation unit 1015 provides a car navigation function. Based on location information obtained from the GPS unit 11 and traffic information obtained from the wireless communication unit 13, the navigation unit 1015 can generate and provide vehicle route guidance. The navigation unit 1015 may, for example, accept the vehicle's destination and waypoints via the HMI control unit 1011, and output information regarding the generated route.

[0073] Assistant unit 1016 provides information to the vehicle occupants by executing a language model capable of natural language interaction. The language model is, for example, LLM (Large Language Model). This could also be a Model. For example, by running an LLM that has information about the vehicle's occupants, it becomes possible to provide information in natural language. If there are multiple occupants in the vehicle, the assistant unit 1016 may obtain a language model corresponding to each person via the network and run any of the language models.

[0074] The storage device 102 is a means for storing information and is composed of storage media such as RAM, magnetic disks, and flash memory. The storage device 102 is executed by the control device 101. The program, the data used by the program, etc., are stored here.

[0075] The communication module 103 is a communication interface for connecting the vehicle ECU 10 to the vehicle network. The communication module 103 is configured to communicate with components of the vehicle 1 via a network such as CAN (Controller Area Network). ru.

[0076] The specific configuration of the vehicle ECU 10 can be appropriately modified, with components omitted, replaced, and added depending on the embodiment. For example, the control device 101 may include multiple hardware processors. The hardware processors may consist of microprocessors, FPGAs, GPUs, etc. Input / output devices other than those exemplified (e.g., optical drives, etc.) may also be added. Furthermore, the vehicle ECU 10 may be composed of multiple computers. In this case, the hardware configurations of each computer may or may not be identical.

[0077] [Interface screen provided by the HMI control unit] Next, the user interface provided by the HMI control unit 1011 will be described.

[0078] The HMI control unit 1011 can generate multiple types of graphical user interface (GUI) screens and output them via the multiple displays mentioned above. In this embodiment, the HMI control unit 1011 aggregates information related to vehicle control on the outer display 23 and aggregates and outputs information related to occupant convenience and comfort on the inner display 22. This allows occupants of the vehicle 1 to intuitively recognize whether the desired information is output to the left or right side. Furthermore, the touch panel operating surfaces of the inner display 22 and outer display 23 are positioned near the steering wheel. This minimizes the amount of eye and hand movement required by the vehicle driver.

[0079] The HMI control unit 1011 can switch between multiple types of screens that are output to the outer display 23. Figure 5 illustrates the transitions between multiple types of screens output to the outer display 23. As shown, the HMI control unit 1011 can output multiple screens, centered around the main screen 501, to the outer display 23.

[0080] (1) Main screen 501 The main screen 501 is the screen that is normally displayed on the outer display while driving. Figure 6(A) shows an example of the main screen 501. The content displayed on the main screen can be customized by the user.

[0081] (2) Headlight control screen 502 This screen is used to set the illumination status of the headlights of vehicle 1. Figure 7(A) shows an example of the headlight control screen 502. The headlight control screen 502 allows the user to select the headlight lighting status (automatic, on, off, small lights, daytime running lights, etc.). The vehicle control unit 1012 generates a user command based on the touch operation acquired via the HMI control unit 1011 and controls the headlight unit 33 by transmitting the user command to the body ECU 30. Furthermore, the vehicle control unit 1012 may acquire data indicating the headlight illumination status from the body ECU 30 and reflect the illumination status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 may display the current headlight status on the vehicle graphic drawn on the screen. The lighting status of the lights may also be reflected.

[0082] (3) Wiper control screen 503 This screen is used to set the operating status of the wipers (front and rear) of vehicle 1. Figure 7(B) shows an example of the wiper control screen 503. On the wiper control screen 503, the operating state of the wipers (stop, automatic operation, intermittent operation, continuous operation, etc.) can be selected. The vehicle control unit 1012 generates a user command based on the touch operation acquired via the HMI control unit 1011 and controls the wiper unit 32 by transmitting the user command to the body ECU 30. Furthermore, the vehicle control unit 1012 may acquire data indicating the wiper's operating status from the body ECU 30 and reflect the operating status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 may reflect the current wiper's operating status in the vehicle graphic drawn on the screen.

[0083] (4) Mirror control screen 504 This screen is used to adjust the deployment state and angle of the side mirrors of vehicle 1. Figure 7(C) shows an example of the mirror control screen 504. On the mirror control screen 504, the status of the side mirror (folded / unfolded state and angle) can be selected. The vehicle control unit 1012 generates a user command based on the touch operation acquired via the HMI control unit 1011 and controls the mirror actuator 34 by transmitting the user command to the body ECU 30. Furthermore, the vehicle control unit 1012 may acquire data indicating the status of the side mirrors from the body ECU 30 and reflect the operating status on the screen via the HMI control unit 1011. For example, the vehicle control unit 1012 may reflect the current extended / retracted state of the side mirrors in the vehicle graphic drawn on the screen.

[0084] (5) Driving position adjustment screen 505 This screen allows you to adjust the position and angle of the seats (seat cushion and backrest) and the steering wheel of vehicle 1. The driving position adjustment screen 505 allows users to set the position and angle of the seat and steering wheel. The vehicle control unit 1012 receives touch operations via the HMI control unit 1011, generates user commands based on these operations, and transmits these user commands to the body ECU 30 to control the seat actuator 35 and other components.

[0085] (6) ADAS settings screen 506 This screen displays the operating status of the advanced driver assistance system in vehicle 1 and allows for setting adjustments. For example, if the advanced driver assistance system of vehicle 1 includes an auto cruise function, the user of vehicle 1 can set the cruising speed, distance between vehicles, whether or not to follow the vehicle in front, whether or not to use lane tracing, etc., via the ADAS setting screen 506. The information included in the ADAS settings screen 506 can be generated by the ADAS control unit 1013.

[0086] (7) Drive mode setting screen 507 This is the screen for setting the drive mode of vehicle 1. Figure 7(D) shows an example of the drive mode setting screen 507. For example, the user of vehicle 1 can select a mode from several modes such as normal mode, sport mode, eco mode, and snow mode via the drive mode setting screen 507. Based on this selection, the vehicle control unit 1012 controls the vehicle's drive mode Change the code.

[0087] (8) Vehicle function setting screen 508 This screen is for using other functions of Vehicle 1. Examples of other functions of Vehicle 1 include opening and closing the electric tailgate and adjusting the suspension.

[0088] (9) Display mode setting screen 509 This screen is for setting the types of information displayed on the meter display 21. For example, if the meter display 21 can display multiple pieces of information other than speed, the user of the vehicle 1 can customize the information displayed on the meter display 21 via the display mode setting screen 509.

[0089] The multiple screens exemplified above can be switched by performing a predetermined operation, such as swiping left or right, on the touch panel provided on the outer display 23. Furthermore, the HMI control unit 1011 can display a shortcut screen 510 on the outer display 23 for jumping to each screen. Figure 6(B) shows an example of the shortcut screen 510. The shortcut screen can be accessed, for example, by performing a predetermined operation (for example, swiping from the bottom of the screen upwards) on each screen, including the main screen 501. The shortcut screen 510 has multiple interfaces (for example, buttons) for transitioning to each screen, and by operating these interfaces, the user can move to the desired screen.

[0090] In this way, the HMI control unit 1011 aggregates and outputs information related to vehicle control to the outer display 23. Information related to vehicle control typically refers to information related to the vehicle's driving control. Examples of information related to vehicle driving control include information for controlling components belonging to the vehicle system and information for controlling components belonging to the drive system. Furthermore, information for controlling components belonging to the electrical system, such as lights and wipers, which are necessary for the vehicle to drive properly, can also be considered information related to vehicle driving control. Furthermore, information related to vehicle control, even if not directly related to vehicle driving control, can be output to the outer display 23. For example, information for controlling the vehicle's electric doors may be output to the outer display 23.

[0091] Similarly, the HMI control unit 1011 can switch the screen output to the inner display 22 to multiple types of screens. Figure 8 illustrates the transitions between multiple types of screens output to the inner display 22. As shown, the HMI control unit 1011 can output multiple screens, centered around the main screen 801, to the inner display 22.

[0092] (1) Main screen 801 The main screen 801 is the screen that is normally displayed on the inner display while driving. Figure 9(A) shows an example of the main screen 801. The content displayed on the main screen can be customized by the user.

[0093] (2) Navigation screen 802 This is a screen related to car navigation. Figure 10(A) shows an example of the navigation screen 802. The information contained in the navigation screen 802 is navigation It is generated by part 1015. Furthermore, when map information from the car navigation system is displayed on the center display 24, the navigation screen 802 may output supplementary information regarding the vehicle 1's route. Examples of supplementary route information include guidance on the next turn, estimated time of arrival, and remaining distance.

[0094] Furthermore, the navigation screen 802 may include some of the information and user interface displayed on the center display 24. For example, frequently used user interfaces such as a button to start directions to home, a button to access bookmarks, and a button to start directions to a charging station can be placed on the navigation screen 802.

[0095] (3) Audio screen 803 This is a screen related to the audio functions of vehicle 1. Figure 10(B) shows an example of the audio screen 803. The information contained in the audio screen 803 is generated by the entertainment control unit 1014. The entertainment control unit 1014 can acquire and play music via a network or media. The audio screen 803 can include information about the music currently being played and an interface for selecting music.

[0096] (4) Air conditioner settings screen 804 This is a screen for controlling the car air conditioner of vehicle 1. Figure 10(C) shows an example of the air conditioner setting screen 804. The air conditioning settings screen 804 allows users to set the temperature and airflow of the car's air conditioning system, as well as the air conditioning devices to be operated (steering wheel heater, seat heater, defroster, etc.). The vehicle control unit 1012 generates user commands based on the operations performed on the screen and transmits these user commands to the body ECU 30 to control the air conditioning unit 31 and other components. Although the car air conditioner is a vehicle 1 piece of equipment controlled by external commands, it relates to the comfort of the occupants, so the air conditioner setting screen 804 is displayed on the inner display 22, not the outer display 23.

[0097] (5) Phone screen 805 This screen is for wirelessly connecting with the user terminal and making hands-free calls. If the wireless communication unit 13 has a module for wireless communication with the user terminal, it can make (or receive) calls from the user terminal via this module.

[0098] (6) AI Partner Screen 806 This is a screen for interacting with a virtual agent (AI partner) capable of natural language dialogue. Figure 10(D) shows an example of the AI ​​partner screen 806. This screen includes a virtual agent character, and the vehicle occupant can obtain information by interacting with this character. The natural language dialogue service is provided by the assistant unit 1016. The AI ​​partner screen 806 may include an interface for giving instructions to the agent. If there are multiple occupants in the vehicle, the assistant unit 1016 may be configured to run any agent corresponding to each person. The AI ​​partner screen 806 may also include interfaces for starting and ending interactions with the agent, and interfaces for switching agents.

[0099] The multiple screens exemplified above can be switched by performing a predetermined operation, such as swiping left or right, on the touch panel provided on the inner display 22. Furthermore, the HMI control unit 1011 can display a shortcut screen 810 on the inner display 22 for jumping to each screen. Figure 9(B) shows an example of the shortcut screen 810. The shortcut screen can be accessed, for example, by performing a predetermined operation (for example, swiping from the bottom of the screen upwards) on each screen, including the main screen 801. The shortcut screen 810 has multiple interfaces (for example, buttons) for transitioning to each screen, and by operating these interfaces, the user can move to the desired screen.

[0100] In this way, the HMI control unit 1011 aggregates and outputs information regarding the convenience and comfort of the occupants to the inner display 22.

[0101] Furthermore, information about components that are part of the vehicle, such as car air conditioners, but relate to the convenience or comfort of the occupants, may be output to the inner display 22 without being treated as information related to vehicle control. Thus, if the information in question falls under both vehicle control information and information related to occupant convenience (or comfort), the information may be selectively output to either the inner display 22 or the outer display 23 according to predetermined criteria.

[0102] [Process to disable touch operation] Next, we will describe the process of disabling touch operations on the other display based on the status of touch operations on the inner display 22 or the outer display 23.

[0103] Figure 11 is a flowchart of the touch operation disabling process performed by the HMI control unit 1011. This process is performed periodically.

[0104] First, in step S11, the HMI control unit 1011 determines whether an operation is currently being performed on the outer display 23 based on the output signal from the touch panel 25. The HMI control unit 1011 may determine that an operation is currently being performed on the outer display 23 if, for example, a predetermined period of time has not elapsed since the last touch operation on the outer display 23.

[0105] If the result in step S11 is positive, the process proceeds to step S12. In step S12, the HMI control unit 1011 performs a process to disable touch operation on the inner display 22. This process may disable touch operation on the entire screen (all areas). Note that "disabling touch operation" may also mean not accepting operation signals corresponding to the target area obtained from the touch panel 25.

[0106] If the result in step S11 is negative, the process proceeds to step S13. In step S13, the HMI control unit 1011 disables touch operation to the inner display 22 if it has been disabled. If touch operation to the display is not disabled, the process proceeds to step S14.

[0107] In step S14, the HMI control unit 1011 determines whether an operation is currently being performed on the inner display 22 based on the output signal from the touch panel 25. The HMI control unit 1011 may determine that an operation is currently being performed on the inner display 22 if, for example, a predetermined period of time has not elapsed since the last touch operation on the inner display 22.

[0108] If the result in step S14 is positive, the process proceeds to step S15. In step S15, the HMI control unit 1011 performs a process to disable touch operation on the outer display 23. This process may disable touch operation on all areas of the screen.

[0109] If the result in step S14 is negative, the process proceeds to step S16. In step S16, the HMI control unit 1011 disables touch operation for the outer display 23 if it is disabled. If touch operation for the display is not disabled, the process ends.

[0110] As described above, in the first embodiment, the HMI control unit 1011 of the vehicle ECU uses the outer display 23 and the inner display 22 to output various information related to the vehicle and driving. The HMI control unit 1011 outputs information related to occupant convenience and comfort (second information) via an inner display positioned closer to the inside of the vehicle, and outputs information related to vehicle control (first information) via an outer display positioned closer to the outside of the vehicle. With this configuration, the driver of vehicle 1 can intuitively recognize which display unit outputs the information they are looking for, minimizing the time they need to focus their attention on the display.

[0111] Furthermore, the HMI control unit 1011 restricts touch operations on the other display (inner display 22 or outer display 23) based on the status of touch operations on the inner display 22 or outer display 23. With this configuration, it becomes possible to suppress inappropriate touch operations by passengers.

[0112] (Second Embodiment) In the first embodiment, when an operation is being performed on the inner display 22 or the outer display 23, touch operation on the other display is disabled, thereby suppressing touch operation by anyone other than the driver. However, in situations where safety is not affected, touch operation by someone other than the driver may be permitted. The second embodiment is an embodiment in which the disabling of touch operation described in the first embodiment is not performed under predetermined conditions.

[0113] Figure 12 is a flowchart of the process for disabling touch operations as performed in this embodiment. In this embodiment, first, in step S10, the HMI control unit 1011 determines whether the status of vehicle 1 meets predetermined conditions. These predetermined conditions can be conditions that are considered to have a high safety margin, such as "vehicle speed is below a predetermined value," "vehicle is stopped," or "steering angle is below a predetermined value." The status of vehicle 1 can be obtained from sensors included in the sensor group 14.

[0114] Furthermore, the predetermined conditions may also be conditions relating to the driver's grip on the steering wheel 4. For example, if it is determined that the driver is gripping the steering wheel 4 with both hands, the vehicle is considered to be running stably. For this reason, for example, a sensor for detecting the driver's grip on the steering wheel 4 may be placed on the steering wheel 4 and the detection result may be obtained.

[0115] In this step, if the status of vehicle 1 meets the predetermined conditions, the processes described in steps S11 to S16 above, namely the process of disabling touch operation, are skipped. If the status of vehicle 1 does not meet the predetermined conditions, the same process as shown in Figure 11 will be performed.

[0116] In this embodiment, the process of disabling touch operation was skipped when the status of vehicle 1 met predetermined conditions. However, multiple levels of the process of disabling touch operation may be provided, and the level to be applied may be dynamically determined according to the status of vehicle 1. For example, as shown in Figure 13, the status of vehicle 1 may be divided into multiple stages according to the driver's workload level, and more elements may be disabled as the driver's workload increases. Note that Figure 13 is just one example, and the driver's workload may be defined by factors such as "whether or not the vehicle is turning right or left," "whether or not the vehicle is changing lanes," or "the volume of surrounding traffic." The vehicle status may also be defined by factors other than the driver's workload. Furthermore, the level of disabling may be, for example, "disabling touch operation on either (or both) the inner display and the outer display," or "classifying multiple operation interfaces displayed on the display by importance and disabling them in order from least important to least important."

[0117] (modified version) The embodiments described above are merely examples, and this disclosure may be modified as appropriate without departing from its essence. For example, the processes and means described in this disclosure can be freely combined and implemented, as long as no technical inconsistencies arise.

[0118] Furthermore, in the first and second embodiments, when an operation is performed on either the inner display 22 or the outer display 23, touch operation on the other display is disabled. However, since it is unlikely that a passenger would perform a touch operation on the outer display 23, the outer display 23 may be excluded from the disabling process. In other words, in the process shown in Figure 11, the processes from step S14 onwards may be omitted.

[0119] Furthermore, in each embodiment, in order to prevent passengers from touching the display, touch operation on the inner display 22 or outer display 23 is disabled when certain conditions are met. However, even if touch operation is disabled, if a notification from the vehicle is output to the inner display 22 or outer display 23, a passenger may attempt to operate the touch screen, which could lead to safety issues. To address this issue, interrupt display output may be suppressed for the relevant display during periods when touch operation is disabled. Furthermore, if the content of the interrupt display is something that can be handled with simple operations, such as presenting a choice between two options, it may be excluded from suppression as it does not pose a safety problem. Furthermore, the suppression of interrupt displays may be applied only to the inner display 22, as it is unlikely that passengers will perform touch operations on the outer display 23.

[0120] Furthermore, while each embodiment describes an example in which touch operations on the touch panel are completely disabled, if multiple operation interfaces are displayed on the target display, only some of them may be disabled. For example, operation interfaces that perform controls important to the vehicle's operation may be excluded from the disablement process.

[0121] Furthermore, in steps S11 and S14, if a predetermined period of time has not elapsed since the last touch on the touch panel, the current operation on the target display is performed. It was determined that the touch operation was being performed on the target display. However, if it can be evaluated that touch operations are being performed on the target display continuously, other criteria may be used to make the determination in that step. For example, if any of the multiple screens illustrated in Figures 5 and 8 are open, it may be determined that touch operations are being performed on the target display continuously.

[0122] Furthermore, while each embodiment illustrates a configuration in which the meter display 21, inner display 22, and outer display 23 are independent of each other, each display does not necessarily have to be physically independent. The display device according to this disclosure may be realized by a single display 1401, for example, as shown in Figure 14. In the example of Figure 14, the display 1401 has three display areas (display sections) separated by dashed lines. The display 1401 may have a structure in which the display surface is bent or curved at the portion indicated by the dashed lines. In the illustrated example, display area 1402 functions as the inner display 22, display area 1403 functions as the outer display 23, and display area 1404 functions as the meter display 21.

[0123] Furthermore, in this embodiment, two types of information were exemplified as the second information: information relating to the convenience of the occupants and information relating to the comfort of the occupants. However, the second information may consist of only one of these, or it may include other types of information. Furthermore, in this embodiment, information related to vehicle control was given as an example of the first information, but the first information may include other types of information.

[0124] Furthermore, a process described as being performed by a single device may be divided and executed by multiple devices. Conversely, a process described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is implemented can be flexibly changed.

[0125] The present disclosure can also be realized by supplying a computer program implementing the functions described in the embodiments above to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer by a non-temporary computer-readable storage medium that can be connected to the computer's system bus, or it may be provided to the computer via a network. Non-temporary computer-readable storage mediums include, for example, any type of disk such as magnetic disks (floppy disks, hard disk drives (HDDs), etc.), optical disks (CD-ROMs, DVDs, Blu-ray discs, etc.), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, and any type of medium suitable for storing electronic instructions. [Explanation of Symbols]

[0126] 1. Vehicle 10. Vehicle ECU 101... Control device 102...Storage device 103...Communication module 21... Meter display 22...Inner display 23. Outer display 24... Center display 25...Touch panel 26..Audio Unit 30...Body ECU

Claims

1. An in-vehicle system including a display device and an information processing device that outputs information via the display device, The aforementioned display device is Located in front of the driver's seat inside the vehicle, A pair of display units, each having a touch panel, are positioned on either side of the steering wheel, protruding further rearward than the instrument panel located in front of the steering wheel. The aforementioned information processing device is To acquire touch operations performed on each of the pair of display units, If it is determined that a touch operation is being performed on either of the pair of display units, the touch operation on the other display unit is disabled at least partially. Having a control unit that performs the following: In-vehicle systems.

2. If the control unit determines that a touch operation is being performed on the first display unit of the pair, which is located closer to the outside of the vehicle, it will at least partially disable the touch operation on the second display unit of the pair, which is located closer to the inside of the vehicle, and if it determines that a touch operation is being performed on the second display unit, it will not disable the touch operation on the first display unit. The in-vehicle system according to claim 1.

3. The control unit outputs the predetermined notification via the second display unit when a predetermined notification to the occupant occurs in the vehicle. The in-vehicle system according to claim 2.

4. The control unit suppresses the output of the predetermined notification via the second display unit when touch operation to the second display unit is disabled. The in-vehicle system according to claim 3.

5. The control unit activates a touch operation on the other display unit if there is no touch operation on the display unit that has been determined to be performing a touch operation for a certain period of time. The in-vehicle system according to claim 1.

6. The control unit shall not disable the touch operation for either of the pair of display units when the vehicle's driving conditions meet predetermined conditions. The in-vehicle system according to claim 1.

7. The aforementioned predetermined conditions include the vehicle's speed being less than or equal to a predetermined value. The in-vehicle system according to claim 6.

8. Disabling the touch operation at least partially includes either completely disabling touch operation on the target display unit, or disabling touch operation on some of the multiple operation interfaces output to the target display unit. The in-vehicle system according to claim 1.

9. A vehicle equipped with the in-vehicle system according to any one of claims 1 to 8.

10. An information processing device provided in the front of the driver's seat inside the vehicle, positioned on both sides of the steering wheel, protruding further rearward than the instrument panel located in front of the steering wheel, and having a display device that includes a pair of display units having touch panels, which provides information via this display device, To acquire touch operations performed on each of the pair of display units, If it is determined that a touch operation is being performed on either of the pair of display units, the touch operation on the other display unit is disabled at least partially. Having a control unit that performs the following: Information processing device.

11. If the control unit determines that a touch operation is being performed on the first display unit of the pair, which is located closer to the outside of the vehicle, it will at least partially disable the touch operation on the second display unit of the pair, which is located closer to the inside of the vehicle, and if it determines that a touch operation is being performed on the second display unit, it will not disable the touch operation on the first display unit. The information processing apparatus according to claim 10.

12. The control unit outputs a predetermined notification via the second display unit when a predetermined notification to the occupant occurs in the vehicle, and suppresses the output of the predetermined notification via the second display unit when touch operation to the second display unit is disabled. The information processing apparatus according to claim 11.

13. The control unit activates a touch operation on the other display unit if there is no touch operation on the display unit that has been determined to be performing a touch operation for a certain period of time. The information processing apparatus according to claim 10.

14. The control unit shall not disable the touch operation for either of the pair of display units when the vehicle speed is below a predetermined value. The information processing apparatus according to claim 10.

15. Disabling the touch operation at least partially includes either completely disabling touch operation on the target display unit, or disabling touch operation on some of the multiple operation interfaces output to the target display unit. The information processing apparatus according to claim 10.

Images