Vehicle system and notification method

Abstract

To improve comfort for an occupant who uses a portable terminal during driving.SOLUTION: A vehicle system comprises: an on-vehicle device mounted on a vehicle; and a portable terminal used by an occupant on the vehicle. The on-vehicle device includes: a vehicle information acquisition unit that acquires vehicle information related to the vehicle; a notification condition determination unit that determines whether or not a notification condition for providing a notification to the occupant is satisfied; and a transmission unit that, when it is determined that the notification condition is satisfied, transmits notification information dependent on the vehicle information to the portable terminal by using the communication device. The portable terminal includes: a reception unit that receives the notification information from the transmission unit by using the communication device; and a notification unit that provides a notification to the occupant on the basis of the notification information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a vehicle system and a notification method. [Background technology]

[0002] While riding in a vehicle, a passenger may browse various types of information using a mobile terminal such as a smartphone or a tablet terminal.

[0003] A wireless communication system including a portable device configured to enable two-way communication and an on-board device installed in a vehicle is known (Patent Document 1). In the wireless communication system described in Patent Document 1, the portable device displays on a display provided in the device the vehicle status (remaining gasoline, voltage of the on-board battery) or its own status, and further, if an action is required in this status, the vehicle status or the action method for its own status obtained via the Internet. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-100645 Summary of the Invention [Problem to be solved by the invention]

[0005] In the wireless communication system described in Patent Document 1, if a passenger uses a mobile terminal while driving, he or she may become car sick if he or she actively looks at the mobile terminal. In other words, if a passenger uses a mobile terminal while driving, the passenger's comfort may be impaired.

[0006] In addition, information such as arrival time to the destination or current location was only provided to passengers when asked by the driver. This could be bothersome for passengers to have to ask the driver for the information, and the driver could not concentrate on driving. In other words, passenger comfort could be impaired while driving.

[0007] There is a demand for improving the comfort of passengers who use portable terminals while driving by making the exchange of information between the vehicle and the portable terminals more appropriate.

[0008] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a vehicle system and notification method that can improve the comfort of occupants who use mobile terminals while driving. [Means for solving the problem]

[0009] The vehicle system and notification method according to the present invention employ the following configuration. (1): A vehicle system according to one embodiment of the present invention comprises an on-board device mounted in a vehicle and a mobile terminal used by an occupant of the vehicle, wherein the on-board device comprises a vehicle information acquisition unit that acquires vehicle information about the vehicle, a notification condition determination unit that determines whether a notification condition for notifying the occupant is satisfied, and a transmission unit that transmits notification information corresponding to the vehicle information to the mobile terminal using a communication device when it is determined that the notification condition is satisfied, and the mobile terminal comprises a receiving unit that receives the notification information from the transmission unit using the communication device, and a notification unit that notifies the occupant based on the notification information.

[0010] (2): In the above aspect (1), the mobile terminal further includes a first display unit, and the notification unit notifies the occupant by displaying information based on the notification information on the first display unit.

[0011] (3): In the above aspect (2), the vehicle-mounted device further includes a gaze determination unit that determines whether the occupant is looking at the mobile terminal, and the vehicle-mounted device further includes a second display unit, and when it is determined that the notification condition is met and the occupant is looking at the mobile terminal, the transmitting unit transmits the notification information to the mobile terminal using a communication device, and when it is determined that the notification condition is met and the occupant is not looking at the mobile terminal, the transmitting unit causes the second display unit to display information based on the notification information.

[0012] (4): In the above aspect (2), the transmitter selects whether to display the information based on the notification information on the first display unit or the second display unit provided in the vehicle, depending on the type of the notification information.

[0013] (5): In any of the above aspects (1) to (4), the mobile terminal further includes an audio output unit, and the notification unit notifies the occupant by causing the audio output unit to output audio based on the notification information.

[0014] (6): In any of the above aspects (1) to (5), the in-vehicle device further includes an estimation unit that estimates, based on the vehicle information, whether the behavior of the vehicle will fluctuate by more than a predetermined amount, and the notification condition is that the behavior of the vehicle is estimated to fluctuate by more than a predetermined amount.

[0015] (7) In the above aspect (6), when it is estimated that the vehicle behavior will fluctuate by more than a predetermined amount, the automatic brake is activated.

[0016] (8): In the above-mentioned aspects (6) or (7), when it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount, this includes when the vehicle is traveling on a route with a series of curves.

[0017] (9): In any of the above aspects (6) to (8), if it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount, the speed limit may be changed.

[0018] (10): In any of the above aspects (6) to (9), when it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount, this includes when the vehicle is traveling through a junction.

[0019] (11): In any of the above aspects (6) to (10), when it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount, this includes when the vehicle is traveling at a lane merging point.

[0020] (12): In any of the above aspects (6) to (11), when it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount, this includes when the vehicle approaches the start point of a traffic jam.

[0021] (13): In any of the above aspects (6) to (12), the notification condition is the arrival of the notification time or the performance of an operation to send a notification to the mobile terminal, and the notification information is information that the occupant wants to know.

[0022] (14): A notification method according to another aspect of the present invention includes an on-board device mounted on a vehicle acquiring vehicle information about the vehicle, the on-board device determining whether notification conditions for notifying an occupant of the vehicle are met, and if the on-board device determines that the notification conditions are met, transmitting notification information corresponding to the vehicle information to a mobile terminal used by the occupant of the vehicle using a communication device, the mobile terminal receiving the notification information from the on-board device using the communication device, and the mobile terminal notifying the occupant based on the notification information. [Effects of the Invention]

[0023] According to aspects (1) to (14), it is possible to improve the comfort of passengers who use mobile terminals while driving. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a diagram showing an example of an overview of a vehicle system 1 according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of a vehicle 2. [Figure 3] 2 is a diagram illustrating an example of the functional configuration of a first control unit 120. FIG. [Figure 4] FIG. 2 is a diagram illustrating an example of a functional configuration of a mobile terminal 8. [Figure 5] FIG. 10 illustrates an example of a notification process. [Figure 6] FIG. 10 is a diagram showing an example of a notification message based on notification information B1. [Figure 7] FIG. 10 illustrates an example of a notification process. [Figure 8] FIG. 10 is a diagram showing an example of a notification message displayed on the in-vehicle display 32. [Figure 9] FIG. 10 illustrates an example of a notification process. [Figure 10] 10 is a diagram showing an example of a display target according to the type of notification information B1. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0025] (Embodiment) Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing an example of an overview of a vehicle system 1 according to this embodiment. The vehicle system 1 includes a vehicle 2 and a mobile terminal 8. As an example, the vehicle 2 is an automobile with an autonomous driving function. The mobile terminal 8 is a smartphone, a tablet terminal, or the like.

[0026] The mobile terminal 8 is used by an occupant of the vehicle 2. An occupant of the vehicle 2 is, for example, a passenger sitting in the front passenger seat or a rear seat. The occupant who uses the mobile terminal 8 also includes the driver. The driver uses the mobile terminal 8, for example, during a period when there is no need to drive due to automatic driving. In the present embodiment, as an example, the mobile terminal 8 is carried and used by the passenger.

[0027] The vehicle 2 and the mobile terminal 8 communicate via wireless communication. When it is determined that the notification conditions are met, the vehicle 2 transmits notification information to the mobile terminal 8 via wireless communication. The mobile terminal 8 notifies the occupant by displaying text or the like based on the notification information on the mobile display unit 84. The mobile display unit 84 includes a display.

[0028] Even when the mobile terminal 8 is used by the driver, if the driver is looking at the mobile terminal 8, the process of stopping the autonomous driving is performed up to conditional autonomous driving (Level 3). Therefore, in the present embodiment, as an example, the driver is not included in the targets of notification up to conditional autonomous driving (Level 3). However, if the driver is fully autonomous driving (Level 4) or higher, the driver is included in the targets of notification.

[0029] The notification condition is, for example, that it is estimated that the behavior of the vehicle 2 will fluctuate by more than a predetermined amount in the near future (for example, in the next few tens of seconds). In this case, the notification information indicates, for example, that the behavior of the vehicle 2 will fluctuate by more than a predetermined amount. Another example of the notification condition is that a time has come when the occupant desires notification. In this case, the notification information is, for example, the arrival time to the destination, the current location, etc.

[0030] The vehicle system 1 may switch whether to display the notification on the mobile display unit 84 or the in-vehicle display 32, depending on whether the occupant using the mobile terminal 8 is looking at the mobile terminal 8. Furthermore, the vehicle system 1 may display the notification on both the mobile display unit 84 and the in-vehicle display 32, or on only one of the mobile display unit 84 and the in-vehicle display 32, depending on the type of notification information.

[0031] [Vehicle 2 Configuration] FIG. 2 is a diagram showing an example of the configuration of a vehicle 2 according to this embodiment. The vehicle 2 is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle. The drive source of the vehicle 2 is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or power discharged from a secondary battery or a fuel cell.

[0032] The vehicle 2 includes a first camera 10, a radar device 11, a LIDAR 12, a second camera 13, an object recognition device 16, a communication device 20, an HMI 30, vehicle sensors 40, a navigation device 50, an MPU 60, a driving operator 70, an automatic driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are connected to each other via multiplexed communication lines such as CAN communication lines, serial communication lines, a wireless communication network, etc. Note that the configuration shown in FIG. 2 is merely an example, and some of the configuration may be omitted, or other configurations may be added.

[0033] The first camera 10 is, for example, a digital camera using a solid-state imaging element such as a CCD or CMOS. The first camera 10 is attached to any location on the vehicle 2. When capturing an image of the front, the first camera 10 is attached to the top of the front windshield or the back of the rearview mirror. The first camera 10, for example, periodically captures images of the surroundings of the vehicle 2. The first camera 10 may be a stereo camera.

[0034] The radar device 11 emits radio waves such as millimeter waves around the vehicle 2 and detects radio waves reflected by objects (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 11 is attached to any location on the vehicle 2. The radar device 11 may detect the position and speed of the object by FM-CW (frequency modulation-continuous wave) method.

[0035] The LIDAR 12 irradiates the area around the vehicle 2 with light (or electromagnetic waves with wavelengths similar to light) and measures the scattered light. The LIDAR 12 detects the distance to the target based on the time between emitting and receiving the light. The irradiated light is, for example, pulsed laser light. The LIDAR 12 may be attached to any location on the vehicle 2.

[0036] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the first camera 10, the radar device 11, and the LIDAR 12 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition results to the autonomous driving control device 100. The object recognition device 16 may output the detection results from the first camera 10, the radar device 11, and the LIDAR 12 to the autonomous driving control device 100 as they are.

[0037] The second camera 13 is, for example, a digital camera that uses a solid-state imaging element such as a CCD or CMOS. The second camera 13 is attached to the bottom of the front windshield, in front of the front seats, or on the surface of the rearview mirror. The second camera 13 captures an image of the eyes of the occupant to detect their line of sight. Multiple second cameras 13 may be attached to capture an image of each of the eyes of multiple occupants.

[0038] The communication device 20 performs wireless communication. For example, the communication device 20 uses a cellular network, a Wi-Fi network, Bluetooth (registered trademark), or DSRC to communicate with other vehicles present around the vehicle 2, or to communicate with various server devices via a wireless base station. The communication device 20 also communicates with the mobile terminal 8 using short-range wireless communication such as Bluetooth (registered trademark). The communication device 20 may communicate with the mobile terminal 8 using the above-mentioned cellular network, Wi-Fi network, or DSRC.

[0039] The HMI 30 presents various information to the occupants of the vehicle 2 and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys, etc. The various display devices include an in-vehicle display 32. The in-vehicle display 32 may include a touch panel. The in-vehicle display 32 is an example of a second display unit. The vehicle information acquisition unit 130 also includes a speaker 34 as a speaker.

[0040] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle 2, an acceleration sensor that detects acceleration, a yaw rate sensor that detects angular velocity around a vertical axis, a direction sensor that detects the direction of the vehicle 2, and the like.

[0041] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as a HDD or flash memory.

[0042] The GNSS receiver 51 determines the position of the vehicle 2 based on signals received from GNSS satellites. The position of the vehicle 2 may be determined or supplemented by an INS that uses the output of the vehicle sensors 40.

[0043] The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 52 may share some or all of its components with the HMI 30 described above.

[0044] The route determination unit 53 determines, for example, a route (hereinafter referred to as a route on a map) from the position of the vehicle 2 identified by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the navigation HMI 52, by referring to the first map information 54.

[0045] The first map information 54 is information that represents road shapes using, for example, links that indicate roads and nodes connected by the links. The first map information 54 may also include information such as road curvature and POI information.

[0046] The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized, for example, by a function of a portable terminal 8 carried by a passenger. The navigation device 50 may transmit the current position and destination to a navigation server via the communication device 20, and obtain a route equivalent to the route on the map from the navigation server.

[0047] The MPU 60 includes, for example, a recommended lane determination unit 61, and stores second map information 62 in a storage device such as an HDD or flash memory.

[0048] The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines, for example, which lane from the left the vehicle should travel in. When there is a branch point on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle 2 can travel on a reasonable route to the branch point.

[0049] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, etc. The second map information 62 may be updated as needed by the communication device 20 communicating with other devices.

[0050] The driving operators 70 include, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, an irregular steering wheel, a joystick, and other operators. The driving operators 70 are equipped with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100 or some or all of the driving force output device 200, the braking device 210, and the steering device 220.

[0051] The automatic driving control device 100 includes a first control unit 120 and a second control unit 180. The first control unit 120 and the second control unit 180 are each realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as a HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the automatic driving control device 100 by inserting the storage medium (non-transitory storage medium) into a drive device. The automatic driving control device 100 is an example of an in-vehicle device.

[0052] The first control unit 120, for example, realizes functions based on AI (artificial intelligence) and functions based on a pre-given model in parallel. For example, the function of "recognizing intersections" may be realized by executing intersection recognition using deep learning or the like and recognition based on pre-given conditions (such as traffic lights and road signs that can be pattern-matched) in parallel, and then scoring and comprehensively evaluating both. This ensures the reliability of autonomous driving. The first control unit 120 also has an ADAS function.

[0053] The second control unit 180 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle 2 passes through the target trajectory generated by the first control unit 120 at the scheduled time.

[0054] [Functional configuration of first control unit 120] Here, the functional configuration of the first control unit 120 will be described in detail with reference to Fig. 3. Fig. 3 is a diagram showing an example of the functional configuration of the first control unit 120 according to this embodiment. The first control unit 120 includes a vehicle information acquisition unit 130, a notification condition determination unit 140, an estimation unit 150, a transmission unit 160, and a line-of-sight determination unit 170.

[0055] The vehicle information acquisition unit 130 acquires vehicle information A1. The vehicle information A1 indicates information related to the vehicle 2. The vehicle information A1 is, for example, navigation guidance, which is guidance generated by the navigation device 50. The navigation guidance is, for example, information indicating that the vehicle 2 will turn at an intersection within a predetermined time (for example, one minute). Other examples of the vehicle information A1 will be described later.

[0056] The notification condition determination unit 140 determines whether or not a notification condition is satisfied. The notification condition is a condition for notifying the occupant.

[0057] The estimation unit 150 estimates whether the behavior of the vehicle 2 will fluctuate by a predetermined amount or more based on the vehicle information A1.

[0058] When it is determined that the notification condition is satisfied, the transmitting unit 160 transmits notification information B1 corresponding to the vehicle information A1 to the mobile terminal 8 using the communication device 20. Here, the communication device 20 transmits the notification information B1 to the mobile terminal 8 by, for example, short-range wireless communication.

[0059] The line of sight determination unit 170 determines whether or not an occupant using the mobile terminal 8 is looking at the mobile terminal 8. The line of sight determination unit 170 determines the line of sight of the occupant from an image of the occupant's eyes captured by the second camera 13. The line of sight determination unit 170 determines the line of sight of the occupant based on image recognition. The image recognition used by the line of sight determination unit 170 is, for example, one or more of AI, pattern matching, and the like.

[0060] [Functional configuration of mobile terminal 8] 4 is a diagram showing an example of the functional configuration of the mobile terminal 8 according to this embodiment. The mobile terminal 8 includes a third control unit 80, a communication unit 82, a mobile display unit 84, an operation unit 86, and an audio output unit 88.

[0061] The third control unit 80 performs various calculations. The third control unit 80 includes a CPU, a RAM, and the like. The third control unit 80 includes a receiving unit 810, a notifying unit 820, and an operation detecting unit 830. The receiving unit 810, the notifying unit 820, and the operation detecting unit 830 are each realized by, for example, a hardware processor such as a CPU executing a program (software). The program is stored in a storage unit (not shown) of the mobile terminal 8. The functional units included in the third control unit 80 will be described later.

[0062] The communication unit 82 performs wireless communication. The communication unit 82 includes a communication interface (I / F) for performing the wireless communication. The wireless communication includes, for example, a cellular network, a Wi-Fi network, or short-range wireless communication such as Bluetooth (registered trademark).

[0063] The mobile display unit 84 displays various types of information. The mobile display unit 84 includes, for example, a liquid crystal display, an organic electroluminescence display, etc. The mobile display unit 84 is an example of a first display unit.

[0064] The operation unit 86 accepts various operations from a user. The user is a passenger in the vehicle 2. The operation unit 86 includes, for example, a touch panel. Note that, as an example, the mobile display unit 84 and the operation unit 86 are provided integrally as a touch panel. The mobile display unit 84 and the operation unit 86 may also be provided as separate units.

[0065] The audio output unit 88 outputs various types of audio and includes a speaker.

[0066] Here, we will explain each functional unit included in the third control unit 80. The receiving unit 810 receives notification information B1 from the transmitting unit 160 using the communication unit 82. Here, the receiving unit 810 causes the communication unit 82 to receive the notification information B1 from the communication device 20, for example, by short-range wireless communication.

[0067] The notification unit 820 notifies the occupant based on the notification information B1. The notification unit 820 notifies the occupant, for example, by displaying a notification message on the mobile display unit 84. The notification message is an example of information based on the notification information B1.

[0068] The operation detection unit 830 detects whether an operation is being performed by a passenger who is a user. The operation detection unit 830 detects whether the passenger is operating the mobile terminal 8 based on whether the operation unit 86 has accepted various operations. When the operation detection unit 830 detects that the passenger is operating the mobile terminal 8, it determines that the passenger is looking at the mobile terminal 8.

[0069] [Notification processing by vehicle system 1] Next, a notification process that is a process for notifying a passenger using the mobile terminal 8 in the vehicle system 1 will be described. FIG. 5 is a diagram showing an example of the notification process according to this embodiment. For example, while the ignition of the vehicle 2 is on, the vehicle system 1 repeatedly executes the notification process shown in FIG. 5. The notification process includes a process executed by the first control unit 120 provided in the vehicle 2 and a process executed by the third control unit 80 provided in the mobile terminal 8.

[0070] Step S10: The vehicle information acquisition unit 130 acquires the vehicle information A1. Depending on the type of the vehicle information A1, the vehicle information acquisition unit 130 acquires the vehicle information A1 from one or more of the first camera 10, the radar device 11, the LIDAR 12, the second camera 13, the object recognition device 16, the communication device 20, the vehicle sensor 40, the navigation device 50, the MPU 60, the driving force output device 200, the braking device 210, and the steering device 220.

[0071] For example, if the vehicle information A1 is navigation guidance, the vehicle information acquisition unit 130 acquires the navigation guidance as the vehicle information A1 from the navigation device 50. The navigation guidance is generated by the navigation device 50 based on the position of the vehicle 2 identified by the GNSS receiver 51, the route determined by the route determination unit 53, the first map information 54, and the like. Note that the navigation device 50 may use the results of information processing by the MPU 60 to generate the navigation guidance. Thereafter, the first control unit 120 executes the process of step S20.

[0072] Step S20: The estimation unit 150 estimates whether the behavior of the vehicle 2 will fluctuate by more than a predetermined amount in the near future (for example, in several tens of seconds). The estimation unit 150 makes this estimation based on the vehicle information A1 acquired by the vehicle information acquisition unit 130. For example, the vehicle information A1 is a navigation guidance, and the navigation guidance indicates that the vehicle 2 will turn at an intersection within a predetermined time (for example, in several tens of seconds). In this case, the estimation unit 150 estimates, based on the navigation guidance, that the vehicle 2 will turn at the intersection within the predetermined time, and therefore the acceleration of the vehicle 2 will change by more than a predetermined amount within the predetermined time. Thereafter, the first control unit 120 executes the processing of step S30.

[0073] Step S30: The notification condition determination unit 140 determines whether or not it is estimated that the behavior of the vehicle 2 will fluctuate by a predetermined amount or more. The notification condition determination unit 140 makes the determination based on the estimation result by the estimation unit 150. In the notification process shown in Fig. 5, the notification condition is that it is estimated that the behavior of the vehicle 2 will fluctuate by a predetermined amount or more.

[0074] If the notification condition determination unit 140 determines that the notification condition is satisfied, the first control unit 120 executes the process of step S40. On the other hand, if the notification condition determination unit 140 determines that the notification condition is not satisfied, the vehicle system 1 ends the notification process.

[0075] Step S40: The transmitting unit 160 transmits notification information B1 corresponding to the vehicle information A1 to the mobile terminal 8 using the communication device 20. That is, when it is determined that the notification condition is satisfied, the transmitting unit 160 transmits notification information B1 corresponding to the vehicle information A1 to the mobile terminal 8 using the communication device 20.

[0076] Here, the transmitting unit 160 determines the occupant to be notified. As an example, the transmitting unit 160 determines the mobile terminal (in this embodiment, the mobile terminal 8) with which the communication device 20 has performed Bluetooth (registered trademark) pairing. The transmitting unit 160 determines that the occupant using the determined mobile terminal 8 is the occupant to be notified. Note that it is preferable that the Bluetooth (registered trademark) pairing is completed before the notification process is started. Thereafter, the third control unit 80 executes the process of step S50.

[0077] Step S50: The receiving unit 810 receives the notification information B1 from the transmitting unit 160 using the communication unit 82. After that, the third control unit 80 executes the process of step S60.

[0078] Step S60: The notification unit 820 displays information based on the notification information B1 on the mobile display unit 84. The information based on the notification information B1 is a message or an image based on the notification information B1. For example, the notification unit 820 displays a notification message based on the notification information B1 on the mobile display unit 84. The notification unit 820 may display an image based on the notification information B1 as the information based on the notification information B1 on the mobile display unit 84. The image is, for example, an arrow indicating a right turn.

[0079] The notification unit 820 may notify the occupant by causing the audio output unit 88 to output audio based on the notification information B1. The notification unit 820 may display information based on the notification information B1 on the mobile display unit 84 and cause the audio output unit 88 to output audio based on the notification information B1. This completes the notification process of the vehicle system 1.

[0080] 6 is a diagram showing an example of a notification message based on the notification information B1 according to the present embodiment. In the example shown in FIG. 6, a mobile notification message C1 is displayed on the mobile display unit 84 as the notification message based on the notification information B1.

[0081] The mobile notification message C1 is text that reads, "Turn right in 100 meters ahead." Another example of the mobile notification message C1 is text that reads, "Turn right here." The notification message is displayed, for example, superimposed on the operation screen or content that was displayed on the mobile display unit 84 before the notification message was displayed.

[0082] In the example shown in FIG. 6, a mobile notification message C1 is displayed on the mobile display unit 84, and a notification sound C2 is output from the sound output unit 88 (not shown in FIG. 6).

[0083] Here, another example of a case where it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount will be described. Cases where it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount include a case where vehicle 2 approaches a traffic light. When vehicle 2 stops at a traffic light, it is estimated that the acceleration will change by more than a predetermined amount. In this case, vehicle information acquisition unit 130 acquires, as vehicle information A1, navigation information indicating that vehicle 2 will approach a traffic light from navigation device 50. Estimation unit 150 estimates that the behavior of vehicle 2 will fluctuate by more than a predetermined amount based on the navigation information. Transmission unit 160 transmits, as notification information B1, notification information indicating that vehicle 2 will approach a traffic light to mobile terminal 8 using communication device 20.

[0084] When it is estimated that the behavior of the vehicle 2 will fluctuate by more than a predetermined amount, a warning may be issued from the automatic driving control device 100. The warning is generated by the automatic driving control device 100 based on the ADAS or the automatic driving system. The warning may include, for example, a warning when the inter-vehicle distance is too small, or a warning when the automatic brake is activated.

[0085] When a warning is issued from the automatic driving control device 100, the vehicle information acquisition unit 130 acquires, as vehicle information A1, warning information indicating the warning from the automatic driving control device 100. The estimation unit 150 estimates, based on the warning information, that the behavior of the vehicle 2 will fluctuate by more than a predetermined amount.

[0086] As described above, cases in which it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount include cases in which the automatic brake will be activated. In such cases, vehicle information acquisition unit 130 acquires warning information indicating that the automatic brake will be activated as vehicle information A1. Estimation unit 150 estimates that the behavior of vehicle 2 will fluctuate by more than a predetermined amount based on the warning information. Transmission unit 160 transmits notification information indicating that the automatic brake will be activated to mobile terminal 8 as notification information B1 using communication device 20.

[0087] As described above, cases in which it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount include cases in which the inter-vehicle distance will decrease. In such cases, vehicle information acquisition unit 130 acquires, as vehicle information A1, warning information indicating that the inter-vehicle distance will decrease. Estimation unit 150 estimates, based on the warning information, that the behavior of vehicle 2 will fluctuate by more than a predetermined amount. Transmission unit 160 transmits, as notification information B1, notification information indicating that the inter-vehicle distance will decrease to mobile terminal 8 using communication device 20.

[0088] Cases where it is estimated that the behavior of the vehicle 2 will fluctuate by more than a predetermined amount include cases where the vehicle 2 is traveling on a route with a series of curves. In these cases, the vehicle information acquisition unit 130 acquires, as vehicle information A1, navigation information indicating that the vehicle 2 is traveling on a route with a series of curves from the navigation device 50. The estimation unit 150 estimates that the behavior of the vehicle 2 will fluctuate by more than a predetermined amount based on the navigation information. The transmission unit 160 transmits, as notification information B1, notification information indicating that the vehicle 2 is traveling on a route with a series of curves to the mobile terminal 8 using the communication device 20.

[0089] Cases where it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount include cases where the speed limit will be changed. In such cases, vehicle information acquisition unit 130 acquires, as vehicle information A1, position information of a sign indicating the speed limit from navigation device 50. Estimation unit 150 estimates that the behavior of vehicle 2 will fluctuate by more than a predetermined amount based on the position information of the sign. Transmission unit 160 transmits, as notification information B1, notification information indicating that the speed limit will be changed to mobile terminal 8 using communication device 20.

[0090] Cases where it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount include cases where vehicle 2 will travel through a junction. In such cases, vehicle information acquisition unit 130 acquires, as vehicle information A1, navigation information indicating that vehicle 2 will travel through a junction or that vehicle 2 is traveling through the junction from navigation device 50. Estimation unit 150 estimates that the behavior of vehicle 2 will fluctuate by more than a predetermined amount based on the navigation information. Transmission unit 160 transmits, as notification information B1, notification information indicating that vehicle 2 will travel through a junction to mobile terminal 8 using communication device 20.

[0091] Cases where it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount include cases where vehicle 2 is traveling through a lane merging point. In such cases, vehicle information acquisition unit 130 acquires, as vehicle information A1, navigation information indicating that vehicle 2 will travel through a lane merging point or that vehicle 2 is traveling through the lane merging point from navigation device 50. Estimation unit 150 estimates that the behavior of vehicle 2 will fluctuate by more than a predetermined amount based on the navigation information. Transmission unit 160 transmits, as notification information B1, notification information indicating that vehicle 2 will travel through a lane merging point to mobile terminal 8 using communication device 20.

[0092] Cases where it is estimated that the behavior of vehicle 2 will fluctuate by more than a predetermined amount include cases where the vehicle is approaching a traffic jam start point. In such cases, vehicle information acquisition unit 130 acquires, as vehicle information A1, navigation information indicating that vehicle 2 will approach a traffic jam start point from navigation device 50. Estimation unit 150 estimates that the behavior of vehicle 2 will fluctuate by more than a predetermined amount based on the navigation information. Transmission unit 160 transmits, as notification information B1, notification information indicating that the vehicle is approaching a traffic jam start point to mobile terminal 8 using communication device 20.

[0093] Here, if the occupant is actively looking at the mobile terminal 8, there is a risk of the occupant becoming car sick. In the vehicle system 1, the occupant is notified of a change in the behavior of the vehicle 2 before the behavior of the vehicle 2 changes by a predetermined amount or more. Therefore, in the vehicle system 1, it is possible to prevent the occupant who is looking at the mobile terminal 8 from becoming car sick.

[0094] (First Modification) Next, a modified example of the notification process will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the notification process according to a first modified example of this embodiment. In the notification process shown in Fig. 7, whether to display the information based on the notification information B1 on the mobile display unit 84 or on the in-vehicle display 32 is selected depending on whether the occupant is looking at the mobile terminal 8.

[0095] Note that the processes from step S110 to step S130 and step S150 to step S170 are similar to the processes from step S10 to step S30 and step S40 to step S60 in Fig. 5, and therefore description thereof will be omitted. However, in the notification process shown in Fig. 7, if the notification condition determination unit 140 determines in step S130 that the notification condition is satisfied, the first control unit 120 executes the process of step S140.

[0096] Step S140: The line-of-sight determination unit 170 determines whether or not the occupant using the mobile terminal 8 is looking at the mobile terminal 8. Even when the occupant is holding the mobile terminal 8 in their hand, they may look at an object other than the mobile terminal 8. An object other than the mobile terminal 8 is the interior of the vehicle 2 or the scenery outside. Note that looking at the mobile terminal 8 means looking at the mobile display unit 84 provided on the mobile terminal 8. The line-of-sight determination unit 170 detects the line of sight of the occupant from the image of the occupant's eyes captured by the second camera 13. The line-of-sight determination unit 170 detects the line of sight of the occupant based on image recognition.

[0097] The line-of-sight determination unit 170 may determine whether the occupant is looking at the portable terminal 8 based on whether the occupant is operating the portable terminal 8. In this case, the line-of-sight determination unit 170 acquires the detection result by the operation detection unit 830 from the portable terminal 8. The detection result indicates whether the occupant is operating the portable terminal 8. When the detection result by the operation detection unit 830 indicates that the occupant is operating the portable terminal 8, the line-of-sight determination unit 170 determines that the occupant is looking at the portable terminal 8. On the other hand, when the detection result by the operation detection unit 830 indicates that the occupant is not operating the portable terminal 8, the line-of-sight determination unit 170 determines that the occupant is not looking at the portable terminal 8.

[0098] When the line-of-sight determination unit 170 determines that the occupant using the portable terminal 8 is looking at the portable terminal 8, the first control unit 120 executes the process of step S150. That is, the transmitting unit 160 transmits notification information B1 corresponding to the vehicle information A1 to the portable terminal 8 using the communication device 20. Therefore, when it is determined that the notification condition is satisfied and that the occupant using the portable terminal 8 is looking at the portable terminal 8, the transmitting unit 160 transmits the notification information B1 to the portable terminal 8 using the communication device 20.

[0099] On the other hand, if the line-of-sight determination unit 170 determines that the occupant using the mobile terminal 8 is not looking at the mobile terminal 8, the first control unit 120 executes the process of step S180.

[0100] Step S180: The transmitting unit 160 displays information based on the notification information B1 on the in-vehicle display 32. Therefore, when it is determined that the notification condition is satisfied and that the occupant using the mobile terminal 8 is not looking at the mobile terminal 8, the transmitting unit 160 displays the information based on the notification information B1 on the in-vehicle display 32. Thereafter, the vehicle system 1 ends the notification process.

[0101] Fig. 8 is a diagram showing an example of a notification message displayed on the in-vehicle display 32 according to this modification. In the example shown in Fig. 8, a vehicle notification message C3 is displayed on the in-vehicle display 32 as a notification message based on the notification information B1. The vehicle notification message C3 is text that reads, "Turn right in 100 m ahead."

[0102] According to the first variant, even if the occupant using the mobile terminal 8 is not looking at the mobile terminal 8, information based on the notification information B1 can be displayed on the in-vehicle display 32, thereby preventing the notification from being overlooked.

[0103] (Second Modification) Next, a modified example of the notification process will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the notification process according to a second modified example of this embodiment. Note that the processes in steps S240 and S250 are the same as the processes up to step S40 and step S50 in Fig. 5, and therefore description thereof will be omitted.

[0104] Step S210: The vehicle information acquisition unit 130 acquires vehicle information A1. The vehicle information A1 is, for example, the arrival time of the vehicle 2 at the destination (or the arrival time required to arrive at the destination). The arrival time is generated by the navigation device 50 based on the position of the vehicle 2 identified by the GNSS receiver 51, the route determined by the route determination unit 53, the first map information 54, and the like. Thereafter, the first control unit 120 executes the process of step S220.

[0105] Step S220: The notification condition determination unit 140 acquires the current time. The notification condition determination unit 140 acquires the current time from a clock unit (not shown in FIGS. 2 and 3). Thereafter, the first control unit 120 executes the process of step S230.

[0106] Step S230: The notification condition determination unit 140 determines whether the notification time has arrived. The notification time is the time for notifying the occupants. The notification condition determination unit 140 makes the determination based on the acquired current time and the notification time list. The notification time list indicates one or more notification times. The notification time is, for example, every 30 minutes after the ignition of the vehicle 2 is turned on. The notification time list is stored in a memory unit (not shown in Figures 2 and 3). Thereafter, the first control unit 120 executes the processing of step S230.

[0107] In the notification process shown in FIG. 9, the notification condition is the arrival of the notification time. The notification time may be a time other than a regular time, such as every 30 minutes. The notification time may be determined by predicting the time when the occupant will check information such as the arrival time based on past check times. The past check times are calculated, for example, from a history of operations performed on the HMI 30 to display the information. The prediction of the time when the occupant will check the information is performed, for example, by the first control unit 120 based on AI.

[0108] Step S260: The notification unit 820 displays information based on the notification information B1 on the mobile display unit 84. The notification unit 820, for example, displays the arrival time as information based on the notification information B1 on the mobile display unit 84. The notification unit 820 may notify the occupant by outputting the arrival time as audio to the audio output unit 88. The notification unit 820 may display the arrival time on the mobile display unit 84 and also output the arrival time as audio to the audio output unit 88. With this, the vehicle system 1 ends the notification process.

[0109] In this modification, an example in which the arrival time is notified as the notification information B1 has been described, but this is not limiting. The notification information B1 may be information indicating the destination, information indicating the distance to the destination, or route information. The route information may include points of interest (such as service areas) on the route that the occupant is interested in, or search results for points of interest along the route.

[0110] When route information is notified as notification information B1, vehicle information A1 is, for example, the route information and the current location of vehicle 2. When route information is notified as notification information B1, the notification condition is, for example, that an operation to display the route information is performed on mobile terminal 8. When operation unit 86 accepts the operation, it transmits an operation signal, which is a signal indicating that the operation has been accepted, to vehicle 2 via communication unit 82. When communication device 20 receives the operation signal, notification condition determination unit 140 determines that an operation to display the route information has been performed on mobile terminal 8.

[0111] The notification unit 820 displays the route information transmitted from the vehicle 2 on the mobile display unit 84 by superimposing it on the map displayed on the mobile display unit 84. Therefore, the occupant using the mobile terminal 8 can operate the mobile terminal 8 to display route information generated by the navigation device 50 or the like provided in the vehicle 2 on the mobile display unit 84. Note that even when the arrival time is notified as the notification information B1, the notification condition may be that an operation has been performed on the mobile terminal 8 to display the arrival time.

[0112] As described above, in this modified example, the notification condition is that the notification time arrives or that an operation is performed to send a notification to the mobile terminal 8, and the notification information B1 is information that the occupant wants to know.

[0113] Conventionally, information such as arrival time to a destination or current location has been communicated to passengers when the driver asks. In the vehicle system 1, this information is automatically displayed on the portable terminal 8 used by the passenger. Therefore, in the vehicle system 1, the driver can concentrate on driving, and the passenger can receive the desired information at the desired time.

[0114] 5 and 9, an example in which information based on notification information B1 is displayed on the mobile display unit 84 has been described, but the present invention is not limited to this. Information based on notification information B1 may be displayed on both the mobile display unit 84 and the in-vehicle display 32.

[0115] Depending on the type of notification information B1, the information based on the notification information B1 may be displayed on only one of the mobile display unit 84 and the in-vehicle display 32. In this case, the transmitting unit 160 selects whether to display the information based on the notification information B1 on the mobile display unit 84 or the in-vehicle display 32 depending on the type of notification information B1. Furthermore, when the user of the mobile terminal 8 is the driver, depending on the type of notification information B1, it may be selected whether to display the information based on the notification information B1 on the mobile display unit 84 or the in-vehicle display 32 depending on whether the driver is looking at the mobile terminal 8.

[0116] FIG. 10 shows an example of a display target according to the type of notification information B1. In FIG. 10, vehicle information and notification conditions are shown along with a display target according to the type of notification information B1. For example, arrival time and route information are displayed only on the mobile display unit 84. Navigation guidance is displayed on both the mobile display unit 84 and the in-vehicle display 32. For warnings from the ADAS or autonomous driving system, whether the information based on the notification information B1 is displayed on the mobile display unit 84 or the in-vehicle display 32 is selected depending on whether the driver is looking at the mobile terminal 8.

[0117] In the vehicle system 1, depending on the type of notification information B1, information based on the notification information B1 can be displayed on only one of the mobile display unit 84 and the in-vehicle display 32, thereby allowing information based on the notification information B1 to be displayed on an appropriate display device for notification.

[0118] In the present embodiment, an example in which one notification condition is determined as the notification condition has been described, but the present invention is not limited to this. The notification condition determination unit 140 may determine multiple notification conditions. In this case, the transmission unit 160 transmits notification information B1 corresponding to the notification condition that is satisfied among the multiple notification conditions to the mobile terminal 8 using the communication device 20.

[0119] In addition, in the present embodiment, an example in which the vehicle 2 has an automatic driving function has been described, but this is not limiting. The vehicle 2 does not have to have an automatic driving function. In addition, in the present embodiment, the vehicle system 1 is provided with only one mobile terminal, the mobile terminal 8, but this is not limiting. The vehicle system 1 may be provided with multiple mobile terminals. These multiple mobile terminals are used, for example, by the driver, the passenger in the front seat, and the passenger in the rear seat, respectively.

[0120] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0121] 1 Vehicle Systems 2 vehicles 8. Mobile devices 130 Vehicle information acquisition unit 140 Notification condition determination section 160 Transmission Department 810 Receiving unit 820 Notification Department A1 Vehicle Information B1 Notification information

Claims

1. an on-board device mounted in a vehicle; a mobile terminal used by an occupant of the vehicle; Equipped with The in-vehicle device a vehicle information acquisition unit that acquires vehicle information related to the vehicle; a notification condition determination unit that determines whether a notification condition for notifying the occupant is satisfied; a transmitting unit that transmits notification information corresponding to the vehicle information to the mobile terminal using a communication device when it is determined that the notification condition is satisfied; a line-of-sight determination unit that determines whether the occupant is looking at the mobile terminal; A second display unit; Equipped with The mobile terminal a receiving unit that receives the notification information from the transmitting unit using a communication device; a notification unit that notifies the occupant based on the notification information; A first display unit; Equipped with the notification unit notifies the occupant by causing the first display unit to display information based on the notification information; the transmitting unit, when it is determined that the notification condition is satisfied and that the occupant is looking at the mobile terminal, transmits the notification information to the mobile terminal using a communication device; When it is determined that the notification condition is satisfied and the occupant is not looking at the mobile terminal, the transmitting unit causes the second display unit to display information based on the notification information. Vehicle systems.

2. an on-board device mounted in a vehicle; a mobile terminal used by an occupant of the vehicle; Equipped with The in-vehicle device a vehicle information acquisition unit that acquires vehicle information related to the vehicle; a notification condition determination unit that determines whether a notification condition for notifying the occupant is satisfied; a transmitting unit that transmits notification information corresponding to the vehicle information to the mobile terminal using a communication device when it is determined that the notification condition is satisfied; Equipped with The mobile terminal a receiving unit that receives the notification information from the transmitting unit using a communication device; a notification unit that notifies the occupant based on the notification information; A first display unit; Equipped with the notification unit notifies the occupant by causing the first display unit to display information based on the notification information; The transmitting unit selects whether to display information based on the notification information on the first display unit or on a second display unit provided in the vehicle, depending on the type of the notification information. Vehicle systems.

3. The mobile terminal further includes an audio output unit, The notification unit notifies the occupant by causing the audio output unit to output a sound based on the notification information.

3. The vehicle system according to claim 1 or 2.

4. the on-vehicle device further includes an estimation unit that estimates whether or not a behavior of the vehicle will fluctuate by a predetermined amount or more based on the vehicle information; The notification condition is that the behavior of the vehicle is estimated to fluctuate by a predetermined amount or more. The vehicle system according to any one of claims 1 to 3.

5. If it is predicted that the vehicle behavior will fluctuate more than a predetermined amount, automatic braking may be activated. The vehicle system of claim 4 .

6. The case where the behavior of the vehicle is estimated to fluctuate by more than a predetermined amount includes a case where the vehicle is traveling on a route with a series of curves.

6. The vehicle system according to claim 4 or claim 5.

7. When it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount, the speed limit may be changed.

7. A vehicle system according to any one of claims 4 to 6.

8. Cases where it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount include cases where the vehicle is traveling through a junction. A vehicle system according to any one of claims 4 to 7.

9. The case where the behavior of the vehicle is estimated to fluctuate by more than a predetermined amount includes a case where the vehicle is traveling at a lane merging point. A vehicle system according to any one of claims 4 to 8.

10. When it is estimated that the behavior of the vehicle will fluctuate by more than a predetermined amount, it includes a case where the vehicle is approaching a traffic jam start point. A vehicle system according to any one of claims 4 to 9.

11. The notification condition is that a notification time has arrived or that an operation for sending a notification has been performed on the mobile terminal, The notification information is information that the occupant wants to know. A vehicle system according to any one of claims 4 to 10.

12. An in-vehicle device mounted on a vehicle acquires vehicle information related to the vehicle, the in-vehicle device determines whether a notification condition for notifying an occupant of the vehicle is satisfied; The in-vehicle device determines whether the occupant is looking at the mobile terminal; When the in-vehicle device determines that the notification condition is satisfied and that the occupant is looking at the mobile terminal, the in-vehicle device transmits notification information corresponding to the vehicle information to the mobile terminal using a communication device; the mobile terminal receives the notification information from the in-vehicle device using a communication device; the portable terminal notifies the occupant by displaying information based on the notification information on a first display unit of the portable terminal; When the in-vehicle device determines that the notification condition is satisfied and that the occupant is not looking at the mobile terminal, the in-vehicle device displays information based on the notification information on a second display unit provided in the in-vehicle device. Notification methods, including:

13. An in-vehicle device mounted on a vehicle acquires vehicle information related to the vehicle, the in-vehicle device determines whether a notification condition for notifying an occupant of the vehicle is satisfied; When it is determined that the notification condition is satisfied, the in-vehicle device transmits notification information corresponding to the vehicle information to a mobile terminal used by an occupant of the vehicle using a communication device; the mobile terminal receives the notification information from the in-vehicle device using a communication device; the portable terminal notifies the occupant by displaying information based on the notification information on a first display unit of the portable terminal; The in-vehicle device selects whether to display information based on the notification information on the first display unit or on a second display unit provided in the vehicle, depending on the type of the notification information. Notification methods, including:

Images