Method and in-vehicle device

Abstract

To provide a method for improving a technique for transmitting notification information to the outside of a vehicle.SOLUTION: The in-vehicle device estimates the intention of the driver based on at least one of the in-vehicle sensor information, the operation information of the driver, and the driving information. Then, the in-vehicle device notifies the outside of the vehicle of output information associated with the intention via one or more output devices.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a method, an in-vehicle device. [Background technology]

[0002] Conventionally, there are known techniques for transmitting notification information to the outside of a vehicle. For example, Patent Document 1 discloses a technique in which an in-vehicle device predicts the future behavior of a pedestrian (for example, crossing the crosswalk), determines an action to suggest to the pedestrian (for example, having the pedestrian cross the crosswalk before the vehicle), and notifies the outside of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-026201 Summary of the Invention [Problem to be solved by the invention]

[0004] There was room for improvement in the technology for sending notification information outside the vehicle.

[0005] In view of the above circumstances, an object of the present disclosure is to improve the technology for transmitting notification information to the outside of a vehicle. [Means for solving the problem]

[0006] According to one embodiment of the present disclosure, a method comprises: A method executed by an on-board device mounted in a vehicle, estimating the driver's intention based on at least one of in-vehicle sensor information, driver operation information, and driving information; notifying the outside of the vehicle of output information according to the intention via one or more output devices; Includes.

[0007] An in-vehicle device according to an embodiment of the present disclosure includes: An in-vehicle device mounted on a vehicle, The in-vehicle device includes a control unit, an output device, Equipped with The control unit Inferring the driver's intention based on at least one of in-vehicle sensor information, driver operation information, and driving information; notifying the one or more output devices of output information according to the intention; The one or more output devices notify the output information to the outside of the vehicle. [Effects of the Invention]

[0008] According to one embodiment of the present disclosure, a technique for transmitting notification information to the outside of a vehicle is improved. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of an in-vehicle device according to an embodiment of the present disclosure. [Figure 2] 4 is a flowchart showing the operation of the in-vehicle device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described.

[0011] (Outline of the embodiment) An overview of an in-vehicle device 10 according to an embodiment of the present disclosure will be described with reference to FIG. 1. The in-vehicle device 10 is equipment mounted on a vehicle. The vehicle may be any vehicle in which a person can ride, including, but not limited to, a passenger car, a small truck, or a large truck. The vehicle may also be, for example, a gasoline-powered vehicle, an electric vehicle (BEV; Battery Electric Vehicle), a hybrid vehicle (HEV; Hybrid Electric Vehicle), a plug-in hybrid electric vehicle (PHEV; Plug-in Hybrid Electric Vehicle), or a fuel cell electric vehicle (FCEV; Fuel Cell Electric Vehicle).

[0012] First, an outline of this embodiment will be described, and details will be described later. The in-vehicle device 10 estimates the driver's intention based on at least one of in-vehicle sensor information, driver operation information, and driving information. The in-vehicle device 10 then notifies the outside of the vehicle of output information associated with the driver's intention via one or more output devices.

[0013] The above-mentioned conventional technology relates to autonomous vehicles, and therefore may not match the intentions of a driver in a manned vehicle. In contrast, according to this embodiment, output information associated with the driver's intentions estimated based on at least one of on-board sensor information, driver operation information, and driving information is notified to the outside of the vehicle via one or more output devices. This increases the likelihood that information consistent with the intentions of the driver of a manned vehicle can be notified to the outside of the vehicle. Therefore, according to this embodiment, the technology for transmitting notification information to the outside of the vehicle is improved in that it increases the likelihood that information consistent with the intentions of the driver of a manned vehicle can be notified to the outside of the vehicle.

[0014] Next, the configuration of the in-vehicle device 10 will be described in detail.

[0015] (Configuration of in-vehicle device) The in-vehicle device 10 includes a communication unit 11 , an output unit 12 , an input unit 13 , a storage unit 14 , a control unit 15 , a sensor unit 16 , and an imaging unit 17 .

[0016] The communication unit 11 includes one or more communication interfaces for connecting to a network. The communication interfaces are compatible with mobile communication standards such as, but not limited to, 4G (4th Generation) or 5G (5th Generation). In this embodiment, the in-vehicle device 10 communicates with a server that provides information about the surroundings of the vehicle, such as a Vehicle Information and Communication System (VICS) (registered trademark), via the communication unit 11 and the network.

[0017] The output unit 12 includes one or more output devices that output information. The output devices include, but are not limited to, at least one of a display that outputs information as a video, a speaker that outputs information as a sound, a robot arm that performs gestures to communicate information to the outside, a holographic display device that displays a hologram, a projector that projects an image, and a lamp that issues a signal. Alternatively, the output unit 12 may include an interface for connecting an external output device.

[0018] The input unit 13 includes one or more input devices that detect an input operation by a user. Examples of the input devices include, but are not limited to, physical keys, capacitive keys, a mouse, a touch panel, a touch screen integrated with the display of the output unit 12, or a microphone. Alternatively, the input unit 13 may include an interface for connecting an external input device.

[0019] The storage unit 14 includes one or more memories. The memories may be, for example, semiconductor memories, magnetic memories, optical memories, etc., but are not limited to these. Each memory included in the storage unit 14 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 14 stores any information used in the operation of the in-vehicle device 10. For example, the storage unit 14 may also store system programs, application programs, embedded software, etc. For example, the information stored in the storage unit 14 may be updated with information obtained from a network via the communication unit 11. The storage unit 14 may store map information and / or road surface information. The storage unit 14 may also store information about the surroundings of the vehicle provided from a server such as the Vehicle Information and Communication System (VICS) (registered trademark).

[0020] The map information may include any road information, for example, including but not limited to, road junctions. The map information or road surface information may include any road surface information, for example, including but not limited to, crosswalks. The map information, road surface information, or information about the vicinity of the vehicle may include information about road conditions, such as past accident information, traffic congestion information, and road construction, provided from a server such as VICS (registered trademark). The map information, road surface information, and / or information about the vicinity of the vehicle may not necessarily be stored in the memory unit 14, and may be acquired by the control unit 15 from a server such as VICS (registered trademark) on an as-needed basis.

[0021] The control unit 15 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor may be, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for a specific process, but is not limited to these. The programmable circuit may be, for example, but is not limited to, an FPGA (Field-Programmable Gate Array). The dedicated circuit may be, for example, but is not limited to, an ASIC (Application Specific Integrated Circuit). The control unit 15 controls the operation of the in-vehicle device 10.

[0022] The sensor unit 16 includes any sensors for acquiring driving information, such as a steering angle sensor, an acceleration sensor, a vehicle speed sensor, a position sensor, and a gyro sensor. The sensor unit 16 may also include, for example, a GPS (Global Positioning System) receiver. The sensor unit 16 includes any sensors for acquiring driver operation information, such as a camera that detects the driver's operation and / or the direction of the driver's face. The sensor unit 16 also includes sensors necessary for acquiring in-vehicle sensor information, such as a lidar or millimeter-wave radar that detects objects present around the vehicle, such as pedestrians and nearby vehicles, and a camera that acquires information about the state of the object, such as the object's speed, direction, and other movement information.

[0023] "In-vehicle sensor information" may include, for example, information on pedestrians and surrounding vehicles, but may also include information on objects present around the vehicle, without being limited to these. Note that the in-vehicle sensor information may also include information on the state of the object, such as the object's movement, such as its speed, and its orientation. Furthermore, the pedestrian information may also include information indicating the pedestrian's age group, such as child, elderly, etc. Furthermore, the surrounding vehicle information may also include information indicating the type of vehicle, such as passenger car, motorcycle, large truck, etc., and information on marks indicating the driver's status, such as a beginner mark, an elderly person mark, etc.

[0024] "Nearby vehicles" include, for example, merging vehicles entering one lane or road from a different lane or road, vehicles behind the vehicle, and oncoming vehicles traveling in the opposite direction to the vehicle, but are not limited to these and may also include any vehicles present around the vehicle.

[0025] "Driver's movement information" may include, for example, information regarding any movement of the driver, including, but not limited to, information regarding at least one movement of the driver's hands, eyes, and face.

[0026] "Driving information" may include, for example, information regarding the vehicle's speed, accelerator position, brake position, turn signals, hazard lights, and brake lights, but may also include any information regarding the driver's driving operations, without being limited to these.

[0027] The photographing unit 17 includes one or more cameras capable of photographing the surroundings. For example, the photographing unit 17 may be used in place of the sensor unit 16 to detect objects present around the vehicle, such as pedestrians and nearby vehicles. The photographing unit 17 may also be used to photograph the rear view projected on the rearview mirror and detect vehicles behind. The photographing unit 17 may also be used to detect whether nearby vehicles have beginner or senior citizen marks. The photographing unit 17 may also be used to obtain sign information.

[0028] (Operation flow of the in-vehicle device) The operation of the in-vehicle device 10 according to this embodiment will be described with reference to FIG.

[0029] S100: The control unit 15 of the in-vehicle device 10 estimates the driver's intention based on at least one of the in-vehicle sensor information, the driver's operation information, and the driving information.

[0030] For example, assume that the in-vehicle sensor information is information about a nearby vehicle, and the nearby vehicle is a merging vehicle. In this case, the control unit 15 may check the driver's behavior information and / or driving information. Then, for example, if the behavior information indicates that the driver's palm is facing up and / or the driving information indicates that the accelerator opening of the host vehicle is decreasing, the control unit 15 may infer that the driver's intention is to "yield to the merging vehicle."

[0031] For example, assume that the in-vehicle sensor information is information about a pedestrian. In this case, the control unit 15 may check the driver's behavior information. For example, if the behavior information indicates that the driver's palms are facing up and / or that the driver's eyes or face are facing toward the pedestrian, the control unit 15 may infer that the driver's intention is to "yield to the pedestrian." Alternatively, the control unit 15 may check the driving information. For example, if the driving information indicates that the accelerator opening is reduced after the vehicle has approached a predetermined distance to a crosswalk and that the vehicle has stopped, the control unit 15 may infer that the driver's intention is to "yield to the pedestrian."

[0032] For example, suppose the in-vehicle sensor information is information about a nearby vehicle, and the nearby vehicle is a rearward vehicle. In this case, the control unit 15 may check the driver's behavior information and driving information. For example, suppose the behavior information indicates that the driver's line of sight or face is facing the rearview mirror of the vehicle, and the driving information indicates that the accelerator pedal position is decreased. In this case, the control unit 15 may infer that the driver's intention is to "urge the rearward vehicle to slow down." Alternatively, if the in-vehicle sensor information is about a rearward vehicle, the control unit 15 may check the brake position information of the vehicle included in the driving information. For example, if the brake position information indicates that the brake position is increased, and the relative speed of the rearward vehicle relative to the vehicle is equal to or greater than a predetermined value, the control unit 15 may infer that the driver's intention is to "urge the rearward vehicle to slow down." The relative speed of the rearward vehicle may be calculated using information about the speed of the vehicle and information about the speed of the rearward vehicle.

[0033] For example, suppose that the in-vehicle sensor information is information about a nearby vehicle, and the nearby vehicle is an oncoming vehicle. In this case, the control unit 15 may check the driver's behavior information. Then, for example, if the driver's behavior information indicates that the palm of the driver's hand facing the oncoming vehicle is facing up or that the driver's line of sight or face is facing the oncoming vehicle, the control unit 15 may infer that the driver's intention is to "yield to the oncoming vehicle."

[0034] The control unit 15 may start acquiring in-vehicle sensor information based on map information or road surface information. For example, the control unit 15 may start acquiring information about surrounding vehicles when it detects from the map information stored in the memory unit 14 that there is a merging point ahead of the vehicle. Alternatively, for example, the control unit 15 may start acquiring information about pedestrians when it detects from the map information or road surface information stored in the memory unit 14 that there is a crosswalk.

[0035] The "driver's intention" may be any driver's intention estimated based on at least one of on-board sensor information, driver operation information, and driving information, and is not limited to the above-mentioned "yield to merging vehicles," "yield to pedestrians," "encourage rear vehicles to slow down," and "yield to oncoming vehicles."

[0036] S101: The control unit 15 determines the timing of notification of output information according to the driver's intention, based on at least one of information on surrounding vehicles, traffic congestion information, and road conditions.

[0037] For example, if the driver intends to "yield to a merging vehicle," the control unit 15 may check whether the merging vehicle has a beginner's mark or an elderly person's mark attached to it. If the merging vehicle has a beginner's mark or an elderly person's mark attached to it, the control unit 15 may determine to notify the output information a predetermined time earlier than usual.

[0038] For example, if the driver's intention is to "encourage the vehicle behind to slow down," the control unit 15 may check whether the point where the driver's intention is estimated corresponds to a location where troubles such as congestion information and accident information frequently occur, based on map information, road surface information, or traffic congestion information and accident information included in information about the vicinity of the vehicle. If the point corresponds to a location where troubles such as traffic congestion information and accident information frequently occur, the control unit 15 may decide to notify the output information a predetermined time earlier than usual.

[0039] For example, if the driver's intention is to "yield to oncoming vehicles," the control unit 15 may confirm whether the point where the driver's intention is estimated corresponds to a location where road conditions such as road construction are being carried out, based on information on road conditions included in map information, road surface information, or information about the vicinity of the vehicle. Alternatively, the control unit 15 may confirm whether the location corresponds to a location where road conditions such as road construction are being carried out, by using the image capture unit 17 to confirm whether information about construction is present on surrounding signs, or whether the road ahead of the vehicle is blocked by a triangular pole or the like. If the location corresponds to a location where road conditions such as road construction are being carried out, the control unit 15 may then decide to notify the output information a predetermined time earlier than usual.

[0040] "Output information according to the driver's intention" (hereinafter referred to as output information) is information for conveying the driver's intention to a target party. For example, if the driver's intention is "to yield to a merging vehicle," the output information may be "please go ahead." For example, if the driver's intention is "to yield to a pedestrian," the output information may be "please go ahead." For example, if the driver's intention is "to request a rear vehicle to slow down," the output information may be "please slow down." For example, if the driver's intention is "to yield to an oncoming vehicle," the output information may be "please go ahead." Note that the output information is not limited to the above, and may be any information that conveys the driver's intention to a target party. Furthermore, the output information is not limited to language, and may include information that can convey the driver's intention that cannot be expressed in language to a target party, such as an image that reflects the driver's intention, the degree of emphasis of a notification, etc.

[0041] The output information may also be changed based on at least one of information about surrounding vehicles, information about the driver's behavior, information about past accidents, driving information, traffic congestion information, and road conditions. For example, if the driver's intention is to "encourage the vehicle behind to slow down," the control unit 15 may calculate the relative speed of the vehicle behind from information about the speed of the vehicle and information about the speed of the vehicle behind. If the relative speed of the vehicle behind is equal to or greater than a predetermined value, the control unit 15 may change the output information from "Please slow down" to "Danger! Please slow down." Alternatively, the faster the relative speed of the vehicle behind is, the more emphasized the notification of the output information may be.

[0042] Furthermore, for example, if the driver's intention is to "yield to pedestrians," the control unit 15 may check whether the location and time at which the driver's intention is estimated corresponds to a time period with a high accident rate based on past accident information. Alternatively, the control unit 15 may check the information about the pedestrian and determine whether the pedestrian is a child or an elderly person. If so, the control unit 15 may increase the degree of emphasis of the notification of the output information.

[0043] The degree of emphasis of the notification of the output information may be expressed in a numerical format using integers in any range, such as 1 to 5, in order of decreasing emphasis. Also, for example, the degree of emphasis of the notification of the output information may be expressed in an alphabetical rank format, such as S, A, B, C, D, in order of decreasing emphasis, instead of a number.

[0044] S102: The control unit 15 determines at least one output device to be used from among one or more output devices based on at least one of the in-vehicle sensor information and the time period.

[0045] For example, if the on-board sensor information is information about a nearby vehicle, and the nearby vehicle is a merging vehicle, the control unit 15 may determine to use at least one of a display, a robot arm, a holographic display device, and a projector that are provided on the side of the vehicle facing the merging vehicle so as to be able to notify the output information.

[0046] For example, if the in-vehicle sensor information is information about a pedestrian, the control unit 15 may determine to use at least one of a display, a robot arm, a hologram display device, a projector, a lamp, and a speaker that are provided on the front of the vehicle so as to be able to notify the output information.

[0047] For example, if the on-board sensor information is information about a nearby vehicle, and the nearby vehicle is an oncoming vehicle, the control unit 15 may determine to use at least one of a display, a robot arm, a hologram display device, a projector, and a lamp that are provided behind the vehicle so as to be able to notify the output information.

[0048] For example, if the on-board sensor information is information about a nearby vehicle and the nearby vehicle is an oncoming vehicle, the control unit 15 may determine to use at least one of a display, a robot arm, a hologram display device, a projector, and a lamp that are provided on the front of the vehicle so as to be able to notify the output information.

[0049] For example, when the time zone is daytime, the control unit 15 may decide not to use the hologram display device.

[0050] Furthermore, control unit 15 may determine the number of output devices to use depending on the degree of notification emphasis included in the output information. For example, assume that the degree of notification emphasis is set to 1 during a time period when no accidents have occurred at the point where the driver's intention is estimated, and the degree of notification emphasis is set to 3 during a time period when the accident rate is high. In this case, for example, control unit 15 may determine to use only the hologram display device when the degree of notification emphasis is 1, and may determine to use both the hologram display device and the robot arm when the degree of notification emphasis is 3.

[0051] S103: The control unit 15 notifies the output information to the outside of the vehicle via at least one output device.

[0052] Specifically, the control unit 15 notifies the output information to the outside of the vehicle via at least one output device determined in S102, based on the notification timing determined in S101.

[0053] For example, if it is determined in S102 that a display will be used, the control unit 15 may notify the outside of the vehicle of the output information by displaying an image using the display. For example, if it is determined in S102 that a speaker will be used, the control unit 15 may notify the outside of the vehicle of the output information by audio output using the speaker. For example, if it is determined in S102 that a robot arm will be used, the control unit 15 may notify the outside of the vehicle of the output information by a gesture using the robot arm. For example, if it is determined in S102 that a hologram display device will be used, the control unit 15 may notify the outside of the vehicle of the output information by displaying a hologram using the hologram display device. For example, if it is determined in S102 that a projector will be used, the control unit 15 may notify the outside of the vehicle of the output information by projecting an image using the projector. For example, if it is determined in S102 that a lamp will be used, the control unit 15 may notify the outside of the vehicle of the output information by a signal using the lamp.

[0054] The signal using the lamps can be any method that transmits notification information to the target party. For example, suppose that the in-vehicle sensor information is information about a surrounding vehicle, the surrounding vehicle is an oncoming vehicle, and the output information is "please go ahead." In this case, the control unit 15 may momentarily flash only the headlights of the oncoming vehicle, making it appear as if the headlights are winking.

[0055] Furthermore, the control unit 15 may change the manner in which the output information is notified to the outside of the vehicle based on the degree of emphasis of the notification of the output information. For example, when displaying an image using a display, the control unit 15 may change the color of the text to black when the degree of emphasis of the notification is 1, and change the color of the text to red and flash when the degree of emphasis of the notification is 5.

[0056] Changes in the form of the output information may include, but are not limited to, the color of the text, the font of the text, the brightness of the image, the addition of an image, the three-dimensionalization of the image, the volume of the audio output, and the volume of the gesture movement.

[0057] As described above, the in-vehicle device 10 estimates the driver's intention based on at least one of the in-vehicle sensor information, the driver's operation information, and the driving information. Then, the in-vehicle device 10 notifies the outside of the vehicle of output information associated with the driver's intention via one or more output devices.

[0058] According to this embodiment, output information associated with the driver's intention estimated based on at least one of on-board sensor information, driver operation information, and driving information is notified to the outside of the vehicle via one or more output devices. This increases the likelihood that information consistent with the driver's intention of the manned vehicle can be notified to the outside of the vehicle. Therefore, this embodiment improves the technology for transmitting notification information to the outside of the vehicle in that it increases the likelihood that information consistent with the driver's intention of the manned vehicle can be notified to the outside of the vehicle.

[0059] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.

[0060] Also, for example, an embodiment is possible in which a general-purpose computer functions as the in-vehicle device 10 according to the above-described embodiment. Specifically, a program describing the processing content for realizing each function of the in-vehicle device 10 according to the above-described embodiment is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor or a non-transitory computer-readable medium storing the program. [Explanation of symbols]

[0061] 10 Onboard equipment 11 Communications Department 12 Output section 13 Input section 14 Storage section 15 Control Unit 16 Sensor section 17 Photography Department

Claims

1. A method executed by an on-board device mounted in a vehicle, estimating the driver's intention based on at least one of in-vehicle sensor information, driver operation information, and driving information; notifying the outside of the vehicle of output information according to the intention via one or more output devices; A method comprising:

2. 10. The method of claim 1, determining a notification timing of the output information based on at least one of information on surrounding vehicles, traffic congestion information, and road conditions; Further comprising: The in-vehicle device notifies the output information based on the notification timing.

3. 10. The method of claim 1, determining at least one output device to be used from among the one or more output devices based on at least one of the on-board sensor information and the time period; Further comprising: The in-vehicle device communicates the output information via the at least one output device.

4. 10. The method of claim 1, the output device includes at least one of a display, a speaker, a robot arm, a hologram display device, a projector, and a lamp mounted on the vehicle; The in-vehicle device notifies the outside of the vehicle of the output information by at least one of image display using the display, audio output using the speaker, gestures using the robot arm, hologram display using the hologram display device, image projection using the projector, and signals using the lamp.

5. An in-vehicle device mounted on a vehicle, The in-vehicle device includes a control unit, an output device, Equipped with The control unit Inferring the driver's intention based on at least one of in-vehicle sensor information, driver operation information, and driving information; notifying the one or more output devices of output information according to the intention; The one or more output devices notify the output information to an outside of the vehicle.

Images