Driver assistance device, driver assistance method, and program

Abstract

To provide a driver assistance device, driver assistance method, and program that reduce the occurrence of accidents by enabling the driver to more accurately recognize the planned operation information entered by the driver. [Solution] A driving support device comprising: an operation schedule receiving unit that receives operation schedule information of a mobile vehicle planned by the driver from the driver of the mobile vehicle; and a notification content determination unit that, based on the operation schedule information, randomly selects at least one operation expression from a plurality of expression candidates relating to the operation schedule information and determines the notification content to be sent to the driver, including the operation expression.

Description

Technical Field

[0001] The present invention relates to a driving support device, a driving support method, and a program.

Background Art

[0002] In driving an automobile, a driver simultaneously performs many operations such as adjusting the accelerator and brakes, checking the traveling direction, operating the direction indicator, and operating the steering wheel. In order to support such complicated operations of the driver, various technologies have been developed, such as simplifying the operations and presenting methods of information necessary for judgment during driving. On the other hand, many accidents occur due to misoperations of the driver, and among driving support technologies, accident prevention technologies due to misoperations are highly important.

[0003] Patent Document 1 discloses a vehicle driving support device that requests a vehicle driver to voice the set position of the shift lever after change or the traveling direction of the vehicle intended by the vehicle driver, and prohibits the vehicle from starting when the content voiced by the driver does not match the set position of the shift lever after change or the traveling direction of the vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The vehicle driving support device described in Patent Document 1 prevents the vehicle from starting in a direction not intended by the driver by requesting the driver to voice the direction in which the driver is trying to start the vehicle. However, the vehicle driving support device described in Patent Document 1 does not prompt the driver to recognize the content of their own voice. Therefore, it cannot prevent malfunction when the driver makes a mistake without paying attention to the voice.

[0006] In light of the aforementioned issues, this disclosure provides a driver assistance device, etc., that reduces the occurrence of accidents by enabling the driver to more accurately recognize the planned operation information entered by the driver. [Means for solving the problem]

[0007] A driver assistance device according to one aspect of this disclosure comprises an operation schedule reception unit and a notification content determination unit. The operation schedule reception unit receives operation schedule information of the mobile vehicle from the driver of the mobile vehicle. Based on the operation schedule information, the notification content determination unit randomly selects at least one operation expression from a plurality of candidate expressions relating to the operation schedule information and determines the notification content to be sent to the driver, including the operation expression.

[0008] A driving assistance method according to one aspect of this disclosure causes a computer to perform the following processes: The computer receives information from the driver of the mobile vehicle about the driver's planned operation of the mobile vehicle. Based on the planned operation information, the computer randomly selects at least one operation expression from a plurality of candidate expressions relating to the planned operation information and determines the content of the notification to the driver, including the operation expression.

[0009] A program according to one aspect of this disclosure causes a computer to perform the following driving assistance method: The computer receives information from the driver of a mobile vehicle about the driver's planned operation of the mobile vehicle. Based on the planned operation information, the computer randomly selects at least one operation expression from a plurality of candidate expressions relating to the planned operation information and determines the content of the notification to the driver, including the operation expression. [Effects of the Invention]

[0010] According to this, it is possible to provide a driver assistance device, driver assistance method, and program that reduce the occurrence of accidents by allowing the driver to more accurately recognize the planned operation information entered by the driver. [Brief explanation of the drawing]

[0011] [Figure 1]This is a block diagram of the driver assistance device according to Embodiment 1. [Figure 2] This is a flowchart of the driver assistance device according to Embodiment 1. [Figure 3] This is a block diagram of the driver assistance device according to Embodiment 2. [Figure 4] This is a schematic diagram of the method for determining the content of notifications using the voice output of the driver assistance device according to Embodiment 2. [Figure 5] This is a schematic diagram of the method for determining the notification mode using the voice output of the driver assistance device according to Embodiment 2. [Figure 6] This is a flowchart of the driving assistance device according to Embodiment 2. [Figure 7] This is a block diagram of the driver assistance device according to Embodiment 3. [Figure 8] This is a flowchart of the driver assistance device according to Embodiment 3. [Figure 9] This is a schematic diagram of the operating environment of the driver assistance device according to Embodiment 4. [Figure 10] This is a flowchart of the driver assistance device according to Embodiment 4. [Figure 11] This is a block diagram illustrating the hardware configuration of a computer. [Modes for carrying out the invention]

[0012] The present invention will be described below through embodiments of the invention, but the invention claimed is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means of solving the problem. For clarity of explanation, the following descriptions and drawings have been omitted and simplified as appropriate. In each drawing, the same elements are denoted by the same reference numerals, and redundant explanations have been omitted where necessary.

[0013] <Embodiment 1> Embodiment 1 according to the present disclosure will be described with reference to FIG. 1. FIG. 1 is a block diagram of the driving support device 10 according to Embodiment 1. The driving support device 10 is a device that supports a driver with respect to the driving of a moving body. Here, the moving body is typically a four-wheel automobile, but is not limited thereto, and includes various moving bodies such as other two-wheel vehicles, railway vehicles, airplanes, ships, and electric mobility that are driven by a driver.

[0014] The driving support device 10 first receives, from the driver, operation schedule information indicating the content of an operation having the intention executed by the driver, independently of the operation executed by the driver with respect to the moving body regarding the driving of the moving body. Then, based on the received operation schedule information, the driving support device 10 notifies the driver of the operation schedule content in a representation different from the representation shown in this operation schedule information. According to this, the driving support device 10 can effectively cause the driver to recognize the operation schedule content input by the driver himself / herself. Therefore, the driver can accurately confirm again the schedule of the operation input by the driver himself / herself, and for example, can prevent performing an inappropriate operation unconsciously due to routinization of work or the like. The driving support device 10 mainly includes an operation schedule reception unit 101 and a notification content determination unit 102.

[0015] The operation schedule reception unit 101 receives operation schedule information of the moving body scheduled by the driver from the driver of the moving body. The operation schedule information is, for example, information indicating the traveling direction of the moving body to be scheduled. Specifically, the operation schedule information is information input in a direction using the driver's utterance such as "forward", or a button or a touch panel. The operation schedule reception unit 101 may be provided with an input interface such as a microphone, a button, a touch panel, or a camera in order to receive the operation schedule information. The operation schedule reception unit 101 may be connected to the input interface and receive the information input to the input interface. Also, communication data in the case of inputting a direction with a remote control, a smartphone, or the like may be received.

[0016] The operation schedule reception unit 101 receives operation schedule information, for example, as the driver's voice or input on a touch panel. Further, the operation schedule reception unit 101 may receive the operation schedule information as image information or video information including motion information such as gestures like the driver's pointing. Note that the operation schedule reception unit 101 may receive operation schedule information regarding the route direction to the left and right from the operation of the direction indicator.

[0017] Based on the operation schedule information, the notification content determination unit 102 irregularly selects at least one operation expression from a plurality of expression candidates regarding the operation schedule information, and determines the notification content to the driver including the operation expression. For example, when the notification content determination unit 102 receives the driver's utterance of "forward" as operation schedule information, it irregularly selects one of a plurality of expression candidates such as "ahead", "advance", "forward", "front", and "drive position" used in the same meaning as "forward", and determines the notification content to the driver including the operation expression.

[0018] First, based on the operation schedule information received by the operation schedule reception unit 101, the notification content determination unit 102 acquires a plurality of expressions used in the same meaning as the operation schedule information as expression candidates. The notification content determination unit 102 may have the expression candidates in a database. The notification content determination unit 102 may connect to an external storage device that is a database of expression candidates and read out the expression candidates. The notification content determination unit 102 may also generate expression candidates using machine learning or an external generator.

[0019] Next, the notification content determination unit 102 irregularly selects at least one of the plurality of expression candidates as an operation expression. Further, the notification content determination unit 102 determines the content to be notified to the driver as the notification content including the selected operation expression. Here, irregular selection means any selection method that the driver does not immediately feel regularity. Therefore, irregular selection may be processed without using a random number. Also, irregular selection may be a selection method having long-term regularity or a selection method that continuously selects the same operation expression at a low frequency. Note that irregular selection may sometimes be expressed as random selection.

[0020] Next, with reference to Figure 2, the processing flow of the driver assistance device 10 will be described. Figure 2 is a flowchart of the driver assistance device 10 according to Embodiment 1. The sequence of flows of the driver assistance device 10 includes steps S11 and S12. The driver assistance device 10 continuously processes the sequence of flows in a loop while the driver operates the mobile vehicle.

[0021] In step S11, the operation schedule reception unit 101 receives information from the driver of the mobile vehicle about the planned operation of the mobile vehicle. The planned operation information is, for example, the planned direction of travel of the mobile vehicle.

[0022] In step S12, the notification content determination unit 102 randomly selects at least one operation expression from a plurality of expression candidates related to the planned operation information, based on the planned operation information, and determines the notification content to be sent to the driver, including the operation expression. First, the notification content determination unit 102 acquires a plurality of expressions used with the same meaning as the planned operation information as expression candidates, based on the planned operation information received by the planned operation reception unit 101. Next, the notification content determination unit 102 randomly selects at least one of the plurality of expression candidates as the operation expression. Furthermore, the notification content determination unit 102 determines the notification content to be sent to the driver, including the selected operation expression. The series of flows of the driver assistance device 10 ends with the completion of step S12.

[0023] According to this, the driver assistance method performed by the driver assistance device 10 can notify the driver of the planned operation using an operation expression that is a rephrased version of the expression entered by the driver. Therefore, the driver can accurately recognize the planned operation information that they themselves entered through the rephrased operation expression. As a result, the driver assistance device 10 can reduce the occurrence of accidents due to errors by making the driver more accurately recognize the planned operation information that they themselves entered.

[0024] The driver assistance device 10 may also have a processor and a memory device, although these are not shown in the diagram. The memory device of the driver assistance device 10 may include, for example, a memory device that includes non-volatile memory such as flash memory or an SSD (Solid State Drive). In this case, the memory device stores a computer program (hereinafter also simply referred to as a program) for executing the above-described method. The processor loads the computer program from the memory device into a buffer memory such as DRAM (Dynamic Random Access Memory) and executes the program.

[0025] Each component of the driver assistance device 10 may be implemented with dedicated hardware. Furthermore, some or all of each component may be implemented by general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may be implemented by a single chip or by multiple chips connected via a bus. Some or all of each component of each device may be implemented by a combination of the aforementioned circuits, etc., and programs. Processors include CPUs (Central Processing Units), GPUs (Graphics Processing Units), FPGAs (Field-Programmable Gate Arrays), etc. Also, at least a portion of the processing performed by the driver assistance device 10 may be provided as SaaS (Software as a Service). The descriptions of the configurations described herein may also apply to other devices or systems described below in this disclosure.

[0026] Furthermore, while the driver assistance system 10 is typically implemented by hardware mounted on the mobile vehicle, some hardware may be installed elsewhere rather than on the mobile vehicle. In this case, the equipment installed elsewhere (e.g., a server) and the equipment installed on the mobile vehicle are connected via a communication network.

[0027] <Embodiment 2> Figure 3 is a block diagram of the driver assistance device 11 according to Embodiment 2. The driver assistance device 11 allows the driver to appropriately recognize the operation schedule information 21 input by the driver by changing the notification mode 34. The driver assistance device 11 also reads the driver's operations on the moving object and appropriately notifies the driver if there is a difference between the operation and the content of the operation schedule information 21. The driver assistance device 11 mainly comprises an operation schedule reception unit 101, a notification content determination unit 102, a notification mode determination unit 113, and an operation reading unit 114.

[0028] The driver assistance device 11 has a configuration that is partially the same as the driver assistance device 10 in Figure 1. Therefore, the explanation of the configuration of the driver assistance device 11, which performs the same processing as the driver assistance device 10, will be omitted. Here, the operation schedule information 21 is received by the operation schedule reception unit 101 via voice, touch panel, or gestures. Specifically, the notification of the notification content 24 determined by the notification content determination unit 102 is communicated using voice output, screen display, or vibration.

[0029] When voice output is used to notify the notification content 24, the expression candidates 22 are, for example, words that have the same meaning as the content of the planned operation information 21, and the notification content 24 is, for example, a reading of a sentence. Figure 4 is a schematic diagram of the method for determining the notification content 24 using the voice output of the driving support device 11 according to Embodiment 2. Referring to Figure 4, when "forward" is voice input as the planned operation information 21, the notification content determination unit 102 acquires words such as "forward," "advance," "front," and "drive position" which have the same meaning as "forward" as expression candidates 22.

[0030] Next, the notification content determination unit 102 randomly selects at least one of the expression candidates 22 as the operation expression 23. In Figure 4, it is assumed that the word "forward" is selected as the operation expression 23. Subsequently, the notification content determination unit 102 determines the notification content 24 so as to naturally include the operation expression 23. In Figure 4, the notification content determination unit 102 determines the notification content 24 to be read aloud as "We are moving forward," so as to include "forward," which was selected as the operation expression 23.

[0031] When a screen display is used to notify the notification content 24, the expression candidate 22 is, for example, a method of displaying the content of the planned operation information 21, and the notification content 24 is, for example, a display on a monitor. Specifically, if the planned operation information 21 intends to start moving forward, the notification content determination unit 102 acquires an upward arrow meaning forward, or the words "forward" or "front," etc., as the expression candidate 22. For example, the notification content determination unit 102 selects an upward arrow as the operation expression 23 from the expression candidate 22 and determines the notification content 24 to display an upward arrow on the windshield. For example, the notification content determination unit 102 selects "forward" as the operation expression 23 from the expression candidate 22 and determines the notification content 24 to be the sentence "Moving forward," which is then displayed on the car navigation monitor.

[0032] When vibration is used in the notification content 24, the expression candidate 22 is, for example, the location or method of generating the vibration, and the notification content 24 is, for example, vibrating a vibration-enabled cushion placed in the driver's seat. Specifically, when the planned operation information 21 indicates an intention to start moving forward, the notification content determination unit 102 acquires the vibration of the front of the vibration-enabled cushion placed in the driver's seat or the top of the steering wheel as the expression candidate 22. For example, the notification content determination unit 102 selects the front of the cushion as the vibration location as the operation expression 23 from the expression candidates 22, and generates vibration on the front of the cushion for 3 seconds as the notification content 24.

[0033] Referring again to Figure 3, the notification mode determination unit 113 determines the notification mode for the notification content 24. Specifically, the notification mode determination unit 113 randomly determines the notification mode 34 from a plurality of notification mode candidates that differ in at least one of the strength, speed, and frequency of the notification of the notification content 24.

[0034] Figure 5 is a schematic diagram of the method for determining the notification mode 34 using the voice output of the driver assistance device 11 according to Embodiment 2. In Figure 5, the notification mode determination unit 113 selects the notification strength, speed, and frequency from the notification mode candidates to determine the notification mode 34. In notification using voice output, the notification strength is the loudness of the notification voice. In Figure 5, the notification mode determination unit 113 sets the notification strength candidate 31 from 40 dB to 80 dB. In notification using voice output, the notification speed is the speed at which the notification content is read aloud. In Figure 5, the notification mode determination unit 113 sets the notification speed candidate 32 from 100 WPM (Words Per Minute) to 180 WPM. In notification using voice output, the notification frequency is the pitch of the voice that reads the notification content. In Figure 5, the notification mode determination unit 113 sets the notification frequency candidate 33 from 400 Hz to 3000 Hz.

[0035] In Figure 5, the notification mode determination unit 113 has decided to notify in a mode of 70 dB, 120 WPM, and 2500 Hz as the notification mode 34. By changing the notification mode 34 irregularly, the driver will pay more attention to the content of the notification and will be able to recognize the planned operation information 21 that the driver has entered more strongly.

[0036] The notification mode determination unit 113 may also determine other modes of notification. For example, if the notification uses voice output, the notification mode determination unit 113 may determine the notification mode by adding elements such as the speaker's gender or age, speaking style, or intonation. The notification mode determination unit 113 may also determine the characteristics of the notification content of the voice output, such as the tone of voice of a specific voice actor, dialect, tone of voice such as cheerful / instructive / firm, or setting it to a melody.

[0037] When the notification uses a display, the notification mode determination unit 113 may also determine elements such as the presence or absence of animation in the display, the display pattern, font, display size, or display color. When the notification uses a display, the notification strength is the size of the displayed characters and figures, the font, the intensity of the display color, the length of the display time, or the display brightness. When the notification uses a display, the notification speed is the speed of scrolling or animation display when displaying characters and figures. When the notification uses a display, the notification frequency is the flashing frequency when displaying characters and figures with flashing. When the notification uses vibration, the notification mode determination unit 113 may also determine elements such as the vibration pattern or the transition of the vibration position. When the notification uses vibration, the notification strength is the magnitude of the vibration and the length of the vibration time. When the notification uses vibration, the notification speed is the length of one period in the transition of the vibration pattern or vibration position. When the notification uses vibration, the notification frequency is the vibration frequency, the frequency at which vibration occurs, or the frequency at which the position at which vibration occurs changes. According to this, the driver assistance device 11 can present the notification content 24 in a way that is less familiar to the driver.

[0038] Referring again to Figure 3, the operation reading unit 114 reads the driver's actions on the mobile vehicle. The operation reading unit 114 may acquire the mobile vehicle's CAN (Controller Area Network) signal and read the driver's actual actions, such as the shift lever position, accelerator opening, or steering input. The operation reading unit 114 may also be equipped with a camera and read the driver's actions on the mobile vehicle from the shift lever position and the driver's movements.

[0039] The notification content determination unit 102 further includes warning information in the notification content 24 if the operation schedule information 21 received by the operation schedule reception unit 101 differs from the operation read by the operation reading unit 114. The warning information includes, for example, at least one of an alarm sound and a flashing warning light. The notification content determination unit 102 may also output a driving prohibition instruction to the vehicle if the operation schedule information 21 received by the operation schedule reception unit 101 differs from the operation read by the operation reading unit 114. For example, if the system receives operation schedule information 21 indicating a reverse operation but reads an operation to move forward, the driving support device 11 outputs a signal to bring the moving vehicle to a sudden stop.

[0040] Next, the processing flow of the driver assistance device 11 will be described with reference to Figure 6. Figure 6 is a flowchart of the driver assistance device 11 according to Embodiment 2. The series of flows of the driver assistance device 11 includes steps S21 to S26. The driver assistance device 11 continuously processes the series of flows in a loop while the driver operates the mobile body.

[0041] In step S21, the operation schedule reception unit 101 receives operation schedule information 21 of the mobile body from the driver of the mobile body. The operation schedule information 21 is, for example, the planned direction of travel of the mobile body.

[0042] In step S22, the notification content determination unit 102 acquires multiple expressions used with the same meaning as the operation schedule information 21 received by the operation schedule reception unit 101 as expression candidates 22. Next, the notification content determination unit 102 randomly selects at least one of the multiple expression candidates 22 as the operation expression 23. Furthermore, the notification content determination unit 102 determines the notification content 24 to be notified to the driver, which includes the selected operation expression 23.

[0043] In step S23, the operation reading unit 114 reads the driver's operation of the mobile body as operation information. Operation information is, for example, a change in the actual direction of travel of the mobile body. The operation reading unit 114 reads the operation information by reading the CAN signal of the mobile body. The operation reading unit 114 may also read operation information using a camera or an external sensor.

[0044] In step S24, the notification content determination unit 102 determines whether the operation schedule information 21 received by the operation schedule reception unit 101 matches the operation information read by the operation reading unit 114. If the operation schedule information 21 and the operation information do not match, the driver assistance device 11 starts step S25. If the operation schedule information 21 and the operation information match, the driver assistance device 11 starts step S26.

[0045] In step S25, the notification content determination unit 102 includes warning information in the notification content 24. The warning information may be, for example, at least one of an alarm sound and a warning light. The warning information may also explain how the received operation plan information 21 differs from the operation information. The warning information may also include information for performing the operation related to the operation plan information 21. In step S25, the driving support device 11 may output a driving prohibition instruction to the moving object. After the completion of step S25, the driving support device 11 starts step S26.

[0046] In step S26, the notification mode determination unit 113 determines a notification mode 34 from the notification mode candidates. The notification mode determination unit 113 randomly determines the notification mode 34 from a plurality of notification mode candidates that differ in at least one of the strength, speed, and frequency of notifying the notification content 24. The notification mode determination unit 113 may also determine other modes of notification. The series of flows of the driver assistance device 11 ends with the completion of step S26.

[0047] According to this, the driver assistance device 11 can notify the driver of the content of the planned operation information 21 entered by the driver using a wider range of expressions. Therefore, the driver will be able to recognize the planned operation information 21 that they themselves entered more strongly as they receive the notification content 24. As a result, the driver assistance device 11 can reduce the occurrence of accidents due to errors by making the driver more appropriately aware of the planned operation information 21 that they themselves entered. In addition, the driver assistance device 11 can detect the difference between the planned operation information 21 entered by the driver and the driver's actual operation, and can further reduce the occurrence of accidents by making the driver aware of the error.

[0048] <Embodiment 3> Figure 7 is a block diagram of the driver assistance device 12 according to Embodiment 3. The driver assistance device 12 enables the driver to appropriately recognize the operation schedule information they have entered by notifying them in a format different from the format of the received operation schedule information. The driver assistance device 12 mainly comprises an operation schedule reception unit 121, a notification content determination unit 122, and an operation schedule reminder unit 125.

[0049] The operation schedule reception unit 121 has multiple input formats for receiving operation schedule information of the mobile vehicle from the driver of the mobile vehicle. The operation schedule information is, for example, the planned direction of travel of the mobile vehicle. The input formats are, for example, voice input, touch panel operation, or gesture input. The operation schedule reception unit 121 may be equipped with an input interface such as a microphone, button, touch panel, or camera in order to receive operation schedule information. The operation schedule reception unit 121 may be connected to an input interface and receive information entered into the input interface.

[0050] The operation schedule reception unit 121 receives operation schedule information, for example, from at least one of the driver's voice and / or touch panel input. The operation schedule reception unit 121 may also receive operation schedule information as image information or video information including action information such as the driver's pointing gesture. Furthermore, the operation schedule reception unit 121 may also receive operation schedule information regarding the left and right directions of travel from the operation of the turn signals.

[0051] The notification content determination unit 122 has multiple notification formats. Notification formats include, for example, voice output, front display display, monitor display, or illumination of interior lights. The notification content determination unit 122 may also notify the driver or passenger by transmitting the notification content to a smartphone or other device held by the driver or passenger. The notification content determination unit 122 irregularly selects at least one notification format from the multiple notification formats that is different from the input format of the planned operation information, and determines the notification content to be sent to the driver using the selected notification format based on the planned operation information.

[0052] First, the notification content determination unit 122 randomly selects at least one notification format from a plurality of notification formats that is different from the input format of the operation schedule information received by the operation schedule reception unit 121. The notification content determination unit 122 may select multiple notification formats. Next, the notification content determination unit 122 determines notification content suitable for the selected notification format based on the operation schedule information received by the operation schedule reception unit 121. In this way, the driver assistance device 12 can provide the driver with a stimulus in a sensory aspect different from the input, thereby directing their attention to the notification. Specifically, when the operation schedule reception unit 121 receives operation schedule information by voice input, the notification content determination unit 122 decides to display the direction of travel input as an arrow on the monitor.

[0053] Here, "irregular selection" means a selection method in which the driver does not immediately perceive any regularity in the operational expressions. Therefore, "irregular selection" does not necessarily mean processing without using random numbers. Furthermore, "irregular selection" may also mean a selection method that exhibits long-term regularity, or a selection method that selects the same operational expression repeatedly at a low frequency. Note that "irregular selection" is sometimes expressed as "random selection."

[0054] Furthermore, the notification content determination unit 122 determines a notification with more information if the input format and content of the planned operation information are simple. Here, "simple" means that it can be performed without special attention. Specifically, the notification content determination unit 122 determines the amount of information in the notification based on the time the planned operation information is received by the planned operation information reception unit 121 or the number of words spoken by the driver. For example, if the number of words spoken by the driver who inputs the planned operation information to the planned operation information reception unit 121 is below a predetermined threshold, the notification content determination unit 122 determines a notification with a long voice output.

[0055] For example, if the driver inputs planned operation information by touching the touch panel once, the notification content determination unit 122 will determine the notification content to be a longer voice message. Specifically, if the driver inputs the direction of travel by touching the upper part of the touch panel once, the voice message will be something like, "Moving forward. Is the shift lever in the drive position?" Also, if the driver inputs planned operation information by speaking a short sentence, the notification content determination unit 122 will determine the notification content to be an arrow and text displayed on the monitor. Specifically, if the driver only says, "Turn left," a large left arrow and text such as, "Turning left. Please check for safety," will be displayed on the monitor.

[0056] On the other hand, if the driver inputs planned operation information by uttering a complex sentence, such as "Forward check complete. Departing," the notification content determination unit 122 will determine a simple notification content, such as displaying only an arrow on the monitor. This allows the system to determine a notification content that more strongly reinforces the driver's awareness of the planned operation information, even if the driver may have input the information unconsciously, thereby preventing erroneous operation.

[0057] The operation schedule prompting unit 125 prompts the driver to input operation schedule information in an input format randomly determined from multiple input formats. The operation schedule prompting unit 125 prompts the driver to input operation schedule information using, for example, voice output, display on monitors, LED illumination, or a buzzer. In this way, the driver assistance device 12 can direct the driver's attention to inputting operation schedule information in the specified input format. Therefore, the driver assistance device 12 can prevent erroneous operations by making the driver strongly aware of the content of the operation schedule through the input of operation schedule information.

[0058] Specifically, if the operation schedule prompting unit 125 selects voice input as the input format, it will output the voice message, "Which direction would you like to go? Please tell us." If the operation schedule prompting unit 125 selects touch panel input as the input format, it will output the voice message, "Please enter your departure direction on the touch panel."

[0059] Next, with reference to Figure 8, the processing flow of the driver assistance device 12 will be described. Figure 8 is a flowchart of the driver assistance device 12 according to Embodiment 3. The sequence of flows of the driver assistance device 12 includes steps S31 to S37. The driver assistance device 12 continuously processes the sequence of flows in a loop while the driver operates the mobile body.

[0060] In step S31, the operation schedule prompting unit 125 irregularly determines an input format from which to accept input of operation schedule information from multiple input formats. The input formats include, for example, voice input, touch panel operation, or gesture input. In step S32, the operation schedule prompting unit 125 prompts the driver to input operation schedule information in the determined input format.

[0061] In step S33, the operation schedule reception unit 121 receives information about the planned operation of the mobile body from the driver of the mobile body. The planned operation information is, for example, the planned direction of travel of the mobile body. The operation schedule reception unit 121 does not have to accept input of operation schedule information other than the input format determined in step S31.

[0062] In step S34, the notification content determination unit 122 irregularly selects a notification format different from the input format of the planned operation information. Notification measures include, for example, audio output, monitor display, or turning on the interior lights. In step S35, the notification content determination unit 122 determines the notification content to be sent to the driver based on the planned operation information, using the notification format determined by the notification content determination unit 122 in step S34.

[0063] In step S36, the notification content determination unit 122 determines whether the input format and content of the planned operation information are simple. The notification content determination unit 122 determines whether the input format and content of the planned operation information are simple based, for example, on the time the planned operation information is received by the planned operation information reception unit 121 or the number of words spoken by the driver. If the notification content determination unit 122 determines that the input format and content of the planned operation information are simple, the driver assistance device 12 starts step S37. Otherwise, the driver assistance device 12 terminates its series of processes at the end of step S36.

[0064] In step S37, the notification content determination unit 122 determines a notification with a larger amount of information. For example, the notification content determination unit 122 makes the notification content with a larger amount of information by making the audio output or displayed text longer. For example, the notification content determination unit 122 increases the amount of information in the notification content by determining a notification content that uses multiple notification formats. The series of processes of the driver assistance device 12 ends at the end of step S37.

[0065] According to this, the driver assistance device 12 can make the driver aware of the input format, thereby enabling the driver to recognize the planned operation information that they themselves are inputting. Furthermore, the driver assistance device 12 can make the driver recognize the planned operation information that they themselves have input by notifying them in a format different from the input planned operation information. In addition, the driver assistance device 12 can make the driver more strongly aware of the planned operation information that they themselves have input when the driver is not paying attention to inputting the planned operation information.

[0066] <Embodiment 4> Figure 9 is a schematic diagram of the operating environment of the driver assistance device 13 according to Embodiment 4. Figure 9 shows the interior of a vehicle equipped with the driver assistance device 13. The driver assistance device 13 receives planned operation information 211 input by the driver 40 of the vehicle and notifies the driver 40 of the notification content using at least one of the voice output 241 and monitor display 242. The driver assistance device 13 is equipped with a microphone 131, a camera 132, and a touch panel 133 as an operation plan reception unit.

[0067] The driver assistance device 13 receives planned operation information 211 from the driver 40, such as "I will move forward," via the microphone 131. The driver assistance device 13 may also receive the driver 40's gestures as planned operation information 211 via the camera 132. The driver assistance device 13 may also receive planned operation information 211 from the driver 40 via the touch panel 133.

[0068] The driver assistance device 13 determines the content of the notification based on the planned operation information 211. The driver assistance device 13 notifies the driver 40 of the determined notification content using at least one of the voice output 241 and the monitor display 242. In this way, the driver assistance device 13 can make the driver 40 aware of the planned operation information 211 that the driver 40 has entered while notifying the driver of the notification content. Therefore, the driver assistance device 13 can reduce the occurrence of accidents by making the driver 40 more aware of the planned operation information 211 that the driver 40 has entered.

[0069] Specifically, in Figure 9, the driver assistance device 13 notifies the driver of the planned forward movement by outputting the voice message 241, "We are moving forward." Also in Figure 9, the driver assistance device 13 notifies the driver of the planned forward movement by displaying an upward arrow on the navigation system monitor via monitor display 242.

[0070] Furthermore, the driver assistance device 13 acquires the vehicle's CAN signal and obtains at least one operation of the steering wheel 41 and the shift lever 42. The driver assistance device 13 may also use the camera 132 to obtain at least one operation of the steering wheel 41 and the shift lever 42. If the obtained operation differs from the content of the received operation schedule information 211, the driver assistance device 13 notifies the driver 40 of the operation error using at least one of the voice output 241 and the monitor display 242. If the difference between the obtained operation and the received operation schedule information 211 occurs when starting the vehicle, the driver assistance device 13 further instructs the vehicle not to move.

[0071] Next, the processing flow of the driver assistance device 13 will be described with reference to Figure 10. Figure 10 is a flowchart of the driver assistance device 13 according to Embodiment 4. The flow of the driver assistance device 13 includes steps S41 to S49.

[0072] In step S41, the driver assistance device 13 is activated when the vehicle's engine is started. In step S42, the driver assistance device 13 receives the driver's (40) direction of travel as planned operation information 211 via the microphone 131 and camera 132, etc. The driver assistance device 13 may also act to prevent the vehicle from starting if it has not received the planned operation information 211.

[0073] In step S43, the driver assistance device 13 generates notification content and notifies the driver of the content using navigation or the like with at least one of the voice output 241 and monitor display 242. This prompts the driver assistance device 13 to confirm the operation and to warn the driver 40. If the planned operation information 211 received by the driver assistance device 13 is a simple input such as a short spoken sentence, the driver assistance device 13 may include a larger amount of information in the notification content. Specifically, the driver assistance device 13 may notify the driver with a voice output such as, "The vehicle has been set to move forward. Please put the shift lever correctly in the drive position and drive carefully, paying attention to your surroundings."

[0074] In step S44, the driver assistance device 13 acquires the vehicle's CAN signal and confirms that the driver 40 has started driving. The driver assistance device 13 may also use the camera 132 to confirm that the driver 40 has started driving.

[0075] In step S45, the driver assistance device 13 checks whether the operation of the driver 40, which was confirmed in step S44, is correct, based on the planned operation information 211 received in step S42. If the driver assistance device 13 confirms that the operation is correct, it starts step S46. Otherwise, the driver assistance device 13 starts step S47.

[0076] In step S46, the driver assistance device 13 generates a notification message to alert the driver. The driver assistance device 13 notifies the driver using at least one of the generated notification message, which is a driving alert, via voice output 241 and monitor display 242. For example, the driver assistance device 13 informs the driver 40 of the situation via voice output 241 such as, "The vehicle is moving forward." After that, the driver assistance device 13 resumes the flow from step S42.

[0077] In step S47, the driver assistance device 13 generates a notification message indicating an operational error. The driver assistance device 13 notifies the driver of the generated error message using at least one of the voice output 241 and the monitor display 242. The driver assistance device 13 may notify the driver of the error in a more attention-grabbing manner than the notification made in step S43. For example, the driver assistance device 13 may notify the driver of the error using multiple notification formats. The notification message regarding the error may continue to be displayed until the driver assistance device 13 accepts a reset of the planned operation information 211, such as the direction of travel, or until the driver assistance device 13 confirms that a correct operation has been performed, such as correcting the position of the shift lever.

[0078] In step S48, the driver assistance device 13 checks whether the erroneous operation occurred during starting. If it did, the driver assistance device 13 starts step S49. If it did not occur during starting, the driver assistance device 13 restarts the flow from step S42.

[0079] In step S49, the driver assistance device 13 issues a command to the vehicle to refrain from driving. This prevents accidents caused by driving in the wrong direction when starting. After that, the driver assistance device 13 resumes the flow from step S42.

[0080] The driver assistance device 13 continues the series of steps, including steps S42 to S49, until the termination process is performed due to engine shutdown. This allows the driver assistance device 13 to more accurately recognize the planned operation information 211 entered by the driver 40, thereby reducing the occurrence of accidents due to erroneous operation.

[0081] <Example hardware configuration> The following describes examples of how each functional configuration of the information processing device in this disclosure is realized through a combination of hardware and software.

[0082] Figure 11 is a block diagram illustrating the hardware configuration of a computer. The information processing device in this disclosure can realize the above-described functions using a computer 500 including the hardware configuration shown in the figure. The computer 500 may be a portable computer such as a smartphone or tablet terminal, or a stationary computer such as a PC. The computer 500 may be a dedicated computer designed to realize each device, or it may be a general-purpose computer. The computer 500 can realize the desired functions by installing a predetermined application.

[0083] Computer 500 includes a bus 502, a processor 504, memory 506, a storage device 508, an input / output interface (I / F) 510, and a network interface (I / F) 512. Bus 502 is a data transmission path for the processor 504, memory 506, storage device 508, input / output interface 510, and network interface 512 to send and receive data to and from each other. However, the method of connecting the processor 504 and other components to each other is not limited to bus connection.

[0084] Processor 504 is a variety of processors such as a CPU, GPU, or FPGA. Memory 506 is main memory implemented using RAM (Random Access Memory), etc.

[0085] The storage device 508 is an auxiliary storage device implemented using a hard disk, SSD, memory card, or ROM (Read Only Memory). The storage device 508 stores a program for realizing a desired function. The processor 504 reads this program into memory 506 and executes it to realize each functional component of each device.

[0086] The input / output interface 510 is an interface for connecting the computer 500 to input / output devices. For example, input devices such as keyboards and output devices such as display devices are connected to the input / output interface 510. The network interface 512 is an interface for connecting the computer 500 to a network.

[0087] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0088] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate. [Explanation of symbols]

[0089] 10, 11, 12, 13 Driving assistance devices 21 Operation Schedule Information 22 Candidate expressions 23 Operation expression 24. News content 31 Candidates for Strength 32 speed candidates 33 frequency candidates 34. Method of reporting 40 Driver 41 Handle 42 Shift lever 101, 121 Operation Schedule Reception Department 102, 122 News Content Decision Department 113 Notification Method Determination Unit 114 Operation and reading section 125 Operation Schedule Reminder Department 131 Mike 132 Cameras 133 Touch Panel 211 Operation Schedule Information 241 Audio Output 242 Monitor Display 500 Computers Bus 502 504 Processors 506 memory 508 Storage Devices 510 Input / Output Interfaces 512 Network Interfaces

Claims

1. An operation schedule receiving unit that receives information on the planned operation of the mobile vehicle from the driver of the mobile vehicle, A notification content determination unit randomly selects at least one operation expression from a plurality of expression candidates relating to the operation schedule information based on the operation schedule information, and determines the notification content to be sent to the driver, including the operation expression. A driver assistance system equipped with the following features.

2. The system further includes a notification mode determination unit that randomly determines a notification mode from a plurality of notification mode candidates, each having at least one of the strength, speed, and frequency of notification of the aforementioned notification content. The driving support device according to claim 1.

3. The system further includes an operation reading unit that reads the driver's operations on the mobile body, The notification content determination unit includes warning information in the notification content if the operation schedule information received by the operation schedule reception unit differs from the operation read by the operation reading unit. The driving support device according to claim 1 or 2.

4. Computers The operator of the mobile vehicle receives information from the operator regarding the planned operation of the mobile vehicle. Based on the planned operation information, at least one operation expression is randomly selected from a plurality of candidate expressions relating to the planned operation information, and the content of the notification to the driver, including the operation expression, is determined. Driving assistance methods.

5. Computers The operator of the mobile vehicle receives information from the operator regarding the planned operation of the mobile vehicle. Based on the planned operation information, at least one operation expression is randomly selected from a plurality of candidate expressions relating to the planned operation information, and the content of the notification to the driver, including the operation expression, is determined. A program for carrying out a task.

Images