Vehicle control device

Abstract

The present invention provides a vehicle control device that can perform operation intervention based on the operation of other passengers when the driver of a vehicle equipped with a device operated and controlled based on voice input is in a state where they are unable to operate or not performing an operation. The vehicle control device includes: an input unit that receives an operation input based on the voice of the vehicle operator; and a determination unit that determines whether the operator is in a predetermined state of being unable to operate or not performing an operation, and when it is determined that the operator is in the predetermined state, the input unit receives an operation input based on the voice of a fellow passenger.

Description

Technical Field

[0001] The present invention relates to a control device for a vehicle. Background Art

[0002] As a control device for a vehicle, various devices have been proposed that recognize the voice of a speaker and perform operation control of devices mounted on the vehicle. For example, Patent Document 1 discloses an automotive voice control device that is a control device that recognizes a voice for control and controls a load device of an automobile based on the recognition result of the voice, and is configured such that a device for controlling an action required for driving recognizes only the voice of the driver and performs control, and control of other devices recognizes the voices of the driver and other people and performs control.

[0003] In addition, Patent Document 2 discloses a vehicle input device that, in a state where the voice of a driver in a vehicle is recognized and reception of an operation input to an operation target device in in-vehicle devices is permitted based on the recognition result, recognizes the movement of a hand in the circumferential direction of a steering wheel based on an image obtained by photographing an area including the steering wheel of the vehicle, and notifies a control instruction corresponding to the recognition result to the operation target device.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: Japanese Patent Laid-Open No. 60-045448

[0007] Patent Document 2: Japanese Patent Laid-Open No. 2017-159692 Summary of the Invention

[0008] Technical Problem

[0009] However, in the devices disclosed in Patent Document 1 and Patent Document 2, when the driver is in a state where he / she is unable to perform driving operations, for example, due to illness, there is a possibility that a passenger other than the driver cannot intervene in the driving operation.

[0010] The present invention has been made in view of the above problems, and an object of the present invention is to provide a control device for a vehicle that can allow operation intervention by other passengers when the driver, who is the driving operation subject of a vehicle equipped with a device that is operationally controlled based on voice input, is in a state where he / she is unable to operate or is not operating.

[0011] Technical Solution

[0012] In order to solve the above problems, according to one aspect of the present invention, there is provided a vehicle control device for controlling a vehicle based on voice input, the vehicle control device including: an input unit that receives an operation input based on the voice of the operator of the vehicle; and a determination unit that determines whether the operator is in a predetermined state where the operator is unable to operate or has not performed an operation. When it is determined that the operator is in the predetermined state, the input unit receives an operation input based on the voice of a passenger.

[0013] In the above vehicle control device, when the posture of the operator is in an abnormal state, the determination unit may determine that the operator is in the predetermined state.

[0014] In the above vehicle control device, when the driving operation of the operator is in an abnormal state, the determination unit may determine that the operator is in the predetermined state.

[0015] In the above vehicle control device, when an intervention operation is performed by a passenger, the determination unit may determine that the operator is in the predetermined state.

[0016] In the above vehicle control device, when the input voice contains words indicating an emergency, the determination unit may determine that the operator is in the predetermined state.

[0017] In the above vehicle control device, when the biological information of the operator is in an abnormal state, the determination unit may determine that the operator is in the predetermined state.

[0018] The above vehicle control device may include a notification unit that, when it is determined that the operator is in the predetermined state, notifies that an operation input based on the voice of a passenger is received.

[0019] In the above vehicle control device, when an intervention release operation is performed after the reception of an operation input based on the voice of a passenger is started, the input unit may end the reception of the operation input based on the voice of a passenger.

[0020] Technical effects

[0021] As described above, according to the present invention, when the driver of a vehicle equipped with a device that is operated and controlled based on voice input is in a state where the driver is unable to operate or has not performed an operation, it is possible to perform an operation intervention by another passenger. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram showing a configuration example of a vehicle equipped with the vehicle control device according to an embodiment of the present invention.

[0023] Figure 2It is a block diagram showing the functional configuration of the vehicle control device according to this embodiment.

[0024] Figure 3 It is a flowchart showing an example of the operation intervention determination process performed by the vehicle control device according to this embodiment.

[0025] Figure 4 It is a flowchart showing an example of the process of determining whether the operator is in a non-drivable state performed by the vehicle control device according to this embodiment.

[0026] Symbol Explanation

[0027] 1 Vehicle

[0028] 11 Operation Intervention Switch

[0029] 13 Camera

[0030] 15 Microphone

[0031] 17 Speaker

[0032] 50 Vehicle Control Device

[0033] 51 Facial Recognition Processing Unit

[0034] 53 Sound Signal Processing Unit

[0035] 55 Sound Direction Determination Processing Unit

[0036] 57 Sound Recognition Processing Unit

[0037] 59 Sound Synthesis Processing Unit

[0038] 61 Speaker Determination Processing Unit

[0039] 63 Operation Intervention Determination Processing Unit

[0040] 65 Operation Input Processing Unit Detailed Embodiment

[0041] Hereinafter, with reference to the drawings, preferred embodiments of the present invention will be described in detail. It should be noted that in this specification and the drawings, components having substantially the same functional structure are denoted by the same reference numerals, and thus redundant descriptions are omitted.

[0042] <1. Overall Configuration Example of Vehicle>

[0043] Refer to Figure 1 , an overall configuration example of a vehicle equipped with a vehicle control device according to an embodiment of the present invention will be described. Figure 1 It is a schematic diagram showing a configuration example of a vehicle 1 equipped with a vehicle control device 50.

[0044] The vehicle 1 to which the vehicle control device 50 is applied is configured as follows: Part or all of the devices for controlling the running of the vehicle 1 are controlled based on voice input. Hereinafter, in the present embodiment, the vehicle 1 will be described as an example in which at least the control of acceleration and deceleration of the vehicle 1 in the running control of the vehicle 1 is performed based on voice input. Therefore, in the following embodiments, the operator of the vehicle 1 refers to the driver who can manually rotate the steering wheel and is in the driver's seat.

[0045] It should be noted that, in the present embodiment, an example in which the vehicle control device 50 is applied to an electric vehicle 1 having two drive motors, a front-wheel drive motor 9F and a rear-wheel drive motor 9R, will be described. However, the vehicle is not limited to the above example.

[0046] For example, the vehicle may be an electric vehicle that uses one drive motor to drive all wheels, or an electric vehicle that uses one drive motor to drive one of the front wheels and the rear wheels. In addition, the vehicle may also be an electric vehicle equipped with in-wheel motors on each drive wheel. In this case, the electric vehicle may be an electric vehicle equipped with a motor as a generator, or a fuel cell vehicle equipped with a fuel cell as a generator.

[0047] Furthermore, the vehicle is not limited to an electric vehicle, and may be a vehicle equipped with an internal combustion engine as a drive source, or a hybrid vehicle equipped with an internal combustion engine and a drive motor as drive sources.

[0048] The vehicle 1 includes a device for controlling the running of the vehicle 1 (hereinafter, also referred to as "running system device"), and a device that is not directly used for running control (hereinafter, also referred to as "non-running system device"). As Figure 1 shown, the vehicle 1 includes a front-wheel drive motor 9F, a rear-wheel drive motor 9R, a steering system 23, and brake devices 25LF, 25RF, 25LR, 25RR (hereinafter, collectively referred to as the brake device 25 when there is no need to distinguish them particularly) as the running system devices.

[0049] The motor 9F for front-wheel drive is driven by electric power supplied from a battery (not shown), and transmits torque to the front-wheel drive shaft 5F via the differential mechanism 7F. In addition, the motor 9R for rear-wheel drive is driven by electric power supplied from the same battery (not shown), and transmits torque to the rear-wheel drive shaft 5R via the differential mechanism 7R. Both the motor 9F for front-wheel drive and the motor 9R for rear-wheel drive have a regenerative function of generating electricity by receiving the rotational torque of each drive wheel 3LF, 3RF, 3LR, 3RR when the vehicle 1 decelerates. The driving of the motor 9F for front-wheel drive and the motor 9R for rear-wheel drive is controlled by controlling a converter (not shown) by a vehicle control device (hereinafter also simply referred to as "control device") 50.

[0050] A steering system 23 is provided on the front-wheel drive shaft 5F, and the orientations of the front wheels 3LF, 3RF are changed as the steering wheel is rotated.

[0051] The braking devices 25LF, 25RF, 25LR, 25RR apply braking forces to the front, rear, left, and right drive wheels 3LF, 3RF, 3LR, 3RR, respectively. The braking devices 25LF, 25RF, 25LR, 25RR are used in combination with the regenerative braking performed by the motor 9F for front-wheel drive and the motor 9R for rear-wheel drive. The braking devices 25LF, 25RF, 25LR, 25RR can be hydraulic braking devices or electric braking devices. For example, in the case of a hydraulic braking device, the braking force is controlled by controlling a hydraulic unit by using the control device 50, and the hydraulic unit controls the hydraulic pressure supplied to each of the braking devices 25LF, 25RF, 25LR, 25RR.

[0052] In addition, the vehicle 1 is provided with an operation intervention switch 11, a camera 13, microphones 15a, 15b, 15c, 15d (hereinafter, collectively referred to as microphones 15 when not particularly distinguished), and speakers 17a, 17b (hereinafter, collectively referred to as speakers 17 when not particularly distinguished).

[0053] The camera 13 is provided in the vehicle compartment so as to be able to take pictures. The camera 13 includes an imaging element such as a CCD (Charged-Coupled Devices) or a CMOS (Complementary Metal-Oxide-Semiconductor), and generates an image signal. The camera 13 can be a single camera or a pair of left and right stereo cameras. The image signal obtained by the camera 13 is output to the control device 50. The control device 50 performs predetermined processing based on the image signal output from the camera 13.

[0054] The microphone 15 collects the sounds or voices inside the vehicle compartment. In the present embodiment, four microphones 15a to 15d are piezoelectric microphones and are arranged near the four corners inside the vehicle compartment. Each of the microphones 15a to 15d is configured to be able to detect the direction of the sound or voice being picked up. The sound signal obtained by the microphone 15 is sent to the control device 50. The control device 50 performs predetermined processing based on the sound signal sent from the microphone 15.

[0055] The speaker 17 generates sounds or voices inside the vehicle compartment. The speaker 17 operates based on the sound signal received from the control device 50, thereby generating sounds or voices.

[0056] The operation intervention switch 11 is provided inside the vehicle compartment in an operable manner. When allowing passengers other than the driver (the operating subject person) in the driver's seat to perform an input operation for the driving control of the vehicle 1 based on voice input, the operation intervention switch 11 is operated by the passengers inside the vehicle compartment. A signal indicating the conduction or cutoff of the operation intervention switch 11 is output to the control device 50. However, the operation intervention switch 11 may also be provided inside the instrument panel, for example, so that the operation intervention switch 11 will not be conducted inadvertently or mischievously.

[0057] In addition, although not shown, various sensors and / or devices for the determination process of whether the driver (the operating subject person) is in a non-drivable state may also be provided. Or, these sensors and / or devices may also be devices worn by the operating subject person. For example, a biological sensor for detecting the biological information of the driver may be provided. The biological sensor may be a known sensor such as a blood pressure sensor, a temperature sensor, a heart rate monitor, an electrocardiogram sensor, a pulse sensor, or an electroencephalogram sensor. In addition, the above-mentioned device for the determination process may also be a camera. The control device 50 can detect abnormalities in the blood pressure, body temperature, heartbeat, pulse, electroencephalogram, or complexion of the operating subject person based on the outputs of these sensors and / or devices, and can also detect the onset of epilepsy or a sudden heart attack.

[0058] <2. Configuration example of vehicle control device>

[0059] Next, with reference to Figure 2 , a configuration example of the vehicle control device 50 of the present embodiment will be described. Figure 2 is a block diagram showing the functional configuration of the control device 50.

[0060] The control device 50 is configured to include, for example, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) and / or a circuit, and a storage element such as a RAM (Random Access Memory) and / or a ROM (Read Only Memory). Part or all of the control device 50 may be constituted by a device capable of updating firmware or the like, or may be a program module or the like executed according to an instruction from a CPU or the like.

[0061] The control device 50 includes a face recognition processing unit 51, a voice signal processing unit 53, a voice direction determination processing unit 55, a voice recognition processing unit 57, a voice synthesis processing unit 59, a speaker determination processing unit 61, an operation intervention determination processing unit 63, an operation input processing unit 65, a driving system operation processing unit 71, and a non-driving system operation processing unit 73. All or part of each of these processing units may be functions implemented by a processor executing a computer program.

[0062] The face recognition processing unit 51 performs face recognition processing of the passengers in the vehicle 1 based on the image signal output by the camera 13. The face recognition processing performed by the face recognition processing unit 51 is executed to grasp the positions of the passengers present in the vehicle compartment. The face recognition processing unit 51 can not only recognize the faces of the people present in the vehicle compartment, but also determine which position the mouth of the person is open, that is, which position the person is speaking. It should be noted that the specific processing content of the face recognition processing may be an appropriate publicly known method and is not particularly limited.

[0063] The voice signal processing unit 53 performs voice signal processing using the voice signals transmitted by the four microphones 15a to 15d. Specifically, the voice signal processing unit 53 performs filtering processing or the like on the voice signals transmitted by the respective microphones 15a to 15d, and calculates the sound pressure, frequency, waveform, etc. through operations. It should be noted that the specific processing content of the voice signal processing may be an appropriate publicly known method and is not particularly limited.

[0064] The voice direction determination processing unit 55 determines the direction of the voice collected by the microphone 15 based on the voice data processed by the voice signal processing unit 53. For example, the voice direction determination processing unit 55 determines the direction of the voice based on the directivity of the voice collected by the respective microphones 15a to 15d. It should be noted that the specific processing content of the voice direction determination processing may be an appropriate publicly known method and is not particularly limited.

[0065] The voice recognition processing unit 57 recognizes the content (speech content) of the voice collected by the microphone 15 based on the voice data processed by the voice signal processing unit 53. The voice recognition processing unit 57 determines, from the speech content, words indicating operation instructions for the driving system devices or the non-driving system devices, and recognizes the operation instructions for these devices. It should be noted that the specific processing content of the voice recognition processing may be an appropriate well-known method and is not particularly limited.

[0066] The voice synthesis processing unit 59 performs voice synthesis processing in order to repeat the operation instructions recognized by the voice recognition processing unit 57 and make them sound from the speaker 17. It should be noted that the specific processing content of the voice synthesis processing may be an appropriate well-known method and is not particularly limited.

[0067] The voice originator determination processing unit 61 determines the voice originator of the voice collected by the microphone 15 based on at least one of the processing results of the face recognition processing unit 51 and the voice direction determination processing unit 55. Specifically, the voice originator determination processing unit 61 discriminates whether the voice originator is the driver or a passenger in the passenger seat or in the back seat based on the direction of the voice determined according to the processing result of the voice direction determination processing unit 55 and the positions of the passengers in the vehicle compartment grasped according to the processing result of the face recognition processing unit 51. The voice originator determination processing unit 61 may also identify which passenger is opening their mouth (speaking) based on the processing result of the face recognition processing unit 51, thereby discriminating the voice originator. It should be noted that the specific processing content of the voice originator determination processing may be an appropriate well-known method and is not particularly limited.

[0068] The operation intervention determination processing unit 63 determines whether to permit an input for the operation control of the vehicle 1 based on a voice input from a passenger other than the driver (the operation subject person). In the present embodiment, the operation intervention determination processing unit 63 corresponds to the determination unit of the present invention. Specifically, the operation intervention determination processing unit 63 determines whether the driver (the operation subject person) is in a predetermined state where they are unable to operate or not operating, and in the case of being in this predetermined state, permits an input for the operation control of the vehicle 1 based on a voice input from a passenger other than the driver (the operation subject person). In the present embodiment, the operation intervention determination processing unit 63 determines whether the driver is in a predetermined state (hereinafter, this state is also referred to as the "unable to drive state") by at least one of the following methods.

[0069] In the first method, when the driver's posture is abnormal, the operation intervention determination processing unit 63 determines that the driver is in a non-drivable state. Specifically, the operation intervention determination processing unit 63 performs image processing using the image signal output from the camera 13 to determine whether the driver's posture is in a state unsuitable for driving the vehicle 1. Examples of situations that can be determined to be in a state unsuitable for driving the vehicle 1 include, for example, a situation where the driver has the face facing downward or to the side for a predetermined time or more, a situation where the driver has the eyes closed for a predetermined time or more, and a situation where the driver is lying on the steering wheel. Considering safety, the predetermined time for determining that these conditions are met can be set to, for example, 1 to 3 seconds. However, conditions other than the above examples can also be set.

[0070] In the second method, when the driver's driving operation is abnormal, the operation intervention determination processing unit 63 determines that the driver is in a non-drivable state. For example, the operation intervention determination processing unit 63 continuously reads information on the operation amounts of the steering wheel 21 and / or the brake pedal, accelerator pedal, etc. When the abnormal state of the operation amount continues for a predetermined time or more, or when the operation amount changes rapidly and repeatedly, etc., it is determined that the driver is in a non-drivable state. In addition, when the input torque to the steering wheel 21 is zero or the steering wheel 21 does not rotate for a predetermined time or more, the operation intervention determination processing unit 63 can also determine that the driver is in a non-drivable state. Considering safety, the predetermined time for determining that these conditions are met can be set to, for example, 1 to 3 seconds. However, conditions other than the above examples can also be set.

[0071] In the third method, when a certain passenger performs an intervention operation, the operation intervention determination processing unit 63 determines that the driver is in a non-drivable state. In the present embodiment, when the operation intervention switch 11 is turned on, the operation intervention determination processing unit 63 determines that the driver is in a non-drivable state as a situation where a passenger other than the driver has performed an intervention operation.

[0072] In the fourth method, when the voice contains words indicating an emergency, the operation intervention determination processing unit 63 determines that the driver is in a non-drivable state. Specifically, the operation intervention determination processing unit 63 determines whether the speech content contains words indicating an emergency based on the voice data processed by the voice recognition processing unit 57. Although examples of words indicating an emergency include, for example, "emergency", "abnormal situation", or "SOS (International distress signal)", appropriate words can also be set from words that are difficult to be included in ordinary conversations other than these.

[0073] In the fifth method, when the driver's biological information is in an abnormal state, the operation intervention determination processing unit 63 determines that the driver is in a state of being unable to drive. Specifically, when the driver's biological information obtained by a biological sensor such as a blood pressure sensor, a temperature sensor, a heart rate meter, or an electrocardiogram sensor shows an abnormal value, the operation intervention determination processing unit 63 determines that the driver is in a state of being unable to drive.

[0074] When the conditions formed by one of the above-mentioned methods from the first to the fifth methods or a combination thereof are met, the operation intervention determination processing unit 63 allows input of operation control of the vehicle 1 based on voice input from passengers other than the driver (operating subject person). In the case where input of operation control of the vehicle 1 based on voice input from passengers other than the driver (operating subject person) is allowed, the operation intervention determination processing unit 63 can operate the notification device to notify the acceptance of operation input based on the voice of passengers other than the driver. In this case, the operation intervention determination processing unit 63 is equivalent to the notification unit of the present invention. The notification method can be, for example, an alarm sound or sound from the speaker 17, a display by a display device, or the lighting of an alarm light.

[0075] Furthermore, when an operation to release the operation intervention of the passengers other than the driver is performed after the input of the operation control of the vehicle 1 based on the voice input from the passengers other than the driver (the main operator) is allowed, the operation intervention determination processing unit 63 may stop the input of the operation control of the vehicle 1 based on the voice input from the passengers other than the driver (the main operator). The operation to release the operation intervention may be, for example, the operation intervention switch 11 being turned off, the release switch other than the operation intervention switch 11 being turned on, or the voice input from the passengers indicating the release of the operation intervention.

[0076] The operation input processing unit 65 receives operation input based on the speech content determined according to the processing result of the speech recognition processing unit 57. The operation input processing unit 65 applies an operation instruction to the driving system operation processing unit 71 according to the received operation input. In this embodiment, the operation input processing unit 65 is equivalent to the input unit of the present invention. The speech content and the executed operation processing content are pre-stored in a storage unit (not shown).

[0077] For example, in the case where the speech content includes the word "acceleration", the operation input processing unit 65 accepts an operation input for driving at least one of the front-wheel drive motor 9F and the rear-wheel drive motor 9R with a predetermined output power. Further, in the case where the speech content includes words such as "deceleration", "stop", or "emergency braking", the operation input processing unit 65 accepts at least one of an operation input for driving the front-wheel drive motor 9F and the rear-wheel drive motor 9R with a predetermined output regeneration or an operation input for operating the braking device 25. In addition, the operation input processing unit 65 may be configured to accept an appropriate operation input corresponding to the speech content with reference to pre-set data.

[0078] In the case where the operation intervention determination processing unit 63 does not determine that the input for operating and controlling the vehicle 1 based on the voice input from a passenger other than the driver (the operation subject person) is permitted, the operation input processing unit 65 accepts only the operation input based on the driver's voice. On the other hand, the operation input processing unit 65 is configured to accept the operation input based on the voice of a passenger other than the operation subject person in the case where the operation intervention determination processing unit 63 determines that the input for operating and controlling the vehicle 1 based on the voice input from a passenger other than the driver (the operation subject person) is permitted.

[0079] That is, the operation input processing unit 65 is configured to accept only the operation input based on the driver's voice during the period when it is presumed that the driver is in a state where normal driving operations can be performed, and perform driving operations based on the will of the driver alone. On the other hand, the operation input processing unit 65 is configured to permit the acceptance of the operation input based on the voice of a passenger other than the driver in the case where it is presumed that the driver is not in a state where normal driving operations can be performed, and enable emergency responses such as deceleration or stop based on the passenger.

[0080] It should be noted that in the case where it is presumed that the driver is not in a state where normal driving operations can be performed, not limited to emergency responses, all passengers may be able to perform driving operations. In this case, the operation input based on the voice of one or more passengers is accepted to continue driving the vehicle. For example, the vehicle can be driven to an appropriate place such as a hospital.

[0081] The driving system operation processing unit 71 executes operation processing of the driving system devices based on operation instructions from the operation input processing unit 65. In the present embodiment, the driving system operation processing unit 71 executes operation control of at least one of the front-wheel drive motor 9F, the rear-wheel drive motor 9R, and the braking device 25. The devices to be operated by the driving system operation processing unit 71 are not limited to the front-wheel drive motor 9F, the rear-wheel drive motor 9R, or the braking device 25, and may also include other driving system devices.

[0082] The non-driving system operation processing unit 73 executes operation processing of the non-driving system devices based on the voice data processed by the voice recognition processing unit 57. The non-driving system devices are devices mounted on the vehicle 1 that are not related to the driving of the vehicle 1, and are not particularly limited.

[0083] It should be noted that although the control device 50 of the present embodiment is illustrated and described as one control device, the control device 50 may also be constituted by a plurality of control devices capable of communicating with each other.

[0084] <3. Action example>

[0085] Next, an example of the control processing of the control device 50 will be described with reference to the flowchart.

[0086] Figure 3 It is a flowchart showing an example of the control processing performed by the operation intervention determination processing unit 63 of the control device 50. First, the operation intervention determination processing unit 63 acquires data signals transmitted from the face recognition processing unit 51 and the voice recognition processing unit 57. In addition, the operation intervention determination processing unit 63 receives the output signal of the operation intervention switch 11, the output signal of the steering angle sensor that detects the rotation angle (steering angle) of the steering wheel 21 provided in the steering system, and the output signals of the stroke sensors that respectively detect the operation amounts of the accelerator pedal and the brake pedal (step S11).

[0087] Next, the operation intervention determination processing unit 63 determines whether the driver (the operating subject person) is in a state of being unable to drive based on the signals and data acquired in step S11 (step S13). Specifically, the operation intervention determination processing unit 63 determines whether any one or a combination of conditions, such as the driver's posture being in an abnormal state, the driver's driving operation being in an abnormal state, an intervening operation being performed by a certain passenger, or the voice containing words indicating an emergency, is satisfied according to the above first method to fifth method.

[0088] Figure 4It is a flowchart showing an example of a process for determining whether a driver (the operating subject person) is in a state where they are unable to drive. First, the operation intervention determination processing unit 63 determines whether the driver's posture is abnormal (step S31). When it is determined that the driver's posture is abnormal (S31 / Yes), the operation intervention determination processing unit 63 determines that the driver is in a state where they are unable to drive (step S43). On the other hand, when it is not determined that the driver's posture is abnormal (S31 / No), the operation intervention determination processing unit 63 determines whether the driver's driving operation is abnormal (step S33). When it is determined that the driver's driving operation is abnormal (S33 / Yes), the operation intervention determination processing unit 63 determines that the driver is in a state where they are unable to drive (step S43).

[0089] On the other hand, when it is not determined that the driver's driving operation is abnormal (S33 / No), the operation intervention determination processing unit 63 determines whether the operation intervention switch 11 has become conductive (step S35). When it is determined that the operation intervention switch 11 has become conductive (S35 / Yes), the operation intervention determination processing unit 63 determines that the driver is in a state where they are unable to drive (step S43). On the other hand, when it is not determined that the operation intervention switch 11 has become conductive (S35 / No), the operation intervention determination processing unit 63 determines whether the speech content collected by the microphone 15 contains words indicating an emergency (step S37). When it is determined that the speech content contains words indicating an emergency (S37 / Yes), the operation intervention determination processing unit 63 determines that the driver is in a state where they are unable to drive (step S43).

[0090] On the other hand, when it is not determined that the speech content contains words indicating an emergency (S37 / No), the operation intervention determination processing unit 63 determines whether the driver's biological information is abnormal (step S39). When it is determined that the driver's biological information is abnormal (S39 / Yes), the operation intervention determination processing unit 63 determines that the driver is in a state where they are unable to drive (step S43). On the other hand, when it is not determined that the driver's biological information is abnormal (S39 / No), the operation intervention determination processing unit 63 determines that the driver is not in a state where they are unable to drive (step S41).

[0091] It should be noted that the order of determining whether the five conditions are satisfied can be appropriately changed. Additionally, in Figure 4 the example of the flowchart shown, it is sequentially determined whether the five conditions are satisfied, and when a certain condition is satisfied, it is determined that the driver is in a state where they are unable to drive. However, it is also possible to determine whether all five conditions are satisfied first, and then determine that the driver is in a state where they are unable to drive when a certain condition or a predetermined number of conditions are satisfied. Furthermore, although Figure 4The flowchart shown is an example in which determination is made as to whether all the five conditions described above are satisfied, but the determination as to whether some of the conditions are satisfied may be omitted.

[0092] return Figure 3 If it is not determined that the driver is in a state of being unable to drive (S13 / No), the operation intervention determination processing unit 63 sets a setting to prohibit the acceptance of operation input based on the voice of a fellow passenger other than the driver (step S17). On the other hand, if it is determined that the driver is in a state of being unable to drive (S13 / Yes), the operation intervention determination processing unit 63 sets a setting to allow the acceptance of operation input based on the voice of a fellow passenger other than the driver (step S15).

[0093] Next, the operation intervention determination processing unit 63 performs a process of notifying the acceptance of the operation input based on the voice of the passenger other than the driver (step S19). For example, the operation intervention determination processing unit 63 can perform the above-mentioned notification by emitting an alarm sound or sound using the speaker 17, or displaying it on a display device, or lighting an alarm lamp.

[0094] Next, the operation intervention determination processing unit 63 determines whether an operation to release the operation intervention has been performed (step S21). For example, when the operation intervention switch 11 is switched off, or when a release switch other than the operation intervention switch 11 is turned on, or when a voice input indicating release of the operation intervention is initiated, the operation intervention determination processing unit 63 determines that an operation to release the operation intervention has been performed.

[0095] If the operation to release the operation intervention is not performed (S21 / No), the operation intervention determination processing unit 63 repeatedly performs the determination of step S21. If the operation to release the operation intervention has been performed (S21 / Yes), the operation intervention determination processing unit 63 stops accepting the operation input based on the voice of the passenger other than the driver (step S23).

[0096] Thus, when the driver is unable to drive, the control device 50 of the present embodiment can perform operation intervention based on the voice of a fellow passenger other than the driver.

[0097] As described above, according to the vehicle control device 50 of the present embodiment, when it is determined that the driver (operating subject) is in a state of being unable to drive, the operation intervention of passengers other than the driver can be performed. By allowing the acceptance of operation input based on voice input of passengers other than the driver, the passengers can use voice input to slow down the vehicle 1 and evade to the roadside and stop the vehicle 1, or go to the hospital, etc. while operating the steering wheel 21. Therefore, even if the driver suddenly becomes ill and loses consciousness, it is possible to evacuate or send to the hospital in an emergency without causing a vehicle accident.

[0098] In addition, the vehicle control device 50 according to the present embodiment determines the non-drivable state of the driver based on the output signals of the camera 13 and the microphone 15 provided inside the vehicle compartment, the operation amounts of the driving system devices, the speech content of the passengers, or the biological information of the driver (the operating subject person), and thus can improve the accuracy of detecting the non-drivable state of the driver.

[0099] In addition, the vehicle control device 50 according to the present embodiment is configured to notify the gist thereof when accepting an operation input based on the voice of a passenger other than the driver (the operating subject person). Therefore, the passenger can easily know that the operation intervention by the passenger can be performed. Therefore, the passenger can perform the driving control of the vehicle 1 instead of the driver, and quickly perform an emergency evasion or send the vehicle 11 to the hospital.

[0100] In addition, the vehicle control device 50 according to the present embodiment is configured to cancel the operation intervention by the passenger when an intervention cancellation operation is performed after starting to accept an operation input based on the voice of a passenger other than the driver (the operating subject person). Therefore, it is possible to prevent a situation where the driving control returns to being mainly by the driver only although the driver has not recovered from the non-drivable state. In addition, in a situation where the driver temporarily closes his eyes or looks around, etc., it is possible to quickly resume the driving control mainly by the driver when the driver is not truly in a non-drivable state.

[0101] As described above, although the preferred embodiments of the present invention have been described in detail with reference to the drawings, the present invention is not limited to such examples. Those skilled in the technical field to which the present invention pertains should understand that various modification examples or correction examples can be conceived within the scope of the technical idea described in the claims, and it is understood that these modification examples or correction examples also naturally belong to the technical scope of the present invention.

[0102] For example, the vehicle 1 described using the above embodiment is a vehicle 1 that performs so-called level 1 autonomous driving control. Although the steering operation in an emergency is manually performed by a passenger or the like, the present invention is not limited to this example. As long as it is a vehicle that performs driving operations with a human as the operating subject, the present invention can be applied. For example, the vehicle can be configured such that, during normal driving, the driver performs all steering operations and speed control in a non-autonomous driving state, while on the other hand, only when the driver is in a non-drivable state, an operation input based on voice is accepted and driving control is executed.

[0103] In addition, depending on the level of autonomous driving of the vehicle, the operating subject person may not be limited to the person sitting in the driver's seat. Therefore, although in the above-described embodiment, the driver is described as the operating subject person, the operating subject person is not limited to the so-called driver, and may also be a person who performs driving control using sound from an arbitrary position inside the vehicle compartment.

[0104] In addition, in the above-described embodiment, the feedback notified to the vehicle compartment when receiving a voice-based operation input is not limited to the voice feedback based on the voice synthesis processing unit 59, and may also be feedback based on other methods such as image display.

Claims

1. A control device for a vehicle, characterized in that, It controls the vehicle based on voice input. The vehicle control device includes: A voice signal processing unit that performs voice signal processing using the voice signal transmitted by the microphone; A voice recognition processing unit that recognizes a first language and a second language based on the voice data processed by the voice signal processing unit. The first language is a language indicating an operation instruction for a driving system device or a non-driving system device, and the second language is a language indicating an emergency, including languages such as emergency, extraordinary situation, or international distress signal; An operation input processing unit that accepts a driving operation input of the vehicle based on the first language in the voice data of the vehicle's operation subject person or a passenger according to the processing result of the voice recognition processing unit; And An operation intervention determination processing unit that determines whether the operation subject person is in a predetermined non-drivable state where they are unable to perform a driving operation or have not performed a driving operation. When the voice data contains the first language from the operation subject person and the passenger but does not contain the second language, the operation intervention determination processing unit determines that the operation subject person is not in the non-drivable state. The operation input processing unit only accepts the driving operation input of the vehicle based on the first language in the voice data of the operation subject person, and performs a driving operation based on the will of the operation subject person alone. When the voice data contains the first language from the passenger but does not contain the second language, the operation intervention determination processing unit determines that the operation subject person is not in the non-drivable state. The operation input processing unit prohibits the acceptance of the driving operation input of the vehicle based on the first language in the voice data of the passenger. When the voice data contains the second language, the operation intervention determination processing unit determines that the operation subject person is in the non-drivable state. When it is determined that the operation subject person is in the non-drivable state, the operation input processing unit allows the acceptance of the driving operation input of the vehicle based on the first language in the voice data from the passenger. After starting to accept the driving operation input of the vehicle based on the first language in the voice data of the passenger, the operation input processing unit sequentially outputs operation instructions to a driving system operation processing unit that performs operation processing on the driving system device of the vehicle according to a plurality of different driving operation inputs of the vehicle accepted. When the driving operation input of the vehicle based on the first language in the voice data of the passenger accepted is an input for commanding the vehicle to accelerate, an operation instruction for setting the output of the vehicle's drive source as a predetermined output is output. When an intervention cancellation operation is performed, the operation input processing unit ends the acceptance of the operation input of the vehicle based on the first language in the voice data of the passenger.

2. The vehicle control device according to claim 1, wherein When the voice data includes the second language indicating the emergency situation and the posture of the operator is in an abnormal state, the operation intervention determination processing unit determines that the operator is in the non-drivable state.

3. The vehicle control device according to claim 1, characterized in that: When the voice data includes the second language indicating the emergency situation and the driving operation of the operator is in an abnormal state, the operation intervention determination processing unit determines that the operator is in the non-drivable state.

4. The vehicle control device according to claim 1, characterized in that: When the voice data includes the second language indicating the emergency situation and a passenger has performed an intervention operation, the operation intervention determination processing unit determines that the operator is in the non-drivable state.

5. The vehicle control device according to claim 1, characterized in that: When the voice data includes the second language indicating the emergency situation and the biometric information of the operator is in an abnormal state, the operation intervention determination processing unit determines that the operator is in the non-drivable state.

6. The vehicle control device according to any one of claims 1 to 5, characterized in that: The vehicle control device is provided with a notification unit. When it is determined that the operator is in the non-drivable state, the notification unit notifies that the driving operation input of the vehicle based on the first language in the voice data of the co-passenger is accepted.

7. The vehicle control device according to any one of claims 1 to 5, characterized in that: The vehicle control device further includes: A face recognition processing unit that determines, based on an image signal output by a camera provided in the vehicle compartment in a manner capable of taking pictures, at which position in the vehicle compartment the mouth of a person is open; A sound direction determination processing unit that determines the direction of sound based on a sound signal transmitted by a microphone that collects the sound in the vehicle compartment; And A voice emitter determination processing unit that determines which one of the operator and the co-passenger emits the sound based on the determination result of the face recognition processing unit and the determination result of the sound direction determination processing unit.

8. The vehicle control device according to claim 6, characterized in that: The vehicle control device further includes: A face recognition processing unit that determines, based on an image signal output by a camera provided in the vehicle compartment in a manner capable of taking pictures, at which position in the vehicle compartment the mouth of a person is open; A sound direction determination processing unit that determines the direction of sound based on a sound signal transmitted by a microphone that collects the sound in the vehicle compartment; And A voice emitter determination processing unit that determines which one of the operator and the co-passenger emits the sound based on the determination result of the face recognition processing unit and the determination result of the sound direction determination processing unit.

Images