Vehicle control device, vehicle control method, and vehicle control program

Abstract

This vehicle control device comprises: an obstacle detection unit (30) that detects an obstacle, which is at least one of a traffic restriction and road construction on a roadway, on the basis of sound information obtained by an acoustic sensor (16) that collects the sounds surrounding a host vehicle, and determines the details of the detected obstacle; a control determination unit (35) that, on the basis of the determination results from the obstacle detection unit, determines execution of at least one type of vehicle control from among reroute control, sending of notification to an occupant of the host vehicle, speed control of the host vehicle, and volume control of an audio device (13) of the host vehicle; and execution units (34, 38) that execute the vehicle control determined by the control determination unit.

Description

Vehicle control device, vehicle control method, and vehicle control program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Patent Application No. 2024-123449, filed on July 30, 2024, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a vehicle control device, a vehicle control method, and a vehicle control program for controlling a vehicle.

[0003] Conventionally, there is known a vehicle control device that uses sound information obtained by collecting sounds around the vehicle using an acoustic sensor mounted on the vehicle to detect traffic regulations and obstacles on the road on which the vehicle is traveling, and notifies the driver of the detection results and performs vehicle control such as driving assistance (for example, Patent Document 1).

[0004] The vehicle control device described in Patent Document 1 is equipped with a camera that captures images in front of the vehicle and an acoustic sensor that collects sounds around the vehicle, and detects traffic regulations based on image information from the camera and sound information from the acoustic sensor, and provides remote support to the vehicle while it is driving autonomously.

[0005] Japanese Patent Application Laid-Open No. 2022-173769

[0006] Although the above-mentioned vehicle control device is capable of providing appropriate remote support for traffic regulations assuming the presence of traffic control officers, there is room for improvement in providing desirable vehicle control tailored to the situation, since the scale and type of road regulations and obstacles vary. Furthermore, this vehicle control device is based on the assumption that a camera is used in addition to an acoustic sensor, and it is difficult to respond to situations where there are no traffic control officers or when some kind of obstacle exists between the vehicle and the location of a traffic regulation or obstacle such as construction until the vehicle is immediately before the regulation or obstacle.

[0007] The present disclosure relates to a vehicle control device, a vehicle control method, and a vehicle control program that can detect obstacles such as traffic restrictions and construction on a road and execute appropriate vehicle control.

[0008] According to one aspect of the present disclosure, a vehicle control device includes: an obstacle detection unit that detects an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained by an acoustic sensor that collects sounds around the vehicle, and determines the content of the detected obstacle; a control decision unit that decides to execute at least one vehicle control from among reroute control, notification to occupants of the vehicle, speed control of the vehicle, and volume control of an audio device of the vehicle, based on the determination result by the obstacle detection unit; and an execution unit that executes the vehicle control determined by the control decision unit.

[0009] In this vehicle control device, an obstacle detection unit detects and determines the nature of obstacles such as traffic restrictions and construction using sound information obtained by an acoustic sensor, and a control decision unit decides to execute vehicle control in accordance with the determination result of the obstacle detection unit. The control decision unit decides to execute at least one of reroute control, notification to occupants, speed control, and volume control of an audio device. The execution unit then executes the vehicle control determined by the control decision unit. This enables the vehicle control device to detect obstacles on the road and execute appropriate vehicle control in accordance with the nature of the obstacle.

[0010] According to one aspect of the present disclosure, a vehicle control method is a vehicle control method that can be used in a vehicle, and includes: detecting an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained by an acoustic sensor that collects sounds around the vehicle, and determining the content of the detected obstacle; and executing at least one vehicle control among rerouting control, notifying an occupant of the vehicle, controlling the speed of the vehicle, and controlling the volume of an audio device of the vehicle, based on the determination result of the content of the detected obstacle.

[0011] This vehicle control method includes detecting and determining the nature of obstacles such as traffic restrictions and construction using sound information obtained by an acoustic sensor, and executing vehicle control in accordance with the determination result. The vehicle control executed is at least one of rerouting control, notification to occupants, speed control, and volume control of an audio device. This makes it possible to detect obstacles on the road and execute appropriate vehicle control in accordance with the nature of the obstacle.

[0012] According to one aspect of the present disclosure, a vehicle control program is a vehicle control program used to control the driving state of the host vehicle or on-board equipment, and causes at least one execution unit to execute processes including: a process of detecting an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained by an acoustic sensor that collects sounds around the host vehicle; a process of determining the content of the detected obstacle; and a process of executing at least one vehicle control among reroute control, notification to an occupant of the host vehicle, speed control of the host vehicle, and volume control of an audio device of the host vehicle, based on the determination result of the content of the obstacle.

[0013] This vehicle control program includes a process for detecting obstacles such as traffic restrictions and construction using sound information obtained by an acoustic sensor, a process for determining the nature of the detected obstacle, and a process for executing vehicle control in accordance with the determination result of the nature of the obstacle. The vehicle control executed is at least one of rerouting control, notification to occupants, speed control, and volume control of an audio device. This results in a vehicle control program that can detect obstacles on the road and execute appropriate vehicle control in accordance with the nature of the obstacle.

[0014] 1 is a block diagram showing an example of a vehicle control system according to an embodiment; FIG. 2 is a block diagram showing a vehicle control unit and related components; FIG. 3 is a flowchart showing first vehicle processing; FIG. 4 is a flowchart showing second vehicle processing; FIG. 5 is a flowchart showing third vehicle processing; FIG. 6 is a flowchart showing sixth vehicle processing;

[0015] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the following embodiments, identical or equivalent parts will be denoted by the same reference numerals.

[0016] (Embodiment) A vehicle control system 1 according to an embodiment will be described. The vehicle control system 1 is mounted on a vehicle such as a general private passenger car, and various functions described below are executed by a vehicle control unit 26 shown in FIG. 1. For ease of explanation, the vehicle on which the vehicle control system 1 is mounted will be referred to as the "host vehicle." The vehicle control system 1 is preferably applied to, for example, an autonomous driving level of 2 or less defined by the Society of Automotive Engineers, i.e., when the driver is a human, but is not limited to this case.

[0017] The vehicle on which the vehicle control system 1 is installed is not limited to a passenger car, but may be a rental car vehicle, a manned taxi vehicle, a ride-sharing vehicle, a freight vehicle, a bus, etc., and may be either a right-hand drive vehicle or a left-hand drive vehicle. Furthermore, each vehicle control described below according to the present disclosure can be optimized as appropriate according to the road traffic laws of each country and region, as well as the steering wheel position of the vehicle, etc.

[0018] [Basic Configuration] As shown in FIG. 1 , the vehicle control system 1 includes an HMI system 100 mounted on the vehicle, various sensors such as a perimeter monitoring sensor 14, various devices such as a cabin camera 20, a cruise control ECU 24, and a vehicle control unit 26, all connected via a communication bus 39. HMI and ECU are abbreviations for Human Machine Interface and Electronic Control Unit, respectively. The various sensors constituting the vehicle control system 1 are capable of communicating with each other via the communication bus 39, for example. However, the connection is not limited to this, and some of the various sensors may be capable of communicating without the communication bus 39. Examples of the various sensors include the perimeter monitoring sensor 14, a wheel speed sensor 18, and an occupant sensor 19. Examples of the various devices include the cabin camera 20, an on-board communication device 21, a locator 22, a navigation device 23, and an air conditioning device 25.

[0019] The HMI system 100 has an input interface function that accepts operations by a vehicle occupant, such as a driver, and an output interface function that presents information to the occupant. The HMI system 100 is configured to include, for example, an HMI control device 10, an input device 11, a display device 12, and an audio device 13.

[0020] The HMI control device 10 is a computer that mainly includes a control circuit equipped with, for example, a CPU, recording media such as ROM and RAM, an input / output interface, and a bus connecting these. The CPU, ROM, and RAM are abbreviations for Central Processing Unit, Read Only Memory, and Random Access Memory, respectively. The HMI control device 10 functions as a presentation control device and comprehensively controls the presentation of information to occupants using the display device 12, audio device 13, etc. The HMI control device 10 presents information related to driving assistance provided by the system in cooperation with the vehicle control unit 26.

[0021] The input device 11 is an input unit that accepts operations by a user such as a driver. Various user operations, such as operations related to setting a destination for route guidance and operations related to setting air conditioning, are input to the input device 11. The input device 11 includes, for example, a steering switch provided on the spokes of the steering wheel, an operating lever provided on the steering column, and a voice input device that recognizes what the passengers are saying.

[0022] The display device 12 is a device that presents information to the occupant's vision by displaying an image or the like. Examples of the display device 12 include a meter display, a center information display (CID), and a head-up display (HUD). For example, in the case of a CID, the display device 12 has a touch panel function and is configured to allow the driver or the like to touch the display screen.

[0023] The audio device 13 has multiple speakers installed in the vehicle cabin and reproduces various sounds, such as notification sounds and voice messages associated with vehicle control by a vehicle control unit 26 (described later) and voice messages associated with user operations on the input device 11, through the speakers.

[0024] The perimeter monitoring sensor 14 is an autonomous sensor that monitors the environment around the vehicle and is composed of, for example, a camera unit 15, an acoustic sensor 16, and an object detection sensor 17 mounted on the vehicle. The perimeter monitoring sensor 14 detects, for example, targets such as moving objects and stationary objects around the vehicle, and obstacles such as traffic restrictions due to construction work or checkpoints, and outputs the detection information to the vehicle control unit 26, etc. The detection information by the perimeter monitoring sensor 14 includes, for example, information on the direction of the target and the distance from the vehicle to the target when a target is detected, and information on the type and scale of the obstacle when an obstacle is detected.

[0025] The camera unit 15 is an imaging device that captures images of the outside of the vehicle. The camera unit 15 includes, for example, a front camera module, a rear camera module, a left side camera module, and a right side camera module, and is configured to capture images of the entire surroundings of the vehicle. The camera unit 15 analyzes image data captured by each camera module using, for example, a known image analysis technique and outputs the analysis information as detection information to the communication bus 39. The detection information from the camera unit 15 includes, for example, information on other objects around the vehicle, such as vehicles, light vehicles, people, obstacles, buildings, road signs, and trees; road surface information such as white lines and road markings painted on the road; and traffic information such as traffic light displays.

[0026] The acoustic sensor 16 is primarily composed of a microphone element that collects sound from outside the vehicle and converts the collected sound into an electrical signal. Examples of microphone elements include a capacitor microphone that converts a change in capacitance caused by a thin diaphragm vibrating due to sound pressure into an electrical signal, and a piezoelectric microphone that converts sound into an electrical signal using a piezoelectric element. Multiple acoustic sensors 16 are provided, for example, on the front, rear, left and right sides, and ceiling of the vehicle. The acoustic sensors 16 collect sounds from the vehicle's surrounding environment, i.e., ambient sounds, and output the collected sound data as sound information to the communication bus 39. The acoustic sensors 16 are particularly used to detect road obstacles, such as road construction and traffic restrictions, as described below. Details of obstacle detection based on sound information will be described later.

[0027] The object detection sensor 17 is a device that detects objects present around the vehicle by transmitting a transmission wave to the outside of the vehicle, receiving a reflected wave generated when the transmission wave is reflected from the outside, and analyzing the reflected wave. The object detection sensor 17 may be, for example, a millimeter-wave radar or LiDAR, which transmits laser light or radio waves as a transmission wave. LiDAR is an abbreviation for Light Detection and Ranging. The object detection sensor 17 is mounted, for example, on the front, rear, left, or right side of the vehicle, and outputs information, such as the presence or absence of objects, such as other vehicles or obstacles, present around the vehicle and the distance to the detected objects, as object information to the communication bus 39.

[0028] The wheel speed sensors 18 are provided, for example, on each wheel of the vehicle, and output an electric signal corresponding to the rotation speed of the wheel to the communication bus 39. The output signal from the wheel speed sensors 18 is used, for example, in the cruise control ECU 24 to calculate the traveling speed of the vehicle, i.e., the vehicle speed.

[0029] The occupant sensor 19 is a device that detects the presence of an occupant in the vehicle cabin. Examples of the occupant sensor 19 include a pressure sensor that detects the load on a seat, and a radio wave or ultrasonic sensor that detects an occupant by transmitting a transmission wave such as an electric wave or an ultrasonic wave into the vehicle cabin and receiving the reflected wave. The occupant sensor 19 outputs information about the presence or absence of the detected occupant to the communication bus 39, for example.

[0030] The interior camera 20 is an imaging device that captures images of the interior of the vehicle. The interior camera 20 captures images of occupants seated in the driver's seat, passenger seat, and rear seats, and analyzes the captured image data using known image analysis techniques to acquire occupant information regarding the type and condition of the occupants. Examples of the occupant type include gender, whether the occupant is an adult or a child, and estimated information about the age of the occupant. Examples of the occupant condition include whether the occupant is awake or not, whether the occupant is calm or not, and the direction of their gaze. The interior camera 20 outputs the acquired occupant information to the communication bus 39 and provides it to the HMI control device 10, the vehicle control unit 26, and the like. In addition to analyzing the captured image data from the interior camera 20, the occupant information may also be acquired based on sounds collected within the vehicle interior by an acoustic sensor installed in the vehicle interior. For example, occupant information may be obtained by identifying the type of occupant based on the occupant's voice collected by an acoustic sensor inside the vehicle cabin, and estimating the occupant's condition, such as whether they are awake or not, or whether they are calm or not, based on whether they are talking and the volume of their voice.

[0031] The in-vehicle communication device 21 is an external communication unit mounted on the vehicle itself, and functions as, for example, a V2X (Vehicle to Everything) communication device. The in-vehicle communication device 21 receives various types of information, such as congestion information, signal information indicating the lighting patterns of traffic signals, and detection information of stopped vehicles, parked vehicles, pedestrians, etc., by transmitting and receiving information via wireless communication with roadside devices installed on the side of the road. The in-vehicle communication device 21 provides the received various types of information to the HMI control device 10, the navigation device 23, the vehicle control unit 26, etc.

[0032] The locator 22 includes a GNSS receiver, an inertial sensor, and the like. GNSS is an abbreviation for Global Navigation Satellite System. The locator 22 sequentially determines the position and traveling direction of the vehicle by combining positioning signals received from multiple positioning satellites by the GNSS receiver, measurement results from the inertial sensor, and vehicle speed information output to the communication bus 39. The locator 22 sequentially outputs position information and direction information of the vehicle based on the positioning results to the communication bus 39 as locator information.

[0033] The locator 22 further includes, for example, a map database storing map data. The map database is mainly composed of a large-capacity storage medium storing a large number of three-dimensional map data and two-dimensional map data. The three-dimensional map data is a so-called HD (High Definition) map and includes road information necessary for driving under the control of the vehicle control system 1. Specifically, the three-dimensional map data includes three-dimensional shape information of roads and detailed information about each lane. The locator 22 can update the three-dimensional map data and two-dimensional map data to the latest information through external communication using the on-board communication device 21. For example, the locator 22 reads map data around the current location from the map database and provides it to the vehicle control unit 26, etc., along with locator information.

[0034] The navigation device 23 acquires information about a destination specified by a passenger such as a driver based on operation information acquired from the HMI control device 10. The navigation device 23 acquires vehicle position information and direction information from the locator 22, and sets a route from the current position to the destination. The navigation device 23 provides route information indicating the set route to the destination to the HMI control device 10, the vehicle control unit 26, etc. The navigation device 23 works in conjunction with the HMI system 100 to provide route guidance to the destination by combining screen displays and voice messages, etc., and notifying the driver of the direction in which the vehicle should travel at intersections, branching points, etc.

[0035] The cruise control ECU 24 is an electronic control device that mainly includes a microcontroller. For example, the cruise control ECU 24 calculates the vehicle's traveling speed based on output signals from the wheel speed sensors 18, generates vehicle speed information indicating the calculated traveling speed, and outputs the generated vehicle speed information to the communication bus 39. The cruise control ECU 24 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The cruise control ECU 24 continuously controls the braking force of each wheel, the output of the powertrain, and the steering angle based on either an operation command based on the driver's driving operation or a control command from the vehicle control unit 26. The powertrain is an engine that generates power for propelling the vehicle, and may be, for example, an internal combustion engine that burns fuel such as gasoline, a drive motor / generator, or a hybrid power unit including multiple power sources.

[0036] The air conditioning device 25 includes, for example, an air conditioning unit mounted on the vehicle and an air conditioning ECU that controls the air conditioning unit. The air conditioning ECU is an electronic control device that mainly includes a microcontroller. The air conditioning device 25 adjusts the operating state of the air conditioning unit, specifically, the temperature, air volume, and direction of the air outlet of the conditioned air, based on operation information acquired from the HMI control device 10, for example.

[0037] The vehicle control unit 26 is a computer that mainly includes a control circuit equipped with, for example, a CPU, recording media such as ROM and RAM, an input / output interface, and a bus connecting these. The vehicle control unit 26 receives output data from the above-mentioned various sensors and on-board devices via a communication bus 39 and reads and executes various vehicle control programs stored in a recording medium (not shown), thereby performing various types of vehicle control. The vehicle control unit 26 corresponds to a vehicle control device that performs various types of vehicle control of the host vehicle, and in the vehicle control described below, controls various devices such as the display device 12, the audio device 13, the navigation device 23, the cruise control ECU 24, and the air conditioning device 25.

[0038] The above is the basic configuration of the vehicle control system 1. The vehicle control system 1 is not limited to the above configuration, and may include other known in-vehicle devices such as an external alarm and an ambient light, as needed.

[0039] 2, the vehicle control unit 26 includes an environment recognition unit 27, a device linkage unit 32, a control decision unit 35, and a control execution unit 38. The vehicle control unit 26 corresponds to an execution unit that reads and executes vehicle control programs for executing first to sixth vehicle controls described below.

[0040] The environment recognition unit 27 recognizes the environment around and inside the vehicle based on output data from, for example, the perimeter monitoring sensor 14, the wheel speed sensor 18, the occupant sensor 19, the cabin camera 20, and the locator 22, and provides the results to the device linkage unit 32 and the control decision unit 35. The environment recognition unit 27 recognizes the driving environment of the vehicle by, for example, combining locator information and map data acquired from the locator 22 with detection information acquired from the perimeter monitoring sensor 14. The environment recognition unit 27 may acquire detection information received by the in-vehicle communication device 21 from the device linkage unit 32 and use it for recognition of the driving environment. The environment recognition unit 27 has, for example, a pre-trained sound discrimination model based on environmental sounds such as construction sounds stored in an internal recording medium, and combines the sound discrimination model with sound information acquired from the perimeter monitoring sensor 14 to recognize obstacles such as road construction and traffic restrictions. The environment recognition unit 27, for example, acquires vehicle speed information indicating the current traveling speed of the vehicle from the wheel speed sensor 18, and acquires occupant information regarding the number and status of occupants in the vehicle from the occupant sensor 19. The environment recognition unit 27 includes, for example, a target recognition unit 28, a road recognition unit 29, an obstacle detection unit 30, and an occupant recognition unit 31.

[0041] The target recognition unit 28 recognizes the relative position, relative speed, etc. of targets such as other vehicles present around the vehicle, based on, for example, analysis information of the image data captured by the camera unit 15 and detection information from the object detection sensor 17. The target recognition unit 28 may recognize the approach of emergency vehicles, including police vehicles, fire engines, and ambulances, to the vehicle, by using detection information based on sound data from the acoustic sensor 16.

[0042] The road recognition unit 29 acquires road information related to the road on which the vehicle is traveling or the road on which the vehicle is scheduled to travel, based on, for example, locator information, map data, and analysis information of image data captured by the camera unit 15. Examples of road information include whether the road on which the vehicle is traveling has multiple lanes on each side, whether the lane on which the vehicle is traveling is a shoulder lane adjacent to the shoulder, and so on.

[0043] The obstacle detection unit 30, for example, analyzes sound data collected by the acoustic sensor 16 using a sound discrimination model to detect the presence or absence of obstacles such as construction and traffic restrictions on the road on which the vehicle is traveling, as well as on roads other than the road on which the vehicle is traveling. The obstacle detection unit 30, for example, combines analysis information of the image data captured by the camera unit 15 with the analysis results of the sound data using the sound discrimination model to estimate whether the obstacle is construction, traffic restrictions, or a checkpoint. When the obstacle detection unit 30 detects an obstacle, it estimates the scale of the construction or traffic restrictions using the sound discrimination model. From the perspective of improving the accuracy of estimating the scale or type, the obstacle detection unit 30 may estimate the scale of the obstacle by combining the analysis information from the sound discrimination model with vehicle speed information acquired by the target recognition unit 28 or analysis information from other on-board devices such as the camera unit 15.

[0044] The sound discrimination model is a learning model that has undergone machine learning in advance using training data that includes various types of audio data generated, for example, at road construction sites, traffic control sites, or checkpoints. The learning model is, for example, configured as a neural network and stored on a recording medium (not shown) in the vehicle control unit 26. The audio data that constitutes the training data is, for example, stored as a database on a recording medium separate from the vehicle control system 1 and used for pre-training the sound discrimination model. Examples of audio data include engine sounds and work sounds from construction vehicles such as work trucks, material transport vehicles, and other large vehicles, as well as horns from traffic controllers and police officers. The sound discrimination model may be downloaded or updated from an external network via the on-board communication device 21, for example. The sound discrimination model is configured to, for example, compare the results of the pre-training with characteristics of the sound, such as the frequency band and occurrence interval, in the sound data collected by the acoustic sensor 16, thereby determining and classifying whether the collected sound is due to construction or traffic control.

[0045] The occupant recognition unit 31 recognizes occupant information regarding the number, type, and status of occupants in the vehicle based on, for example, the output signal of the occupant sensor 19 and analysis information of the image data captured by the vehicle interior camera 20 .

[0046] The environment recognition unit 27 sequentially provides the control decision unit 35 with the information detected / recognized by the target recognition unit 28, road recognition unit 29, obstacle detection unit 30, and occupant recognition unit 31, i.e., the recognition results of the environment around and inside the vehicle.

[0047] The device linkage unit 32 enables information linkage between various in-vehicle devices, such as the display device 12, the audio device 13, the in-vehicle communication device 21, the navigation device 23, the air conditioning device 25, and the HMI control device 10, and the vehicle control unit 26, and controls the various in-vehicle devices. The device linkage unit 32 acquires route information from the navigation device 23 and provides the acquired route information to the environment recognition unit 27 and the control decision unit 35. For example, when the control decision unit 35 performs rerouting control (detour control), the device linkage unit 32 requests the navigation device 23 to change the planned driving route by outputting a rerouting request to the navigation device 23. The device linkage unit 32 provides detection information received by the in-vehicle communication device 21 to the environment recognition unit 27. The device linkage unit 32 includes, for example, a route determination unit 33 and a device control unit 34.

[0048] When the obstacle detection unit 30 detects a road obstacle, the route determination unit 33 determines whether the road determined to have the obstacle matches the road on which the vehicle is currently traveling or the road on which the vehicle is scheduled to travel. The route determination unit 33 determines whether the vehicle will travel on a road with an obstacle by, for example, comparing the detection information by the obstacle detection unit 30 with road information on the road on which the vehicle is currently traveling, obtained from the locator 22, or route information, obtained from the navigation device 23. The determination result by the route determination unit 33 is provided to the control decision unit 35 and used for vehicle control.

[0049] The device control unit 34 requests various in-vehicle devices to perform predetermined processing corresponding to the vehicle control determined by the control determination unit 35. The device control unit 34 executes, for example, a request to the display device 12 and / or the audio device 13 to notify the driver of an alert within the vehicle, a request to the audio device 13 to adjust the volume, a request to the navigation device 23 to perform reroute control, etc. The device control unit 34 corresponds to an execution unit that executes the vehicle control determined by the control determination unit 35.

[0050] The control decision unit 35 decides the control content of the vehicle based on the recognition results of the driving environment of the vehicle and the occupants by the environment recognition unit 27. When the obstacle detection unit 30 determines that there is an obstacle on the road, the control decision unit 35 decides to execute at least one vehicle control from among reroute control, notification to the occupants of the vehicle, speed control of the vehicle, and volume control of the audio device 13. The control decision unit 35 includes, for example, a driving control unit 36 ​​and a volume control unit 37.

[0051] When it is determined that there is an obstacle on the road, the driving control unit 36 ​​outputs a request signal corresponding to at least one of reroute control, notification to the occupants of the vehicle, and speed control of the vehicle. When it is determined that reroute control or notification to the occupants is to be performed, for example, the driving control unit 36 ​​outputs a request signal to the equipment control unit 34, and executes reroute control in the navigation device 23 or control of notification to the occupants by the display device 12 or the like via the equipment control unit 34. When it is determined that speed control of the vehicle is to be performed, for example, the driving control unit 36 ​​outputs a request signal to the control execution unit 38.

[0052] For example, when it is determined that there is construction or traffic restrictions of a predetermined scale or larger on a road, the driving control unit 36 ​​performs reroute control to guide the vehicle to a road other than the road on which the obstacle is detected, or performs notification control to warn large vehicles. In this case, when it is determined that the occupants include a child who is awake, the driving control unit 36 ​​may perform notification control to confirm whether or not to perform reroute control before performing reroute control. Furthermore, when it is determined that there is construction or traffic restrictions of a smaller scale than a predetermined scale on a road, the driving control unit 36 ​​performs notification control to the occupants and suggests changing to another appropriate lane on the road currently being traveled. For example, when it is determined that there is an obstacle on the road, the driving control unit 36 ​​performs deceleration control to reduce the vehicle speed of the vehicle when the vehicle passes near a road with construction or traffic restrictions, or performs speed adjustment control to increase the distance from other vehicles ahead.

[0053] When it is determined that there is an obstacle on the road and it is decided to control the volume of the audio device 13, the volume control unit 37 outputs a request signal to the device control unit 34 to increase or decrease the volume of the audio device 13. When the obstacle detection unit 30 estimates that the obstacle is construction, the volume control unit 37 requests that the volume of the audio device 13 be increased to match the noise level of the construction noise so that the occupants can easily hear the sound of the audio device 13. When the obstacle detection unit 30 estimates that the obstacle is a checkpoint, the volume control unit 37 requests that the volume of the audio device 13 be decreased to a predetermined level so that the driver and others can easily hear instructions at the checkpoint. Note that when it is estimated that the obstacle is a checkpoint, the volume control unit 37 may request the device control unit 34 to decrease the volume of the audio device 13 as well as to decrease the output of the air conditioning unit 25 to reduce the sound inside the vehicle cabin caused by the air conditioning.

[0054] For example, when a speed control request is made by the cruise control unit 36, the control execution unit 38 generates a control command signal corresponding to decelerating the host vehicle or adjusting the distance between the host vehicle and the preceding vehicle, and outputs the generated signal to the cruise control ECU 24. In this case, the control execution unit 38 generates, for example, a command signal for deceleration control when no preceding vehicle is detected, and a command signal for adjusting the distance between the host vehicle and the preceding vehicle when a preceding vehicle is detected. Similar to the device control unit 34, the control execution unit 38 corresponds to an execution unit that executes the vehicle control determined by the control determination unit 35.

[0055] The basic configuration of the vehicle control unit 26 has been described above.

[0056] [First Vehicle Control] Next, the first vehicle control executed by the vehicle control system 1 will be described.

[0057] The vehicle control system 1 executes the control flow shown in FIG. 3 when a predetermined start condition is met, for example, when the ignition of the vehicle is turned on.

[0058] In step S110, external sound is collected by the acoustic sensor 16 mounted on the vehicle. The acoustic sensor 16 outputs sound data corresponding to the collected external sound as sound information to the vehicle control unit 26 via the communication bus 39.

[0059] In step S120, for example, the obstacle detection unit 30 determines the type of sound contained in the acquired external sound based on the sound information acquired in step S110 and the machine-learned sound discrimination model.

[0060] In step S130, the obstacle detection unit 30 determines whether an obstacle has been detected, for example, based on whether the determination result in step S120 includes sounds resulting from construction work, traffic restrictions, or checkpoints. If the determination in step S130 is affirmative, the vehicle control unit 26 proceeds to step S140, and if the determination in step S130 is negative, the vehicle control unit 26 returns to step S110.

[0061] In step S140, for example, the obstacle detection unit 30 determines whether the scale of the detected obstacle is equal to or greater than a predetermined scale based on the sound information acquired in step S110. The scale of the obstacle can be defined as, for example, a small scale (less than a predetermined scale) when the obstacle is contained within a single lane, and a large scale (above a predetermined scale) when the obstacle spans multiple lanes. For example, the determination in step S140 can be made based on whether the sound pressure of the sound determined to be due to construction, traffic restrictions, or a checkpoint is equal to or greater than a predetermined scale. If the determination in step S140 is affirmative, the vehicle control unit 26 proceeds to step S150, and if the determination in step S140 is negative, the vehicle control unit 26 proceeds to step S160.

[0062] In step S150, for example, the control decision unit 35 decides to execute a detour proposal notification and sends a notification request to the device control unit 34. Then, the device control unit 34 outputs a notification command signal to, for example, the display device 12 or the audio device 13, or both, proposing to change the planned driving route and take a detour. As a result, the driver of the vehicle is notified of the detour proposal message by image, sound, or both, making it easier for the driver to avoid the obstacle.

[0063] In step S151, for example, the control decision unit 35 determines whether the driver of the vehicle has agreed to the detour proposal made in step S150. The determination in step S151 can be made based on whether consent has been input to the input device 11 within a predetermined time period since the execution of the process of step S150. If the determination in step S151 is affirmative, the vehicle control unit 26 proceeds to step S152, and if the determination in step S151 is negative, the vehicle control unit 26 proceeds to step S153.

[0064] In step S152, for example, the control decision unit 35 decides to execute reroute control and requests the device control unit 34 to execute reroute control. Then, the device control unit 34 outputs a command signal to the navigation device 23 to reset the planned route for the host vehicle to a road other than the road on which the obstacle was detected. This allows the host vehicle to bypass the road with the obstacle, enabling a comfortable drive.

[0065] In step S153, for example, the control decision unit 35 decides to execute a notification to warn the driver of large vehicles and sends a notification request to the device control unit 34. Then, the device control unit 34 outputs a notification command signal to warn the driver of large vehicles, for example, to the display device 12 or the audio device 13, or both. This makes it possible to warn the driver of large vehicles and improve driving safety in situations where the scale of an obstacle such as construction work is large and there is a high possibility that a large vehicle such as a construction vehicle may be present.

[0066] In the following step S154, for example, the environment recognition unit 27 determines whether or not the vehicle speed of the host vehicle is equal to or greater than a predetermined value based on the vehicle speed information from the wheel speed sensor 18, and outputs a signal according to the determination result to the control decision unit 35. Then, for example, the vehicle control unit 26 advances the process to step S155 if the determination in step S154 is affirmative, or advances the process to step S156 if the determination in step S154 is negative.

[0067] In step S155, for example, the control decision unit 35 decides to execute deceleration control of the host vehicle and requests the control execution unit 38 to execute deceleration control. Then, the control execution unit 38 outputs a command signal to the cruise control ECU 24 to reduce the vehicle speed of the host vehicle. This makes it possible to keep the vehicle speed below a predetermined value and improve driving safety. After step S155, for example, the vehicle control unit 26 advances the process to step S156.

[0068] In step S156, for example, the environment recognition unit 27 determines whether or not another vehicle is present ahead of the vehicle based on the detection information from the object detection sensor 17, and outputs a signal according to the determination result to the control decision unit 35. Then, for example, the vehicle control unit 26 proceeds to step S157 if the determination in step S156 is affirmative, and skips the processing of step S157 if the determination in step S156 is negative.

[0069] In step S157, for example, the control decision unit 35 decides to execute inter-vehicle distance control between the host vehicle and the preceding vehicle, and requests vehicle speed control from the control execution unit 38. Then, the control execution unit 38 outputs, for example, a command signal to the cruise control ECU 24 to decrease the relative speed of the host vehicle with respect to the preceding vehicle. This increases the inter-vehicle distance between the host vehicle and the preceding vehicle, making it possible to improve driving safety.

[0070] In step S160, for example, the control decision unit 35 decides to notify the driver suggesting a lane change and requests the device control unit 34 to do so. The device control unit 34 then outputs a notification command signal suggesting a lane change to, for example, the display device 12, the audio device 13, or both. At this time, for example, the environment recognition unit 27 determines the direction of an obstacle relative to the vehicle based on sound information from the acoustic sensor 16 and identifies the location of the lane on the opposite side of the obstacle on the road on which the vehicle is traveling, i.e., an appropriate driving position. The device control unit 34 then outputs a notification command suggesting that the vehicle change to a lane opposite the obstacle, based on, for example, the obstacle information and road information. This allows the system to prompt the driver to change vehicle location and ensure a comfortable driving experience that avoids the obstacle when the obstacle, such as construction work, is small and does not require a detour.

[0071] Then, the vehicle control unit 26 executes any one of steps S152, S153, S155, S157, and S160, and then returns the process to step S110. The vehicle control unit 26 also repeats the above series of processes until a predetermined termination condition is met, such as the position of the shift lever of the vehicle being in the parking position.

[0072] The vehicle control system 1 estimates the presence and scale of an obstacle through the first vehicle control, and executes appropriate vehicle control such as suggesting a change in driving position, issuing a warning, rerouting control, controlling the distance between vehicles, or deceleration control, depending on the scale of the obstacle, the speed of the host vehicle, and the presence or absence of a vehicle ahead. Note that, although the above describes an example in which the processes are executed in the order of determining the speed of the host vehicle (step S154) and determining the presence of a vehicle ahead (step S156), the order may be reversed, and the order of each process in the first vehicle control may be changed as appropriate within a possible range.

[0073] [Variation of First Vehicle Control] The obstacle scale estimation process in step S140 may be based on the sound information from the acoustic sensor 16, as well as on the vehicle speed information of the host vehicle or detection information from other sensors. For example, the obstacle detection unit 30 may estimate that the scale of the obstacle is greater than or equal to a predetermined scale when the sound pressure of the detected sound is greater than or equal to a predetermined scale and the vehicle speed of the host vehicle is less than a predetermined scale, and may estimate that the scale of the obstacle is less than or equal to a predetermined scale when the vehicle speed is greater than or equal to the predetermined scale. This is because, when the vehicle speed is less than the predetermined scale, it is assumed that the obstacle has a large impact on traffic and congestion is occurring, i.e., the obstacle is large in scale, whereas when the vehicle speed is greater than or equal to the predetermined scale, it is assumed that the obstacle has a small impact on traffic and the obstacle is small in scale.

[0074] Furthermore, for example, the obstacle detection unit 30 may estimate that the scale of the obstacle is greater than or equal to a predetermined value if the sound pressure is greater than or equal to a predetermined value and a large object such as a crane truck is detected by the camera unit 15 or the object detection sensor 17, and may estimate that the scale of the obstacle is less than or equal to a predetermined value if no large object is detected.

[0075] As described above, the vehicle control system 1 may estimate the scale of the fault based on the vehicle speed or detection information from other sensors, in addition to the sound information from the acoustic sensor 16. This improves the accuracy of the fault scale estimation compared to when sound information alone is used. In this case, for example, the vehicle control unit 26 may execute a process of acquiring vehicle speed information or detection information from other sensors, before the fault scale estimation process (step S140).

[0076] [Second Vehicle Control] Next, the second vehicle control will be described, but details of parts common to the first vehicle control will be omitted. For example, similar to the first vehicle control, when a predetermined start condition is satisfied, the vehicle control unit 26 executes the control flow shown in FIG. 4. As with the first vehicle control, the vehicle control unit 26 executes the processes of steps S110, S120, and S130 in this order. Then, if the vehicle control unit 26 makes a positive determination in step S130, it proceeds to step S210, and if the vehicle control unit 26 makes a negative determination in step S130, it returns the process to step S110.

[0077] In step S210, for example, the obstacle detection unit 30 determines whether the external vehicle sound determined in step S120 is construction noise. If the determination in step S210 is affirmative, the vehicle control unit 26 proceeds to step S220, and if the determination in step S210 is negative, the vehicle control unit 26 proceeds to step S230.

[0078] In step S220, for example, the control decision unit 35 decides to execute a notification to alert the driver to the large vehicle and makes a notification request to the device control unit 34. Then, the device control unit 34 outputs a notification command to the display device 12 or the like, similar to step S153 in the first vehicle control.

[0079] In step S230, for example, the obstacle detection unit 30 determines whether the external sound determined in step S120 is a sound originating from a checkpoint. If the determination in step S230 is affirmative, the vehicle control unit 26 proceeds to step S250, and if the determination in step S230 is negative, the vehicle control unit 26 proceeds to step S240.

[0080] In step S240, for example, the control decision unit 35 decides to execute a notification to alert the driver to traffic regulations and makes a notification request to the device control unit 34. Then, the device control unit 34 outputs a notification command to the display device 12 or the like, similar to step S220.

[0081] In step S250, for example, the control decision unit 35 decides to notify the driver that a checkpoint has been detected, and issues a notification request to the device control unit 34. Then, the device control unit 34 outputs a notification command to the display device 12 or the like, similar to step S220.

[0082] In the following step S251, for example, similar to step S154 in the first vehicle control, the environment recognition unit 27 determines the vehicle speed of the host vehicle and outputs a signal to the control decision unit 35. Then, for example, if the determination in step S251 is affirmative, the vehicle control unit 26 proceeds to step S252, and if the determination in step S251 is negative, the vehicle control unit 26 proceeds to step S253.

[0083] In step S252, for example, the vehicle control unit 26 executes the same deceleration control process as in step S155 in the first vehicle control. After step S252, for example, the vehicle control unit 26 advances the process to step S253.

[0084] In step S253, for example, similar to step S156 in the first vehicle control, the environment recognition unit 27 determines whether or not there is a vehicle ahead and outputs a signal to the control decision unit 35. Then, for example, if the determination in step S253 is affirmative, the vehicle control unit 26 proceeds to step S254, and if the determination in step S253 is negative, the vehicle control unit 26 skips the processing in step S254.

[0085] In step S254, for example, the vehicle control unit 26 executes the same process of inter-vehicle distance control as in step S157 in the first vehicle control.

[0086] Then, after executing one of steps S220, S240, S250, S252, and S254, the vehicle control unit 26 returns the process to step S110 and repeats the above series of processes until a predetermined termination condition is met.

[0087] The vehicle control system 1 estimates the presence and type of an obstacle through the second vehicle control, and executes appropriate vehicle control, such as warning the driver of a large vehicle or traffic regulations, controlling the distance between vehicles, or controlling deceleration, depending on the type of obstacle, the vehicle speed, and the presence or absence of a preceding vehicle. In the second vehicle control, the vehicle speed determination (step S251) and the presence or absence of a preceding vehicle (step S253) are executed in this order. However, the order may be reversed, and the order of the steps may be changed as appropriate within the possible range. For example, in the second vehicle control, the obstacle type determination process may be executed in the order of a checkpoint (step S230) and construction work (step S210), or a process of determining whether or not a traffic regulation exists may be executed.

[0088] [Variation of Second Vehicle Control] The obstacle type determination process in steps S210 and S230 may be based on the sound information from the acoustic sensor 16, as well as on the vehicle speed information of the vehicle or detection information from other sensors. For example, in addition to the result of determining the detected sound, the obstacle detection unit 30 may determine that the type of obstacle is a traffic restriction or a checkpoint if the vehicle speed is below a predetermined value, and that the type of obstacle is construction if the vehicle speed is above a predetermined value. Furthermore, for example, in addition to the result of determining the sound, the obstacle detection unit 30 may determine that the type of obstacle is a checkpoint if the camera unit 15 detects a police officer or a police vehicle, or that the type of obstacle is a traffic restriction if the camera unit 15 detects a traffic guide other than a police officer, and that the type of obstacle is construction otherwise.

[0089] As described above, the vehicle control system 1 may estimate the type of obstacle based on the vehicle speed or detection information from other sensors, etc., in addition to the sound discrimination result by the environment recognition unit 27. This improves the accuracy of obstacle type estimation compared to when sound information alone is used. In this case, for example, the vehicle control unit 26 may acquire vehicle speed information, etc., from other sensors, etc., before the obstacle type estimation process (steps S210 and S230).

[0090] [Third Vehicle Control] Next, the third vehicle control will be described with reference to FIG. 5, but details of parts common to the first and second vehicle controls will be omitted. In the third vehicle control, for example, similar to the second vehicle control, when a predetermined start condition is satisfied, the processing of steps S110 to S130 and S210 is executed sequentially. The control flow shown in FIG. 5 shows the processing of the third vehicle control after it is determined that the type of obstacle is construction. In the third vehicle control, for example, when it is determined that the type of obstacle is construction, the vehicle control unit 26 proceeds to the processing of step S310.

[0091] In step S310 , for example, the occupant recognition unit 31 acquires occupant information including an output signal from the occupant sensor 19 and analysis information from the vehicle interior camera 20 .

[0092] In the next step S320, for example, the occupant recognition unit 31 determines whether or not the occupants of the vehicle include children based on the occupant information acquired in step S310. If the determination in step S320 is affirmative, the vehicle control unit 26 proceeds to step S330, and if the determination is negative, the vehicle control unit 26 proceeds to step S360.

[0093] In step S330, for example, the occupant recognition unit 31 determines whether or not a child occupant is awake based on the occupant information acquired in step S310. If the determination in step S330 is affirmative, the vehicle control unit 26 proceeds to step S340, and if the determination is negative, the vehicle control unit 26 proceeds to step S360.

[0094] In step S340, for example, the control decision unit 35 decides to execute a notification that an obstacle has been detected, that the obstacle is construction work, and that if the child is interested in the construction work, it is recommended that the child continue driving without taking a detour, and issues a notification request to the device control unit 34. Then, the device control unit 34 outputs a notification command to the display device 12 or the like, similar to step S153 in the first vehicle control.

[0095] In step S350, for example, the vehicle control unit 26 determines whether the driver agrees with the proposal not to take the detour in step S340, similar to step S151 in the first vehicle control. If the determination in step S350 is affirmative, the vehicle control unit 26 does not perform any particular processing, whereas if the determination is negative, the processing proceeds to step S380. This enables flexible vehicle control, such as driving around the construction site without taking the detour if a child is interested in the construction site, even when the determination in step S350 is affirmative.

[0096] In step S360, for example, similar to step S150 in the first vehicle control, the control decision unit 35 decides to execute a notification suggesting a detour around the road where an obstacle (construction) has been detected, and the equipment control unit 34 outputs a notification command to the display device 12, etc.

[0097] In step S370, for example, the control decision unit 35 determines whether the driver has agreed to the detour proposal made in step S360, similar to step S151 in the first vehicle control. Then, for example, the vehicle control unit 26 proceeds to step S380 if the determination in step S370 is affirmative, and skips the process of step S380 if the determination in step S370 is negative.

[0098] In step S380, for example, similar to step S152 in the first vehicle control, the control decision unit 35 decides to execute the reroute control, and the equipment control unit 34 outputs a command signal for the reroute control to the navigation device 23. Then, the vehicle control unit 26 repeats the above series of processes until, for example, a predetermined termination condition is satisfied.

[0099] The vehicle control system 1 estimates the presence and type of an obstacle through the third vehicle control, and when the obstacle is construction and a child is involved, suggests not taking a detour or suggesting a detour depending on the child's condition, and executes rerouting control as necessary. Note that, although the above describes an example in which the occupant information acquisition process (step S310) is executed after an obstacle is detected, this is not limiting, and the process may be executed before an obstacle is detected, or the order of the processes may be changed within a possible range.

[0100] [Fourth Vehicle Control] Next, the fourth vehicle control will be described with reference to Figure 6, but details of parts common to any of the first to third vehicle controls will be omitted. For example, in the fourth vehicle control, the vehicle control unit 26 sequentially executes the processes of steps S110 to S130 and S210, as in the third vehicle control, and if it is determined that a fault exists, proceeds to step S410.

[0101] In step S410, for example, similar to step S210 in the second vehicle control, the obstacle detection unit 30 determines whether the obstacle is construction work or not, and if the vehicle control unit 26 makes a positive judgment, the processing proceeds to step S420, and if the vehicle control unit 26 makes a negative judgment, the processing proceeds to step S430.

[0102] In step S420, for example, the control decision unit 35 decides to increase the volume of the audio device 13, and the volume control unit 37 issues a control request to increase the volume to the device control unit 34 based on this decision. Then, in response to the request from the volume control unit 37, the device control unit 34 outputs a command signal to increase the volume to the audio device 13. At this time, for example, a plurality of values ​​corresponding to the sound pressure of the detected construction noise are set in advance in a data table or the like for the volume of the audio device 13 so that the occupants can hear the construction noise without any problems even if there is construction noise.

[0103] In step S430, for example, similar to step S410, the obstacle detection unit 30 determines whether the obstacle is a checkpoint, and if the vehicle control unit 26 makes a positive judgment, the processing proceeds to step S440, and if the vehicle control unit 26 makes a negative judgment, the processing of step S440 is skipped.

[0104] In step S440, for example, the opposite of step S420 is that control decision unit 35 decides to lower the volume of audio device 13, and volume control unit 37 issues a control request to device control unit 34 to reduce the volume based on this decision. Then, device control unit 34 outputs a command signal to audio device 13 to lower the volume in response to the request from volume control unit 37. At this time, for example, the volume of audio device 13 is preset to a value that does not at least obstruct the ability to hear instructions, etc., at checkpoints. Then, vehicle control unit 26 repeats the above series of processes until, for example, a predetermined termination condition is satisfied.

[0105] Vehicle control system 1 estimates the presence and type of obstacle through the fourth vehicle control, and controls to increase the volume of audio device 13 if the obstacle is construction work, and to decrease the volume of audio device 13 if the obstacle is a checkpoint. This makes it easier to hear the sound of audio device 13 in the case of construction work that generates noise, and reduces the volume of audio device 13 so as not to disturb the driver in the case of a checkpoint where instructions from a police officer, etc., need to be heard, thereby ensuring a comfortable driving experience.

[0106] Although the above describes an example of automatically executing volume control to increase the volume of audio device 13 when construction is detected and decrease the volume of audio device 13 when a checkpoint is detected, this is not limiting. For example, if the sound pressure of the detected construction noise is below a predetermined level, i.e., if the noise level is below a predetermined level, the volume of audio device 13 may not be increased. Furthermore, the fourth vehicle control may be executed when volume control is required based on a processing start condition such as "music, etc., is being played on audio device 13" or "audio device 13 is on." Furthermore, in the fourth vehicle control, an example has been described in which the process of determining a detected fault is executed in the order of construction (step S410) and checkpoint (step S430). However, this order may be reversed, and the order of each process may be changed as appropriate within the scope of feasibility.

[0107] [Fifth Vehicle Control] Next, the fifth vehicle control will be described with reference to Figure 7, but details of parts common to any of the first to fourth vehicle controls will be omitted. For example, in the fifth vehicle control, the vehicle control unit 26 sequentially executes the processes of steps S110 to S120, as in the first vehicle control, and then proceeds to step S510.

[0108] In step S510, for example, the obstacle detection unit 30 determines whether or not an obstacle exists based on the results of determining sound information from the acoustic sensor 16. If the determination in step S510 is affirmative, the vehicle control unit 26 proceeds to step S520, and if the determination in step S510 is negative, the vehicle control unit 26 returns the process to step S110.

[0109] In step S520, for example, similar to step S153 in the first vehicle control, the control decision unit 35 decides to notify the driver that an obstacle has been detected and requests the notification from the equipment control unit 34, and the equipment control unit 34 outputs a notification command to the display device 12, etc.

[0110] In the next step S530, for example, the environment recognition unit 27 acquires detection information and the like from various in-vehicle devices such as the camera unit 15 and the object detection sensor 17, and recognizes the driving environment of the vehicle.

[0111] In step S540, for example, the obstacle detection unit 30 determines whether or not an obstacle exists based on detection information from various sensors other than the acoustic sensor 16. If the determination in step S540 is affirmative, the vehicle control unit 26 proceeds to step S550, and if the determination in step S540 is negative, the vehicle control unit 26 returns the process to step S530.

[0112] In step S550, for example, the vehicle control unit 26 executes a process after detecting a fault in at least one of the first to fourth vehicle control modes described above. The vehicle control unit 26 then repeats the above series of processes until a predetermined termination condition is met.

[0113] The vehicle control system 1 executes a two-stage determination process using the fifth vehicle control, which includes a first fault determination based on sound information and a second fault determination based on information other than sound. The first fault detection results in a notification, and the second fault detection results in vehicle control. This improves the fault detection accuracy of the vehicle control system 1, enabling more appropriate vehicle control. While the above description includes an example in which the process returns to step S530 when a negative determination is made in step S540, the present invention is not limited to this example, and the process may also return to step S110. Furthermore, in the fifth vehicle control, the driving environment recognition process (step S520) may be executed before the sound-based fault detection determination (step S510), and the order of each process may be changed as appropriate within the scope of feasible limits.

[0114] [Sixth Vehicle Control] Next, the sixth vehicle control will be described with reference to Figure 8, but details of parts common to any of the first to fifth vehicle controls will be omitted. For example, in the sixth vehicle control, the vehicle control unit 26 sequentially executes the processes of steps S110 to S130, as in the first vehicle control, and if a fault is detected, proceeds to step S610.

[0115] In step S610, for example, the obstacle detection unit 30 determines whether or not there is an obstacle on the road on which the vehicle is currently traveling, based on whether or not the angular difference between the direction of the sound determined to be an obstacle and the traveling direction of the vehicle is equal to or smaller than a predetermined value. If the determination in step S610 is affirmative, the vehicle control unit 26 proceeds to step S620, and if the determination in step S610 is negative, the vehicle control unit 26 proceeds to step S630.

[0116] In step S620, for example, the vehicle control unit 26 executes at least one of the first to fifth vehicle controls described above, which is a process to be performed after detecting a fault.

[0117] In step S630, for example, the obstacle detection unit 30 identifies a road on which an obstacle may exist based on the direction of the sound determined to be an obstacle, and determines whether the identified road matches the road along which the vehicle is scheduled to travel according to the navigation device 23. If the determination in step S630 is affirmative, the vehicle control unit 26 proceeds to step S640, and if the determination in step S630 is negative, the vehicle control unit 26 skips step S640.

[0118] In step S640, for example, similar to step S152 in the first vehicle control, the control decision unit 35 decides to execute reroute control and requests the device control unit 34 to execute reroute control, and the device control unit 34 outputs a command signal to the navigation device 23 to reset a detour route. As a result, before the vehicle enters the road with the obstacle, a road other than the road with the obstacle is reset to the planned driving route, so that the road with the obstacle can be detoured in advance, enabling a comfortable drive.

[0119] The vehicle control system 1 executes the sixth vehicle control to determine whether there is an obstacle, whether the vehicle is traveling on a road where an obstacle has occurred, and whether there is a planned trip, and executes vehicle control according to the situation. As a result, appropriate vehicle control is performed when there is an obstacle on the road the vehicle is traveling on, and rerouting control is performed when there is an obstacle on the road the vehicle is scheduled to travel on, thereby realizing a more comfortable driving condition.

[0120] The vehicle control system 1 according to the embodiment includes an obstacle detection unit 30 that determines the presence, scale, and type of obstacle on the road based on sound information from the acoustic sensor 16 of the vehicle, and a control decision unit 35 that decides to execute various vehicle controls when it is determined that an obstacle exists. The vehicle control unit 26, which corresponds to a vehicle control device, executes at least one of vehicle control functions including rerouting control, notifying the vehicle occupants, controlling the speed of the vehicle, and controlling the volume of the audio device 13 of the vehicle. This enables the vehicle control system 1 to detect obstacles on the road based on the sound information from the acoustic sensor 16 and to execute appropriate vehicle control according to the type of obstacle. Furthermore, because the vehicle control system 1 determines the presence or absence of an obstacle based on sound information, it is possible to detect the obstacle and control the vehicle even in situations where an obstacle, such as a large vehicle, is present between the vehicle and the obstacle and the occupants cannot see the obstacle. Furthermore, the vehicle control system 1 also achieves the following effects.

[0121] (1) The fault detection unit 30 estimates the scale of the detected fault. Based on the estimated fault scale, the vehicle control unit 26 executes one of the above-described vehicle controls that corresponds to the estimated scale. This enables the vehicle control system 1 to execute more appropriate vehicle control according to the scale of the fault.

[0122] (2) When the obstacle detection unit 30 estimates that the scale of the obstacle is equal to or greater than a predetermined level, the vehicle control unit 26 executes reroute control to guide the vehicle to a road other than the road on which the obstacle is detected. As a result, the vehicle control system 1 guides the vehicle to a road without obstacles, providing a comfortable driving environment.

[0123] (3) If the obstacle detection unit 30 estimates that the scale of the obstacle is less than a predetermined value, the vehicle control unit 26 notifies the occupant of the vehicle and suggests guiding the occupant to a lane other than the lane the vehicle is currently traveling on, on the road where the obstacle is detected. In this way, the vehicle control system 1 provides a comfortable driving environment by encouraging the occupant to change lanes without having to detour the vehicle onto a road other than the road the vehicle is currently traveling on.

[0124] (4) The obstacle detection unit 30 estimates the scale of the detected obstacle based on the sound information from the acoustic sensor 16, as well as road information obtained by capturing images with the camera unit 15 or vehicle speed information of the vehicle obtained by the wheel speed sensor 18. This further improves the accuracy of estimating the scale of the obstacle in the vehicle control system 1, enabling more appropriate vehicle control.

[0125] (5) When the obstacle detection unit 30 estimates that the scale of the obstacle is equal to or greater than a predetermined scale, the vehicle control unit 26 notifies the driver of the vehicle to be careful of large vehicles. As a result, when the scale of the obstacle is estimated to be large, the vehicle control system 1 improves driving safety by urging the driver to be careful of large vehicles.

[0126] (6) When the obstacle detection unit 30 detects an obstacle, the vehicle control unit 26 executes a speed adjustment to slow down the speed of the host vehicle or to increase the distance between the host vehicle and the preceding vehicle to a predetermined value or more when passing through the road on which the obstacle is detected. This enables the vehicle control system 1 to slow down the host vehicle or increase the distance between the host vehicle and the preceding vehicle when traveling on the road on which the obstacle is detected, thereby improving driving safety.

[0127] (7) The fault detection unit 30 estimates the type of the detected fault. Then, the vehicle control unit 26 notifies the occupant according to the estimated type of fault. This enables the vehicle control system 1 to provide an appropriate notification to the occupant according to the type of the detected fault.

[0128] (8) The obstacle detection unit 30 estimates the type of detected obstacle based on the sound information from the acoustic sensor 16, as well as road information obtained by capturing images with the camera unit 15 or vehicle speed information of the vehicle obtained by the wheel speed sensor 18. This further improves the accuracy of the obstacle type estimation in the vehicle control system 1, enabling more appropriate vehicle control.

[0129] (9) When the obstacle detection unit 30 estimates that the obstacle is construction work, the vehicle control unit 26 notifies the driver of the vehicle to be careful of large vehicles. As a result, when the type of obstacle is estimated to be construction work, the vehicle control system 1 improves driving safety by urging the driver to be careful of large vehicles.

[0130] (10) When the obstacle detection unit 30 estimates that the obstacle is a checkpoint, the vehicle control unit 26 notifies the driver of the obstacle estimation result and executes speed adjustment to slow down the speed of the vehicle or to increase the distance between the vehicle and the preceding vehicle to a predetermined value or more. As a result, when the obstacle is estimated to be a checkpoint, the vehicle control system 1 can slow down the vehicle or increase the distance between the vehicle and the preceding vehicle, thereby improving driving safety.

[0131] (11) The vehicle control unit 26 further includes an occupant recognition unit 31 that acquires occupant information, including whether or not the occupants of the vehicle are children, from the vehicle interior camera 20. When the obstacle detection unit 30 detects an obstacle, the vehicle control unit 26 changes the content of the notification to the occupants of the vehicle based on the occupant information. This enables the vehicle control system 1 to provide notifications that take into account the occupant information of the vehicle, allowing the driver to select more appropriate vehicle control.

[0132] (12) When the obstacle detection unit 30 detects an obstacle, it estimates the type of the detected obstacle, and the vehicle control unit 26 controls the volume of the audio device 13 based on the estimated type of obstacle. This allows the vehicle control system 1 to control the volume of the audio device 13 in accordance with the type of obstacle detected, thereby providing a comfortable driving environment for the occupants.

[0133] (13) The obstacle detection unit 30 detects the presence or absence of an obstacle based on sound information, as well as image information captured by the camera unit 15 or reflected wave information obtained by the object detection sensor 17. The vehicle control unit 26 notifies the occupant when the obstacle detection unit 30 detects an obstacle based on sound information, and after the notification, executes at least one vehicle control when the obstacle detection unit 30 detects an obstacle based on information other than sound information. This improves the obstacle detection accuracy of the vehicle control system 1, enabling more appropriate vehicle control to be executed.

[0134] (14) The vehicle control unit 26 further includes a route determination unit 33 that, when the obstacle detection unit 30 detects an obstacle on a road different from the road on which the vehicle is traveling, determines whether the vehicle will travel on the different road. When the route determination unit 33 determines that the vehicle will travel on the different road, the vehicle control unit 26 executes reroute control to guide the vehicle to a road other than the different road before entering the different road. This allows the vehicle control system 1 to use reroute control to detour the road with the obstacle in advance before the vehicle enters the road with the obstacle, thereby enabling a comfortable journey.

[0135] (Other Embodiments) While the present disclosure has been described with reference to examples, it is understood that the present disclosure is not limited to those examples or structures. The present disclosure also encompasses various modifications and modifications within the scope of equivalents. In addition, various combinations and forms, as well as other combinations and forms including only one element, more than one, or less than one, are also within the scope and spirit of the present disclosure.

[0136] Each processing unit in the above embodiment is, for example, hardware for arithmetic processing coupled to a RAM, and includes at least one arithmetic core, such as a CPU and a GPU. GPU is an abbreviation for Graphics Processing Unit. Each processing unit may further include an FPGA, an NPU, and an IP core with other dedicated functions. FPGA and NPU are abbreviations for Field-Programmable Gate Array and Neural Network Processing Unit, respectively. Each processing unit may be individually mounted on a printed circuit board, or may be mounted on an ASIC, FPGA, or the like. ASIC is an abbreviation for Application Specific Integrated Circuit. In this way, the hardware configuration of each processing unit can be changed as appropriate.

[0137] In the above embodiment, the form of the recording medium (non-transitory tangible storage medium) on which various programs, including the vehicle control programs corresponding to the first to sixth vehicle controls, are recorded may be changed as appropriate. Furthermore, the recording medium is not limited to being provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to a control circuit such as the vehicle control unit 26. Furthermore, the storage medium may be an optical disk, hard disk drive, solid state drive, or the like, from which programs are copied or distributed to the vehicle control unit 26, etc.

[0138] The vehicle control unit 26 and the methods described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the vehicle control unit 26 and the methods described herein may be implemented by a special-purpose computer configured with a processor configured with one or more dedicated hardware logic circuits. Alternatively, the vehicle control unit 26 and the methods described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium.

[0139] It goes without saying that in each of the above embodiments, the elements constituting the embodiments are not necessarily essential unless they are specifically stated as essential or are clearly considered essential in principle. Furthermore, in each of the above embodiments, when numerical values ​​such as the number, values, amounts, and ranges of the components of the embodiments are mentioned, they are not limited to the specific numbers unless they are specifically stated as essential or are clearly limited to a specific number in principle. Furthermore, in each of the above embodiments, when the shapes, positional relationships, etc. of the components are mentioned, they are not limited to the shapes, positional relationships, etc., unless they are specifically stated or are clearly limited to a specific shape, positional relationship, etc. in principle.

[0140] (Aspects of the Present Disclosure) The above-described present disclosure can be understood from the following aspects, for example. [First Aspect] A vehicle control device comprising: an obstacle detection unit (30) that detects an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained by an acoustic sensor (16) that collects sounds around the vehicle, and determines the content of the detected obstacle; a control decision unit (35) that decides to execute at least one vehicle control from among rerouting control, notification to an occupant of the vehicle, speed control of the vehicle, and volume control of an audio device (13) of the vehicle, based on a determination result by the obstacle detection unit; and an execution unit (34, 38) that executes the vehicle control decided by the control decision unit. [Second Aspect] The vehicle control device according to the first aspect, wherein the obstacle detection unit estimates the scale of the detected obstacle, and the control decision unit decides to execute one of a plurality of vehicle controls corresponding to the estimated scale based on a result of the estimation of the scale of the obstacle by the obstacle detection unit. [Third Aspect] The vehicle control device according to the second aspect, wherein, when the obstacle detection unit estimates that the scale of the obstacle is equal to or greater than a predetermined scale, the control decision unit decides to execute the reroute control to guide the vehicle to a road other than the road on which the obstacle is detected. [Fourth Aspect] The vehicle control device according to the second or third aspect, wherein, when the obstacle detection unit estimates that the scale of the obstacle is less than a predetermined scale, the control decision unit decides to notify an occupant of the vehicle of a suggestion to guide the vehicle to a lane on the road on which the obstacle is detected that is different from the lane on which the vehicle is traveling. [Fifth Aspect] The vehicle control device according to any one of the second to fourth aspects, wherein the obstacle detection unit estimates the scale of the detected obstacle based on, in addition to sound information from the acoustic sensor, road information obtained by imaging with a camera unit (15) mounted on the vehicle or vehicle speed information of the vehicle obtained by an on-board sensor (18). [Sixth Aspect] A vehicle control device according to any one of the second to fifth aspects, wherein, when the obstacle detection unit estimates that the scale of the obstacle is equal to or greater than a predetermined value, the control decision unit decides to execute a notification to warn the driver of the vehicle about large vehicles.[Seventh Aspect] The vehicle control device according to any one of the second to sixth aspects, wherein, when the obstacle detection unit detects the obstacle, the control decision unit decides to execute a speed adjustment to reduce the speed of the host vehicle or to execute a speed adjustment to maintain a predetermined inter-vehicle distance between the host vehicle and a preceding vehicle or more when passing through a road on which the obstacle has been detected. [Eighth Aspect] The vehicle control device according to any one of the first to seventh aspects, wherein the obstacle detection unit estimates a type of the detected obstacle, and the control decision unit decides to execute a notification to an occupant in accordance with the estimated type of obstacle. [Ninth Aspect] The vehicle control device according to the eighth aspect, wherein the obstacle detection unit estimates the type of the detected obstacle based on road information obtained by imaging with a camera unit (15) mounted on the host vehicle or vehicle speed information of the host vehicle obtained by an on-board sensor (18), in addition to sound information from the acoustic sensor. [Tenth Aspect] The vehicle control device according to the eighth or ninth aspect, wherein, when the obstacle detection unit estimates the obstacle to be construction work, the control decision unit decides to issue a notification to the driver of the host vehicle to warn the driver of the host vehicle of a large vehicle. [Eleventh Aspect] The vehicle control device according to any one of the eighth to tenth aspects, wherein, when the obstacle detection unit estimates the obstacle to be a checkpoint, the control decision unit notifies the driver of the host vehicle of the estimated obstacle and decides to implement a speed adjustment to reduce the speed of the host vehicle or to implement a speed adjustment to set a distance between the host vehicle and a preceding vehicle to a predetermined value or more. [Twelfth Aspect] The vehicle control device according to any one of the first to eleventh aspects, further comprising an occupant recognition unit (33) that acquires occupant information including the presence or absence of children among the occupants of the host vehicle using a cabin camera (20) of the host vehicle, and when the obstacle detection unit detects the obstacle, the control decision unit changes the content of the notification to the occupants of the host vehicle based on the occupant information. [Thirteenth Aspect] The vehicle control device according to any one of the first to twelfth aspects, wherein the obstacle detection unit estimates a type of the detected obstacle, and when the obstacle detection unit detects the obstacle, the control decision unit decides to control the volume of the audio device based on the estimated type of obstacle.[14th Aspect] The vehicle control device according to any one of the first to 13th aspects, wherein the obstacle detection unit detects the presence or absence of the obstacle based on, in addition to the sound information, image information captured by a camera unit (15) that captures an image of the periphery of the vehicle, or reflected wave information obtained by an on-board sensor (17) that transmits a transmission wave to the periphery of the vehicle and acquires a reflected wave of the transmission wave, and the control decision unit decides to execute a notification to the occupant when the obstacle detection unit detects the obstacle based on the sound information, and after the notification is executed, decides to execute at least one of the vehicle controls when the obstacle detection unit detects the obstacle based on information other than the sound information. [Fifteenth Aspect] A vehicle control device according to any one of the first to fourteenth aspects, further comprising a route determination unit (33) that, when the obstacle detection unit detects the obstacle on a road different from the road on which the host vehicle is traveling, determines whether the host vehicle will travel on the different road, and when the route determination unit determines that the host vehicle will travel on the different road, the control decision unit decides to execute the reroute control to guide the host vehicle to a road other than the different road before entering the different road. [Sixteenth Aspect] A vehicle control method that can be used in a vehicle, comprising: detecting an obstacle that is at least one of traffic restrictions and construction on a road based on sound information obtained by an acoustic sensor (16) that collects sounds around the host vehicle, and determining the content of the detected obstacle; and executing at least one of vehicle control among reroute control, notifying an occupant of the host vehicle, controlling the speed of the host vehicle, and controlling the volume of an audio device (13) of the host vehicle based on a determination result of the content of the detected obstacle.[Seventeenth Aspect] A vehicle control program used to control the driving state of a vehicle or on-board equipment (12, 13, 23, 24), the vehicle control program causing at least one execution unit (26) to execute processes including: a process of detecting an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained by an acoustic sensor (16) that collects sounds around the vehicle; a process of determining the content of the detected obstacle; and a process of executing at least one vehicle control out of reroute control, notification to an occupant of the vehicle, speed control of the vehicle, and volume control of an audio device (13) of the vehicle, based on the determination result of the content of the obstacle.

Claims

1. A vehicle control device comprising: an obstacle detection unit (30) that detects an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained from an acoustic sensor (16) that collects sounds around the vehicle, and determines the content of the detected obstacle; a control decision unit (35) that decides to execute at least one vehicle control from among reroute control, notification to occupants of the vehicle, speed control of the vehicle, and volume control of an audio device (13) of the vehicle, based on the determination result by the obstacle detection unit; and an execution unit (34, 38) that executes the vehicle control determined by the control decision unit.

2. A vehicle control device as described in claim 1, wherein the fault detection unit estimates the scale of the detected fault, and the control decision unit decides to execute the vehicle control corresponding to the estimated scale among the multiple vehicle controls based on the result of the fault detection unit's estimation of the scale of the fault.

3. A vehicle control device as described in claim 2, wherein, when the obstacle detection unit estimates that the scale of the obstacle is equal to or greater than a predetermined value, the control decision unit decides to execute the reroute control to guide the vehicle to a road other than the road on which the obstacle is detected.

4. A vehicle control device as described in claim 2, wherein, when the obstacle detection unit estimates that the scale of the obstacle is less than a predetermined value, the control decision unit decides to notify an occupant of the vehicle suggesting that the occupant be guided to a lane on the road on which the obstacle is detected that is different from the lane in which the vehicle is traveling.

5. A vehicle control device as described in claim 2, wherein the obstacle detection unit estimates the scale of the detected obstacle based on, in addition to sound information from the acoustic sensor, road information obtained by imaging with a camera unit (15) mounted on the vehicle or vehicle speed information of the vehicle obtained by an on-board sensor (18).

6. A vehicle control device as described in claim 2, wherein, when the fault detection unit estimates that the scale of the fault is equal to or greater than a predetermined value, the control decision unit decides to issue a notification to the driver of the vehicle to warn him or her of a large vehicle.

7. A vehicle control device as described in claim 2, wherein, when the obstacle detection unit detects the obstacle, the control decision unit decides to perform speed adjustment to reduce the speed of the vehicle when passing through the road on which the obstacle is detected, or to perform speed adjustment to make the inter-vehicle distance between the vehicle and a preceding vehicle equal to or greater than a predetermined value.

8. The vehicle control device according to claim 1, wherein the fault detection unit estimates the type of the detected fault, and the control decision unit decides to notify the occupant according to the estimated type of fault.

9. A vehicle control device as described in claim 8, wherein the obstacle detection unit estimates the type of the detected obstacle based on, in addition to sound information from the acoustic sensor, road information obtained by imaging with a camera unit (15) mounted on the vehicle or vehicle speed information of the vehicle obtained by an on-board sensor (18).

10. A vehicle control device according to claim 8, wherein, when the obstacle detection unit estimates that the obstacle is construction work, the control decision unit decides to execute a notification to warn the driver of the vehicle of a large vehicle.

11. A vehicle control device as described in claim 8, wherein, when the obstacle detection unit estimates that the obstacle is a checkpoint, the control decision unit notifies the driver of the vehicle of the estimated obstacle and decides to perform speed adjustment to reduce the speed of the vehicle, or to perform speed adjustment to make the inter-vehicle distance between the vehicle and a preceding vehicle equal to or greater than a predetermined value.

12. A vehicle control device as described in claim 1, further comprising an occupant recognition unit (31) that acquires occupant information including the presence or absence of children among the occupants of the vehicle using a cabin camera (20) of the vehicle, and when the obstacle detection unit detects the obstacle, the control decision unit changes the content of the notification to the occupants of the vehicle based on the occupant information.

13. The vehicle control device described in claim 1, wherein the fault detection unit estimates the type of the detected fault, and when the fault detection unit detects the fault, the control decision unit decides to execute volume control of the audio device based on the estimated type of fault.

14. A vehicle control device as described in claim 1, wherein the obstacle detection unit detects the presence or absence of the obstacle based on, in addition to the sound information, image information captured by a camera unit (15) that captures images of the surroundings of the vehicle, or reflected wave information obtained by an on-board sensor (17) that transmits a transmission wave to the surroundings of the vehicle and acquires a reflected wave of the transmission wave, and the control decision unit decides to execute a notification to the occupant when the obstacle detection unit detects the obstacle based on the sound information, and after the notification is executed, decides to execute at least one of the vehicle controls when the obstacle detection unit detects the obstacle based on information other than the sound information.

15. A vehicle control device as described in claim 1, further comprising a route determination unit (33) that, when the obstacle detection unit detects an obstacle on a road different from the road on which the vehicle is traveling, determines whether the vehicle will travel on the different road, and when the route determination unit determines that the vehicle will travel on the different road, the control decision unit decides to execute the reroute control that guides the vehicle to a road other than the different road before entering the different road.

16. A vehicle control method usable in a vehicle, comprising: detecting an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained from an acoustic sensor (16) that collects sounds around the vehicle, and determining the content of the detected obstacle; and, based on the determination result of the content of the detected obstacle, executing at least one of vehicle control including rerouting control, notifying an occupant of the vehicle, speed control of the vehicle, and volume control of an audio device (13) of the vehicle.

17. A vehicle control program used to control the driving state of the vehicle or on-board equipment (12, 13, 23, 24), which causes at least one execution unit (26) to execute processes including: a process of detecting an obstacle, which is at least one of traffic restrictions and construction work on a road, based on sound information obtained by an acoustic sensor (16) that collects sounds around the vehicle; a process of determining the content of the detected obstacle; and a process of executing at least one vehicle control among reroute control, notification to an occupant of the vehicle, speed control of the vehicle, and volume control of an audio device (13) of the vehicle, based on the determination result of the content of the obstacle.

Images