Controller, control method, and control program

Abstract

To early and widely promote a widespread of a high-level driving support vehicle or an automatic driving vehicle.SOLUTION: A controller (2) comprises a detection part (2001) for detecting an obstacle around a vehicle, and an operation determination part (2002) for determining an operation content of an on-vehicle system according to a determination result of a collision possibility while determining the collision possibility of the detected obstacle with the vehicle. The operation determination part determines that collision possibility notification of giving notice of the collision possibility is executed, when there is the collision possibility; determines whether or not an occupant executes a collision avoidance operation, after executing the collision possibility notification; and determines that neither the collision avoidance operation is executed, nor a collision avoidance support operation is executed by the on-vehicle system when the obstacle exhibiting the collision possibility is a low-height falling object on a road, over which the vehicle possibly runs.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The present disclosure relates to a control device, a control method, and a control program to be mounted on a vehicle. [Background technology]

[0002] In recent years, research and development of so-called driving assistance technology or automatic driving technology for vehicles such as automobiles has been actively carried out, and various proposals have been made regarding this (see, for example, Patent Document 1, etc.). Driving assistance technology is technology that reduces the burden on the driver and enables comfortable and safe driving, and includes, for example, following distance control, lane keeping assistance control, lane change assistance control, parking assistance control, obstacle warning, collision avoidance assistance control, etc. Automatic driving technology is technology that allows a vehicle to run automatically, that is, autonomously, without the need for driving operation by the driver. A vehicle equipped with such driving assistance technology or automatic driving technology is equipped with sensing devices such as cameras and radars to grasp the situation of the area ahead and the surrounding area. Then, vehicle control such as steering, driving, and braking is automatically performed based on the road conditions, etc. acquired by the traveling vehicle using sensing devices such as cameras and map information, etc. It is possible to improve safety and reliability by using a high-precision map including road data for each lane. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 202397 Summary of the Invention [Problem to be solved by the invention]

[0004] Research and development of these types of driving assistance technology and autonomous driving technology has been accelerating in recent years from various perspectives, such as improving the accuracy of detection or recognition by sensing devices and improving convenience. In other words, by further improving such performance, it is possible to promote the early and widespread dissemination of so-called advanced driving assistance vehicles and autonomous driving vehicles. The present disclosure has been made in consideration of the circumstances exemplified above. [Means for solving the problem]

[0005] The control device (2) recited in claim 1 is configured to control the operation of an in-vehicle system (1) mounted on a vehicle (V). This control device is A detection unit (2001) that detects obstacles around the vehicle; an operation determination unit (2002) that determines a possibility of a collision between the obstacle detected by the detection unit and the vehicle, and determines an operation content of the in-vehicle system according to a result of the determination of the possibility of the collision; Equipped with The operation determination unit is When the possibility of a collision is detected, a determination is made to execute a collision possibility notification to notify the occupant (D) of the vehicle of the possibility of a collision by display and / or audio using an alarm device (7) provided in the in-vehicle system; determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; If the collision avoidance operation is not performed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass over, a decision is made not to perform a collision avoidance assistance operation by a motion control device (9) provided in the in-vehicle system to control the movement of the vehicle. The control method described in claim 12 is a method executed by a control device (2) that controls the operation of an in-vehicle system (1) mounted on a vehicle (V), and includes the following steps or processes: Detecting obstacles around the vehicle; determining a possibility of a collision between the detected obstacle and the vehicle; When the possibility of a collision is detected, a determination is made to execute a collision possibility notification to notify the occupant (D) of the vehicle of the possibility of a collision by display and / or audio using an alarm device (7) provided in the in-vehicle system; determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; If the collision avoidance operation is not performed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass over, a decision is made not to perform a collision avoidance assistance operation by a motion control device (9) provided in the in-vehicle system to control the movement of the vehicle. The control program according to claim 13 is a computer program executed by a control device (2) for controlling an operation of an in-vehicle system (1) mounted on a vehicle (V), The process executed by the control device is A process of detecting obstacles around the vehicle; A process of determining a possibility of a collision between the detected obstacle and the vehicle; a process of determining whether to execute a collision possibility notification, when the collision possibility exists, to notify the vehicle occupant (D) of the collision possibility by display and / or audio using an alarm device (7) provided in the in-vehicle system; A process of determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; a process of determining not to execute a collision avoidance support operation by a motion control device (9) provided in the in-vehicle system so as to control the motion of the vehicle when the collision avoidance operation is not executed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass through; Includes.

[0006] In addition, in each section of the application documents, each element may be given a reference number in parentheses. In this case, the reference number merely indicates an example of the correspondence between the element and the specific configuration described in the embodiment described later. Therefore, the present disclosure is not limited in any way by the description of the reference number. [Brief description of the drawings]

[0007] [Figure 1] 1 is a plan view showing a schematic configuration of a vehicle equipped with an in-vehicle system including a driving ECU as a control device according to an embodiment. [Diagram 2] FIG. 2 is a schematic block diagram showing the overall configuration of the in-vehicle system shown in FIG. [Diagram 3] FIG. 3 is a diagram showing an outline of target detection using the sonar sensor shown in FIG. 2. [Figure 4] FIG. 3 is a diagram showing an outline of target detection using the sonar sensor shown in FIG. 2. [Diagram 5] 3 is a diagram showing an outline of a target detection range of each of the plurality of sonar sensors shown in FIG. 2. [Figure 6] 3 is a diagram showing an outline of parking space detection using the sonar sensor shown in FIG. 2. [Figure 7] 3 is a diagram showing an outline of target detection using the radar sensor shown in FIG. 2. [Figure 8] 3 is a diagram showing an outline of target detection using the laser radar sensor shown in FIG. 2. [Figure 9] 9 is a diagram illustrating an outline of the configuration and operation of the laser radar sensor illustrated in FIG. 8. [Figure 10] FIG. 3 is a diagram showing an outline of target detection using the camera shown in FIG. 2. [Figure 11] FIG. 3 is a diagram showing an outline of target detection using the camera shown in FIG. 2. [Figure 12] FIG. 3 is a diagram showing an outline of driver state detection using the driver state monitor shown in FIG. 2. [Figure 13] FIG. 3 is a diagram showing an outline of driver state detection using the driver state monitor shown in FIG. 2. [Figure 14] FIG. 3 is a diagram showing a schematic configuration of the head-up display shown in FIG. 2. [Figure 15] 15 is a schematic diagram showing a road surface ahead of the vehicle and a part of the dashboard seen through the front windshield from the viewpoint of the driver shown in FIG. 14. [Figure 16] 3 is a block diagram showing an example of a schematic functional configuration realized by an operation ECU shown in FIG. 2. [Figure 17] 17 is a block diagram showing an example of a schematic functional configuration realized by a recognition unit shown in FIG. 16. [Figure 18] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 19] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 20] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 21] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 22] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Diagram 23] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 24] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Diagram 25] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 26] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 27] 2 is a schematic diagram showing a specific example of a target to be recognized in the embodiment. FIG. [Figure 28] FIG. 4 is a schematic diagram showing an example of a control scene using a target recognition result in the embodiment. [Figure 29] FIG. 11 is a schematic diagram showing another example of a control scene using a target recognition result in the embodiment. [Diagram 30] FIG. 2 is a conceptual diagram showing an overview of collision avoidance assistance control in the embodiment. [Diagram 31] 4 is a flowchart showing an outline of a first specific example of collision avoidance assistance control in the embodiment. [Diagram 32]4 is a flowchart showing an outline of a first specific example of collision avoidance assistance control in the embodiment. [Diagram 33] 4 is a flowchart showing an outline of a first specific example of collision avoidance assistance control in the embodiment. [Diagram 34] 10 is a flowchart showing an outline of a second specific example of collision avoidance assistance control in the embodiment. [Diagram 35] 13 is a flowchart showing an outline of a third specific example of collision avoidance assistance control in the embodiment. [Diagram 36] 11 is a table showing an outline of a fourth specific example of collision avoidance assistance control in the embodiment. [Figure 37] 13 is a table showing an overview of a fifth specific example of collision avoidance assistance control in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] (Embodiment) Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings as appropriate. Note that the configuration, functions, and implementation examples of the embodiments can be modified as appropriate. Therefore, multiple embodiments and variations may be described in the specification. In this case, in the following description, the same reference symbols will be given to parts that are identical or equivalent between one embodiment and another embodiment, between one embodiment and its variation, or between one variation and another variation. And, the preceding description of such parts may be appropriately applied to other embodiments or variations that follow, unless there is a technical contradiction or a special additional description.

[0009] (Overall vehicle configuration) Referring to FIG. 1, the system-equipped vehicle V, which is a vehicle to which the present disclosure is applied, is a so-called four-wheeled vehicle, and includes a vehicle body V1 that is substantially rectangular in plan view. Hereinafter, a virtual line that passes through the center of the system-equipped vehicle V in the vehicle width direction and is parallel to the vehicle overall length direction of the system-equipped vehicle V is referred to as a vehicle width center line LC1. Also, a virtual line that passes through the center of the system-equipped vehicle V in the vehicle overall length direction and is parallel to the vehicle width direction of the system-equipped vehicle V is referred to as a vehicle length center line LC2. The vehicle width center line LC1 and the vehicle length center line LC2 pass through a vehicle center point VC. The vehicle center point VC is the center position of the vehicle body V1 in the three-dimensional direction. In FIG. 1, the vehicle width direction is the left-right direction in the figure. The vehicle overall length direction is a direction perpendicular to the vehicle width direction and perpendicular to the vehicle height direction. The vehicle height direction is a direction that defines the vehicle height of the system-equipped vehicle V, and is a direction parallel to the direction of gravity when the system-equipped vehicle V is stably placed on a horizontal plane in a drivable state. Also, for ease of explanation, the "front," "rear," "left," and "right" of the system-equipped vehicle V are defined as shown by the arrows in FIG. 1. In other words, the vehicle overall length direction is synonymous with the front-to-rear direction. Also, the vehicle width direction is synonymous with the left-to-right direction. Note that the vehicle height direction may not be parallel to the direction of gravity depending on the mounting conditions or running conditions of the system-equipped vehicle V. However, in most cases, the vehicle height direction is a direction along the direction of gravity.

[0010] Inside the vehicle body V1, a vehicle compartment V2 is formed, which constitutes a riding space for occupants including the driver D. Wheels V3 are provided at each of the four corners of the vehicle body V1. That is, a left front wheel V3a is provided at the left front part of the vehicle body V1. A right front wheel V3b is provided at the right front part of the vehicle body V1. A left rear wheel V3c is provided at the left rear part of the vehicle body V1. A right rear wheel V3d is provided at the right rear part of the vehicle body V1. The system-equipped vehicle V is not limited to a four-wheeled vehicle, but may be a three-wheeled vehicle, or a six- or eight-wheeled vehicle such as a cargo truck. There is no particular limitation on the position or number of drive wheels.

[0011] A front bumper V12 is attached to a front portion V11, which is the front end of the vehicle body V1. A rear bumper V14 is attached to a rear portion V13, which is the rear end of the vehicle body V1. The side and upper portions of the vehicle body V1 are covered by a vehicle body panel V15. The vehicle body panel V15 includes a door panel V16. In the specific example shown in FIG. 1, a total of four door panels V16 are provided, two on each side. A door mirror V17 is attached to each of the pair of left and right door panels V16 on the front side. In addition, the front side of the vehicle interior V2 is covered by a front windshield V18. The front windshield V18 is formed in a plate shape from translucent glass or synthetic resin. The front windshield V18 is inclined so that the upper end is located further rearward than the lower end when viewed from the side in a line of sight parallel to the vehicle width direction.

[0012] A dashboard V21 is provided at the front end of the vehicle interior V2. A plurality of seats V22 are arranged behind the dashboard V21. Of the plurality of seats V22, a handle V24 is provided in front of a driver's seat V23 in which the driver D sits, for the driver D to grasp and operate when steering. The handle V24 is typically a so-called steering wheel formed in a substantially circular, elliptical, or polygonal ring shape, but may be rod-shaped or control stick-shaped.

[0013] (Overview of the in-vehicle system) The system-equipped vehicle V is equipped with an in-vehicle system 1. The in-vehicle system 1 is configured to function as a driving automation system in the system-equipped vehicle V by being installed in the system-equipped vehicle V. The system-equipped vehicle V may be referred to as the "own vehicle" hereinafter. The "driving automation system" is a system for realizing a driving automation level corresponding to at least one of levels 1 to 5 defined in the standard "SAE J3016" published by SAE International. SAE stands for Society Of Automotive Engineers. Level X in "SAE J3016" is hereinafter simply referred to as "SAE level X". X is any one of 0 to 5. The larger the value of X in SAE level X, or the more dynamic driving tasks the driving automation system is in charge of, i.e., executes, the higher the driving automation level is expressed as being. In addition, a change in the driving automation level to a higher level is referred to as an "increase" in the driving automation level. In contrast, the smaller the value of X, or the fewer dynamic driving tasks that the driving automation system is responsible for, that is, executes, the lower the level of driving automation is expressed. Also, a change to a lower level of driving automation is referred to as a "degradation" of the level of driving automation.

[0014] (Definition of Driving Assistance and Autonomous Driving) The contents of SAE levels 0 to 5 are specifically described below. In the following description, the driver D is a passenger who is in charge of or performs the dynamic driving task. The "dynamic driving task" refers to all operational and tactical functions that need to be performed in real time when operating a vehicle in road traffic, excluding strategic functions. Driving behavior in general can be classified into three types of functions: strategic, tactical, and operational. The "strategic" functions include itinerary planning, route selection, etc., and specifically include deciding or selecting itinerary plans such as "whether to go or not, when, where, and how to go." The "tactical" functions relate to vehicle operation in traffic situations, such as deciding whether and when to overtake or change lanes during the trip, selecting an appropriate speed, and checking mirrors. The "operational" functions relate to instantaneous reactions, such as making minor corrections to steering, braking, accelerator, and acceleration operations to maintain the position of the road lane or to avoid sudden obstacles or dangerous events in the vehicle's path. "OEDR" is an abbreviation for Object and Event Detection and Response, and is also referred to as "detection and response of objects and events." OEDR includes monitoring of the driving environment. Monitoring of the driving environment includes detection, recognition, and classification of objects and events. Monitoring of the driving environment also includes preparation to respond to objects and events as necessary. A "limited domain" is a specific condition in which a driving automation system or its functions are designed to operate, and is also referred to as an operational design domain or ODD. ODD stands for Operational Design Domain. A limited domain includes at least one of multiple constraints, such as geographical, environmental, speed, and time.

[0015] Level 0: Manual driving…Driver D performs all dynamic driving tasks. Level 1: Driving assistance: The driving automation system continuously performs either the longitudinal vehicle motion control subtask or the lateral vehicle motion control subtask of the dynamic driving task in a specific limited area. The longitudinal vehicle motion control subtask is starting, accelerating / decelerating, and stopping. The lateral vehicle motion control subtask is steering. However, the driving automation system does not perform both the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask simultaneously. Level 2: Advanced driving assistance: The driving automation system continuously performs the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask of the dynamic driving task in a specific limited area. The driver D is expected to supervise the driving automation system by performing the OEDR, which is a subtask of the dynamic driving task. Level 3: Conditional Autonomous Driving: The driving automation system continuously performs all dynamic driving tasks in specific limited areas. In principle, driver D is not obligated to perform OEDR such as periphery monitoring. Periphery monitoring is the monitoring of the traffic environment around the vehicle. However, if it becomes difficult to continue this driving automation level, the driving automation system will request driver D to take over driving with ample time to spare. Driver D must respond to the request appropriately. Level 4: Highly automated driving: The driving automation system continuously performs all dynamic driving tasks in specific limited areas. If it becomes difficult to maintain the driving automation level in a limited area, the driving automation system will take action. Level 5: Fully automated driving: The driving automation system performs all dynamic driving tasks continuously, without being limited to specific restricted areas. If it becomes difficult to continue with this level of driving automation, the driving automation system will also respond without being limited to specific restricted areas.

[0016] The in-vehicle system 1 is configured to be capable of executing various driving controls during driving of the vehicle and various associated notification operations. Specifically, the in-vehicle system 1 has a configuration as a driving automation system for realizing or implementing driving assistance or automatic driving in the vehicle. "Autonomous driving" refers to a driving automation level corresponding to SAE levels 3 to 5 in which the driving automation system is responsible for, i.e., executes, all dynamic driving tasks. In contrast, "driving assistance" refers to a driving automation level corresponding to SAE levels 1 to 2 in which the driving automation system is responsible for, i.e., executes, some dynamic driving tasks. Hereinafter, the expression "driving assistance" includes "advanced driving assistance" of SAE level 2, unless it is clearly stated as "SAE level 1 driving assistance" or clearly distinguished from "advanced driving assistance" such as "driving assistance or advanced driving assistance".

[0017] In this embodiment, the in-vehicle system 1 is configured to be capable of performing at least SAE level 3 or higher autonomous driving, SAE level 2 advanced driving assistance, and SAE level 1 driving assistance. Therefore, the in-vehicle system 1 according to this embodiment can be called an "autonomous driving system" during autonomous driving, while it can be called a "driving assistance system" during driving assistance or advanced driving assistance. The advanced driving assistance that can be performed by the in-vehicle system 1 according to this embodiment includes so-called "hands-off driving". In hands-off driving, the in-vehicle system 1 automatically executes control of starting, steering, acceleration / deceleration, lane change, and stopping, on the condition that the driver D responds appropriately to an intervention request from the in-vehicle system 1. In hands-off driving, the driver D is not required to be in a hands-on state, but is required to monitor the road conditions, traffic conditions, and obstacle presence conditions around the vehicle. The "hands-on state" is a state in which the driver D can interfere with the steering of the vehicle, i.e., the lateral vehicle motion control subtask. The hands-on state is typically a state in which the driver D is seated in the driver's seat V23 in a position in which he or she can drive the vehicle and can operate the steering wheel V24 with his or her hands. In the hands-on state, the driver D usually holds the steering wheel V24 with his or her hands, but even if the driver D's hands are touching the steering wheel V24 in a state in which the driver D can immediately hold it, this state can be considered to be a "hands-on state." For example, a state in which the driver D is performing a steering operation, that is, actively operating the steering wheel V24, is considered to be a hands-on state. In addition, a state in which the driver D holds the steering wheel V24 against the steering control by the in-vehicle system 1 is also considered to be a hands-on state.

[0018] (In-vehicle system configuration) The overall configuration of the in-vehicle system 1 will be described below with reference to FIG. 1 and FIG. 2. The in-vehicle system 1 includes a driving ECU 2, a driving information input unit 3, an in-vehicle communication device 4, an HD map 5, a navigation device 6, an HMI device 7, lighting devices 8, and a motion control device 9. ECU stands for Electronic Control Unit. HD stands for High Definition. HMI stands for Human Machine Interface. The in-vehicle system 1 is configured by connecting the driving ECU 2, the driving information input unit 3, the in-vehicle communication device 4, the HD map 5, the navigation device 6, the HMI device 7, lighting devices 8, and the motion control device 9 to each other via an in-vehicle communication line 10. The in-vehicle communication line 10 is configured to comply with a predetermined communication standard such as CAN (international registered trademark: international registration number 1048262A). CAN (international registered trademark) stands for Controller Area Network. In addition to the main network conforming to CAN (international registered trademark), the in-vehicle communication line 10 may have a sub-network conforming to LIN, FlexRay, etc. LIN is an abbreviation for Local Interconnect Network.

[0019] (Drive ECU) The driving ECU 2 as a control device according to the present disclosure is configured to control the overall operation of the in-vehicle system 1 by being mounted in the system-mounted vehicle V. That is, the driving ECU 2 has a configuration as a so-called AD / ADAS ECU, which is a controller for driving assistance or automatic driving. AD stands for automatic driving. ADAS stands for Advanced Driver-Assistance Systems. Specifically, the driving ECU 2 has a configuration as an in-vehicle computer including a processor 21 and a memory 22 connected to the processor 21. More specifically, the driving ECU 2 has at least one processor 21 and at least one memory 22.

[0020] The processor 21 has a configuration as a CPU or MPU. CPU stands for Central Processing Unit. MPU stands for Micro Processing Unit. At least one memory 22 includes at least a ROM and a RAM among various non-transient substantial recording media such as ROM, RAM, non-volatile rewritable memory, etc. ROM stands for Read Only Memory. RAM stands for Random Access Memory. A "recording medium" may also be called a "storage medium." A non-volatile rewritable memory is a storage device that can rewrite information while the power is on, but retains information in an unrewritable manner while the power is off, such as a flash ROM or an EEPROM. EEPROM stands for Electronically Erasable and Programmable ROM. Data and program instructions used by the processor 21 are stored in advance in the ROM or non-volatile rewritable memory included in the at least one memory 22. The driving ECU 2 is configured to execute various processes or operations such as vehicle control operations and information notification to occupants by having the processor 21 read and execute program instructions stored in the memory 22.

[0021] (Driving information input section) The driving information input unit 3 is provided to input information required for various processing operations in the driving ECU 2 to the driving ECU 2. Specifically, the driving information input unit 3 includes a sonar sensor 31, a radar sensor 32, a laser radar sensor 33, a camera 34, an operation sensor 35, a behavior sensor 36, a driver state monitor 37, an operation switch 38, and a locator 39. The sonar sensor 31, the radar sensor 32, the laser radar sensor 33, and the camera 34 may be collectively referred to as a "periphery monitoring sensor" or an "ADAS sensor." Each element provided as the driving information input unit 3 will be described below in order.

[0022] (Sonar sensor) The sonar sensor 31 is an ultrasonic distance measuring sensor and is attached to the vehicle body V1. As shown in FIG. 3, the sonar sensor 31 is a so-called ultrasonic sensor, and is configured to detect the target B by irradiating sonar search waves Wsp in an ultrasonic band toward the space outside the vehicle and receiving sonar reflected waves Wsr that are reflected by the target B. Specifically, the sonar sensor 31 is configured to calculate the distance to the target B based on the TOF and the speed of sound. TOF is an abbreviation for Time of Flight. That is, TOF is the time from the transmission time of the sonar search wave Wsp to the reception time of the sonar reflected wave Wsr in the propagation path Ls of the sonar search wave Wsp, which is an ultrasonic wave, and the sonar reflected wave Wsr, and may also be referred to as the propagation time.

[0023] As shown in FIG. 4, when there are a pair of sonar sensors 31, a first sonar sensor 311 and a second sonar sensor 312, which are arranged at different positions in the horizontal direction, it is possible to calculate the relative position of the target B with respect to the vehicle by so-called triangulation. Here, in FIG. 4, the X-axis is set along a straight line connecting the first sonar sensor 311 and the second sonar sensor 312, and the Y-axis is set in a direction perpendicular to the X-axis. Both the X-axis and the Y-axis are assumed to be horizontal, that is, perpendicular to the vehicle height direction. In addition, the first sonar sensor 311 transmits a sonar search wave Wsp, and the reflected wave by the target B is received by the first sonar sensor 311 and the second sonar sensor 312. In this case, the position of the target B in the XY two-dimensional coordinate system is calculated based on the TOF in the first propagation path Ls1 and the TOF in the second propagation path Ls2. The first propagation path Ls1 is a propagation path of an ultrasonic wave that returns to the first sonar sensor 311 via the target B from the first sonar sensor 311. The ultrasonic waves propagating through the first propagation path Ls1 are referred to as "direct waves." The second propagation path Ls2 is a propagation path of ultrasonic waves that travels from the first sonar sensor 311 through the target B to the second sonar sensor 312. The ultrasonic waves propagating through the second propagation path Ls2 are referred to as "indirect waves."

[0024] As shown in FIG. 1, a vehicle body V1 is equipped with a plurality of sonar sensors 31. Specifically, a first front sonar SF1, a second front sonar SF2, a third front sonar SF3, and a fourth front sonar SF4 are mounted on a front bumper V12 as the sonar sensors 31. Similarly, a first rear sonar SR1, a second rear sonar SR2, a third rear sonar SR3, and a fourth rear sonar SR4 are mounted on a rear bumper V14 as the sonar sensors 31. In addition, a first side sonar SS1, a second side sonar SS2, a third side sonar SS3, and a fourth side sonar SS4 are mounted on a side portion of the vehicle body V1 as the sonar sensors 31. When it is not specified that the sonar sensor is one of the first front sonar SF1 to the fourth side sonar SS4, the singular expression "sonar sensor 31" or the expression "plurality of sonar sensors 31" may be used hereinafter. FIG. 5 shows the detection range of each of the plurality of sonar sensors 31. Each of the multiple sonar sensors 31 will be described in turn below with reference to FIGS.

[0025] The first front sonar SF1 is mounted on the front bumper V12 at a position close to the left end in the vehicle width direction so as to transmit a sonar search wave Wsp to the left front of the vehicle. The second front sonar SF2 is mounted on the front bumper V12 at a position close to the right end in the vehicle width direction so as to transmit a sonar search wave Wsp to the right front of the vehicle. The first front sonar SF1 and the second front sonar SF2 are disposed symmetrically across the vehicle width center line LC1. The front corner sonar detection range Rsc, which is the detection range of the first front sonar SF1 and the second front sonar SF2, is set to have a detection distance of, for example, about 60 cm. The "detection distance" is the maximum distance that can be measured from the sonar sensor 31.

[0026] The third front sonar SF3 and the fourth front sonar SF4 are arranged in the vehicle width direction at a position closer to the center of the front bumper V12 in the vehicle width direction. The third front sonar SF3 is arranged between the first front sonar SF1 and the vehicle width center line LC1 in the vehicle width direction so as to transmit a sonar search wave Wsp approximately in front of the vehicle. The fourth front sonar SF4 is arranged between the second front sonar SF2 and the vehicle width center line LC1 in the vehicle width direction so as to transmit a sonar search wave Wsp approximately in front of the vehicle. The third front sonar SF3 and the fourth front sonar SF4 are arranged symmetrically with respect to the vehicle width center line LC1. The front sonar detection range Rsf, which is the detection range of the third front sonar SF3 and the fourth front sonar SF4, is set to a detection distance of, for example, about 1 m.

[0027] The first front sonar SF1 and the third front sonar SF3 mounted on the left side of the vehicle body V1 are arranged at different positions in the horizontal direction. The first front sonar SF1 and the third front sonar SF3, which are adjacent to each other in the vehicle width direction, are arranged in a positional relationship such that the reflected wave of the sonar search wave Wsp emitted by one of them by the target B can be received as a received wave by the other. That is, the first front sonar SF1 is arranged so as to be able to receive both the direct wave corresponding to the sonar search wave Wsp emitted by itself and the indirect wave corresponding to the sonar search wave Wsp emitted by the third front sonar SF3. Similarly, the third front sonar SF3 is arranged so as to be able to receive both the direct wave corresponding to the sonar search wave Wsp emitted by itself and the indirect wave corresponding to the sonar search wave Wsp emitted by the first front sonar SF1.

[0028] Similarly, the third front sonar SF3 and the fourth front sonar SF4, which are mounted toward the center of the vehicle body V1 in the vehicle width direction, are disposed at different positions from each other in the horizontal direction. Moreover, the third front sonar SF3 and the fourth front sonar SF4, which are adjacent to each other in the vehicle width direction, are disposed in a positional relationship such that the one can receive the sonar search wave Wsp emitted by the other sonar and reflected by the target B as a received wave.

[0029] Similarly, the second front sonar SF2 and the fourth front sonar SF4 mounted on the right side of the vehicle body V1 are positioned at different positions from each other in the horizontal direction. The second front sonar SF2 and the fourth front sonar SF4, which are adjacent to each other in the vehicle width direction, are positioned in such a way that the reflected wave of the sonar search wave Wsp emitted by one of them by the target B can be received as a received wave by the other sonar.

[0030] The first rear sonar SR1 is mounted on the rear bumper V14 at a position close to the left end in the vehicle width direction so as to transmit a sonar search wave Wsp to the left rear of the vehicle. The second rear sonar SR2 is mounted on the rear bumper V14 at a position close to the right end in the vehicle width direction so as to transmit a sonar search wave Wsp to the right rear of the vehicle. The first rear sonar SR1 and the second rear sonar SR2 are disposed symmetrically across the vehicle width center line LC1. The rear corner sonar detection range Rsd, which is the detection range of the first rear sonar SR1 and the second rear sonar SR2, is set to a detection distance of, for example, about 60 cm.

[0031] The third rear sonar SR3 and the fourth rear sonar SR4 are arranged in the vehicle width direction at a position closer to the center of the rear bumper V14 in the vehicle width direction. The third rear sonar SR3 is arranged between the first rear sonar SR1 and the vehicle width center line LC1 in the vehicle width direction so as to transmit a sonar search wave Wsp approximately behind the vehicle. The fourth rear sonar SR4 is arranged between the second rear sonar SR2 and the vehicle width center line LC1 in the vehicle width direction so as to transmit a sonar search wave Wsp approximately behind the vehicle. The third rear sonar SR3 and the fourth rear sonar SR4 are arranged symmetrically with respect to the vehicle width center line LC1. The rear sonar detection range Rsr, which is the detection range of the third rear sonar SR3 and the fourth rear sonar SR4, is set to a detection distance of, for example, about 1.5 m.

[0032] The first rear sonar SR1 and the third rear sonar SR3 mounted on the left side of the vehicle body V1 are arranged at different positions in the horizontal direction. In addition, the first rear sonar SR1 and the third rear sonar SR3, which are adjacent to each other in the vehicle width direction, are arranged in a positional relationship such that the reflected wave of the sonar search wave Wsp emitted by one of them by the target B can be received as a received wave by the other. That is, the first rear sonar SR1 is arranged so as to be able to receive both the direct wave corresponding to the sonar search wave Wsp emitted by itself and the indirect wave corresponding to the sonar search wave Wsp emitted by the third rear sonar SR3. Similarly, the third rear sonar SR3 is arranged so as to be able to receive both the direct wave corresponding to the sonar search wave Wsp emitted by itself and the indirect wave corresponding to the sonar search wave Wsp emitted by the first rear sonar SR1.

[0033] Similarly, the third rear sonar SR3 and the fourth rear sonar SR4, which are mounted toward the center of the vehicle body V1 in the vehicle width direction, are disposed at different positions from each other in the horizontal direction. In addition, the third rear sonar SR3 and the fourth rear sonar SR4, which are adjacent to each other in the vehicle width direction, are disposed in a positional relationship such that the reflected wave of the sonar search wave Wsp emitted by one of them by the target B can be received as a received wave by the other.

[0034] Similarly, the second rear sonar SR2 and the fourth rear sonar SR4 mounted on the right side of the vehicle body V1 are disposed at different positions from each other in the horizontal direction. The second rear sonar SR2 and the fourth rear sonar SR4, which are adjacent to each other in the vehicle width direction, are disposed in a positional relationship such that the reflected wave of the sonar search wave Wsp emitted by one of them by the target B can be received as a received wave by the other.

[0035] The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are provided so as to transmit sonar search waves Wsp from the side of the vehicle body V1 to the side of the vehicle. The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are provided so as to be able to receive only direct waves. The side sonar detection range Rss, which is the detection range of the first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4, is set to be, for example, a detection distance of about 2 to 3 m.

[0036] The first side sonar SS1 is disposed between the left door mirror V17 and the first front sonar SF1 in the longitudinal direction so as to transmit sonar search waves Wsp to the left of the vehicle. The second side sonar SS2 is disposed between the right door mirror V17 and the second front sonar SF2 in the longitudinal direction so as to transmit sonar search waves Wsp to the right of the vehicle. The first side sonar SS1 and the second side sonar SS2 are provided symmetrically across the vehicle width center line LC1. The first side sonar SS1 and the second side sonar SS2 may be attached to the vehicle body panel V15, or may be attached to both ends of the front bumper V12 in the vehicle width direction, which are extended rearward along the vehicle overall length.

[0037] The third side sonar SS3 is disposed between the left rear door panel V16 and the first rear sonar SR1 in the front-rear direction so as to transmit sonar search waves Wsp to the left of the vehicle. The fourth side sonar SS4 is disposed between the right rear door panel V16 and the second rear sonar SR2 in the front-rear direction so as to transmit sonar search waves Wsp to the right of the vehicle. The third side sonar SS3 and the fourth side sonar SS4 are provided symmetrically across the vehicle width center line LC1. The third side sonar SS3 and the fourth side sonar SS4 may be attached to the vehicle body panel V15, or may be attached to both ends of the rear bumper V14 in the vehicle width direction, which are extended forward along the vehicle overall length.

[0038] FIG. 6 shows an example of detecting a parking space PS for parallel parking on the left side of the vehicle using a first side sonar SS1, which is one of the multiple sonar sensors 31. In the figure, a sonar detection point Psr is a point on the surface of the target B that is estimated to have reflected a sonar search wave Wsp emitted from the sonar sensor 31, and may also be called a "distance measurement point." As shown in FIG. 6, the parking space PS can be detected based on the distribution state of the sonar detection points Psr acquired based on the sonar reflection wave Wsr from the target B, which is a parked vehicle lined up along the traveling direction of the vehicle traveling straight. Specifically, for example, in the case of parallel parking shown in FIG. 6, the parking space PS can be detected at a location where the width of the section where the arrangement of the sonar detection points Psr lined up along the traveling direction of the vehicle is interrupted is equal to or greater than a predetermined width that exceeds the overall width of the vehicle.

[0039] (Radar Sensor) 1 and 2 again, the radar sensor 32 is configured to detect the target B by transmitting and receiving radar waves, which are millimeter waves or submillimeter waves. In this embodiment, the radar sensor 32 is mounted on the center of the front part V11 of the vehicle body V1 in the vehicle width direction. As shown in FIG. 7, the radar sensor 32 is a so-called long-distance radar having a detection range of about 10 to 250 m and a radar scanning angle range θr1 centered on the vehicle's forward direction Df, and has a phased array type or beam forming type configuration. The vehicle's forward direction Df is the direction in which a virtual straight line extending the vehicle width center line LC1 forward of the vehicle extends, and corresponds to the traveling direction when the vehicle is traveling forward, that is, when the vehicle is traveling in a shift position other than reverse shift. The vehicle's forward direction Df coincides with the traveling direction when the vehicle is traveling straight, and becomes the tangent direction of the traveling trajectory when the vehicle is traveling on a curve. The radar sensor 32 is configured to detect the target B within the radar detection range Rg1 by emitting radar search waves Wrp while scanning within the radar scan angle range θr1 and receiving radar reflected waves Wrr that are waves reflected by the target B. The radar detection range Rg1 is a sector-shaped range with the radar scan angle range θr1 as its central angle.

[0040] Specifically, the radar sensor 32 is an FMCW radar device equipped with a so-called array antenna, and is configured to be able to detect the distance to the target B, the azimuth angle θ, and the relative speed of the target B with respect to the vehicle, based on the frequency difference and phase difference of the transmitted and received millimeter waves. The azimuth angle θ is an angle between an imaginary line extending forward of the vehicle from the vehicle width center line LC1, which forms the center line of the radar detection range Rg1, and an imaginary line connecting the radar sensor 32 and the target B. The relative speed is the difference between the target moving speed vb and the vehicle speed vm, which is the traveling speed of the vehicle. More specifically, the radar sensor 32 transmits a radar search wave Wrp generated based on a transmission signal of a predetermined modulation frequency, receives a radar reflected wave Wrr, which is a wave reflected by the target B, and acquires a reception signal indicating the frequency characteristic of the radar reflected wave Wrr. The radar sensor 32 generates a beat signal, which is the deviation between the frequency of the transmission signal and the frequency of the reception signal. The radar sensor 32 performs a fast Fourier transform on the beat signal to obtain a frequency spectrum, analyzes the frequency spectrum to obtain a beat frequency, and calculates the distance and relative speed based on the beat frequency. In addition to the long-range radar sensor 32, so-called medium-range and short-range radar sensors 32 may also be provided in the in-vehicle system 1. The detection distance of the medium-range type is about 1 to 100 m, and the detection distance of the short-range type is about 15 cm to 30 m.

[0041] (Laser radar sensor) As shown in FIG. 8, the laser radar sensor 33 is configured to detect the target B by emitting a detection light Lp, which is a laser light in the infrared band, to the outside of the vehicle and receiving a reflected light Lr by the target B. The laser radar sensor 33 may also be called a LIDAR. LIDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging. In the vehicle-mounted system 1 according to this embodiment, a long-distance laser radar sensor 33 having a configuration as a so-called scanning LIDAR is mounted in the center of the front part V11 of the vehicle body V1 in the vehicle width direction, as shown in FIG. 1. The laser radar sensor 33 is configured to detect the target B within the LIDAR detection range Rg2 by scanning the detection light Lp in the horizontal direction within the LIDAR scan angle range θr2 centered on the vehicle forward direction Df. The LIDAR detection range Rg2 is a sector-shaped range with a radius of 200 m or more and with the LIDAR scan angle range θr2 as a central angle.

[0042] Fig. 9 shows a schematic configuration of the laser radar sensor 33. As shown in Fig. 9, the laser radar sensor 33 includes a light emitting unit 331, a scanning unit 332, and a light receiving unit 333. The light emitting unit 331 is configured to generate detection light Lp. The scanning unit 332 has a configuration as a so-called MEMS mirror. MEMS is an abbreviation for Micro Electro Mechanical Systems. That is, the scanning unit 332 is configured to scan the detection light Lp along the horizontal scanning direction Ds and the vertical scanning direction Dh by electrically controlling the orientation of a reflecting mirror that reflects the detection light Lp emitted from the light emitting unit 331 by a MEMS mechanism.

[0043] The light receiving unit 333 includes a light receiving sensor 334 that is a two-dimensional image sensor. That is, in the light receiving sensor 334, a plurality of light receiving elements 335 are two-dimensionally arranged in the vertical and horizontal directions, that is, in the directions corresponding to the horizontal scanning direction Ds and the vertical scanning direction Dh. The light receiving elements 335 are configured as light detection elements such as APD, SPAD, and photodiodes. APD stands for avalanche photodiode. SPAD stands for Single Photon Avalanche Diode. The laser radar sensor 33 is configured to generate detection point cloud data based on the reflected light Lr received by the light receiving unit 333, and detect the target B based on the detection point cloud data. The detection point cloud data is a two-dimensional array of a plurality of lidar detection points Prr arranged in an image-like manner in a two-dimensional direction corresponding to the horizontal scanning direction Ds and the vertical scanning direction Dh in FIG. 9. The LIDAR detection point Prr is a point on the target B that is estimated to have reflected the detection light Lp corresponding to the reflected light Lr detected by one light receiving element 335, and includes position information and distance measurement information in the above-mentioned two-dimensional array. Therefore, the detection point cloud data may also be called "distance measurement point cloud data." In this way, the laser radar sensor 33 acquires detection point cloud data in which a plurality of LIDAR detection points Prr are two-dimensionally arranged like an image. Therefore, the laser radar sensor 33 can be said to be a type of image sensor.

[0044] (camera) 1 and 2 again, the camera 34 as an image sensor is mounted on the vehicle so as to move with the movement of the vehicle while capturing images of the surroundings of the vehicle. That is, the camera 34 is configured to generate image information corresponding to the captured images of the surroundings of the vehicle. The image information may also be referred to as image data. The camera 34 is a so-called digital camera device and includes an image sensor such as a CCD or a CMOS. CCD stands for Charge Coupled Device. CMOS stands for Complementary Metal Oxide Semiconductor.

[0045] In this embodiment, the system-equipped vehicle V is equipped with multiple cameras 34, namely, a front camera CF, a rear camera CB, a left camera CL, and a right camera CR. When not indicating a specific one of the front camera CF, the rear camera CB, the left camera CL, and the right camera CR, hereinafter, the singular expression "camera 34" or the expression "multiple cameras 34" may be used.

[0046] The front camera CF is disposed at a central position in the vehicle width direction, i.e., on the vehicle width center line LC1 in a plan view, near the upper end of the front windshield V18 at the upper part of the vehicle interior V2 so as to obtain image information corresponding to an image in front of the vehicle. The front camera CF may be mounted on the front part V11 of the vehicle body V1. The rear camera CB is disposed at a central position in the vehicle width direction at the rear part V13 of the vehicle body V1 so as to obtain image information corresponding to an image behind the vehicle. The left camera CL is mounted on the left door mirror V17 so as to obtain image information corresponding to an image to the left of the vehicle. The right camera CR is mounted on the right door mirror V17 so as to obtain image information corresponding to an image to the right of the vehicle.

[0047] Based on an image captured by the camera 34, it is possible to recognize the target B, that is, to determine at least the presence and type of the target B. FIG. 10 shows an overview of an operation for recognizing the target B included in the foreground based on an image of the foreground including the road Rd in front of the vehicle captured by the front camera CF, as an example. Specifically, as shown in FIG. 10, a detection area Aw is set in the captured image. The detection area Aw is all or a part of the captured image area, i.e., the angle of view area of ​​the camera 34. Based on pixel feature parameters in the image within the detection area Aw, a feature point image Gp can be acquired. The pixel feature parameters are feature parameters of pixels constituting an image, such as brightness, contrast, color, etc. The "brightness" may also be called "lightness". The "color" may also be called "saturation". The feature point image Gp is ​​an image in which feature points Pt extracted based on the difference in pixel feature parameters between adjacent pixels, the gradient of change, etc. are arranged in an image shape. The feature points Pt are points that characterize the shape of the target B in the captured image. In other words, the feature points Pt are characteristic points, i.e., pixels, in the image within the detection area Aw. Then, by pattern matching between a feature point group Pg, which is a collection of the feature points Pt in the feature point image Gp, and a pre-stored pattern, it is possible to determine the type of the target B corresponding to the feature point group Pg.

[0048] It should be noted that the method of extracting the feature points Pt is well known at the time of filing the present application. Specifically, as the method of extracting the feature points Pt, for example, well-known methods such as a Sobel filter, a Laplacian filter, and a Canny method can be used. Therefore, in this specification, a detailed description of the method of extracting the feature points Pt is omitted. Also, the "extraction" of the feature points Pt can be expressed as "detection."

[0049] Also, it is possible to estimate the relative position or distance of the recognized target B with respect to the vehicle itself based on an image captured by the camera 34. FIG. 11 shows an overview of calculating an estimated point Pb, which is an estimation result of the three-dimensional position of a feature point Pt corresponding to the target B, by a technique called monocular moving stereo or SFM. SFM is an abbreviation for Structure from Motion. Note that the present disclosure is not limited to such monocular moving stereo. That is, the camera 34 may be a so-called compound eye stereo camera.

[0050] 11, it is assumed that the camera 34 moves in the host vehicle movement direction Dv as the host vehicle travels, and the position of the camera 34 at time t1 is a first camera position Pc1, and the position of the camera 34 at time t2 is a second camera position Pc2. The first captured image A1 is assumed to be an image captured by the camera 34 at the first camera position Pc1. The second captured image A2 is assumed to be an image captured by the camera 34 at the second camera position Pc2.

[0051] The first feature point Pt1 is a feature point Pt extracted from the first captured image A1. The second feature point Pt2 is a feature point Pt extracted from the second captured image A2 and estimated to correspond to the first feature point Pt1 at time t1. That is, the second feature point Pt2 corresponds to a point on the target B corresponding to the first feature point Pt1 at time t1, which is estimated to have moved after (t2-t1) seconds. It is possible to determine that the first feature point Pt1 and the second feature point Pt2 are the same, i.e., corresponding points, by a publicly known or well-known method such as optical flow. Then, the intersection of a first straight line L1 passing through the first camera position Pc1 and the first feature point Pt1 and a second straight line L2 passing through the second camera position Pc2 and the second feature point Pt2 is estimated as an estimated point Pb. Such an estimated point Pb is a point on the target B corresponding to the first feature point Pt1 and the second feature point Pt2 in a three-dimensional coordinate system based on the vehicle. If the estimated point Pb is a stationary point, it satisfies the epipolar constraint condition. The epipolar constraint condition is a condition defined by epipolar geometry, which states that the first camera position Pc1, the second camera position Pc2, and the estimated point Pb exist on the same plane Π at both times t1 and t2.

[0052] (Various sensors) 1 and 2 again, the operation sensor 35 is provided to generate outputs corresponding to various quantities related to the driving operation state of the vehicle by the driver D. The "various quantities related to the driving operation state" are, for example, an accelerator operation amount, a brake operation amount, a shift position, a steering angle, and the like. That is, the operation sensor 35 is a collective name for well-known sensors such as an accelerator pedal sensor, a brake pedal sensor, a shift position sensor, a steering angle sensor, a steering torque sensor, and the like, for the sake of simplicity of illustration and explanation. In addition, the operation sensor 35 may be provided with a so-called steering wheel sensor that detects whether the driver D is gripping the steering wheel V24.

[0053] The behavior sensor 36 is provided to generate outputs corresponding to various quantities related to the driving behavior of the vehicle. The "various quantities related to the driving behavior" include physical quantities related to the behavior or motion state of the vehicle, such as vehicle speed, yaw rate, longitudinal acceleration, lateral acceleration, etc. In other words, the behavior sensor 36 is a collective term for well-known sensors such as a vehicle speed sensor, a yaw rate sensor, an acceleration sensor, etc., for the sake of simplicity of illustration and explanation.

[0054] (Driver status monitor) 1 and 2, and also FIG. 12 and FIG. 13, the driver state monitor 37 is configured to sequentially detect the driver state based on the captured image of the driver D. The driver state includes, for example, whether or not the driver is awake. Specifically, the driver state monitor 37 includes a driver monitor camera provided in the vehicle interior V2 so that at least the head D1 of the driver D seated in the driver's seat V23 in a posture that allows the driver to drive the vehicle appropriately can be captured in the field of view so as to capture the face D2 from the front. The driver monitor camera is configured as a near-infrared camera. The driver state monitor 37 detects the driver state by performing image recognition processing on the captured image by the driver monitor camera.

[0055] The detection contents using the driver state monitor 37 include, for example, the direction of the face D2 of the driver D, the degree of opening of the eyes D3, the position of the pupils D4, and the like, as shown in FIG. 12 and FIG. 13. The direction of the face D2 includes a yaw angle θy and a pitch angle θp. The yaw angle θy represents the rotation angle of the face D2 of the driver D about a vertical axis Dx1 extending in the up-down direction. The yaw angle θy is 0° when the face D2 faces forward. The yaw angle θy is a positive value when the face D2 faces leftward, and a negative value when the face D2 faces rightward. The pitch angle θp represents the rotation angle of the face D2 about a horizontal axis Dx2 extending in the left-right direction. The pitch angle θp is 0° when the face D2 faces horizontally. The pitch angle θp is a positive value when the face D2 faces downward, and a negative value when the face D2 faces upward. FIG. 12 shows an example of a captured image when the driver D is in an awake state. Fig. 13 shows an example of a captured image when the driver D is not awake. As shown in Fig. 12 and Fig. 13, it is possible to detect the driver's state, including whether or not the driver is awake, the line of sight D5, etc., based on the direction of the face D2, the degree of opening of the eyes D3, the position of the pupils D4, etc.

[0056] (Operation switch) 1 and 2 again, the operation switches 38 are various switches operated by the driver D when driving the vehicle, and are provided in various locations in the vehicle compartment V2. The operation switches 38 correspond to switches that are not subject to detection of the operation amount or operation state by the operation sensor 35, and include at least an ignition switch 381, a blinker switch 382, ​​and an AD / ADAS switch 383. The ignition switch 381 is a switch for turning on and off the activation of the system-equipped vehicle V, i.e., the in-vehicle system 1, and may also be called a start switch or a power switch. The blinker switch 382 is a switch for detecting the operation state of a blinker lever. The AD / ADAS switch 383 is a switch for the driver D to input instructions such as start, end, level setting, and function selection related to driving assistance or automatic driving.

[0057] (locator) The locator 39 is configured to acquire highly accurate position information of the vehicle. Specifically, the locator 39 is a so-called composite positioning system, and includes a GNSS receiver 391, an inertial acquisition unit 392, and a locator ECU 393. GNSS is an abbreviation for Global Navigation Satellite System. The "highly accurate position information" is, for example, position information having a position accuracy of SAE level 2 or higher that can be used for advanced driving assistance or automated driving, specifically, an error of less than 10 cm. Note that, as the locator 39, a commercially available system such as the "POSLV" positioning and orientation system for land vehicles manufactured by Applanix, Inc., can be used.

[0058] The GNSS receiver 391 is provided so as to be capable of receiving a positioning signal transmitted from at least one positioning satellite, i.e., an artificial satellite. In this embodiment, the GNSS receiver 391 is configured so as to be capable of receiving a positioning signal from a positioning satellite in at least one of satellite positioning systems such as GPS, QZSS, GLONASS, Galileo, IRNSS, and the Beidou satellite navigation system. GPS is an abbreviation for Global Positioning System. QZSS is an abbreviation for Quasi-Zenith Satellite System. GLONASS is an abbreviation for Global Navigation Satellite System. IRNSS is an abbreviation for Indian Regional Navigation Satellite System.

[0059] The inertia acquisition unit 392 is configured to acquire the acceleration and angular velocity acting on the host vehicle. In this embodiment, the inertia acquisition unit 392 is provided as a three-axis gyro sensor and a three-axis acceleration sensor built into a box-shaped housing of the locator 39. The locator ECU 393 has a configuration as a so-called in-vehicle microcomputer including a CPU, a ROM, a RAM, an input / output interface, and the like. The locator ECU 393 is configured to sequentially determine the position, direction, and the like of the host vehicle based on the positioning signal received by the GNSS receiver 391, the acceleration and angular velocity acquired by the inertia acquisition unit 392, and the like.

[0060] (In-vehicle communication device) The in-vehicle communication device 4 is an in-vehicle communication module also called DCM, and is provided so as to be able to communicate information with base stations around the vehicle by wireless communication conforming to a communication standard such as LTE or 5G. DCM stands for Data Communication Module. LTE stands for Long Term Evolution. 5G stands for 5th Generation. Specifically, for example, the in-vehicle communication device 4 acquires traffic information such as congestion information from a probe server on the cloud and / or a predetermined database. The "traffic congestion information" includes the location and length of the congestion section. Specifically, the traffic congestion information includes the head position of the congestion, the tail position of the congestion, the estimated congestion distance, the estimated congestion time, and the like. The traffic information is also called "road traffic information." In addition, the in-vehicle communication device 4 acquires the latest HD map information from the above-mentioned probe server and stores it in the HD map 5.

[0061] (HD Map) The HD map 5 is mainly composed of a non-volatile rewritable memory so that the HD map information can be rewritably stored and the stored contents can be retained even when the power is cut off. The HD map information can also be called high-precision map data. The HD map information includes map information with higher precision than the map information in the SD map 601 used in the navigation device 6, which has a position error of about several meters. SD stands for Standard. Specifically, the HD map 5 stores information that can be used for advanced driving assistance or automatic driving, such as three-dimensional road shape information, lane number information, and regulation information, in accordance with a predetermined standard such as the ADASIS standard. ADASIS stands for Advanced Driver Assistance Systems Interface Specification.

[0062] (Navigation device) The navigation device 6 is configured to calculate a planned driving route from the current position of the vehicle to the destination. In this embodiment, the navigation device 6 is configured to calculate a planned driving route based on the destination set by the driver D or the like, map information stored in the HD map 5 or SD map 601, position information of the vehicle acquired from the locator 39, and the like. The navigation device 6 is also configured to be able to provide various information including route information, which is the calculation result, to each part such as the driving ECU 2 and the HMI device 7 via the in-vehicle communication line 10. That is, the navigation device 6 is configured to be able to display a navigation screen on the HMI device 7 for map display, route display, and the like.

[0063] (HMI device) The HMI device 7 is a so-called vehicle HMI, and has a configuration for realizing information transmission between the vehicle and the occupants including the driver D. Specifically, the HMI device 7 is configured to present various information related to the vehicle to the occupants at least visually, and to receive input operations by the occupants corresponding to the presented contents. The presented information includes, for example, various kinds of guidance, input operation instructions, input operation content notification, warnings, and the like. Typically, the HMI device 7 has a configuration as a so-called "dashboard HMI" mainly composed of devices attached to the dashboard V21 and the steering wheel V24. However, some of the input / output devices provided in the HMI device 7 may be provided in a place other than the dashboard V21, for example, in the ceiling part of the vehicle interior V2, or in the center console between the driver's seat V23 and the seat V22 adjacent thereto, i.e., the passenger seat, or the like.

[0064] The HMI device 7 includes an HCU 701, a meter panel 702, a main display device 703, a head-up display 704, a speaker 705, and an operation device 706. HCU is an abbreviation for HMI Control Unit. The HCU 701 has a configuration as a so-called in-vehicle microcomputer equipped with a CPU, ROM, RAM, an input / output interface, etc., and is configured to control display output and audio output by the HMI device 7. That is, the HCU 701 is provided to control the operations of the meter panel 702, the main display device 703, the head-up display 704, the speaker 705, etc.

[0065] The meter panel 702 is provided in front of the driver's seat V23 on the dashboard V21. The meter panel 702 is configured to execute meter display of the vehicle speed, coolant temperature, remaining fuel amount, etc., as well as various information display such as date and time, outside air temperature, mileage, radio station, etc.

[0066] The main display device 703 is a display device also called a CID device, and is disposed in the approximate center of the dashboard V21 in the vehicle width direction so that the display screen can be viewed by the occupant. CID stands for Center Information Display. The main display device 703 can display a navigation screen for map display and route display by the navigation device 6. The main display device 703 can also display information and content different from the navigation screen. Specifically, the main display device 703 is configured to be able to execute a display for setting operations related to driving modes such as "comfort", "normal", "sports", and "circuit". The main display device 703 is also configured to be able to execute a display related to a second task that the driver D can use during automatic driving. The second task is a task other than driving operations that the driver D executes, and includes, for example, reading, operating a mobile communication terminal, watching video content, and the like. The "video content" is, for example, a movie, a concert video, a music video, a television broadcast, and the like. The second task is also called a "non-driving task" or a "secondary activity".

[0067] 1 and 2, and also with reference to FIG. 14 and FIG. 15, the head-up display 704 is provided to display a display image M including characters, figures, and / or symbols in the forward field of vision of the driver D. In this embodiment, the head-up display 704 is configured to display the display image M as a virtual image in front of the driver D using AR technology, thereby enabling the display image M to be superimposed on the foreground including the road surface FR ahead of the vehicle. AR is an abbreviation for Augmented Reality. The "superimposed display" refers to displaying related information in a positional correspondence with a target of interest included in the foreground, for example, displaying related information so as to be superimposed on the target of interest or displaying it in the vicinity of the target of interest. The "target of interest" is, for example, a target object to which attention should be paid or attention should be paid during driving, such as a road marking, a road sign, a vehicle ahead, a pedestrian, etc. The "road marking" may also be referred to as a "road marking". For example, various displays such as a route display, a traveling direction display, and a traffic information display on the road surface FR ahead as an object of attention correspond to "superimposed display".

[0068] As shown in FIG. 14, the head-up display 704 is configured so that the depression angle AD is a positive value of 0 degrees or more. In FIG. 14, the depression angle AD is an angle between a virtual horizontal line passing through the viewpoint EP of the driver D and a virtual straight line connecting the viewpoint EP and the upper end of the projection range AP. The depression angle AD is a positive value when the upper end of the projection range AP is looked down from the viewpoint EP, and a negative value when the upper end is looked up. In addition, the head-up display 704 is configured so that the projection range AP is a horizontally elongated approximately rectangular shape as shown in FIG. 15 by making the horizontal angle of view larger than the vertical angle of view AV. The vertical angle of view AV is an angle between a virtual straight line connecting the viewpoint EP and the upper end of the projection range AP and a virtual straight line connecting the viewpoint EP and the lower end of the projection range AP in FIG. That is, the vertical angle of view AV is an angle range in the vertical direction in which the display image M can be viewed from the viewpoint EP, and may also be referred to as a viewing angle in the vertical direction.

[0069] As shown in FIG. 14, the head-up display 704 includes a projector 741 and a magnifying optical system 742. The projector 741 is configured to generate a display image light LV based on a display image signal generated by the HCU 701 and emit the display image light LV toward the magnifying optical system 742. The magnifying optical system 742 includes a plurality of optical elements including a concave mirror and an actuator for controlling the alignment of the optical elements. The magnifying optical system 742 is configured to adjust the projection state of the display image light LV onto the front windshield V18 by controlling the above alignment with the actuator according to the viewpoint EP detected by the driver state monitor 37. The head-up display 704 projects the display image light LV onto a projection range AP on the front windshield V18 and allows the driver D to visually recognize the reflected light, thereby realizing the display of the display image M.

[0070] FIG. 15 shows an example of a display state of the head-up display 704. The head-up display 704 can display an information display M1 and a graphic display M2. The information display M1 is a display content that displays various information such as the maximum speed on the road Rd on which the vehicle is traveling, the current vehicle speed of the vehicle, the distance to the destination, the estimated time of arrival at the destination, building names, intersection names, etc. The information display M1 is typically a display using characters, road sign-like figures, or symbols. The graphic display M2 is a display content using graphics such as lines and arrows, and is used to display, for example, information related to the traveling direction of the vehicle and lane selection.

[0071] The head-up display 704 can display superimposed content and non-superimposed content. The "superimposed content" is a display image content that is associated with or related to a specific target of attention included in the foreground and is superimposed on the target of attention. In contrast, the "non-superimposed content" is a display image content that is not associated with a specific target of attention included in the foreground and is not superimposed on the specific target of attention. In the display example of FIG. 15, a linear graphic display M2 is superimposed as superimposed content on the lane in which the vehicle is currently traveling, and an information display M1 indicating the maximum speed is displayed as non-superimposed content at an appropriate position that does not overlap with the graphic display M2. In this way, the HMI device 7 has a configuration as a notification unit that notifies a passenger such as the driver D of the vehicle of information.

[0072] 1 and 2 again, speaker 705 is provided to output audio corresponding to the display contents on meter panel 702, main display device 703, and head-up display 704. Note that speaker 705 can also output audio that does not correspond to the display contents on meter panel 702, main display device 703, and head-up display 704, such as music, radio audio, and the like.

[0073] The operation device 706 is an input device that is not included in the operation switch 38 and is not subject to detection of the operation amount or operation state by the operation sensor 35. Specifically, for example, switches provided around the display screen of the main display device 703 and a transparent touch panel provided so as to cover such a display screen correspond to the operation device 706. Also, for example, switches provided on the spokes of the handle V24 of the steering wheel structure correspond to the operation device 706. Furthermore, a pointing device such as a touch panel provided on the center console corresponds to the operation device 706.

[0074] (Lighting device) The lighting device 8 includes a body ECU 801, a headlight 802, and a blinker 803. The body ECU 801 has a configuration as a so-called in-vehicle microcomputer equipped with a CPU, a ROM, a RAM, an input / output interface, etc. The body ECU 801 is configured to control the lighting state of the headlights 802 and the blinkers 803 based on information input from the driving ECU 2 and the driving information input unit 3.

[0075] (Motion Control Device) The motion control device 9 is provided to control the motion, i.e., the driving behavior, of the vehicle based on information input from the driving ECU 2 and the driving information input unit 3. Specifically, the motion control device 9 includes a drive device 91, a shift device 92, a braking device 93, and a steering device 94.

[0076] The drive device 91 includes a drive system ECU 911 and a drive mechanism 912. The drive system ECU 911 has a configuration as a so-called in-vehicle microcomputer including a CPU, ROM, RAM, an input / output interface, etc., and is configured to control the operation of the drive mechanism 912. The drive mechanism 912 that generates the driving force for running the system-equipped vehicle V includes an engine and / or an electric motor. That is, the system-equipped vehicle V may be any of a gasoline engine vehicle, a diesel engine vehicle, a biofuel vehicle, a hydrogen engine vehicle, a hybrid vehicle, a BEV, a fuel cell vehicle, etc. BEV is an abbreviation for Battery Electric Vehicle.

[0077] The shift device 92 includes a shift ECU 921 and a shift mechanism 922. The shift ECU 921 has a configuration as a so-called in-vehicle microcomputer including a CPU, a ROM, a RAM, an input / output interface, etc., and is configured to control the operation of the shift mechanism 922. The shift mechanism 922 is provided between the wheels V3 functioning as drive wheels and the drive device 91, and is configured to switch the shift position, that is, switch between forward running and reverse running, and change the gear ratio during forward running. The shift device 92 may have a configuration as a so-called shift-by-wire.

[0078] The braking device 93 includes a brake ECU 931 and a brake mechanism 932. The brake ECU 931 has a configuration as a so-called on-board microcomputer including a CPU, a ROM, a RAM, an input / output interface, etc., and is configured to control the operation of the brake mechanism 932. The brake mechanism 932 that generates a braking force in the system-equipped vehicle V may include a regenerative brake mechanism in addition to a friction brake mechanism. The braking device 93 may have a so-called brake-by-wire configuration.

[0079] The steering device 94 includes a steering ECU 941 and a steering mechanism 942. The steering ECU 941 has a configuration as a so-called on-board microcomputer including a CPU, a ROM, a RAM, an input / output interface, and the like, and is configured to control the operation of the steering mechanism 942. The steering mechanism 942 is configured to be able to change the orientation of the left front wheel V3a and the right front wheel V3b, which are steered wheels. The steering mechanism 942 may also be able to change the orientation of the left rear wheel V3c and the right rear wheel V3d. That is, the host vehicle may be a so-called four-wheel steering vehicle. The steering device 94 may also have a configuration as a so-called steer-by-wire.

[0080] (Driving control based on target recognition results) 16 and 17 show an example of a functional configuration realized by the driving ECU 2 by the processor 21 executing the program instructions stored in the memory 22. Hereinafter, an overview of the driving control of the host vehicle executed by the driving ECU 2 based on the recognition result of the target B around the host vehicle using the ADAS sensor will be described. Hereinafter, in this specification, "recognition" and "detection" are defined as follows. "Recognition" is a concept including "detection", "identification" and "recognition". "Detection" refers to finding the target B based on images and point cloud data. The "detected" target B is only judged to exist, and the shape, meaning, etc. are not judged. "Identification" refers to classifying the shape, meaning, etc. of the detected target B. The "identified" target B is judged not only to exist, but also to be what it is, for example, whether it is a human, a car, or a building. "Recognition" refers to judging the detected and identified target B as something that should be taken into consideration in the driving control of the host vehicle, such as a pedestrian, an oncoming vehicle, etc. It should be noted that "detection" includes "detection."

[0081] 16, the driving ECU 2 has a recognition unit 2001, an operation determination unit 2002, and a control output unit 2003 as functional configurations realized by the processor 21. The recognition unit 2001 is adapted to execute a recognition operation of a target B around the vehicle based on information input from a surrounding monitoring sensor, an operation sensor 35, a behavior sensor 36, and the like. The operation determination unit 2002 is adapted to determine the content of a control operation required for the vehicle at the present time based on the recognition result of the target B by the recognition unit 2001 and information input from the operation sensor 35, the behavior sensor 36, and the like. The content of such a control operation is, for example, a collision avoidance operation, an emergency stop operation, a warning operation to the driver D, and the like. The control output unit 2003 is adapted to generate a control output according to the control operation content determined by the operation determination unit 2002, and output it to each unit in the in-vehicle system 1. The control output is, for example, a steering amount, a braking amount, a warning message code, and the like.

[0082] 17, the recognition unit 2001 has an input information acquisition function 2101, an input information processing function 2102, a target recognition function 2103, a vehicle lane recognition function 2104, an intersection recognition function 2105, and a surrounding environment recognition function 2106. The input information acquisition function 2101 holds a predetermined amount of information input to the recognition unit 2001 in time series. The input information processing function 2102 executes various processes such as noise removal and coordinate conversion on the information acquired, i.e., held, by the input information acquisition function 2101. The target recognition function 2103 executes a recognition operation of the target B based on the information processed by the input information processing function 2102. The target recognition function 2103 has a lane marking recognition function 2131, a road marking recognition function 2132, a roadside structure recognition function 2133, a traffic light recognition function 2134, a road sign recognition function 2135, a lane recognition function 2136, a pedestrian recognition function 2137, a surrounding vehicle recognition function 2138, and an obstacle recognition function 2139.

[0083] The recognition targets by each recognition function in the recognition unit 2001 are illustrated in FIG. 18, FIG. 19, etc. Referring to FIG. 18, the recognition targets B include, broadly speaking, a traffic-related three-dimensional object B1, another vehicle B2, a general three-dimensional object B3, and a road marking B4. Referring to FIG. 19, the recognition targets B include a parking frame line B5, a wheel stopper B6, and a parked vehicle B7 as objects related to the detection of the parking space PS and parking assistance. The traffic-related three-dimensional object B1 is a three-dimensional object for use in road traffic, such as a traffic light B11 and a road sign B12. The other vehicle B2 is a vehicle that may be a following target or an obstacle during the travel of the host vehicle, and does not include the parked vehicle B7 shown in FIG. 19. The general three-dimensional object B3 is a three-dimensional object other than the traffic-related three-dimensional object B1, the other vehicle B2, the wheel stopper B6, and the parked vehicle B7, and mainly constitutes an "obstacle." Furthermore, for example, it is possible to recognize the lane LN based on the recognition result of the target B using an image. That is, the parking space PS and the lane LN are different from the target B as a direct recognition target by the ADAS sensor, but can be said to be indirect recognition targets based on the recognition result of the target B. The lane LN as such an indirect recognition target can include, for example, the own lane LNm in which the own vehicle is currently traveling, the oncoming lane LNc, etc. Further details of each recognition target in the recognition unit 2001 will be described later.

[0084] The own lane recognition function 2104 recognizes the position of the own vehicle in the width direction of the road Rd on which the own vehicle is currently traveling, based on the recognition result of the target B by the target recognition function 2103. Hereinafter, the width direction of the road Rd is referred to as the "road width direction". In contrast, the direction perpendicular to the road width direction is referred to as the "road extension direction". The road extension direction is a direction extending along the road Rd, and is also referred to as the "road extension direction" or "road extension direction". When such a road Rd includes multiple lanes LN, the own lane recognition function 2104 recognizes which of the multiple lanes LN lined up in the road width direction the own lane LNm is. The intersection recognition function 2105 recognizes an intersection Xr around the own vehicle, based on the recognition result of the target B by the target recognition function 2103. Specifically, the intersection recognition function 2105 recognizes, for example, the presence or absence of a traffic light B11 and the color of the light, the presence of road markings B4 such as a stop line B42, the positions of the intersection entrance Xr1 and the intersection exit Xr2, the course division, etc., of the intersection Xr that the vehicle is currently approaching. The surrounding environment recognition function 2106 recognizes the driving environment around the vehicle, that is, for example, the presence or absence of obstacles, etc., based on the recognition result of the target B by the target recognition function 2103. These recognition results are provided to the operation determination unit 2002 shown in FIG. 16 to determine the content of the control operation currently required for the vehicle.

[0085] FIG. 18 shows an example of a foreground view from the vehicle in a situation where the vehicle is approaching an intersection Xr that forms a crossroads and has a traffic light B11. The intersection Xr that forms a crossroads is hereinafter simply referred to as a "crossroads". The recognition results of the traffic light B11 and the road sign B12 included in the traffic-related three-dimensional object B1 can be used, for example, for information notification or warning to the driver D by the HMI device 7. In addition, such recognition results can be used for the motion control of the vehicle during automatic driving or driving assistance. In addition to the traffic light B11 and the road sign B12, the traffic-related three-dimensional object B1 includes roadside structures B13 such as guardrails and curbs. The roadside structures B13 are recognized as obstacles when controlling the running of the vehicle. In addition, the traffic-related three-dimensional object B1 includes road studs B14 and poles B15 as shown in FIG. 20. Depending on the height and installation situation, the road studs B14 are recognized as obstacles when controlling the running of the vehicle. The pole B15 is recognized as an obstacle during driving control of the host vehicle.

[0086] 18 again, the general three-dimensional object B3 includes a fallen object B31 on the road, a pedestrian B32, a cyclist B33, a structure B34, etc. The fallen object B31 on the road, the pedestrian B32, and the cyclist B33 are recognized as obstacles when controlling the running of the vehicle. The structure B34 is not recognized as an obstacle because it is usually located outside the road Rd, but it can be a target for superimposed display when notifying the driver D of information by the head-up display 704, for example.

[0087] The road markings B4 include a crosswalk B41, a stop line B42, and a road dividing line B43. As shown in FIG. 21, the road markings B4 also include a character marking B44 and a graphic marking B45. The character markings B44 include, for example, a number indicating a maximum speed, a vehicle traffic division such as "bus only", and a regulatory character such as "stop". The graphic markings B45 include, for example, an arrow indicating a possible travel direction and a graphic for calling attention to the crosswalk B41 ahead. The recognition result of the road markings B4 can be used, for example, to estimate the positions of an intersection entrance Xr1, an intersection exit Xr2, and an intersection center Xrc. The intersection entrance Xr1 is the edge of the intersection of interest on the side where the vehicle enters. The intersection of interest is an intersection Xr that the vehicle is currently approaching and is closest to the vehicle, where the vehicle is scheduled to pass or is highly likely to pass. An intersection Xr that "the vehicle plans to pass through" is an intersection Xr on the planned travel route when the destination and planned travel route of the vehicle are set in the navigation device 6. An intersection Xr that "is likely to be passed through" is an intersection Xr that is estimated with a high probability that the vehicle will pass through, based on the current vehicle speed of the vehicle and the distance to the intersection Xr, when the destination and planned travel route of the vehicle are not set in the navigation device 6. The intersection exit Xr2 is the edge of the intersection of interest on the side from which the vehicle will exit. The intersection center Xrc is the center of the intersection of interest.

[0088] As shown in Figs. 20 to 22, the road division line B43 includes a roadway outer edge line B431, a center line B432, and a lane boundary line B433. The roadway outer edge line B431 is provided to indicate the outer edge line of the roadway in the road width direction. The "roadway" is a part of the road Rd on which vehicles travel. The center line B432 is provided to divide the road Rd, which is a two-way road, i.e., a non-one-way road, by traffic direction. The center line B432 may be a single line, a double line, or a triple line. The lane boundary line B433 is provided to divide multiple lanes LN of the same traffic direction on a road Rd with two or more lanes on one side. The road division line B43 may be painted white or yellow. In the example of Fig. 20, the center line B432 has a triple line pattern with yellow solid lines on both sides of a white solid line, and road studs B14 and poles B15 are provided on the central white solid line. In the example of Fig. 21, the center line B432 has a double line pattern with white solid lines, and the lane boundary line B433 has a yellow solid line portion beyond the white broken line portion. From the recognition results of the roadside structures B13 and road division lines B43, etc., it becomes possible to recognize the driving lane LNd, the oncoming lane LNc, the passing lane LNp, the right-turn lane LNr, the road shoulder LNs, the emergency parking zone EZ, etc.

[0089] The road markings B4 include a guidance strip marking B461, a safety zone marking B462, a no entry sign B463, a no stopping sign B464, and the like, as shown in Figs. 23 to 26. The guidance strip marking B461 shown in Fig. 23 is provided to indicate a guidance strip for guiding vehicles to travel safely and smoothly. Although the guidance strip is a marking for guiding vehicles not to pass through, the entry of vehicles into the guidance strip itself is not prohibited by law. The guidance strip marking B461 is provided, for example, near an intersection Xr, a junction, or a branch. The safety zone marking B462 shown in Fig. 24 is provided to indicate a safety zone. The safety zone is an area provided on the road Rd for the purpose of ensuring the safety of pedestrians B32, etc., and vehicles are not allowed to enter inside the safety zone. The no entry sign B463 shown in Fig. 25 is provided to indicate a no entry area. A no-entry area is an area that is not permitted for vehicle traffic, and vehicles are not permitted to enter inside the area. The no-stop sign B464 shown in Fig. 26 is provided to indicate a no-stop area. A no-stop area is an area that a vehicle is not permitted to enter if it is likely to be forced to stop due to the situation ahead of the path it is about to travel. In other words, a vehicle is not permitted to stop inside the no-stop sign B464 when waiting at a traffic light or in a traffic jam.

[0090] FIG. 27 shows an example of a road sign B12. The road sign B12 is a signboard that is installed beside or above the road Rd and provides necessary information to users through its design. The design includes an external shape, a color, and a symbol. The external shape is a circle, a triangle, an inverted triangle, a square, a diamond, an octagon, etc. The colors are white, black, red, blue, yellow, etc. In FIG. 27, the cross-hatched portion indicates red, the diagonal hatched portion indicates blue, the dot-hatched portion indicates yellow, and the blackened portion indicates black. The symbols include letters, figures, and combinations thereof.

[0091] In FIG. 27, examples of signs in Japan, Europe, and the United States are shown, categorized by meaning. For European examples, examples from Germany and France are used. For example, the design of "No entry" is almost the same in each country. In contrast, the color and symbol of "Stop" are common, but the external shape is almost the same in Europe and the United States, while the Japanese sign is different from Europe and the United States. The design of maximum speed signs is almost the same in Japan and Europe, except for the color of the letters, but the American sign is completely different from Japan and Europe in terms of external shape and color. The design of "Watch out for railroad crossings" is low in common between Japan, Europe, and the United States. The design of "No right turns" is similar in Europe and the United States, but there is no corresponding sign in Japan. However, for example, if a "No going in any direction other than the specified direction" sign, which shows the straight-ahead direction and the left-turn direction with white arrows, is provided at a cross intersection, as shown in FIG. 27, such a sign essentially indicates "No right turns."

[0092] Accurate recognition of the road signs B12 satisfactorily realizes automated driving and advanced driving assistance, but as described above, the design of the road signs B12 may differ greatly from country to country. Therefore, a database for recognizing the meaning indicated by the detected road signs B12 by pattern matching is stored in advance in the memory 22 shown in FIG. 2. Such a database may be country-specific or region-specific, or may be common to the entire world. Note that such a database may be stored in a host computer connected to the vehicle via the in-vehicle communication device 4 in place of or together with the memory 22. V2X is an abbreviation for Vehicle to X.

[0093] FIG. 28 shows an example of a scene in which the recognition result of the target B is used, in which the vehicle is approaching a crossroads. In this scene, for example, an information display M1 such as "10m to stop position" showing the distance between the vehicle and the stop line B42 can be superimposed on the stop line B42. In addition, a voice notification such as "10m to stop position" can be given. This makes it possible to effectively support the driver D using the driving support function to stop at the stop line B42 without fail. Alternatively, by notifying the passengers including the driver D using the automatic driving function of the scheduled occurrence of a stop event at an appropriate timing, it is possible to effectively avoid impairing the comfort of the passengers.

[0094] FIG. 29 shows another example of a scene in which the recognition result of the target B is used, in which the vehicle is approaching a crossroads with a traffic light B11 where the vehicle is scheduled to turn right. In this scene, for example, an information display M1 such as "20m ahead, turn right" indicating the distance to the right turn point can be superimposed on the road surface FR in front of the field of vision of the driver D in the right turn lane LNr, which is the vehicle lane LNm. In addition, a voice notification such as "turn right at the intersection 20m ahead" can be given. Furthermore, an arrow graphic display M2 indicating the planned travel route of the vehicle can be superimposed on the road surface FR so as to pass through the vicinity of the intersection center Xrc from the right turn lane LNr, which is the vehicle lane LNm, and reach the road Rd where the vehicle is to turn right. This allows the vehicle's course guidance and lateral acceleration generation behavior to be properly notified to the driver D and other occupants, thereby making it possible to properly avoid impairing the comfort of the occupants.

[0095] 17 again, the target recognition function 2103 recognizes the type and state of the target B based on the recognition result using the ADAS sensor and the HD map information stored in the HD map 5. The "state" includes the position. The "position" includes the distance from the vehicle, the relative position with respect to the vehicle, the lane position, i.e., one of the multiple lanes LN, and the like.

[0096] The lane marking recognition function 2131 recognizes road markings B43 provided around the vehicle, including on the road surface FR of the road Rd where the vehicle is heading. For example, the lane marking recognition function 2131 recognizes whether the outer roadway line B431, center line B432, lane boundary line B433, etc., as shown in Figs. 20 to 23, are white or yellow, solid or broken, single or multiple lines, etc. Technology for recognizing road markings B43 was well known at the time of filing the present application, and therefore detailed description thereof will be omitted in this specification.

[0097] The road marking recognition function 2132 recognizes the type, meaning, and position of road markings B4 provided around the vehicle, other than the road dividing line B43. Recognition targets by the road marking recognition function 2132 include, for example, a crosswalk B41, a stop line B42, and a graphic marking B45.

[0098] The roadside structure recognition function 2133 recognizes the type and position of the roadside structure B13 provided around the vehicle. At least one of the image recognition result using the camera 34, the recognition result using the laser radar sensor 33, and HD map information is used to recognize the roadside structure B13. The traffic light recognition function 2134 recognizes at least the position and light color of the traffic light B11 provided at the travel destination of the vehicle. The road sign recognition function 2135 recognizes the road signs B12 provided around the vehicle. The lane recognition function 2136 recognizes the lane, that is, recognizes the number and type of the lanes LN on the road Rd on which the vehicle is traveling. At least one of the image recognition result using the camera 34, i.e., the front camera CF, the recognition result using the laser radar sensor 33, and HD map information is used to recognize the lane. Specifically, in lane recognition, the image recognition results obtained using the front camera CF are usually used, but so-called "sensor fusion" that uses other information as well can be performed as necessary.

[0099] The pedestrian recognition function 2137 recognizes a pedestrian B32 and a cyclist B33 around the vehicle. The surrounding vehicle recognition function 2138 recognizes another vehicle B2 around the vehicle. The obstacle recognition function 2139 recognizes obstacles such as a fallen object B31 on the road around the vehicle. In the following description, among the targets B that are recognition targets for collision avoidance, those excluding the protection target targets are referred to as "obstacles". The "protection target targets" are pedestrians B32 and cyclists B33, including "people" as protection targets. The pedestrian B32 as the protection target target also includes so-called "lying person" who has fallen on the road Rd. Note that the "obstacle" may include other vehicles B2 including the parked vehicle B7, regardless of whether they have an occupant. The so-called "sensor fusion" may also be performed in the recognition of the recognition target in the pedestrian recognition function 2137, the surrounding vehicle recognition function 2138, and the obstacle recognition function 2139. Note that the recognition methods of the traffic light recognition function 2134, the road sign recognition function 2135, the lane recognition function 2136, the pedestrian recognition function 2137, the surrounding vehicle recognition function 2138, and the obstacle recognition function 2139 are well known at the time of filing of this application, and therefore further detailed explanations of these recognition methods will be omitted in this specification.

[0100] (Collision avoidance assistance control) Hereinafter, the details of the collision avoidance support control by the driving ECU 2 during execution of driving support corresponding to SAE levels 1 to 2 according to this embodiment will be described. The "collision avoidance support control" includes, for example, brake assist control that assists a collision avoidance operation by an occupant such as the driver D of the vehicle. In addition, the "collision avoidance support control" also includes, for example, collision avoidance control, that is, control of braking, steering, etc. by the in-vehicle system 1 when there is no collision avoidance operation by an occupant such as the driver D of the vehicle. In addition, when describing the effects of the driving ECU 2 as a control device according to this embodiment, the control method and control program executed thereby, and the recording medium on which the control program is recorded, these may be collectively referred to as "this embodiment" below.

[0101] In this embodiment, a collision avoidance support operation is performed based on the result of detection or recognition of a target B, i.e., an obstacle, ahead of the vehicle. The "collision avoidance support operation" includes a collision possibility notification and / or an avoidance movement support operation. The "collision possibility notification" is a notification by the HMI device 7 to the occupants, including the driver D, of the vehicle, that there is a possibility of a collision between the vehicle and a recognized obstacle, by display and / or voice. The collision possibility notification is displayed, for example, using the meter panel 702 or the head-up display 704. Hereinafter, the collision possibility notification is simply referred to as "notification". Note that the "notification" also includes an "alarm". The "alarm" is a stronger type of "alarm", i.e., a higher degree of attracting the attention of the occupants. The "avoidance movement support operation" is an intervention operation in the vehicle movement for collision avoidance by the movement control device 9, and specifically includes at least one of brake control, brake assist, steering control, and the like.

[0102] 2 and 16, the recognition unit 2001 functioning as a detection unit according to this embodiment detects and recognizes obstacles around the vehicle. That is, the recognition unit 2001 recognizes the presence, type, size, and movement state of obstacles around the vehicle. The "movement state" includes information such as the relative speed with respect to the vehicle and the relative movement direction. Based on these information and the information on the movement state of the vehicle, it is possible to calculate the ground speed, i.e., absolute speed and movement direction of the obstacle. The ground speed and movement direction of the obstacle can be calculated by the recognition unit 2001 or the operation determination unit 2002. The operation determination unit 2002 determines the possibility of a collision between the obstacle detected and recognized by the recognition unit 2001 and the vehicle. In addition, the operation determination unit 2002 determines, i.e., determines the operation content of each unit in the in-vehicle system 1, for example, the HMI device 7 and the motion control device 9, according to the determination result of the collision possibility. Specifically, the operation determination unit 2002 determines that the HMI device 7 should execute a notification when there is a possibility of a collision. Furthermore, when there is a possibility of a collision, the operation determination unit 2002 determines to execute an avoidance operation support operation by the motion control device 9, if necessary. Then, the control output unit 2003 generates a control output according to the control operation content determined by the operation determination unit 2002, and outputs it to each unit in the in-vehicle system 1.

[0103] FIG. 30 shows a situation where a road fallen object B31 exists on the road Rd ahead of the vehicle. As shown in FIG. 30, the road fallen object B31 can be detected by the radar sensor 32 and the camera 34. Here, for example, the road fallen object B31 detected by the radar sensor 32 and the camera 34 may be a low-profile object that the vehicle can climb over and pass through. Such a low-profile object has, for example, a height of about 15 to about 50 cm and a width and depth of about 30 cm to about 2.2 m. Note that these dimensional examples are for distinguishing the road fallen object B31 as a low-profile object from a step, a speed bump, a road stud, and the like, and the present disclosure is not limited to such values. Even if such a low-profile object is determined by the driving ECU 2, i.e., the operation determination unit 2002, as having a possibility of colliding with the vehicle and a notification is given by the HMI device 7, there may be a case where the driver D or other occupant does not perform a collision avoidance operation. In addition, such a low-profile object may be difficult to detect by the radar sensor 32 or the camera 34. Furthermore, if the object is not detected by either the radar sensor 32 or the camera 34, it can be said that the presence of an obstacle is uncertain. In such a situation, if the in-vehicle system 1 performs collision avoidance operations such as braking control or steering control on its own, there is a concern that the passengers may feel uncomfortable.

[0104] Therefore, in this embodiment, even if an obstacle with a possibility of collision is detected, if it is a low-profile object and the driver D or other occupant does not perform a collision avoidance operation, the motion control device 9 determines not to perform the collision avoidance support operation. Specifically, this embodiment determines whether the occupant performed the collision avoidance operation after the notification. Then, if the occupant does not perform the collision avoidance operation after the notification and the obstacle that was the subject of the notification is a low-profile object, the collision avoidance support operation is not performed. That is, in this case, the collision avoidance operation is left to the operation of the occupant such as the driver D. In this way, in this embodiment, the operation status of the occupant and the determination of whether the obstacle is a low-profile road-fallen object B31 are incorporated as control elements. As a result, in a situation where the presence of an obstacle is uncertain, the intervention of the in-vehicle system 1 is suppressed and the decision and operation of the driver D or other occupant are left to take appropriate measures. Therefore, according to this embodiment, the occurrence of inadvertent intervention in vehicle driving control by the in-vehicle system 1 is well avoided, thereby improving the convenience of the in-vehicle system 1.

[0105] (Example 1) Figures 31 to 33 show a specific example of an operation flow corresponding to this embodiment. In the illustrated flowcharts, "S" is an abbreviation of "step." Each process in the operation flow shown in Figures 31 to 33 is executed by the processor 21 shown in Figure 2 reading and starting a control program from the memory 22.

[0106] First, referring to FIG. 31, in step 101, the processor 21 determines whether the driver's state is normal based on the detection result of the driver's state by the driver state monitor 37 or the like. That is, the processor 21 determines whether the driver's state is such that the host vehicle needs to be urgently evacuated to the road shoulder LNs shown in FIG. 21 or the emergency parking zone EZ shown in FIG. 22. If the driver's state is not normal (i.e., step 101=NO), the processor 21 advances the process to step 102 and activates the EDSS. EDSS is an abbreviation for Emergency Driving Stop System. That is, the in-vehicle system 1 executes emergency evacuation of the host vehicle to the road shoulder LNs or the emergency parking zone EZ. On the other hand, if the driver's state is normal (i.e., step 101=YES), it is assumed that the driver D is in an appropriate driving posture according to the currently executed driving automation level, or is in a state where the driver D can immediately take the appropriate driving posture even if the driver D is not temporarily in the appropriate driving posture. Therefore, in this case, the processor 21 advances the process to step 103.

[0107] In step 103, the processor 21 determines whether the radar sensor 32 and the camera 34 are normal, i.e., whether an abnormality or malfunction of the radar sensor 32 or the camera 34 has been detected in advance. If an abnormality or malfunction of the radar sensor 32 or the camera 34 has been detected in advance (i.e., step 103=NO), the processor 21 temporarily ends this operation flow. On the other hand, if an abnormality or malfunction of the radar sensor 32 or the camera 34 has not been detected in advance (i.e., step 103=YES), the processor 21 advances the process to step 104.

[0108] In step 104, the processor 21 determines whether or not an obstacle is detected. If an obstacle is not detected (i.e., step 104=NO), the processor 21 temporarily ends this operation flow. On the other hand, if an obstacle is detected (i.e., step 104=YES), the processor 21 advances the process to step 105.

[0109] In step 105, processor 21 determines whether or not the detected obstacle is one with which there is a possibility of collision with the vehicle. Since the determination of the possibility of collision with an obstacle is already well known at the time of filing of the present application, detailed description thereof will be omitted in this specification. If there is no possibility of collision (i.e., step 105=NO), processor 21 temporarily ends this operation flow. On the other hand, if there is a possibility of collision (i.e., step 105=YES), processor 21 advances the process to step 106. In step 106, processor 21 determines whether or not the obstacle determined in step 105 as one with which there is a possibility of collision was detected by camera 34.

[0110] When the determination result in step 106 is "YES", the processor 21 advances the process to step 201 shown in FIG. 32. Referring to FIG. 32, in step 201, the processor 21 determines whether or not a collision avoidance operation is performed by an occupant such as the driver D t1 seconds before the expected collision time. t1 can be set to, for example, 3 seconds, but can be changed appropriately according to the relative speed and the like. When the determination result in step 201 is "YES", the processor 21 advances the process to step 202. In step 202, the processor 21 determines that the in-vehicle system 1 does not execute an alarm operation or a brake control operation. As a result, the collision avoidance operation is left to the collision avoidance operation by an occupant such as the driver D. On the other hand, when the determination result in step 201 is "NO", the processor 21 advances the process to step 203 and step 204.

[0111] In step 203, the processor 21 determines that a warning is to be issued by the in-vehicle system 1. As a result, a warning is issued using the HMI device 7. In step 204, the processor 21 determines whether or not a braking operation, which is a collision avoidance operation, has been performed by an occupant such as the driver D. If a braking operation has been performed (i.e., step 204=YES), the processor 21 advances the process to step 205. In step 205, the processor 21 determines that a brake assist operation is to be performed by the in-vehicle system 1. As a result, a brake assist operation is executed using the braking device 93. On the other hand, if a braking operation has not been performed (i.e., step 204=NO), the processor 21 advances the process to step 206.

[0112] In step 206, the processor 21 determines whether or not the obstacle detected as having a possibility of collision with the host vehicle is a low-profile object B31 on the road that the host vehicle can overcome and pass over. If the determination result in step 206 is "YES", the processor 21 advances the process to step 207. In step 207, the processor 21 determines that the vehicle-mounted system 1 will not execute brake control. In this case, brake control using the braking device 93, i.e., collision avoidance braking, is not executed. On the other hand, if the determination result in step 206 is "NO", the processor 21 advances the process to step 208. In step 208, the processor 21 determines that the vehicle-mounted system 1 will execute brake control. As a result, brake control using the braking device 93, i.e., collision avoidance braking, is executed.

[0113] If the determination result in step 106 is "NO", the processor 21 advances the process to step 301 shown in Fig. 33. Referring to Fig. 33 below, in step 301, the processor 21 determines whether or not a collision scene assumed in the case where no collision avoidance operation is performed with an obstacle detected as having a possibility of collision with the host vehicle is a frontal collision scene. A frontal collision refers to a collision of the front part V11 of the host vehicle with an obstacle. If it is not a frontal collision scene (i.e., step 301 = NO), the processor 21 advances the process to step 302. In step 302, the processor 21 determines not to execute brake control by the in-vehicle system 1. On the other hand, if it is a frontal collision scene (i.e., step 301 = YES), the processor 21 advances the process to step 303.

[0114] In step 303, the processor 21 determines whether or not a collision avoidance operation is performed by an occupant such as the driver D t2 seconds before the expected collision time. t2 can be set to, for example, 3 seconds, but can be appropriately changed according to the relative speed and the like. If the determination result in step 303 is "YES", the processor 21 advances the process to step 302. As a result, the brake control by the in-vehicle system 1 is not executed. On the other hand, if the determination result in step 303 is "NO", the processor 21 advances the process to steps 304 and 305. In step 304, the processor 21 determines that an alarm is to be issued by the in-vehicle system 1. As a result, an alarm is issued using the HMI device 7. In step 305, the processor 21 determines whether or not an obstacle detected as having a possibility of collision with the vehicle is in a moving state. That is, the processor 21 determines whether or not the ground speed of the obstacle is equal to or higher than a predetermined speed.

[0115] If the obstacle is in a stopped state (i.e., step 305=NO), the processor 21 advances the process to step 302. As a result, the brake control by the in-vehicle system 1 is not executed. However, the warning is executed by the process of step 304. That is, the operation determination unit 2002 determines only to execute the collision possibility notification when the ground speed is less than the predetermined speed. On the other hand, if the obstacle is in a moving state (i.e., step 305=YES), the processor 21 advances the process to step 306. In step 306, the processor 21 determines whether or not the brake operation is performed by an occupant such as the driver D. If the brake operation is performed (i.e., step 306=YES), the processor 21 advances the process to step 307. In step 307, the processor 21 determines to execute the brake assist operation by the in-vehicle system 1. On the other hand, if the brake operation is not performed (i.e., step 306=NO), the processor 21 advances the process to step 308. In step 308, the processor 21 leaves the brake control to the collision avoidance operation by the driver D or other occupant.

[0116] (Example 2) Fig. 34 is a partial modification of Fig. 31. That is, Fig. 32, Fig. 33, and Fig. 34 constitute a specific example different from the above-mentioned specific example 1. The contents of each process of step 401 to step 404 in the operation flow shown in Fig. 34 are the same as the contents of each process of step 101 to step 104 in the operation flow shown in Fig. 31. Therefore, for the explanation of each process of step 401 to step 404, the explanation of each process of step 101 to step 104 in the operation flow shown in Fig. 31 is cited.

[0117] If an obstacle is detected (i.e., step 404=YES), the processor 21 advances the process to step 405. In step 405, the processor 21 determines whether or not the detected obstacle is one that may cause a collision with the vehicle. That is, the determination content in step 405 is the same as that in step 105 in the operation flow shown in FIG. 31. If there is no possibility of a collision (i.e., step 405=NO), the processor 21 temporarily ends this operation flow. On the other hand, if there is a possibility of a collision (i.e., step 405=YES), the processor 21 advances the process to steps 406 and 407. In step 406, the processor 21 determines that the in-vehicle system 1 should issue a notification with a lower intensity than an alarm. As a result, a notification is issued using the HMI device 7. Subsequently, in step 407, the processor 21 determines whether or not the obstacle determined in step 405 as one that may cause a collision is one that the camera 34 could detect. That is, the process contents including the determination contents in step 407 are the same as those in step 106 in the operation flow shown in FIG.

[0118] As is clear from the above description, this specific example is obtained by adding or inserting a notification process for the possibility of collision in step 406 between step 105 and step 106 in the operation flow shown in FIG. 31 corresponding to the above specific example 1. Therefore, in this specific example, after the notification process in step 406, a collision avoidance operation determination process in step 201 or step 303 is executed. That is, in this specific example, the operation determination unit 2002 determines whether the occupant performed a collision avoidance operation after the notification of the possibility of collision. Furthermore, after the process of step 406, and further after the determination result in step 407 is "YES" and the process proceeds to step 201, the process of step 206 is executed. That is, in this specific example, if no collision avoidance operation is performed even after a predetermined time has elapsed since the notification, the operation determination unit 2002 determines whether the obstacle with the possibility of collision is a low-profile fallen object B31 on the road.

[0119] In this way, this specific example checks the occupant's behavior after the notification, and after checking that the occupant has not performed a collision avoidance operation in response to the notification, judges whether the obstacle is a low-profile fallen object B31 on the road. Then, this specific example changes the avoidance movement support operation mode by the in-vehicle system 1 according to the result of the judgment of whether the obstacle is a low-profile fallen object B31 on the road. Therefore, according to this specific example, the occurrence of inadvertent intervention in vehicle driving control by the in-vehicle system 1 is more effectively avoided, thereby improving the convenience of the in-vehicle system 1.

[0120] (Example 3) FIG. 35 is a partial modification of FIG. 33. That is, for example, another specific example different from the above-mentioned specific example 1 is constituted by FIG. 31, FIG. 32, and FIG. 35. Alternatively, for example, still another specific example is constituted by FIG. 32, FIG. 34, and FIG. 35. The contents of each process of step 501 and step 502 in the operation flow shown in FIG. 35 are the same as the contents of each process of step 301 and step 302 in the operation flow shown in FIG. 33. Therefore, for the explanation of each process of step 501 and step 502, the explanation of each process of step 301 and step 302 in the operation flow shown in FIG. 33 is cited.

[0121] In step 503, the processor 21 determines whether or not a collision avoidance operation is performed by an occupant such as the driver D t2 seconds before the expected collision time. t2 can be set to, for example, 3 seconds, but can be appropriately changed according to the relative speed and the like. If the determination result in step 503 is "YES", the processor 21 advances the process to step 502. As a result, the brake control by the in-vehicle system 1 is not executed. On the other hand, if the determination result in step 503 is "NO", the processor 21 advances the process to step 504. In step 504, the processor 21 determines whether or not an obstacle detected as having a possibility of a frontal collision with the host vehicle is in a moving state. That is, the processor 21 determines whether or not the ground speed of the obstacle is equal to or higher than a predetermined speed.

[0122] If the obstacle is in a stopped state (i.e., step 504=NO), the processor 21 advances the process to step 502. As a result, the brake control by the in-vehicle system 1 is not executed. On the other hand, if the obstacle is in a moving state (i.e., step 504=YES), the processor 21 advances the process to steps 505 and 506. In step 505, the processor 21 determines that the in-vehicle system 1 issues an alarm. As a result, an alarm is issued using the HMI device 7. Then, in step 506, the processor 21 determines whether or not a brake operation is performed by an occupant such as the driver D. The contents of each process in steps 506 to 508 in the operation flow shown in FIG. 35 are the same as the contents of each process in steps 306 to 308 in the operation flow shown in FIG. 33. Therefore, for the explanation of each process in steps 506 to 508, the explanation of each process in steps 306 to 308 in the operation flow shown in FIG. 33 is cited. In this way, in this specific example, the order of the warning process in step 304 and the moving state determination process in step 305 in FIG. 33 is reversed.

[0123] (Example 4) FIG. 36 shows an example of changing the processing contents according to the height of an obstacle with a possibility of collision. In this specific example, the operation determination unit 2002 determines that an obstacle with a possibility of collision with the host vehicle is a low-profile object B31 on the road as described above when the height of the obstacle is equal to or greater than the first threshold h1 and equal to or less than the second threshold h2. The first threshold h1 is, for example, 15 cm. The second threshold h2 is, for example, 50 cm. In this specific example, as shown in FIG. 36, the operation determination unit 2002 determines that a collision avoidance support operation is not performed when the height of an obstacle with a possibility of collision with the host vehicle is less than the first threshold h1, and determines that a collision avoidance support operation is performed when the height of the obstacle exceeds the second threshold h2. Note that "h1-h2" in the middle row in the table shown in FIG. 36 indicates that the height is equal to or greater than the first threshold h1 and equal to or less than the second threshold h2, that is, that the obstacle is a low-profile object that the host vehicle can pass over. In this case, the processing is performed as shown in each of the above specific examples.

[0124] (Example 5) FIG. 37 shows an example of changing the control contents and control timing according to the type and detection state of an obstacle that may collide with the vehicle. In FIG. 37, "sensor detection state" indicates the detection state of an obstacle in the radar sensor 32 and the camera 34. "◯" indicates that it is detected, "△" indicates that it was detected once but was lost later, and "-" indicates that it is not detected. Note that a situation in which both the radar sensor 32 and the camera 34 are "△", i.e., lost, is not possible in the PCS control according to this embodiment, and is not included in the table of FIG. 37. A situation in which both are lost is synonymous with an obstacle not being detected, and is not a situation in which the collision avoidance support operation according to this embodiment is executed. PCS is an abbreviation for pre-crash safety. "Ground speed" indicates the ground speed of the obstacle. "Obstacle type" indicates the type of the detected obstacle. "Vehicle" is another vehicle B2. "Structure" is a roadside structure B13 such as a guardrail, a building B34, or the like. "Road Object" is the road object B31. If it was detected, it is shown with "〇", and if it was not detected, it is shown with "-". "Scene" shows the corresponding collision type and driving scene.

[0125] Referring to FIG. 37, in this specific example, the following first to sixth cases are defined. First case: The detection state of the radar sensor 32 and the camera 34 is "◯" or "△", and the obstacle is another vehicle B2 or a structure. Second case: The detection state of the radar sensor 32 and the camera 34 is "◯" or "△" and the obstacle is an object fallen on the road B31. Third case: Of the radar sensor 32 and the camera 34, only the radar sensor 32 can detect the obstacle, the ground speed of the obstacle is less than a predetermined speed (i.e., slow or stopped), and the type of obstacle is unknown. Fourth Case: An obstacle is detected only by the radar sensor 32 out of the radar sensor 32 and the camera 34, the ground speed of the obstacle is equal to or greater than a predetermined speed, and the type of obstacle is unknown. Fifth Case: Of the radar sensor 32 and the camera 34, only the camera 34 is able to detect the obstacle, and the obstacle is another vehicle B2 or a structure. Sixth case: Of the radar sensor 32 and the camera 34, only the camera 34 has been able to detect the obstacle, and the obstacle is an object B31 that has fallen onto the road.

[0126] In the first case, this specific example issues an alarm and activates automatic braking if the occupant does not operate the brakes. The timing of the alarm and automatic braking is standard timing. The standard timing is, for example, 3 seconds before the collision for the alarm and about 1.4 seconds before the collision for the automatic braking, but it is possible to change, i.e. correct, depending on the relative speed, etc. Assumed scenes are, for example, a head-on collision, crossing, turning right, oncoming, cutting in, meeting, after turning, etc. "Crossing" is a scene in which a moving object as an obstacle (e.g., another vehicle B2) crosses in front of the vehicle, that is, a scene in which the moving object moves in a direction intersecting with the traveling direction of the vehicle. "Turning right" is a scene in which a right-turning vehicle collides with a vehicle moving straight. "Oncoming" is a scene in which a collision occurs with another vehicle B2, which is an oncoming vehicle. "Cutting in" is a scene in which a vehicle cuts in front of the vehicle. "Meeting" is a scene in which the vehicle may collide with another vehicle B2 proceeding from a different fork in the intersection toward the intersection. "After Turning" is a scene in which the vehicle collides with a pedestrian or motorcycle after turning right or left at an intersection.

[0127] In the second case, this specific example issues a warning and activates a brake assist if the occupant applies the brakes, but does not perform any brake control if the occupant does not apply the brakes. The timing of the warning and brake assist is standard. The standard timing is, for example, 3 seconds before the collision for the warning and about 2 seconds before the collision for the brake assist, but these can be changed, i.e., corrected, depending on the relative speed, etc. The assumed scene is a frontal collision.

[0128] In the third case, this specific example only issues an alarm and does not control the brakes. The timing of the alarm is delayed from the standard timing. The assumed scene is a frontal collision. In the fourth case, this specific example issues an alarm and activates automatic braking if the occupant does not operate the brakes. The timing of the alarm and automatic braking is delayed. The assumed scene is a frontal collision. In the third and fourth cases, the above-mentioned delay timing is, for example, 2 seconds before the collision for the alarm and about 1.2 seconds before the collision for the automatic braking, but can be changed, i.e., corrected, depending on the relative speed, etc. In this way, in this specific example, the operation determination unit 2002 delays the timing of executing the collision avoidance support operation when an obstacle with a possibility of collision is detected only by the radar sensor 32 compared to when the obstacle is detected by both the radar sensor 32 and the camera 34.

[0129] In the fifth case, this specific example issues an alarm, and activates automatic braking if the occupant does not apply the brakes. The timing of the alarm and automatic braking is standard timing. The assumed scene types are the same as in the first case, such as a frontal collision, crossing, turning right, oncoming, cutting in, meeting head-on, after turning, etc. In the sixth case, this specific example issues an alarm, and activates brake assist if the occupant applies the brakes, but does not perform brake control if the occupant does not apply the brakes. The timing of the alarm and brake assist is standard timing. The assumed scene is a frontal collision.

[0130] (Modification) The present disclosure is not limited to the above embodiment. Therefore, the above embodiment can be modified as appropriate. Representative modified examples will be described below. In the following description of the modified examples, differences from the above embodiment will be mainly described. In addition, the same reference numerals are used for parts that are the same or equivalent to each other in the above embodiment and the modified examples. Therefore, in the following description of the modified examples, the description of the above embodiment can be appropriately used for components having the same reference numerals as the above embodiment, unless there is a technical contradiction or special additional explanation.

[0131] The present disclosure is not limited to the specific device configurations shown in the above embodiments. That is, for example, the shape and structure of the vehicle body V1 in the system-equipped vehicle V are not limited to a box shape, i.e., a substantially rectangular shape in a plan view. In addition, the vehicle body panel V15 does not need to be provided on the upper side of the vehicle interior V2. Alternatively, the vehicle body panel V15 covering the upper side of the vehicle interior V2 may be detachable. There are also no particular limitations on the use of the system-equipped vehicle V, the positions of the driver's seat V23 and the steering wheel V24, the number of occupants, and the like.

[0132] The definition, level, or category of driving automation is not limited to those specified in "SAE J3016". Specifically, the "automated driving" in the above embodiment is a driving automation level corresponding to levels 3 to 5 in "SAE J3016" where the driving automation system is responsible for, i.e., executes, all dynamic driving tasks. For this reason, the definition of "automated driving" in the above embodiment naturally includes the fact that the driver D is not required to monitor the surroundings. However, the present disclosure is not limited to such an aspect. That is, for example, depending on the definition of "automated driving", not only "automated driving without the obligation to monitor the surroundings" but also "automated driving with the obligation to monitor the surroundings" can be conceived. Specifically, for example, the hands-off driving in the above embodiment can be interpreted as "automated driving with the obligation to monitor the surroundings". In this case, "automated driving" is a concept that includes so-called "partially automated driving" in which the driver D is responsible for, i.e., executes, some dynamic driving tasks such as the obligation to monitor the surroundings. "Partially automated driving" can be evaluated as being substantially synonymous with "advanced driving assistance". In addition, in the road traffic system of each country, the type and conditions of automated driving (for example, feasible roads, driving speed range, lane change availability, etc.) may be appropriately considered according to domestic circumstances, etc. For this reason, the present disclosure may be implemented with specifications that are compatible with the road traffic system of each country.

[0133] The configuration of the in-vehicle system 1 is not limited to the above example. For example, there is no particular limitation on the number and mounting positions of so-called ADAS sensors such as the sonar sensor 31, the radar sensor 32, and the camera 34. That is, for example, by reducing the number of radar sensors 32 and laser radar sensors 33, which are relatively expensive at the time of filing the present application, as much as possible, it is possible to promote the early spread of autonomous vehicles. Therefore, for example, the radar sensor 32 or the laser radar sensor 33 may be omitted.

[0134] Locator 39 is not limited to a configuration having inertial acquisition unit 392 built therein. Specifically, inertial acquisition unit 392 may receive output information from an angular velocity sensor and an acceleration sensor provided outside locator 39 as behavior sensor 36. Locator 39 may be integrated with HD map 5. Locator 39 is not limited to "POSLV" manufactured by Applanix.

[0135] The navigation device 6 may be connected to the HMI device 7 so as to be capable of communicating information via a sub-communication line different from the in-vehicle communication line 10. Alternatively, the navigation device 6 may have a display screen dedicated to displaying a navigation screen, separate from the HMI device 7. Alternatively, the navigation device 6 may be provided as a part of the HMI device 7. Specifically, for example, the navigation device 6 may be integrated with the main display device 703.

[0136] The HMI device 7 is not limited to a configuration including a meter panel 702, a main display device 703, and a head-up display 704. That is, for example, the meter panel 702 and the main display device 703 may be integrated into one display device. In this case, the meter panel 702 may be provided as a part of a display area in one display device that is a liquid crystal or organic EL display. That is, the meter panel 702 may be realized by displaying images of bezels, pointers, scales, etc. corresponding to a tachometer, a speedometer, a water temperature gauge, etc. Also, the HMI device 7 does not have to include the head-up display 704.

[0137] In the above embodiment, each ECU has a configuration as a so-called in-vehicle microcomputer including a CPU, an MPU, and the like. However, the present disclosure is not limited to such a configuration. That is, all or a part of the ECU may be configured to include a digital circuit configured to enable the above-mentioned operation, such as an ASIC or an FPGA. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array. That is, in the ECU, an in-vehicle microcomputer portion and a digital circuit portion may coexist.

[0138] The program according to the present disclosure, which enables the execution of various operations, procedures, or processes described in the above embodiment, may be downloaded or upgraded via V2X communication by the in-vehicle communication device 4. Alternatively, such a program may be downloaded or upgraded via a terminal device provided in a vehicle manufacturing plant, a maintenance plant, a dealer, or the like. Such a program may be stored on a memory card, an optical disk, a magnetic disk, or the like.

[0139] In this way, each of the above functional configurations and methods may be realized by a dedicated computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, each of the above functional configurations and methods may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, each of the above functional configurations and methods may be realized by one or more dedicated computers configured by combining a processor and a memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored in a computer-readable non-transitive tangible recording medium as instructions to be executed by a computer. In other words, each of the above functional configurations and methods may be expressed as a computer program including a procedure for realizing the same, or as a non-transitive tangible recording medium storing the program.

[0140] The present disclosure is not limited to the specific operation and processing modes shown in the above embodiment. That is, for example, although the present specification is described on the premise of so-called "vehicles driving on the left side" that complies with the road traffic laws in Japan, the present disclosure is not limited thereto.

[0141] The flowcharts shown in Fig. 31 and the like can also be modified as appropriate. That is, for example, before the processing of step 206 shown in Fig. 32, it may be determined whether or not the time required for the collision between the vehicle and an obstacle with which there is a possibility of collision, i.e., TTC, is within a threshold time. In this case, the operation determination unit 2002 calculates the required time, and if the required time is within the threshold time and no collision avoidance operation is performed even after a predetermined time has elapsed since the collision possibility notification, it determines whether the obstacle with which there is a possibility of collision is a low-profile object B31 that has fallen on the road. This further effectively prevents the in-vehicle system 1 from intervening in the vehicle driving control inadvertently.

[0142] The process of step 308 in Fig. 33 and the process of step 508 in Fig. 35 may be changed to a process of determining whether brake control is to be executed by the in-vehicle system 1. In this case, the operation determination unit 2002 operates as follows: if an obstacle with a possibility of collision can be detected only by the radar sensor 32 out of the radar sensor 32 and the camera 34, it determines whether the ground speed of the obstacle is equal to or greater than a predetermined speed. If the ground speed is equal to or greater than the predetermined speed, it determines to execute a possibility of collision notification, and if no collision avoidance operation is performed after the possibility of collision notification, it determines to execute a collision avoidance support operation. This further effectively prevents the in-vehicle system 1 from inadvertently intervening in vehicle driving control.

[0143] In the warning process such as step 203 in FIG. 32 or step 304 in FIG. 33, or in the notification process of step 406 in FIG. 34, the operation determination unit 2002 may change the manner of the collision possibility notification according to the type of obstacle with which there is a collision possibility. The "type of obstacle" here refers to, for example, another vehicle B2, a roadside object B31, or a building B34. Specifically, for example, when the obstacle is a roadside object B31, the operation determination unit 2002 may change the manner of the collision possibility notification according to the type of the roadside object B31. The "type of roadside object B31" here refers to the height of the roadside object B31, whether the vehicle can overcome it, and the like. This makes it possible to issue an appropriate notification or warning to the occupant.

[0144] The definition of a low-height object, that is, the values ​​of the first threshold value h1 and the second threshold value h2 and the ranges of the width and depth dimensions, can be changed as appropriate.

[0145] Similar expressions such as "obtain," "calculate," "estimate," "detect," "sensing," and "determine" may be substituted for each other as appropriate to the extent that they are not technically inconsistent. Furthermore, "information" and "signal" may be substituted for each other as appropriate to the extent that they are not technically inconsistent. Furthermore, "above the threshold" and "exceed the threshold" may be substituted for each other as appropriate to the extent that they are not technically inconsistent. Similarly, "below the threshold" and "below the threshold" may be substituted for each other as appropriate to the extent that they are not technically inconsistent.

[0146] Needless to say, the elements constituting the above-mentioned embodiments are not necessarily essential, except when expressly stated as essential or when clearly considered essential in principle. In addition, when the numbers, amounts, ranges, etc. of the components are mentioned, the present disclosure is not limited to the specific values, except when expressly stated as essential or when clearly limited to a specific value in principle. Similarly, when the shapes, directions, positional relationships, etc. of the components are mentioned, the present disclosure is not limited to the shapes, directions, positional relationships, etc., except when expressly stated as essential or when clearly limited to a specific shape, direction, positional relationship, etc. in principle.

[0147] The modified examples are not limited to the above examples. For example, multiple specific examples may be combined with each other as long as there is no technical contradiction. Furthermore, multiple modified examples may be combined with each other as long as there is no technical contradiction. Furthermore, all or part of the above embodiment and all or part of the modified examples may be combined with each other as long as there is no technical contradiction.

[0148] (Disclosure Perspective) As is apparent from the above description of the embodiments and modifications, at least the following disclosure items are disclosed in this specification.

[0149] [Point 1-1] A control device (2) for controlling the operation of an in-vehicle system (1) mounted on a vehicle (V), A detection unit (2001) that detects obstacles around the vehicle; an operation determination unit (2002) that determines a possibility of a collision between the obstacle detected by the detection unit and the vehicle, and determines an operation content of the in-vehicle system according to a result of the determination of the possibility of the collision; Equipped with The operation determination unit is When the possibility of a collision is detected, a determination is made to execute a collision possibility notification to notify the occupant (D) of the vehicle of the possibility of a collision by display and / or audio using an alarm device (7) provided in the in-vehicle system; determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; When the collision avoidance operation is not performed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass through, a determination is made not to perform a collision avoidance support operation by a motion control device (9) provided in the in-vehicle system to control the motion of the vehicle. Control device. [Point 1-2] When the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification, the operation determination unit determines whether the obstacle with the collision possibility is the low-profile fallen object on the road. A control device as described in aspect 1-1. [Points 1-3] The operation determination unit is Calculating a time required until a collision between the obstacle having the collision possibility and the vehicle; If the required time is within a threshold time and the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification, it is determined whether the obstacle with which the collision possibility exists is a low-profile object that has fallen onto the road. A control device as described in aspect 1-1. [Points 1-4] the operation determination unit determines, when a brake operation, which is the collision avoidance operation, is performed, to execute a brake assist operation, which is the collision avoidance support operation. The control device according to any one of Aspects 1-1 to 1-3. [Points 1-5] the operation determination unit determines to execute the collision avoidance assistance operation when the obstacle with the collision possibility is not the low-profile fallen object on the road. The control device according to any one of Aspects 1-1 to 1-4. [Points 1-6] The operation determination unit determines that the obstacle with the collision possibility is the low-profile fallen object on the road when the height of the obstacle is equal to or greater than a first threshold value and equal to or less than a second threshold value. The control device according to any one of Aspects 1-1 to 1-5. [Point 1-7] The operation determination unit determines whether the height of the obstacle having the collision possibility is If the difference is less than the first threshold, it is determined that the collision avoidance assist operation is not to be performed; If the second threshold is exceeded, it is determined that the collision avoidance assistance operation is to be performed. The control device according to aspects 1-6. [Point 1-8] The operation determination unit changes a manner of the collision possibility notification depending on a type of the obstacle with which the collision possibility exists. The control device according to any one of Aspects 1-1 to 1-7. [Point 1-9] The vehicle-mounted system includes a radar sensor (32) and a camera (34); The operation determination unit is When the obstacle with the possibility of collision is detected only by the radar sensor out of the radar sensor and the camera, it is determined whether the ground speed of the obstacle is equal to or greater than a predetermined speed; determining whether to execute the collision possibility notification when the ground speed is equal to or greater than the predetermined speed, and determining whether to execute the collision avoidance assistance operation when the collision avoidance operation has not been performed after the collision possibility notification. The control device according to any one of Aspects 1-1 to 1-8. [Points 1-10] the operation determination unit determines only to execute the collision possibility notification when the ground speed is less than the predetermined speed. The control device according to aspects 1-9. [Point 1-11] the operation determination unit delays the timing of executing the collision avoidance support operation in a case where the obstacle with the possibility of collision can be detected only by the radar sensor compared to a case where the obstacle can be detected by both the radar sensor and the camera. A control device according to aspect 1-9 or 1-10.

[0150] [Point 2-1] A control method executed by a control device (2) that controls an operation of an in-vehicle system (1) mounted on a vehicle (V), comprising: Detecting obstacles around the vehicle; determining a possibility of a collision between the detected obstacle and the vehicle; When the possibility of a collision is detected, a determination is made to execute a collision possibility notification to notify the occupant (D) of the vehicle of the possibility of a collision by display and / or audio using an alarm device (7) provided in the in-vehicle system; determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; When the collision avoidance operation is not performed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass through, a determination is made not to perform a collision avoidance support operation by a motion control device (9) provided in the in-vehicle system to control the motion of the vehicle. Control methods. [Point 2-2] If the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification, it is determined whether the obstacle with the possibility of collision is a low-profile object that has fallen on the road. The control method according to aspect 2-1. [Point 2-3] Calculating a time required until a collision between the obstacle having the collision possibility and the vehicle; If the required time is within a threshold time and the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification, it is determined whether the obstacle with which the collision possibility exists is a low-profile object that has fallen onto the road. The control method according to aspect 2-1. [Point 2-4] When a brake operation, which is the collision avoidance operation, is performed, it is determined that a brake assist operation, which is the collision avoidance support operation, is to be performed. The control method according to any one of Aspects 2-1 to 2-3. [Point 2-5] When the obstacle with the possibility of collision is not the low-profile fallen object on the road, it is determined that the collision avoidance assistance operation is to be performed. The control method according to any one of Aspects 2-1 to 2-4. [Point 2-6] When the obstacle having the collision possibility has a height equal to or greater than a first threshold value and equal to or less than a second threshold value, the obstacle is determined to be a low-profile object that has fallen onto the road. The control method according to any one of Aspects 2-1 to 2-5. [Point 2-7] The height of the obstacle with the possibility of collision is If the difference is less than the first threshold, it is determined that the collision avoidance assist operation is not to be performed; If the second threshold is exceeded, it is determined that the collision avoidance assistance operation is to be performed. The control method according to aspect 2-6. [Point 2-8] changing a manner of the collision possibility notification depending on a type of the obstacle with which the collision possibility exists; The control method according to any one of Aspects 2-1 to 2-7. [Point 2-9] The vehicle-mounted system includes a radar sensor (32) and a camera (34); When the obstacle with the possibility of collision is detected only by the radar sensor out of the radar sensor and the camera, it is determined whether the ground speed of the obstacle is equal to or greater than a predetermined speed; determining whether to execute the collision possibility notification when the ground speed is equal to or greater than the predetermined speed, and determining whether to execute the collision avoidance assistance operation when the collision avoidance operation has not been performed after the collision possibility notification. The control method according to any one of Aspects 2-1 to 2-8. [Point 2-10] When the ground speed is less than the predetermined speed, it is determined only to execute the collision possibility warning. The control method according to aspect 2-9. [Point 2-11] a timing for executing the collision avoidance support operation is delayed in a case where the obstacle with the possibility of collision can be detected only by the radar sensor compared to a case where the obstacle can be detected by both the radar sensor and the camera; The control method according to aspect 2-9 or 2-10.

[0151] [Point 3-1] A control program executed by a control device (2) for controlling an operation of an in-vehicle system (1) mounted on a vehicle (V), The process executed by the control device is A process of detecting obstacles around the vehicle; A process of determining a possibility of a collision between the detected obstacle and the vehicle; a process of determining whether to execute a collision possibility notification, when the collision possibility exists, to notify the vehicle occupant (D) of the collision possibility by display and / or audio using an alarm device (7) provided in the in-vehicle system; A process of determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; a process of determining not to execute a collision avoidance support operation by a motion control device (9) provided in the in-vehicle system so as to control the motion of the vehicle when the collision avoidance operation is not executed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass through; Including, Control program. [Point 3-2] and a process of determining whether the obstacle with the possibility of collision is a low-profile object that has fallen on the road when the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification. A control program according to aspect 3-1. [Point 3-3] A process of calculating a required time until a collision between the obstacle having the collision possibility and the vehicle; a process of determining whether the obstacle with the possibility of collision is a low-profile object that has fallen on the road when the required time is within a threshold time and the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification; Including, A control program according to aspect 3-1. [Point 3-4] A process of determining whether to execute a brake assist operation, which is the collision avoidance support operation, when a brake operation, which is the collision avoidance operation, is performed. The control program according to any one of Aspects 3-1 to 3-3. [Point 3-5] a process of determining whether to execute the collision avoidance assistance operation when the obstacle with the possibility of collision is not the low-profile fallen object on the road; The control program according to any one of Aspects 3-1 to 3-4. [Point 3-6] a process of determining that the obstacle with the possibility of collision is a low-profile object that has fallen onto the road when the height of the obstacle is equal to or greater than a first threshold value and equal to or less than a second threshold value; The control program according to any one of Aspects 3-1 to 3-5. [Point 3-7] a process of determining not to execute the collision avoidance support operation when the height of the obstacle with which there is a possibility of collision is less than a first threshold value, and determining to execute the collision avoidance support operation when the height of the obstacle exceeds a second threshold value, A control program according to any one of claims 3-6. [Point 3-8] A process of changing a manner of the collision possibility notification depending on a type of the obstacle with which the collision possibility exists. The control program according to any one of Aspects 3-1 to 3-7. [Point 3-9] a process of determining whether the ground speed of the obstacle with which there is a possibility of collision is equal to or greater than a predetermined speed when the obstacle is detected only by the radar sensor out of a radar sensor (32) and a camera (34) provided in the in-vehicle system; a process of determining whether to execute the collision possibility notification when the ground speed is equal to or greater than the predetermined speed, and determining whether to execute the collision avoidance assistance operation when the collision avoidance operation has not been performed after the collision possibility notification; Including, The control program according to any one of Aspects 3-1 to 3-8. [Point 3-10] a process of determining only to execute the collision possibility warning when the ground speed is less than the predetermined speed, A control program according to any one of claims 3-9. [Point 3-11] a process of delaying a timing of executing the collision avoidance support operation when the obstacle with the possibility of collision can be detected only by the radar sensor compared to ... both by the radar sensor and the camera; A control program according to aspect 3-9 or 3-10.

[0152] [Point 4] A computer-readable non-transitive tangible recording medium storing the control program according to any one of Aspects 3-1 to 3-11.

[0153] [Point 5] An in-vehicle system (1) mounted on a vehicle (V), Detecting an obstacle ahead of the vehicle; determining a possibility of a collision between the obstacle and the vehicle; When there is a possibility of a collision, a collision possibility notification is executed to notify an occupant of the vehicle of the possibility of a collision. determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; determining whether the obstacle with the collision possibility is a low-profile fallen object on the road (B31) that the vehicle can overcome and pass over; When the collision avoidance operation is not performed and the obstacle with the possibility of collision is the low-profile fallen object on the road, the motion control device (9) does not perform a collision avoidance support operation. In-vehicle systems.

[0154] [Point 6] An in-vehicle system (1) mounted on a vehicle (V), Detecting an obstacle ahead of the vehicle; determining a possibility of a collision between the obstacle and the vehicle; When there is a possibility of a collision, a collision possibility notification is executed to notify an occupant of the vehicle of the possibility of a collision. determining whether the obstacle with the collision possibility is a low-profile fallen object on the road (B31) that the vehicle can overcome and pass over; determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; When the collision avoidance operation is not performed and the obstacle with the possibility of collision is the low-profile fallen object on the road, the motion control device (9) does not perform a collision avoidance support operation. In-vehicle systems. [Explanation of symbols]

[0155] 1. In-vehicle systems 2. Control device 21 Processors 2001 Recognition unit (detection unit) 2002 Operation judgment section 7 HMI device (alarm device) 9. Motion Control Device B31 Fallen object on the road D Driver (passenger) V Vehicle

Claims

1. A control device (2) for controlling an operation of an in-vehicle system (1) mounted on a vehicle (V), A detection unit (2001) that detects obstacles around the vehicle; an operation determination unit (2002) that determines a possibility of a collision between the obstacle detected by the detection unit and the vehicle, and determines an operation content of the in-vehicle system according to a result of the determination of the possibility of the collision; Equipped with The operation determination unit is When the possibility of a collision is detected, a determination is made to execute a collision possibility notification to notify the occupant (D) of the vehicle of the possibility of a collision by display and / or audio using an alarm device (7) provided in the in-vehicle system; determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; When the collision avoidance operation is not performed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass through, a motion control device (9) provided in the in-vehicle system is judged not to perform a collision avoidance support operation to control the motion of the vehicle. Control device.

2. When the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification, the operation determination unit determines whether the obstacle with the collision possibility is the low-profile fallen object on the road. The control device according to claim 1 .

3. The operation determination unit is Calculating a time required until a collision between the obstacle having the collision possibility and the vehicle; If the required time is within a threshold time and the collision avoidance operation is not performed even after a predetermined time has elapsed since the collision possibility notification, it is determined whether the obstacle with which the collision possibility exists is a low-profile object that has fallen onto the road. The control device according to claim 1 .

4. the operation determination unit determines, when a brake operation, which is the collision avoidance operation, is performed, to execute a brake assist operation, which is the collision avoidance support operation. The control device according to any one of claims 1 to 3.

5. the operation determination unit determines to execute the collision avoidance assistance operation when the obstacle with the collision possibility is not the low-profile fallen object on the road. The control device according to claim 1 .

6. The operation determination unit determines that the obstacle with the collision possibility is the low-profile fallen object on the road when the height of the obstacle is equal to or greater than a first threshold value and equal to or less than a second threshold value. The control device according to claim 1 .

7. The operation determination unit determines whether the height of the obstacle having the collision possibility is If the difference is less than the first threshold, it is determined that the collision avoidance assist operation is not to be performed; If the second threshold is exceeded, it is determined that the collision avoidance assistance operation is to be performed. The control device according to claim 6.

8. The operation determination unit changes a manner of the collision possibility notification depending on a type of the obstacle with which the collision possibility exists. The control device according to claim 1 .

9. The vehicle-mounted system includes a radar sensor (32) and a camera (34); The operation determination unit is When the obstacle with the possibility of collision is detected only by the radar sensor out of the radar sensor and the camera, it is determined whether the ground speed of the obstacle is equal to or greater than a predetermined speed; determining whether to execute the collision possibility notification when the ground speed is equal to or greater than the predetermined speed, and determining whether to execute the collision avoidance assistance operation when the collision avoidance operation has not been performed after the collision possibility notification. The control device according to claim 1 .

10. the operation determination unit determines only to execute the collision possibility notification when the ground speed is less than the predetermined speed. The control device according to claim 9.

11. the operation determination unit delays the timing of executing the collision avoidance support operation in a case where the obstacle with the possibility of collision can be detected only by the radar sensor compared to a case where the obstacle can be detected by both the radar sensor and the camera. The control device according to claim 9 or 10.

12. A control method executed by a control device (2) that controls an operation of an in-vehicle system (1) mounted on a vehicle (V), comprising: Detecting obstacles around the vehicle; determining a possibility of a collision between the detected obstacle and the vehicle; When the possibility of a collision is detected, a determination is made to execute a collision possibility notification to notify the occupant (D) of the vehicle of the possibility of a collision by display and / or audio using an alarm device (7) provided in the in-vehicle system; determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; When the collision avoidance operation is not performed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass through, a motion control device (9) provided in the in-vehicle system is judged not to perform a collision avoidance support operation to control the motion of the vehicle. Control methods.

13. A control program executed by a control device (2) for controlling an operation of an in-vehicle system (1) mounted on a vehicle (V), The process executed by the control device is A process of detecting obstacles around the vehicle; A process of determining a possibility of a collision between the detected obstacle and the vehicle; a process of determining whether to execute a collision possibility notification, when the collision possibility exists, for notifying an occupant (D) of the vehicle of the collision possibility by display and / or audio using an alarm device (7) provided in the in-vehicle system; A process of determining whether the occupant has performed a collision avoidance operation after the collision possibility notification; a process of determining not to execute a collision avoidance support operation by a motion control device (9) provided in the in-vehicle system so as to control the motion of the vehicle when the collision avoidance operation is not executed and the obstacle with the possibility of collision is a low-profile fallen object (B31) on the road that the vehicle can overcome and pass over; Including, Control program.

Images