Control device, control method, and control program

Abstract

This control device (2) mounted on a vehicle comprises a processor (21) and a memory (22) which is a non-transitory tangible recording medium for storing program instructions used by the processor. The processor executes the following processes by executing the program instructions stored in the memory. The processor executes lane deviation prevention control on the basis of the result of recognizing the road situation at a travel destination of the vehicle. The processor estimates, on the basis of the result of recognizing a zebra zone adjacent to the lane in which a vehicle is currently traveling, an entrance accuracy corresponding to the probability the vehicle will enter the zebra zone. The processor suppresses the execution of the lane deviation prevention control to a greater degree when the entrance accuracy is higher.

Description

Control device, control method, and control program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Japanese Patent Application No. 2023-135583, filed on August 23, 2023, the contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a control device, a control method, and a control program to be mounted on a vehicle.

[0003] In recent years, research and development of so-called driver assistance technologies or autonomous driving technologies for automobiles and other vehicles has been actively promoted, and various proposals have been made (see, for example, Patent Document 1). Driving assistance technologies are technologies that reduce the burden on the driver and enable comfortable and safe driving, and include, for example, adaptive cruise control, lane keeping assist control, lane change assist control, parking assist control, obstacle warnings, collision avoidance assist control, and the like. Automated driving technologies are technologies that allow vehicles to travel automatically or autonomously without the need for driver operation. Vehicles equipped with such driving assistance technologies or autonomous driving technologies are equipped with sensing devices such as cameras and radars to grasp the conditions of the area ahead and surrounding areas. Based on road conditions and other information acquired by the vehicle using sensing devices such as cameras and map information, vehicle control such as steering, driving, and braking is automatically performed while the vehicle is traveling. Using high-precision maps including road data for each lane can improve safety and reliability.

[0004] International Publication No. 2019 / 202397

[0005] Research and development of these types of driving assistance technologies and autonomous driving technologies has accelerated in recent years from various perspectives, such as improving the accuracy of detection or recognition by sensing devices and improving convenience. In other words, further improving such performance will enable the early and widespread adoption of so-called advanced driving assistance vehicles and autonomous driving vehicles. The present disclosure has been made in consideration of the circumstances exemplified above.

[0006] According to one aspect of the present disclosure, a control device (2) mounted on a vehicle (V) comprises: a processor (21); and a memory (22) which is a non-transient tangible recording medium that stores program instructions used by the processor, wherein the processor executes the program instructions stored in the memory to perform lane departure prevention control based on the recognition results of the road conditions ahead of the vehicle, recognizes a zebra zone (B461) adjacent to the vehicle's own lane (LNm), which is the lane in which the vehicle is currently traveling, in the road width direction as the road conditions, and estimates an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition results of the zebra zone, and the higher the entry probability, the more the execution of the lane departure prevention control is suppressed. According to another aspect of the present disclosure, the control method is executed by a processor (21) provided in a control device (2) mounted on a vehicle (V), and executes lane departure prevention control based on the recognition results of road conditions ahead of the vehicle, recognizes a zebra zone (B461) adjacent to the vehicle's own lane (LNm), which is the lane in which the vehicle is currently traveling, in the road width direction as the road conditions, and estimates an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition results of the zebra zone, and the higher the entry probability, the more the execution of the lane departure prevention control is suppressed. According to yet another aspect of the present disclosure, the control program is a computer-readable program executed by a processor (21) provided in a control device (2) mounted on a vehicle (V), and the processing executed by the processor includes: executing lane departure prevention control based on a recognition result of road conditions ahead of the vehicle; recognizing, as the road conditions, a zebra zone (B461) adjacent to the host lane (LNm) in which the vehicle is currently traveling in the road width direction; estimating, based on the recognition result of the zebra zone, an entry probability corresponding to the possibility that the vehicle will enter the zebra zone; and suppressing the execution of the lane departure prevention control as the entry probability increases. Note that in each column of the application documents, each element may be assigned a reference symbol in parentheses.In this case, the reference symbols merely indicate an example of the correspondence between the same elements and the specific configurations described in the embodiments described later, and therefore the present disclosure is not limited in any way by the descriptions of the reference symbols.

[0007] 14 is a plan view showing a schematic configuration of a vehicle equipped with an on-board system including a driving ECU as a control device according to an embodiment. FIG. 15 is a schematic block diagram showing the overall configuration of the on-board system shown in FIG. 1. FIG. 16 is a diagram showing an outline of target detection using a sonar sensor shown in FIG. 2. FIG. 17 is a diagram showing an outline of target detection using a sonar sensor shown in FIG. 2. FIG. 18 is a diagram showing an outline of target detection ranges for each of a plurality of sonar sensors shown in FIG. 2. FIG. 19 is a diagram showing an outline of parking space detection using a sonar sensor shown in FIG. 2. FIG. 19 is a diagram showing an outline of target detection using a radar sensor shown in FIG. 2. FIG. 20 is a diagram showing an outline of target detection using a laser radar sensor shown in FIG. 2. FIG. 21 is a diagram showing an outline of the configuration and operation of the laser radar sensor shown in FIG. 8. FIG. 19 is a diagram showing an outline of target detection using a camera shown in FIG. 2. FIG. 21 is a diagram showing an outline of target detection using a camera shown in FIG. 2. FIG. 22 is a diagram showing an outline of driver state detection using a driver state monitor shown in FIG. 2. FIG. 23 is a diagram showing an outline of driver state detection using a driver state monitor shown in FIG. 2. FIG. 24 is a diagram showing a schematic configuration of the head-up display shown in FIG. 2. FIG. 15 is a schematic diagram showing the road surface ahead of the host vehicle and a part of the dashboard viewed through the front windshield from the viewpoint of the driver shown in FIG. 14. FIG. 17 is a block diagram showing an example of a schematic functional configuration realized by the driving ECU shown in FIG. 2. FIG. 18 is a block diagram showing an example of a schematic functional configuration realized by the recognition unit shown in FIG. 16. FIG. 19 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 20 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 21 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 22 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 23 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 24 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 25 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 26 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 27 is a schematic diagram showing a specific example of a target to be recognized in an embodiment. FIG. 28 is a schematic diagram showing a specific example of a target to be recognized in an embodiment.FIG. 1 is a schematic diagram showing another example of a control scene using a target recognition result in an embodiment. FIG. 2 is a plan view showing an overview of lane departure prevention support operation in an embodiment. FIG. 3 is a plan view showing an overview of lane departure prevention support operation in an embodiment. FIG. 4 is a plan view showing an overview of lane departure prevention support operation in an embodiment. FIG. 5 is a plan view showing an overview of lane departure prevention support operation in an embodiment. FIG. 6 is a plan view showing an example of a zebra zone. FIG. 7 is a plan view showing another example of a zebra zone. FIG. 8 is a plan view showing yet another example of a zebra zone. FIG. 9 is a plan view showing yet another example of a zebra zone. FIG. 10 is a flowchart showing an overview of lane departure prevention support operation in an embodiment. FIG. 11 is a flowchart showing an overview of lane departure prevention support operation in an embodiment. FIG. 12 is a plan view showing an overview of lane departure prevention support operation in an embodiment.

[0008] (Embodiments) Exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. Note that the configuration, functions, and implementation examples of the embodiments may be modified as appropriate. Therefore, multiple embodiments and variations may be described in the specification. In this case, in the following description, identical or equivalent parts between one embodiment and another embodiment, between one embodiment and its variation, or between one variation and another variation will be assigned the same reference numeral. Furthermore, the preceding description of such parts may be appropriately incorporated into other embodiments or variations that follow, unless there is a technical contradiction or additional explanation.

[0009] (Overall Vehicle Configuration) Referring to FIG. 1 , a system-equipped vehicle V, which is a vehicle to which the present disclosure is applied, is a so-called four-wheeled automobile and includes a vehicle body V1 that is substantially rectangular in plan view. Hereinafter, an imaginary line that passes through the center of the system-equipped vehicle V in the vehicle width direction and is parallel to the vehicle length direction of the system-equipped vehicle V will be referred to as a vehicle width center line LC1. Furthermore, an imaginary line that passes through the center of the system-equipped vehicle V in the vehicle length direction and is parallel to the vehicle width direction of the system-equipped vehicle V will be referred to as a vehicle length center line LC2. The vehicle width center line LC1 and the vehicle length center line LC2 pass through a vehicle center point VC. The vehicle center point VC is the center position of the vehicle body V1 in the three-dimensional direction. In FIG. 1 , the vehicle width direction is the left-right direction in the figure. The vehicle length direction is a direction that is perpendicular to the vehicle width direction and perpendicular to the vehicle height direction. The vehicle height direction is a direction that defines the vehicle height of the system-equipped vehicle V, and is a direction parallel to the direction of gravity when the system-equipped vehicle V is stably placed on a horizontal surface in a drivable state. For ease of explanation, the "front," "rear," "left," and "right" of the system-equipped vehicle V are defined as indicated by the arrows in FIG. 1 . That is, the vehicle overall length direction is synonymous with the front-to-rear direction. Furthermore, the vehicle width direction is synonymous with the left-to-right direction. Note that the vehicle height direction may not be parallel to the direction of gravity depending on the placement conditions or driving conditions of the system-equipped vehicle V. However, in many cases, the vehicle height direction is a direction that follows the direction of gravity.

[0010] Inside the vehicle body V1, a vehicle cabin V2 is formed, which constitutes a passenger space for occupants including the driver D. Wheels V3 are provided at each of the four corners of the vehicle body V1. That is, a left front wheel V3a is provided at the left front portion of the vehicle body V1. A right front wheel V3b is provided at the right front portion of the vehicle body V1. A left rear wheel V3c is provided at the left rear portion of the vehicle body V1. A right rear wheel V3d is provided at the right rear portion of the vehicle body V1. Note that the system-equipped vehicle V is not limited to a four-wheeled vehicle, but may be a three-wheeled vehicle, or a six- or eight-wheeled vehicle such as a cargo truck. There are also no particular limitations on the position or number of drive wheels.

[0011] A front bumper V12 is attached to a front surface V11, which is the front end of the vehicle body V1. A rear bumper V14 is attached to a rear surface V13, which is the rear end of the vehicle body V1. The side and top surfaces of the vehicle body V1 are covered by vehicle body panels V15. The vehicle body panels V15 include door panels V16. In the specific example shown in FIG. 1 , a total of four door panels V16 are provided, two on each side. Door mirrors V17 are attached to each of the pair of left and right front door panels V16. The front side of the passenger compartment V2 is covered by a front windshield V18. The front windshield V18 is formed in a plate shape from translucent glass or synthetic resin. The front windshield V18 is inclined so that the upper end is located further rearward than the lower end in a side view seen from a line of sight parallel to the vehicle width direction.

[0012] A dashboard V21 is provided at the front end of the passenger compartment V2. A plurality of seats V22 are arranged behind the dashboard V21. Of the plurality of seats V22, a steering wheel V24 is provided in front of a driver's seat V23 in which a driver D sits, for the driver D to grip and operate when steering. The steering wheel V24 is typically a so-called steering wheel formed in a substantially circular, elliptical, or polygonal ring shape, but may also be rod-shaped or control stick-shaped.

[0013] (Overview of the In-Vehicle System) A system-equipped vehicle V is equipped with an in-vehicle system 1. When installed in the system-equipped vehicle V, the in-vehicle system 1 is configured to function as a driving automation system for the system-equipped vehicle V. The system-equipped vehicle V may be referred to as the "host vehicle" hereinafter. A "driving automation system" is a system for achieving a driving automation level corresponding to at least one of levels 1 to 5 specified in the standard "SAE J3016" published by SAE International. SAE stands for Society of Automotive Engineers. Level X in "SAE J3016" is hereinafter simply referred to as "SAE Level X." X is any value between 0 and 5. The larger the value of X in SAE Level X, or the more dynamic driving tasks the driving automation system takes on, i.e., executes, the higher the driving automation level is expressed. Furthermore, a change in the driving automation level toward a higher level is referred to as an "increase" in the driving automation level. In contrast, the smaller the value of X, or the fewer dynamic driving tasks the driving automation system is responsible for or executes, the lower the level of driving automation is expressed. Furthermore, a change to a lower level of driving automation is referred to as a "degradation" of the level of driving automation.

[0014] (Definition of Driver Assistance and Automated Driving) The specific details of SAE Levels 0 to 5 are as follows. In the following description, driver D is a vehicle occupant who is responsible for or performs the dynamic driving task. The "dynamic driving task" refers to all operational and tactical functions that must be performed in real time when operating a vehicle in road traffic, excluding strategic functions. Driving behavior in general can be categorized into three types of functions: strategic, tactical, and operational. "Strategic" functions include route planning, route selection, etc., and specifically include deciding or selecting a route plan, such as "whether to go or not, when, where, and how to go." "Tactical" functions relate to vehicle operation in traffic situations, such as deciding whether and when to overtake or change lanes during the route, selecting an appropriate speed, and checking mirrors. "Operational" functions relate to instantaneous reactions, such as making minor corrections to steering, braking, accelerating, and accelerating operations to maintain lane position or avoid sudden obstacles or hazards in the vehicle's path. "OEDR" stands for Object and Event Detection and Response, and is also referred to as "object and event detection and response." OEDR includes monitoring the driving environment. Monitoring the driving environment includes detecting, recognizing, and classifying objects and events. Monitoring the driving environment also includes preparing to respond to objects and events as needed. A "constrained domain" is a specific condition in which a driving automation system or its functions are designed to operate, and is also referred to as an operational design domain or ODD. ODD stands for Operational Design Domain. A constrained domain includes at least one of multiple constraints, such as geographical, environmental, speed, and time.

[0015] Level 0: Manual driving...Driver D performs all dynamic driving tasks. Level 1: Driving assistance...The driving automation system continuously performs either the longitudinal vehicle motion control subtask or the lateral vehicle motion control subtask of the dynamic driving task in a specific limited area. The longitudinal vehicle motion control subtask is starting, accelerating / decelerating, and stopping. The lateral vehicle motion control subtask is steering. However, the driving automation system does not simultaneously perform both the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask. Level 2: Advanced driving assistance...The driving automation system continuously performs the longitudinal vehicle motion control subtask and the lateral vehicle motion control subtask of the dynamic driving task in a specific limited area. Driver D is expected to supervise the driving automation system by performing the OEDR, a subtask of the dynamic driving task. Level 3: Conditional automated driving...The driving automation system continuously performs all dynamic driving tasks in a specific limited area. In principle, driver D is not obligated to perform OEDR such as periphery monitoring. Periphery monitoring involves monitoring the traffic environment around the vehicle. However, if it becomes difficult to maintain the driving automation level, the driving automation system will request driver D to take over driving with sufficient time to spare. Driver D must respond appropriately to this request. - Level 4: Highly automated driving...The driving automation system continuously performs all dynamic driving tasks in specific limited areas. If it becomes difficult to maintain the driving automation level in the limited areas, the driving automation system will take action. - Level 5: Fully automated driving...The driving automation system continuously performs all dynamic driving tasks indefinitely, not limited to specific limited areas. If it becomes difficult to maintain the driving automation level, the driving automation system will also take action indefinitely, not limited to specific limited areas.

[0016] The in-vehicle system 1 is configured to be able to perform various driving controls and various associated notification operations while the vehicle is being driven. Specifically, the in-vehicle system 1 is configured as a driving automation system for realizing or implementing driving assistance or autonomous driving in the vehicle. "Automated driving" refers to a driving automation level corresponding to SAE Levels 3 to 5, in which the driving automation system is responsible for, or performs, all dynamic driving tasks. In contrast, "driving assistance" refers to a driving automation level corresponding to SAE Levels 1 to 2, in which the driving automation system is responsible for, or performs, some dynamic driving tasks. Hereinafter, the term "driving assistance" is intended to include "advanced driving assistance" of SAE Level 2, unless it is clearly stated as "SAE Level 1 driving assistance" or clearly distinguished from "advanced driving assistance" by using terms such as "driving assistance or advanced driving assistance."

[0017] In this embodiment, the in-vehicle system 1 is configured to be capable of performing at least SAE Level 3 or higher autonomous driving, SAE Level 2 advanced driving assistance, and SAE Level 1 driving assistance. Therefore, the in-vehicle system 1 according to this embodiment can be referred to as an "autonomous driving system" while performing autonomous driving, and as a "driving assistance system" while performing driving assistance or advanced driving assistance. The advanced driving assistance that can be performed by the in-vehicle system 1 according to this embodiment includes so-called "hands-off driving." Hands-off driving is a mode in which the in-vehicle system 1 automatically controls starting, steering, acceleration / deceleration, lane changes, and stopping, provided that the driver D appropriately responds to intervention requests from the in-vehicle system 1. Hands-off driving does not require the driver D to be in a hands-on state, but does require the driver D to monitor road conditions, traffic conditions, the presence of obstacles, and the like around the vehicle. The "hands-on state" refers to a state in which the driver D can interfere with the steering of the vehicle, i.e., the lateral vehicle motion control subtask. The hands-on state is typically a state in which the driver D is seated in the driver's seat V23 in a position capable of driving the vehicle and can operate the steering wheel V24 with his / her hands. In the hands-on state, the driver D usually holds the steering wheel V24 with his / her hands, but even if the driver D's hands are touching the steering wheel V24 in a state in which they can be immediately grasped, this can be considered a "hands-on state." For example, a state in which the driver D is performing a steering operation, that is, actively operating the steering wheel V24, is considered a hands-on state. In addition, a state in which the driver D holds the steering wheel V24 against the steering control by the in-vehicle system 1 is also considered a hands-on state.

[0018] (In-Vehicle System Configuration) The overall configuration of the in-vehicle system 1 will be described below with reference to FIGS. 1 and 2 . The in-vehicle system 1 includes a driving ECU 2, a driving information input unit 3, an in-vehicle communication device 4, an HD map 5, a navigation device 6, an HMI device 7, lighting devices 8, and a motion control device 9. ECU stands for Electronic Control Unit. HD stands for High Definition. HMI stands for Human Machine Interface. The in-vehicle system 1 is configured by interconnecting the driving ECU 2, the driving information input unit 3, the in-vehicle communication device 4, the HD map 5, the navigation device 6, the HMI device 7, the lighting devices 8, and the motion control device 9 via an in-vehicle communication line 10. The in-vehicle communication line 10 is configured to comply with a predetermined communication standard such as CAN (international registered trademark: International Registration Number 1048262A). CAN (international registered trademark) stands for Controller Area Network. The in-vehicle communication line 10 may have a sub-network conforming to LIN, FlexRay, etc., in addition to a main network conforming to CAN (an international registered trademark). LIN is an abbreviation for Local Interconnect Network.

[0019] (Driving ECU) The driving ECU 2 as a control device according to the present disclosure is configured to be mounted in a system-equipped vehicle V and to control the overall operation of the in-vehicle system 1. That is, the driving ECU 2 is configured as a so-called AD / ADAS ECU, which is a driving assistance or automatic driving controller. AD stands for automatic driving. ADAS stands for Advanced Driver-Assistance Systems. Specifically, the driving ECU 2 is configured as an in-vehicle computer including a processor 21 and a memory 22 connected to the processor 21. More specifically, the driving ECU 2 has at least one processor 21 and at least one memory 22.

[0020] The processor 21 is configured as a CPU or MPU. CPU stands for Central Processing Unit. MPU stands for Micro Processing Unit. At least one memory 22 includes at least a ROM and a RAM among various non-transient physical recording media such as ROM, RAM, and non-volatile rewritable memory. ROM stands for Read Only Memory. RAM stands for Random Access Memory. A "recording medium" may also be referred to as a "storage medium." A non-volatile rewritable memory is a storage device that allows information to be rewritten while the power is on but retains information in an unrewritable manner while the power is off, such as a flash ROM or an EEPROM. EEPROM stands for Electronically Erasable and Programmable ROM. Data and program instructions used by the processor 21 are pre-stored in the ROM or non-volatile rewritable memory included in the at least one memory 22. The driving ECU 2 is configured to execute various processes or operations such as vehicle control operations and information notification to occupants by having the processor 21 read and execute program instructions stored in the memory 22.

[0021] (Driving Information Input Unit) The driving information input unit 3 is provided to input information necessary for various processing operations in the driving ECU 2 to the driving ECU 2. Specifically, the driving information input unit 3 includes a sonar sensor 31, a radar sensor 32, a laser radar sensor 33, a camera 34, an operation sensor 35, a behavior sensor 36, a driver state monitor 37, an operation switch 38, and a locator 39. The sonar sensor 31, the radar sensor 32, the laser radar sensor 33, and the camera 34 may also be collectively referred to as a "periphery monitoring sensor" or an "ADAS sensor." Each element provided as the driving information input unit 3 will be described below in order.

[0022] (Sonar Sensor) The sonar sensor 31 is an ultrasonic distance measuring sensor mounted on the vehicle body V1. As shown in FIG. 3 , the sonar sensor 31 is a so-called ultrasonic sensor that detects the target B by emitting sonar search waves Wsp in the ultrasonic band toward the space outside the vehicle and receiving sonar reflection waves Wsr that are reflected by the target B. Specifically, the sonar sensor 31 calculates the distance to the target B based on the TOF and the speed of sound. TOF stands for Time of Flight. That is, TOF is the time from the transmission time of the sonar search waves Wsp to the reception time of the sonar reflection waves Wsr, which are ultrasonic waves, along the propagation path Ls of the sonar search waves Wsp and the sonar reflection waves Wsr, and may also be referred to as the propagation time.

[0023] As shown in FIG. 4 , when a pair of sonar sensors 31, a first sonar sensor 311 and a second sonar sensor 312, are located at different positions in the horizontal direction, it is possible to calculate the relative position of target B with respect to the vehicle by so-called triangulation. In FIG. 4 , the X axis is defined along the line connecting the first sonar sensor 311 and the second sonar sensor 312, and the Y axis is defined perpendicular to the X axis. Both the X axis and the Y axis are assumed to be horizontal, i.e., perpendicular to the vehicle height direction. Furthermore, the first sonar sensor 311 emits a sonar search wave Wsp, and the reflected wave from target B is received by the first sonar sensor 311 and the second sonar sensor 312. In this case, the position of target B in an XY two-dimensional coordinate system is calculated based on the TOF along the first propagation path Ls1 and the TOF along the second propagation path Ls2. The first propagation path Ls1 is a propagation path of ultrasonic waves that travels from the first sonar sensor 311 via the target B and back to the first sonar sensor 311. The ultrasonic waves that travel along the first propagation path Ls1 are called "direct waves." The second propagation path Ls2 is a propagation path of ultrasonic waves that travel from the first sonar sensor 311 via the target B and reach the second sonar sensor 312. The ultrasonic waves that travel along the second propagation path Ls2 are called "indirect waves."

[0024] As shown in FIG. 1 , a vehicle body V1 is equipped with multiple sonar sensors 31. Specifically, the front bumper V12 is equipped with a first front sonar SF1, a second front sonar SF2, a third front sonar SF3, and a fourth front sonar SF4 as sonar sensors 31. Similarly, the rear bumper V14 is equipped with a first rear sonar SR1, a second rear sonar SR2, a third rear sonar SR3, and a fourth rear sonar SR4 as sonar sensors 31. Furthermore, the side portions of the vehicle body V1 are equipped with a first side sonar SS1, a second side sonar SS2, a third side sonar SS3, and a fourth side sonar SS4 as sonar sensors 31. Hereinafter, unless it is specified that a specific one of the first front sonar SF1 to the fourth side sonar SS4 is being referred to, the singular expression "sonar sensor 31" or the expression "multiple sonar sensors 31" may be used. 5 shows the detection range of each of the plurality of sonar sensors 31. Each of the plurality of sonar sensors 31 will be described below in order with reference to FIGS.

[0025] The first front sonar SF1 is mounted near the left edge of the front bumper V12 in the vehicle width direction so as to emit sonar search waves Wsp to the left front of the vehicle. The second front sonar SF2 is mounted near the right edge of the front bumper V12 in the vehicle width direction so as to emit sonar search waves Wsp to the right front of the vehicle. The first front sonar SF1 and the second front sonar SF2 are arranged symmetrically across the vehicle width center line LC1. The front corner sonar detection range Rsc, which is the detection range of the first front sonar SF1 and the second front sonar SF2, is set to a detection distance of, for example, approximately 60 cm. The "detection distance" refers to the maximum distance that can be measured from the sonar sensor 31.

[0026] The third front sonar SF3 and the fourth front sonar SF4 are arranged in the vehicle width direction at a position closer to the center of the front bumper V12 in the vehicle width direction. The third front sonar SF3 is arranged between the first front sonar SF1 and the vehicle width center line LC1 in the vehicle width direction so as to emit sonar search waves Wsp approximately ahead of the vehicle. The fourth front sonar SF4 is arranged between the second front sonar SF2 and the vehicle width center line LC1 in the vehicle width direction so as to emit sonar search waves Wsp approximately ahead of the vehicle. The third front sonar SF3 and the fourth front sonar SF4 are arranged symmetrically across the vehicle width center line LC1. The front sonar detection range Rsf, which is the detection range of the third front sonar SF3 and the fourth front sonar SF4, is set to a detection distance of, for example, approximately 1 meter.

[0027] The first front sonar SF1 and the third front sonar SF3 mounted on the left side of the vehicle body V1 are positioned at different positions in the horizontal direction. Furthermore, the first front sonar SF1 and the third front sonar SF3, which are adjacent to each other in the vehicle width direction, are positioned so that the reflected waves of the sonar search waves Wsp emitted by one sonar sonar SF1 and the reflected waves of the target B can be received by the other sonar SF1. That is, the first front sonar SF1 is positioned so that it can receive both the direct waves corresponding to the sonar search waves Wsp it emitted and the indirect waves corresponding to the sonar search waves Wsp emitted by the third front sonar SF3. Similarly, the third front sonar SF3 is positioned so that it can receive both the direct waves corresponding to the sonar search waves Wsp it emitted and the indirect waves corresponding to the sonar search waves Wsp emitted by the first front sonar SF1.

[0028] Similarly, the third front sonar SF3 and the fourth front sonar SF4, which are mounted toward the center of the vehicle body V1 in the vehicle width direction, are positioned at different positions in the horizontal direction. Furthermore, the third front sonar SF3 and the fourth front sonar SF4, which are adjacent to each other in the vehicle width direction, are positioned in a mutually relativistic manner such that the sonar search wave Wsp emitted by one sonar sonar sensor can be received by the other sonar sensor.

[0029] Similarly, the second front sonar SF2 and the fourth front sonar SF4 mounted on the right side of the vehicle body V1 are positioned at different positions in the horizontal direction. The second front sonar SF2 and the fourth front sonar SF4 are adjacent to each other in the vehicle width direction, and are positioned in a mutually relativistic manner such that the sonar search wave Wsp emitted by one sonar sensor can be received by the other sensor as a reflected wave from the target B.

[0030] The first rear sonar SR1 is mounted near the left edge of the rear bumper V14 in the vehicle width direction so as to emit sonar search waves Wsp to the left rear of the vehicle. The second rear sonar SR2 is mounted near the right edge of the rear bumper V14 in the vehicle width direction so as to emit sonar search waves Wsp to the right rear of the vehicle. The first rear sonar SR1 and the second rear sonar SR2 are arranged symmetrically across the vehicle width center line LC1. The rear corner sonar detection range Rsd, which is the detection range of the first rear sonar SR1 and the second rear sonar SR2, is set to a detection distance of, for example, approximately 60 cm.

[0031] The third rear sonar SR3 and the fourth rear sonar SR4 are arranged in the vehicle width direction at a position closer to the center of the rear bumper V14 in the vehicle width direction. The third rear sonar SR3 is arranged between the first rear sonar SR1 and the vehicle width center line LC1 in the vehicle width direction so as to emit sonar search waves Wsp approximately rearward of the vehicle. The fourth rear sonar SR4 is arranged between the second rear sonar SR2 and the vehicle width center line LC1 in the vehicle width direction so as to emit sonar search waves Wsp approximately rearward of the vehicle. The third rear sonar SR3 and the fourth rear sonar SR4 are arranged symmetrically across the vehicle width center line LC1. The rear sonar detection range Rsr, which is the detection range of the third rear sonar SR3 and the fourth rear sonar SR4, is set to a detection distance of, for example, approximately 1.5 m.

[0032] The first rear sonar SR1 and the third rear sonar SR3 mounted on the left side of the vehicle body V1 are positioned at different positions in the horizontal direction. Furthermore, the first rear sonar SR1 and the third rear sonar SR3, which are adjacent to each other in the vehicle width direction, are positioned so that the reflected waves of the sonar search waves Wsp emitted by one sonar sonar SR1 from a target B can be received by the other sonar sonar SR1. That is, the first rear sonar SR1 is positioned so that it can receive both the direct waves corresponding to the sonar search waves Wsp it emitted and the indirect waves corresponding to the sonar search waves Wsp emitted by the third rear sonar SR3. Similarly, the third rear sonar SR3 is positioned so that it can receive both the direct waves corresponding to the sonar search waves Wsp it emitted and the indirect waves corresponding to the sonar search waves Wsp emitted by the first rear sonar SR1.

[0033] Similarly, the third rear sonar SR3 and the fourth rear sonar SR4, which are mounted toward the center of the vehicle body V1 in the vehicle width direction, are positioned at different positions in the horizontal direction. Furthermore, the third rear sonar SR3 and the fourth rear sonar SR4, which are adjacent to each other in the vehicle width direction, are positioned in a mutually relativistic manner such that the sonar search wave Wsp emitted by one sonar sonar can be received by the other sonar sonar.

[0034] Similarly, the second rear sonar SR2 and the fourth rear sonar SR4 mounted on the right side of the vehicle body V1 are positioned at different positions in the horizontal direction. Furthermore, the second rear sonar SR2 and the fourth rear sonar SR4 are adjacent to each other in the vehicle width direction, and are positioned in a mutually relativistic manner such that the sonar search wave Wsp emitted by one sonar sonar can be received by the other sonar sonar.

[0035] The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are configured to emit sonar search waves Wsp from the sides of the vehicle body V1 toward the sides of the vehicle. The first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4 are each configured to be able to receive only direct waves. The side sonar detection range Rss, which is the detection range of the first side sonar SS1, the second side sonar SS2, the third side sonar SS3, and the fourth side sonar SS4, is set to a detection distance of, for example, approximately 2 to 3 meters.

[0036] The first side sonar SS1 is disposed between the left door mirror V17 and the first front sonar SF1 in the longitudinal direction so as to emit sonar search waves Wsp to the left of the vehicle. The second side sonar SS2 is disposed between the right door mirror V17 and the second front sonar SF2 in the longitudinal direction so as to emit sonar search waves Wsp to the right of the vehicle. The first side sonar SS1 and the second side sonar SS2 are disposed symmetrically across the vehicle width center line LC1. The first side sonar SS1 and the second side sonar SS2 may be mounted on the vehicle body panel V15, or may be mounted on both ends of the front bumper V12 in the vehicle width direction, extending rearward along the vehicle length.

[0037] The third side sonar SS3 is disposed between the left rear door panel V16 and the first rear sonar SR1 in the longitudinal direction so as to emit sonar search waves Wsp to the left of the vehicle. The fourth side sonar SS4 is disposed between the right rear door panel V16 and the second rear sonar SR2 in the longitudinal direction so as to emit sonar search waves Wsp to the right of the vehicle. The third side sonar SS3 and the fourth side sonar SS4 are disposed symmetrically across the vehicle width center line LC1. The third side sonar SS3 and the fourth side sonar SS4 may be attached to the vehicle body panel V15 or to both ends of the rear bumper V14 in the vehicle width direction, extending forward along the vehicle length.

[0038] FIG. 6 shows an example of detecting a parking space PS for perpendicular parking on the left side of the host vehicle using a first side sonar sensor SS1, which is one of the multiple sonar sensors 31. In the figure, sonar detection points Psr are points on the surface of the target B that are estimated to have reflected sonar search waves Wsp emitted from the sonar sensor 31, and may also be referred to as "distance measurement points." As shown in FIG. 6 , the parking space PS can be detected based on the distribution of sonar detection points Psr acquired based on sonar reflection waves Wsr from targets B, which are parked vehicles lined up along the direction of travel of the host vehicle while traveling straight. Specifically, for example, in the case of perpendicular parking shown in FIG. 6 , the parking space PS can be detected at a location where the width of the section where the array of sonar detection points Psr lined up along the direction of travel of the host vehicle is interrupted is equal to or greater than a predetermined width that exceeds the overall width of the host vehicle.

[0039] (Radar Sensor) Referring again to FIGS. 1 and 2 , the radar sensor 32 is configured to detect the target B by transmitting and receiving radar waves, which are millimeter waves or submillimeter waves. In this embodiment, the radar sensor 32 is mounted in the center of the front portion V11 of the vehicle body V1 in the vehicle width direction. As shown in FIG. 7 , the radar sensor 32 is a so-called long-range radar with a detection range of approximately 10 to 250 m and a radar scan angle range θr1 centered on the vehicle's forward direction Df, and has a phased array or beamforming configuration. The vehicle's forward direction Df is the direction in which an imaginary straight line extending from the vehicle width centerline LC1 extends forward of the vehicle, and corresponds to the traveling direction of the vehicle when traveling forward, i.e., when the shift position is other than reverse. The vehicle's forward direction Df coincides with the vehicle's traveling direction when traveling straight, and is a tangent direction to the traveling trajectory when traveling around a curve. The radar sensor 32 is configured to detect the target B within the radar detection range Rg1 by scanning and emitting a radar search wave Wrp within the radar scan angle range θr1 and receiving radar reflected waves Wrr that are waves reflected by the target B. The radar detection range Rg1 is a sector-shaped range with the radar scan angle range θr1 as its central angle.

[0040] Specifically, the radar sensor 32 is an FMCW radar device equipped with a so-called array antenna, and is configured to detect the distance to the target B, the azimuth angle θ, and the relative speed of the target B with respect to the host vehicle based on the frequency difference and phase difference of the transmitted and received millimeter waves. The azimuth angle θ is the angle between an imaginary line extending forward from the vehicle's widthwise centerline LC1, which forms the center line of the radar detection range Rg1, and an imaginary line connecting the radar sensor 32 and the target B. The relative speed is the difference between the target moving speed vb and the host vehicle speed vm, which is the traveling speed of the host vehicle. More specifically, the radar sensor 32 transmits radar search waves Wrp generated based on a transmission signal with a predetermined modulation frequency, receives radar reflected waves Wrr reflected by the target B, and acquires a received signal indicating the frequency characteristics of the radar reflected waves Wrr. The radar sensor 32 generates a beat signal, which is the deviation between the frequency of the transmission signal and the frequency of the received signal. The radar sensor 32 then performs a fast Fourier transform on the beat signal to obtain a frequency spectrum, analyzes the frequency spectrum to obtain a beat frequency, and calculates the distance and relative velocity based on the beat frequency. In addition to the long-range radar sensor 32, so-called medium-range and short-range radar sensors 32 may also be provided in the in-vehicle system 1. The medium-range radar sensor has a detection distance of approximately 1 to 100 m, and the short-range radar sensor has a detection distance of approximately 15 cm to 30 m.

[0041] (Laser Radar Sensor) As shown in FIG. 8 , the laser radar sensor 33 is configured to detect a target B by emitting detection light Lp, which is a laser beam in the infrared band, to the outside of the vehicle and receiving reflected light Lr from the target B. The laser radar sensor 33 may also be referred to as a LIDAR. LIDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging. In the onboard system 1 according to this embodiment, a long-range laser radar sensor 33 configured as a so-called scanning LIDAR is mounted in the center of the front portion V11 of the vehicle body V1 in the vehicle width direction, as shown in FIG. 1 . The laser radar sensor 33 is configured to detect a target B within a LIDAR detection range Rg2 by scanning the detection light Lp horizontally within a LIDAR scan angle range θr2 centered in the forward direction Df of the vehicle. The lidar detection range Rg2 is a sector-shaped range with a radius of 200 m or more and a central angle defined by the lidar scanning angle range θr2.

[0042] FIG. 9 shows a schematic configuration of the laser radar sensor 33. As shown in FIG. 9, the laser radar sensor 33 includes a light-emitting unit 331, a scanning unit 332, and a light-receiving unit 333. The light-emitting unit 331 is configured to generate detection light Lp. The scanning unit 332 has a configuration as a so-called MEMS mirror. MEMS stands for Micro Electro Mechanical Systems. That is, the scanning unit 332 is configured to scan the detection light Lp along the horizontal scanning direction Ds and the vertical scanning direction Dh by electrically controlling, using a MEMS mechanism, the orientation of a reflecting mirror that reflects the detection light Lp emitted from the light-emitting unit 331.

[0043] The light receiving unit 333 includes a light receiving sensor 334, which is a two-dimensional image sensor. The light receiving sensor 334 has a plurality of light receiving elements 335 two-dimensionally arranged in the vertical and horizontal directions, i.e., in directions corresponding to the horizontal scanning direction Ds and the vertical scanning direction Dh. The light receiving elements 335 are configured as photodetection elements such as APDs, SPADs, and photodiodes. APD stands for avalanche photodiode. SPAD stands for single photon avalanche diode. The laser radar sensor 33 generates detection point cloud data based on the reflected light Lr received by the light receiving unit 333 and detects the target B based on the detection point cloud data. The detection point cloud data is a two-dimensional array of a plurality of lidar detection points Prr arranged in an image-like manner in two-dimensional directions corresponding to the horizontal scanning direction Ds and the vertical scanning direction Dh in FIG. 9 . The LIDAR detection point Prr is a point on the target B that is estimated to have reflected the detection light Lp corresponding to the reflected light Lr detected by a certain light receiving element 335, and includes position information and ranging information in the above-mentioned two-dimensional array. Therefore, the detection point cloud data may also be referred to as "ranging point cloud data." In this way, the laser radar sensor 33 acquires detection point cloud data in which multiple LIDAR detection points Prr are arranged two-dimensionally like an image. Therefore, the laser radar sensor 33 can be considered a type of image sensor.

[0044] (Camera) Referring again to Figures 1 and 2, the camera 34 as an image sensor is mounted on the vehicle so as to capture images of the surroundings of the vehicle and move in accordance with the movement of the vehicle. That is, the camera 34 is configured to generate image information corresponding to the captured images of the surroundings of the vehicle. The image information may also be referred to as image data. The camera 34 is a so-called digital camera device and is equipped with an image sensor such as a CCD or CMOS. CCD stands for Charge Coupled Device. CMOS stands for Complementary Metal Oxide Semiconductor.

[0045] In this embodiment, the system-equipped vehicle V is equipped with multiple cameras 34, namely, a front camera CF, a rear camera CB, a left camera CL, and a right camera CR. When not specifying which of the front camera CF, rear camera CB, left camera CL, and right camera CR is being used, the singular expression "camera 34" or the expression "multiple cameras 34" may be used hereinafter.

[0046] The front camera CF is disposed in the upper part of the passenger compartment V2 near the upper end of the front windshield V18, at a central position in the vehicle width direction, i.e., on the vehicle width centerline LC1 in a plan view, so as to acquire image information corresponding to an image of the area ahead of the vehicle. The front camera CF may also be mounted on the front part V11 of the vehicle body V1. The rear camera CB is disposed in the central position in the vehicle width direction on the rear part V13 of the vehicle body V1 so as to acquire image information corresponding to an image of the area behind the vehicle. The left camera CL is mounted on the left door mirror V17 so as to acquire image information corresponding to an image of the left side of the vehicle. The right camera CR is mounted on the right door mirror V17 so as to acquire image information corresponding to an image of the right side of the vehicle.

[0047] Based on an image captured by the camera 34, it is possible to recognize the target B, i.e., to determine at least the presence and type of the target B. FIG. 10 shows, as an example, an overview of an operation for recognizing the target B included in the foreground, based on an image of the foreground including the road Rd ahead of the vehicle captured by the front camera CF. Specifically, as shown in FIG. 10 , a detection area Aw is set in the captured image. The detection area Aw is all or part of the captured image area, i.e., the angle of view area of ​​the camera 34. A feature point image Gp can be acquired based on pixel feature parameters in the image within the detection area Aw. The pixel feature parameters are feature parameters of the pixels constituting the image, such as brightness, contrast, color, etc. "Brightness" may also be referred to as "lightness." "Color" may also be referred to as "saturation." The feature point image Gp is ​​an image in which feature points Pt are extracted based on the difference in pixel feature parameters between adjacent pixels, the gradient of change, etc., and arranged in an image. The feature points Pt characterize the shape of the target B in the captured image. In other words, the feature points Pt are characteristic points, i.e., pixels, in the image within the detection area Aw. Then, by pattern matching a feature point group Pg, which is a collection of the feature points Pt in the feature point image Gp, with a pre-stored pattern, it is possible to determine the type of target B corresponding to the feature point group Pg.

[0048] Note that methods for extracting feature points Pt are well known at the time of filing this application. Specifically, well-known methods such as a Sobel filter, a Laplacian filter, or a Canny method can be used as methods for extracting feature points Pt. Therefore, in this specification, detailed explanations of methods for extracting feature points Pt will be omitted. Furthermore, the "extraction" of feature points Pt can also be expressed as "detection."

[0049] Furthermore, it is possible to estimate the relative position or distance of the recognized target B with respect to the vehicle based on an image captured by the camera 34. FIG. 11 shows an overview of calculating an estimated point Pb, which is an estimation result of the three-dimensional position of a feature point Pt corresponding to the target B, using a technique known as monocular moving stereo or SFM. SFM stands for Structure from Motion. Note that the present disclosure is not limited to such monocular moving stereo. In other words, the camera 34 may be a so-called compound eye stereo camera.

[0050] 11 , it is assumed that the camera 34 is moving in the vehicle movement direction Dv as the vehicle travels, and the position of the camera 34 at time t1 is defined as a first camera position Pc1, and the position of the camera 34 at time t2 is defined as a second camera position Pc2. The first captured image A1 is defined as an image captured by the camera 34 at the first camera position Pc1. The second captured image A2 is defined as an image captured by the camera 34 at the second camera position Pc2.

[0051] The first feature point Pt1 is a feature point Pt extracted from the first captured image A1. The second feature point Pt2 is a feature point Pt extracted from the second captured image A2 that is estimated to correspond to the first feature point Pt1 at time t1. That is, the second feature point Pt2 corresponds to a point to which the point on the target B corresponding to the first feature point Pt1 at time t1 is estimated to have moved (t2-t1) seconds later. It is possible to determine that the first feature point Pt1 and the second feature point Pt2 are identical, i.e., corresponding, to each other using a publicly known or well-known method such as optical flow. The intersection of a first line L1 passing through the first camera position Pc1 and the first feature point Pt1 and a second line L2 passing through the second camera position Pc2 and the second feature point Pt2 is estimated as an estimated point Pb. The estimated point Pb is a point on the target B that corresponds to the first feature point Pt1 and the second feature point Pt2 in a three-dimensional coordinate system based on the host vehicle. If the estimated point Pb is a stationary point, it satisfies the epipolar constraint, which is a condition defined by epipolar geometry, and requires that the first camera position Pc1, the second camera position Pc2, and the estimated point Pb exist on the same plane Π at both times t1 and t2.

[0052] 1 and 2 again, the operation sensor 35 is configured to generate outputs corresponding to various quantities related to the driving operation state of the vehicle by the driver D. The "various quantities related to the driving operation state" are, for example, the accelerator operation amount, the brake operation amount, the shift position, the steering angle, etc. That is, the operation sensor 35 is a general term for well-known sensors such as an accelerator pedal sensor, a brake pedal sensor, a shift position sensor, a steering angle sensor, and a steering torque sensor, for the sake of simplicity of illustration and explanation. Furthermore, the operation sensor 35 may be a so-called steering wheel sensor that detects whether the driver D is gripping the steering wheel V24.

[0053] The behavior sensor 36 is configured to generate outputs corresponding to various quantities related to the driving behavior of the vehicle. The "various quantities related to the driving behavior" include physical quantities related to the behavior or motion state of the vehicle, such as vehicle speed, yaw rate, longitudinal acceleration, lateral acceleration, etc. In other words, the behavior sensor 36 is a general term for well-known sensors such as a vehicle speed sensor, yaw rate sensor, and acceleration sensor, for the sake of simplicity of illustration and explanation.

[0054] (Driver State Monitor) Referring to FIGS. 12 and 13 in addition to FIGS. 1 and 2, the driver state monitor 37 is configured to sequentially detect the driver state based on captured images of the driver D. The driver state includes, for example, whether or not the driver is awake. Specifically, the driver state monitor 37 includes a driver monitor camera installed in the vehicle interior V2 so as to capture an image of the face D2 from the front by including within its angle of view at least the head D1 of the driver D seated in the driver's seat V23 in a posture that allows the driver to appropriately drive the vehicle. The driver monitor camera is configured as a near-infrared camera. The driver state monitor 37 detects the driver state by performing image recognition processing on the images captured by the driver monitor camera.

[0055] The detection contents using the driver state monitor 37 include, for example, the orientation of the driver D's face D2, the degree of opening of the eyes D3, the position of the pupils D4, etc., as shown in FIGS. 12 and 13 . The orientation of the face D2 includes a yaw angle θy and a pitch angle θp. The yaw angle θy represents the rotation angle of the driver D's face D2 around a vertical axis Dx1 extending in the up-down direction. The yaw angle θy is 0° when the face D2 faces forward. The yaw angle θy is a positive value when the face D2 faces leftward and a negative value when the face D2 faces rightward. The pitch angle θp represents the rotation angle of the face D2 around a horizontal axis Dx2 extending in the left-right direction. The pitch angle θp is 0° when the face D2 faces horizontally. The pitch angle θp is a positive value when the face D2 faces downward and a negative value when the face D2 faces upward. FIG. 12 shows an example of a captured image in which the driver D is awake. Fig. 13 shows an example of a captured image when the driver D is in a waking state. As shown in Fig. 12 and Fig. 13, it is possible to detect the driver's state, including whether the driver is awake or not, the line of sight D5, etc., based on the direction of the face D2, the degree of opening of the eyes D3, the position of the pupils D4, etc.

[0056] 1 and 2 , the operation switches 38 are various switches operated by the driver D when driving the vehicle, and are provided in various locations within the vehicle compartment V2. The operation switches 38 correspond to switches that are not subject to detection of the operation amount or operation state by the operation sensor 35, and include at least an ignition switch 381, a blinker switch 382, ​​and an AD / ADAS switch 383. The ignition switch 381 is a switch for turning on and off the activation of the system-equipped vehicle V, i.e., the in-vehicle system 1, and may also be referred to as a start switch or power switch. The blinker switch 382 is a switch for detecting the operation state of a blinker lever. The AD / ADAS switch 383 is a switch for the driver D to input instructions regarding start, end, level setting, function selection, etc., related to driving assistance or autonomous driving.

[0057] (Locator) The locator 39 is configured to acquire highly accurate position information of the vehicle. Specifically, the locator 39 is a so-called composite positioning system and includes a GNSS receiver 391, an inertial acquisition unit 392, and a locator ECU 393. GNSS stands for Global Navigation Satellite System. "Highly accurate position information" refers to position information having a position accuracy of a level usable for advanced driving assistance or automated driving, for example, SAE Level 2 or higher, specifically, an error of less than 10 cm. Note that the locator 39 can be, for example, a commercially available system such as the "POSLV" positioning and orientation system for land vehicles manufactured by Applanix.

[0058] The GNSS receiver 391 is configured to be able to receive positioning signals transmitted from at least one positioning satellite, i.e., artificial satellite. In this embodiment, the GNSS receiver 391 is configured to be able to receive positioning signals from positioning satellites in at least one of satellite positioning systems such as GPS, QZSS, GLONASS, Galileo, IRNSS, and the Beidou satellite navigation system. GPS is an abbreviation for Global Positioning System. QZSS is an abbreviation for Quasi-Zenith Satellite System. GLONASS is an abbreviation for Global Navigation Satellite System. IRNSS is an abbreviation for Indian Regional Navigation Satellite System.

[0059] The inertia acquisition unit 392 is configured to acquire the acceleration and angular velocity acting on the host vehicle. In this embodiment, the inertia acquisition unit 392 is provided as a three-axis gyro sensor and a three-axis acceleration sensor built into a box-shaped housing of the locator 39. The locator ECU 393 has a configuration as a so-called on-board microcomputer including a CPU, ROM, RAM, an input / output interface, etc. The locator ECU 393 is configured to sequentially determine the position, direction, etc. of the host vehicle based on the positioning signal received by the GNSS receiver 391, the acceleration and angular velocity acquired by the inertia acquisition unit 392, etc.

[0060] (In-Vehicle Communication Device) The in-vehicle communication device 4 is an in-vehicle communication module also referred to as DCM, and is capable of communicating information with base stations around the vehicle via wireless communication compliant with communication standards such as LTE or 5G. DCM stands for Data Communication Module. LTE stands for Long Term Evolution. 5G stands for 5th Generation. Specifically, for example, the in-vehicle communication device 4 acquires traffic information such as congestion information from a probe server on the cloud and / or a predetermined database. The "congestion information" includes the location and length of a congestion section. Specifically, the congestion information includes the beginning and end positions of the congestion, the approximate congestion distance, the approximate congestion time, etc. The traffic information is also referred to as "road traffic information." The in-vehicle communication device 4 also acquires the latest HD map information from the probe server and stores it in the HD map 5.

[0061] (HD Map) The HD map 5 is primarily configured with a nonvolatile rewritable memory so that it can rewrite HD map information and retain the stored contents even when the power is turned off. The HD map information may also be referred to as high-precision map data. The HD map information includes map information with higher precision than the map information in the SD map 601 used in the navigation device 6, which has a position error of about several meters. SD stands for Standard. Specifically, the HD map 5 stores information that can be used for advanced driving assistance or automated driving, such as three-dimensional road shape information, lane number information, and traffic regulation information, in accordance with predetermined standards such as the ADASIS standard. ADASIS stands for Advanced Driver Assistance Systems Interface Specification.

[0062] (Navigation Device) The navigation device 6 is configured to calculate a planned driving route from the current position of the vehicle to a destination. In this embodiment, the navigation device 6 is configured to calculate a planned driving route based on the destination set by the driver D or the like, map information stored in the HD map 5 or the SD map 601, position information of the vehicle acquired from the locator 39, and the like. The navigation device 6 is also configured to be able to provide various information including the route information resulting from the calculation to each component, such as the driving ECU 2 and the HMI device 7, via the in-vehicle communication line 10. In other words, the navigation device 6 is configured to display a navigation screen on the HMI device 7 for displaying a map, a route, and the like.

[0063] (HMI Device) The HMI device 7 is a so-called vehicle HMI and is configured to realize information transmission between the vehicle and occupants including the driver D. Specifically, the HMI device 7 is configured to at least visually present various information related to the vehicle to the occupants and to accept input operations by the occupants corresponding to the presented information. The various pieces of information to be presented include, for example, various guidance messages, input operation instructions, input operation content notifications, warnings, etc. Typically, the HMI device 7 is configured as a so-called "dashboard HMI" mainly composed of devices attached to the dashboard V21 and the steering wheel V24. However, some of the input / output devices included in the HMI device 7 may be provided in locations other than the dashboard V21, such as the ceiling of the passenger compartment V2 or a center console between the driver's seat V23 and the adjacent seat V22, i.e., the passenger seat.

[0064] The HMI device 7 includes an HCU 701, a meter panel 702, a main display device 703, a head-up display 704, a speaker 705, and an operation device 706. HCU stands for HMI Control Unit. The HCU 701 has a configuration as a so-called in-vehicle microcomputer including a CPU, ROM, RAM, an input / output interface, etc., and is configured to control the display output and audio output by the HMI device 7. In other words, the HCU 701 is configured to control the operations of the meter panel 702, the main display device 703, the head-up display 704, the speaker 705, etc.

[0065] The meter panel 702 is provided on the dashboard V21 in front of the driver's seat V23. The meter panel 702 is configured to display various information such as the vehicle speed, coolant temperature, and remaining fuel level of the vehicle, as well as the date and time, outside air temperature, mileage, and radio stations.

[0066] The main display device 703, also referred to as a CID device, is located approximately in the center of the dashboard V21 in the vehicle width direction, with its display screen visible to the occupants. CID stands for Center Information Display. The main display device 703 is capable of displaying a navigation screen for displaying maps and routes by the navigation device 6. The main display device 703 is also capable of displaying information and content different from the navigation screen. Specifically, the main display device 703 is configured to execute a display for setting operations related to driving modes, such as "Comfort," "Normal," "Sport," and "Circuit." The main display device 703 is also configured to execute a display related to a second task that the driver D can use during autonomous driving. The second task is a task other than driving operations that the driver D performs, and includes, for example, reading, operating a mobile communication terminal, watching video content, etc. "Video content" includes, for example, movies, concert footage, music videos, television broadcasts, etc. The second task is also referred to as a "non-driving task" or a "secondary activity."

[0067] 1 and 2 , and also with reference to FIGS. 14 and 15 , the head-up display 704 is configured to display a display image M including letters, figures, and / or symbols within the forward field of vision of the driver D. In this embodiment, the head-up display 704 is configured to use AR technology to virtually display the display image M in front of the driver D, thereby superimposing the display image M on a foreground including a road surface FR ahead of the vehicle. AR stands for Augmented Reality. "Superimposed display" refers to displaying related information in positional correspondence with a target of interest included in the foreground. For example, the related information is displayed so as to overlap the target of interest or in the vicinity of the target of interest. The "target of interest" refers to a target object that requires attention or attention while driving, such as a road marking, a road sign, a vehicle ahead, or a pedestrian. "Road markings" may also be referred to as "road markings." For example, various displays such as a route display, a direction of travel display, and traffic information display on the road surface FR ahead as the focus of attention correspond to "superimposed display."

[0068] As shown in FIG. 14 , the head-up display 704 is configured so that the depression angle AD is a positive value equal to or greater than 0 degrees. In FIG. 14 , the depression angle AD is the angle between a virtual horizontal line passing through the viewpoint EP of the driver D and a virtual line connecting the viewpoint EP and the upper end of the projection range AP. The depression angle AD is a positive value when looking down at the upper end of the projection range AP from the viewpoint EP, and a negative value when looking up. The head-up display 704 is configured so that the horizontal angle of view is larger than the vertical angle of view AV, so that the projection range AP is a horizontally elongated, approximately rectangular shape as shown in FIG. 15 . The vertical angle of view AV is the angle between the virtual line connecting the viewpoint EP and the upper end of the projection range AP and the virtual line connecting the viewpoint EP and the lower end of the projection range AP in FIG. That is, the vertical angle of view AV is the angle range in the vertical direction in which the display image M can be viewed from the viewpoint EP, and may also be referred to as the viewing angle in the vertical direction.

[0069] As shown in FIG. 14 , the head-up display 704 includes a projector 741 and a magnifying optical system 742. The projector 741 is configured to generate display image light LV based on a display image signal generated by the HCU 701 and emit the display image light LV toward the magnifying optical system 742. The magnifying optical system 742 includes multiple optical elements, including a concave mirror, and an actuator that controls the alignment of these elements. The magnifying optical system 742 is configured to adjust the projection state of the display image light LV onto the front windshield V18 by controlling the alignment using the actuator in accordance with the viewpoint EP detected by the driver state monitor 37. The head-up display 704 projects the display image light LV onto a projection range AP on the front windshield V18 and allows the driver D to view the reflected light, thereby displaying a display image M.

[0070] FIG. 15 shows an example of a display state of the head-up display 704. The head-up display 704 is capable of displaying an information display M1 and a graphic display M2. The information display M1 is display content that displays various information such as the maximum speed on the road Rd on which the vehicle is traveling, the current speed of the vehicle, the distance to the destination, the estimated time of arrival at the destination, the names of buildings, and the names of intersections. The information display M1 is typically displayed using characters, road sign-like figures, or symbols. The graphic display M2 is display content using graphics such as lines and arrows, and is used to display, for example, information regarding the vehicle's traveling direction and lane selection.

[0071] The head-up display 704 can also display superimposed content and non-superimposed content. "Superimposed content" refers to display image content that is associated with or related to a specific target of interest in the foreground and is superimposed on the target of interest. In contrast, "non-superimposed content" refers to display image content that is not associated with a specific target of interest in the foreground and is not superimposed on the specific target of interest. In the display example of FIG. 15 , a linear graphic display M2 is displayed as superimposed content on the lane in which the vehicle is currently traveling, and an information display M1 indicating the maximum speed is displayed as non-superimposed content at an appropriate position that does not overlap with the graphic display M2. In this way, the HMI device 7 is configured as a notification unit that notifies information to an occupant of the vehicle, such as the driver D.

[0072] 1 and 2 again, speaker 705 is provided to output audio corresponding to the display contents on meter panel 702, main display device 703, and head-up display 704. Note that speaker 705 can also output audio that does not correspond to the display contents on meter panel 702, main display device 703, and head-up display 704, such as music or radio audio.

[0073] The operation device 706 is an input device that is not included in the operation switch 38 and is not subject to detection of the operation amount or operation state by the operation sensor 35. Specifically, for example, switches provided around the display screen of the main display device 703 and a transparent touch panel provided to cover such a display screen correspond to the operation device 706. Also, for example, switches provided on the spokes of the handle V24 of the steering wheel structure correspond to the operation device 706. Furthermore, a pointing device such as a touch panel provided on the center console corresponds to the operation device 706.

[0074] (Lighting Device) The lighting device 8 includes a body ECU 801, headlights 802, and blinkers 803. The body ECU 801 is configured as a so-called on-board microcomputer, including a CPU, ROM, RAM, an input / output interface, etc. The body ECU 801 is configured to control the lighting state of the headlights 802 and blinkers 803 based on information input from the driving ECU 2 and the driving information input unit 3.

[0075] (Motion Control Device) The motion control device 9 is configured to control the motion, i.e., driving behavior, of the vehicle based on information input from the driving ECU 2 and the driving information input unit 3. Specifically, the motion control device 9 includes a drive device 91, a shift device 92, a braking device 93, and a steering device 94.

[0076] The drive device 91 includes a drive system ECU 911 and a drive mechanism 912. The drive system ECU 911 is configured as a so-called on-board microcomputer including a CPU, ROM, RAM, an input / output interface, etc., and is configured to control the operation of the drive mechanism 912. The drive mechanism 912, which generates the driving force for running the system-equipped vehicle V, includes an engine and / or an electric motor. In other words, the system-equipped vehicle V may be any of a gasoline engine vehicle, a diesel engine vehicle, a biofuel vehicle, a hydrogen engine vehicle, a hybrid vehicle, a BEV, a fuel cell vehicle, etc. BEV is an abbreviation for Battery Electric Vehicle.

[0077] The shift device 92 includes a shift ECU 921 and a shift mechanism 922. The shift ECU 921 is configured as a so-called on-board microcomputer including a CPU, ROM, RAM, an input / output interface, etc., and is configured to control the operation of the shift mechanism 922. The shift mechanism 922 is provided between the wheels V3 that function as drive wheels and the drive device 91, and is configured to switch the shift position, i.e., switch between forward and reverse driving, and change the gear ratio during forward driving. The shift device 92 may also have a so-called shift-by-wire configuration.

[0078] The braking device 93 includes a brake ECU 931 and a brake mechanism 932. The brake ECU 931 has a configuration as a so-called on-board microcomputer including a CPU, ROM, RAM, an input / output interface, etc., and is configured to control the operation of the brake mechanism 932. The brake mechanism 932 that generates braking force in the system-equipped vehicle V may include a regenerative brake mechanism in addition to a friction brake mechanism. The braking device 93 may also have a so-called brake-by-wire configuration.

[0079] The steering device 94 includes a steering ECU 941 and a steering mechanism 942. The steering ECU 941 has a configuration as a so-called on-board microcomputer including a CPU, ROM, RAM, an input / output interface, etc., and is configured to control the operation of the steering mechanism 942. The steering mechanism 942 is configured to be able to change the direction of the left front wheel V3a and the right front wheel V3b, which are steered wheels. Note that the steering mechanism 942 may also be able to change the direction of the left rear wheel V3c and the right rear wheel V3d. In other words, the host vehicle may be a so-called four-wheel steering vehicle. The steering device 94 may also have a so-called steer-by-wire configuration.

[0080] (Driving control based on target recognition results) Figures 16 and 17 show an example of a functional configuration realized by the driving ECU 2 when the processor 21 executes program instructions stored in the memory 22. Below, an overview of driving control of the host vehicle executed by the driving ECU 2 based on the recognition results of targets B around the host vehicle using the ADAS sensor will be described. Hereinafter, in this specification, "recognition" and "detection" will be defined separately as follows. "Recognition" is a concept that includes "detection," "identification," and "recognition." "Detection" refers to finding a target B based on images or point cloud data. A "detected" target B is merely determined to exist, and its shape, meaning, etc. are not determined. "Identification" refers to classifying the shape, meaning, etc. of the detected target B. An "identified" target B is not only determined to exist, but also determined to be what it is, for example, whether it is a human, a vehicle, or a building. "Recognition" refers to determining that the detected and identified object B is something that should be taken into consideration in driving control of the vehicle, such as a pedestrian, an oncoming vehicle, etc. Note that "detection" includes "detection."

[0081] 16 , the driving ECU 2 has a recognition unit 2001, an operation determination unit 2002, and a control output unit 2003 as functional components realized by the processor 21. The recognition unit 2001 performs a recognition operation of a target B around the vehicle based on information input from the surrounding monitoring sensor, the operation sensor 35, the behavior sensor 36, etc. The operation determination unit 2002 determines the content of a control operation currently required for the vehicle based on the recognition result of the target B by the recognition unit 2001 and information input from the operation sensor 35, the behavior sensor 36, etc. Examples of the content of such control operation include a collision avoidance operation, an emergency stop operation, and an alert operation to the driver D. The control output unit 2003 generates a control output corresponding to the content of the control operation determined by the operation determination unit 2002 and outputs the control output to each unit in the in-vehicle system 1. Examples of such control output include a steering amount, a braking amount, and a warning message code.

[0082] 17 , the recognition unit 2001 has an input information acquisition function 2101, an input information processing function 2102, a target object recognition function 2103, a current lane recognition function 2104, an intersection recognition function 2105, and a surrounding environment recognition function 2106. The input information acquisition function 2101 holds a predetermined amount of information input to the recognition unit 2001 in chronological order. The input information processing function 2102 performs various processes such as noise removal and coordinate conversion on the information acquired, i.e., held, by the input information acquisition function 2101. The target object recognition function 2103 performs a recognition operation for target object B based on the information processed by the input information processing function 2102. The target recognition function 2103 has a dividing line recognition function 2131, a road marking recognition function 2132, a roadside structure recognition function 2133, a traffic light recognition function 2134, a road sign recognition function 2135, a lane recognition function 2136, a pedestrian recognition function 2137, a surrounding vehicle recognition function 2138, and an obstacle recognition function 2139.

[0083] Examples of objects to be recognized by each recognition function in the recognition unit 2001 are shown in FIGS. 18 and 19 . Referring to FIG. 18 , the recognition objects B are broadly categorized as traffic-related three-dimensional objects B1, other vehicles B2, general three-dimensional objects B3, and road markings B4. Referring to FIG. 19 , the recognition objects B include parking frame lines B5, wheel chocks B6, and parked vehicles B7, which are related to the detection of parking spaces PS and parking assistance. The traffic-related three-dimensional objects B1 are three-dimensional objects used for road traffic, such as traffic lights B11 and road signs B12. The other vehicles B2 are vehicles that may be followed by or become obstacles while the host vehicle is traveling, but do not include the parked vehicles B7 shown in FIG. 19 . The general three-dimensional objects B3 are three-dimensional objects other than the traffic-related three-dimensional objects B1, other vehicles B2, wheel chocks B6, and parked vehicles B7, and primarily constitute "obstacles." Furthermore, for example, it is possible to recognize the lane LN based on the recognition result of the target B using an image. In other words, the parking space PS and the lane LN are different from the target B as a direct recognition target by the ADAS sensor, but can be said to be indirectly recognized targets based on the recognition result of the target B. Examples of the lane LN as an indirect recognition target include the lane LNm in which the host vehicle is currently traveling and the oncoming lane LNc. Further details of each recognition target in the recognition unit 2001 will be described later.

[0084] The own lane recognition function 2104 recognizes the position of the own vehicle in the width direction of the road Rd on which the own vehicle is currently traveling, based on the recognition result of the target B by the target recognition function 2103. Hereinafter, the width direction of the road Rd will be referred to as the "road width direction." In contrast, the direction perpendicular to the road width direction will be referred to as the "road extension direction." The road extension direction is the direction extending along the road Rd, and is also referred to as the "road extension direction" or "road extension direction." If the road Rd includes multiple lanes LN, the own lane recognition function 2104 recognizes which of the multiple lanes LN lined up in the road width direction the own lane LNm is. The intersection recognition function 2105 recognizes an intersection Xr around the own vehicle, based on the recognition result of the target B by the target recognition function 2103. Specifically, the intersection recognition function 2105 recognizes, for example, the presence or absence of a traffic light B11 and the color of its light, the presence or absence of road markings B4 such as a stop line B42, the positions of the intersection entrance Xr1 and the intersection exit Xr2, the route divisions, etc., for the intersection Xr that the host vehicle is currently approaching. The surrounding environment recognition function 2106 recognizes the driving environment around the host vehicle, i.e., the presence or absence of obstacles, etc., based on the recognition result of the target B by the target recognition function 2103. These recognition results are used by the operation determination unit 2002 shown in FIG. 16 to determine the content of the control operation currently required for the host vehicle.

[0085] FIG. 18 shows an example of a view in front of the vehicle as the vehicle approaches an intersection Xr that forms a crossroad and has a traffic light B11. The intersection Xr that forms a crossroad will be referred to simply as a "crossroad intersection" hereinafter. The recognition results of the traffic light B11 and road signs B12 included in the traffic-related three-dimensional object B1 can be used, for example, by the HMI device 7 to provide information or warnings to the driver D. Furthermore, such recognition results can be used for controlling the vehicle's motion during automated driving or driver assistance. In addition to the traffic light B11 and road signs B12, the traffic-related three-dimensional object B1 also includes roadside structures B13 such as guardrails and curbs. The roadside structures B13 are recognized as obstacles during vehicle navigation control. The traffic-related three-dimensional object B1 also includes road studs B14 and poles B15, as shown in FIG. 20. The road studs B14 are recognized as obstacles during vehicle navigation control depending on their height and installation conditions. The pole B15 is recognized as an obstacle when controlling the running of the host vehicle.

[0086] 18 again, the general three-dimensional objects B3 include a fallen object B31 on the road, a pedestrian B32, a cyclist B33, a building B34, etc. The fallen object B31 on the road, the pedestrian B32, and the cyclist B33 are recognized as obstacles during driving control of the host vehicle. The building B34 is not recognized as an obstacle because it is normally located outside the road Rd, but it can be a target for superimposed display when notifying the driver D of information via the head-up display 704, for example.

[0087] The road markings B4 include a crosswalk B41, a stop line B42, and a road dividing line B43. The road markings B4 also include a character marking B44 and a graphic display B45, as shown in FIG. 21 . The character markings B44 include, for example, numbers indicating a maximum speed limit, vehicle traffic classifications such as "buses only," and regulatory characters such as "stop." The graphic display B45 includes, for example, an arrow indicating a possible direction of travel or a graphic to alert the driver to the crosswalk B41 ahead. The recognition results of the road markings B4 can be used, for example, to estimate the positions of an intersection entrance Xr1, an intersection exit Xr2, and an intersection center Xrc. The intersection entrance Xr1 is the edge of the target intersection on the side where the vehicle is entering. The target intersection is the closest intersection Xr that the vehicle is currently approaching, the intersection that the vehicle is scheduled to pass through, or the intersection that the vehicle is likely to pass through. The intersection Xr "that the host vehicle plans to pass through" is an intersection Xr on the planned driving route when the destination and planned driving route of the host vehicle are set in the navigation device 6. The intersection Xr "that the host vehicle is likely to pass through" is an intersection Xr that is estimated with a high probability to be passed by the host vehicle based on the current vehicle speed and the distance to the intersection Xr when the destination and planned driving route of the host vehicle are not set in the navigation device 6. The intersection exit Xr2 is the edge of the target intersection from which the host vehicle will exit. The intersection center Xrc is the center of the target intersection.

[0088] As shown in Figures 20 to 22, the road dividing line B43 includes a roadway outer boundary line B431, a center line B432, and a lane boundary line B433. The roadway outer boundary line B431 is provided to indicate the outer edge of the roadway in the road width direction. The "roadway" is the portion of the road Rd on which vehicles travel. The center line B432 is provided to divide a two-way road Rd, i.e., a road that is not one-way, by direction of travel. The center line B432 may be a single line, a double line, or a triple line. The lane boundary line B433 is provided to divide multiple lanes LN with the same direction of travel on a road Rd with two or more lanes on each side. The road dividing line B43 may be painted white or yellow. In the example of Fig. 20, the center line B432 has a triple line pattern with yellow solid lines on both sides of a white solid line, and road studs B14 and poles B15 are provided on the central white solid line. In the example of Fig. 21, the center line B432 has a double line pattern with white solid lines, and the lane boundary line B433 has a yellow solid line beyond a white dashed line. Based on the recognition results of roadside structures B13 and road dividing lines B43, it is possible to recognize the driving lane LNd, oncoming lane LNc, overtaking lane LNp, right-turn lane LNr, road shoulder LNs, emergency parking zone EZ, etc.

[0089] Furthermore, the road markings B4 include guidance strip markings B461, safety zone markings B462, no entry markings B463, no stopping markings B464, and the like, as shown in Figures 23 to 26. The guidance strip marking B461 shown in Figure 23 is provided to indicate a guidance strip to guide vehicles for safe and smooth driving. Although guidance strips are markings to prevent vehicles from passing through, vehicles are not prohibited from entering the guidance strip by law. The guidance strip marking B461 is provided, for example, near intersections Xr, merging points, and forks. The safety zone marking B462 shown in Figure 24 is provided to indicate a safety zone. A safety zone is an area on road Rd established to ensure the safety of pedestrians B32 and others, and vehicles are not permitted to enter within the area. The no entry marking B463 shown in Figure 25 is provided to indicate a restricted area. A no-entry area is an area that is not intended for use by vehicles, and vehicles are not permitted to enter inside the area. The no-stopping sign B464 shown in Figure 26 is provided to indicate a no-stopping area. A no-stopping area is an area that a vehicle is not permitted to enter if there is a risk that the vehicle will have to stop due to conditions ahead of the path it is attempting to travel. In other words, a vehicle is not permitted to stop inside the no-stopping sign B464 when waiting at a traffic light or in a traffic jam.

[0090] FIG. 27 shows an example of a road sign B12. The road sign B12 is a signboard installed beside or above a road Rd, and provides necessary information to users through its design. The design includes an outer shape, a color, and a symbol. The outer shape may be a circle, a triangle, an inverted triangle, a square, a diamond, an octagon, etc. The color may be white, black, red, blue, yellow, etc. In FIG. 27, cross-hatched areas indicate red, diagonally hatched areas indicate blue, dotted areas indicate yellow, and solid black areas indicate black. The symbol includes letters, figures, and combinations thereof.

[0091] Figure 27 shows examples of signs from Japan, Europe, and the United States, categorized by meaning. The European examples use examples from Germany and France. For example, the design of "No Entry" is almost the same in each country. In contrast, while the color and symbol of "Stop" are common, the external shape is almost identical between Europe and the United States, while the Japanese sign is different. The design of maximum speed signs is almost the same between Japan and Europe, except for the color of the text, but the American sign is completely different from Japan and Europe in terms of external shape and color. The design of "Watch out for railroad crossings" is less common among Japan, Europe, and the United States. While the "No Right Turns" sign is similar between Europe and the United States, there is no corresponding sign in Japan. However, for example, if a "No Proceeding Outside the Specified Direction" sign, which indicates straight ahead and left turns with white arrows, is installed at a crossroads, as shown in the lower left of Figure 27, such a sign essentially indicates "No Right Turns."

[0092] Accurate recognition of road signs B12 effectively realizes automated driving and advanced driving assistance. However, as described above, the designs of road signs B12 may vary significantly from country to country. Therefore, the memory 22 shown in FIG. 2 pre-stores a database for recognizing the meaning indicated by the detected road signs B12 through pattern matching. Such a database may be country-specific or region-specific, or may be a database common to the entire world. Note that such a database may be stored in a host computer connected via V2X via the on-board communication device 4, instead of or in addition to the memory 22. V2X stands for Vehicle to X.

[0093] FIG. 28 shows an example of a scenario in which the recognition result of target object B is used, in which the vehicle is approaching a crossroads. In this scenario, for example, an information display M1 indicating the distance between the vehicle and the stop line B42, such as "10 m to stop position," can be superimposed on the stop line B42. In addition, a voice notification such as "10 m to stop position" can be provided. This effectively assists the driver D, who is using the driving assistance function, to make a safe temporary stop at the stop line B42. Alternatively, by notifying the occupants, including the driver D, who are using the automated driving function, of the scheduled occurrence of a temporary stop event at an appropriate time, it is possible to effectively avoid impairing the comfort of the occupants.

[0094] FIG. 29 shows another example of a scenario in which the recognition result of the target object B is used. The scenario shows a situation in which the vehicle is approaching a cross-junction with a traffic light B11 where the vehicle is planning to turn right. In this scenario, for example, an information display M1 indicating the distance to the right turn point, such as "Turn right in 20 meters," can be superimposed on the road surface FR in front of the driver D's field of vision in the right-turn lane LNr, which is the vehicle's own lane LNm. In addition, an audio notification, such as "Turn right at the intersection 20 meters ahead," can be provided. Furthermore, an arrow graphic display M2 indicating the vehicle's planned travel path can be superimposed on the road surface FR so that the arrow travels from the right-turn lane LNr, which is the vehicle's own lane LNm, through the vicinity of the intersection center Xrc, and reaches the road Rd where the vehicle is to turn right. This allows for good route guidance and advance notice of the lateral acceleration behavior of the vehicle to the driver D and thereby effectively avoids impairing the vehicle's comfort.

[0095] 17 again, the target object recognition function 2103 recognizes the type and status of the target object B based on the recognition results obtained using the ADAS sensor and the HD map information stored in the HD map 5. The "status" includes the position. The "position" includes the distance from the vehicle, the relative position with respect to the vehicle, the lane position, i.e., one of the multiple lanes LN, etc.

[0096] The lane marking recognition function 2131 recognizes road markings B43 around the vehicle, including on the road surface FR of the road Rd where the vehicle is traveling. For example, the lane marking recognition function 2131 recognizes whether the outer roadway line B431, center line B432, lane boundary line B433, etc., as shown in Figures 20 to 23, are white or yellow, solid or broken, single or multiple lines, etc. Technology for recognizing road markings B43 was well known at the time of filing this application, and therefore a detailed description of the technology therefor will be omitted in this specification.

[0097] The road marking recognition function 2132 recognizes the type, meaning, and position of road markings B4 provided around the vehicle, other than the road dividing lines B43. Recognition targets of the road marking recognition function 2132 include, for example, crosswalks B41, stop lines B42, and graphic markings B45.

[0098] The roadside structure recognition function 2133 recognizes the type and position of roadside structures B13 located around the host vehicle. At least one of the image recognition results using the camera 34, the recognition results using the laser radar sensor 33, and HD map information is used to recognize the roadside structures B13. The traffic light recognition function 2134 recognizes the position and light color of at least a traffic light B11 located ahead of the host vehicle. The road sign recognition function 2135 recognizes road signs B12 located around the host vehicle. The lane recognition function 2136 recognizes lanes, i.e., recognizes the number and type of lanes LN on the road Rd on which the host vehicle is traveling. At least one of the image recognition results using the camera 34, i.e., the front camera CF, the recognition results using the laser radar sensor 33, and HD map information is used to recognize the lane. Specifically, lane recognition typically uses the image recognition results obtained using the front camera CF, but if necessary, so-called "sensor fusion" can be performed using other information as well.

[0099] The pedestrian recognition function 2137 recognizes a pedestrian B32 present around the host vehicle. The surrounding vehicle recognition function 2138 recognizes other vehicles B2 present around the host vehicle. The obstacle recognition function 2139 recognizes obstacles such as fallen objects B31 on the road and pedestrians B32 present around the host vehicle. The recognition methods used by the traffic light recognition function 2134, road sign recognition function 2135, lane recognition function 2136, pedestrian recognition function 2137, surrounding vehicle recognition function 2138, and obstacle recognition function 2139 are well known at the time of filing this application. Therefore, detailed descriptions of these recognition methods will be omitted in this specification.

[0100] (Lane Departure Prevention Control) Hereinafter, details of lane keeping assist control, i.e., lane departure prevention control, performed by the driving ECU 2 according to this embodiment will be described. When describing the effects of the driving ECU 2 as a control device according to this embodiment, the control method and control program executed thereby, and the recording medium on which the control program is recorded, these may be collectively referred to as the "present embodiment." This embodiment executes lane departure prevention control based on the recognition results of the road conditions ahead of the host vehicle, for example, the recognition results of road markings B4, i.e., road dividing lines B43, provided on road Rd. The lane departure prevention control includes at least one of operational control of the HMI device 7 for issuing a warning or other notification to an occupant, such as the driver D, of the host vehicle, and operational control of the motion control device 9 corresponding to a vehicle motion control subtask in at least the lateral direction of the host vehicle.

[0101] The driving ECU 2 can execute lane departure prevention control while predetermined execution conditions are met. For example, the execution conditions can be set as follows, but the present disclosure is not limited to these conditions, and some of the following conditions may be omitted or replaced with other conditions. Among the following conditions, condition C12-3 and condition C12-4 relate to conditions for canceling control by so-called "override." "Override" refers to driver D obtaining control of the driving operation at his or her own discretion. In other words, if at least one of condition C12-3 and condition C12-4 is not met, it is presumed that the driving operation is being performed at the driver D's will, and therefore the driving operation by driver D takes priority over system intervention. Condition for executing lane departure prevention control = either condition C11 or condition C12 below is met. Condition C11 = The vehicle is in automated driving mode. Condition C12 = All of the following conditions C12-1 to C12-5 are met. Condition C12-1 = An occupant such as driver D of the vehicle has selected to execute a predetermined driving mode (for example, hands-off driving) that includes the execution of lane departure prevention control, and no system abnormality such as a sensor abnormality has occurred that would prevent the execution of such a driving mode. Condition C12-2 = The white line recognition function, i.e., the function for recognizing road dividing line B43 by the ADAS sensor, is operating normally. Condition C12-3 = The amount of brake operation and the amount of accelerator operation are less than a predetermined amount. Condition C12-4 = The steering torque, which indicates the operation state of the steering wheel V24, is less than a predetermined value. Condition C12-5 = The speed of the vehicle is within a predetermined lane departure prevention control range.

[0102] Furthermore, the driving ECU 2 can execute lane change assist control while predetermined execution conditions are met. For example, such execution conditions can be set as follows, but the present disclosure is not limited to these, and some of the following conditions may be omitted or replaced with other conditions. Of the following conditions, conditions C22-3 and C22-4 relate to conditions for canceling control by so-called "override." In other words, if these conditions are not met, the driving operation by the driver D takes priority over system intervention. Condition for executing lane change assist control = Either condition C21 or condition C22 below is met. Condition C21 = The vehicle is in automatic driving mode. Condition C22 = All of the following conditions C22-1 to C22-8 are met. Condition C22-1 = An occupant of the vehicle, such as driver D, has selected to execute a predetermined driving mode (e.g., hands-off driving) that includes the execution of lane change assist control, and no system abnormality has occurred that would prevent the execution of such a driving mode. Condition C22-2 = The white line recognition function is operating normally. Condition C22-3 = The amount of brake operation and the amount of accelerator operation are less than a predetermined amount. Condition C22-4 = The steering torque, which indicates the operating state of the steering wheel V24, is less than a predetermined value. Condition C22-5 = An occupant of the vehicle, such as driver D, has performed an operation to change lanes (e.g., operating the turn signal switch 382). Condition C22-6 = The lane boundary line B433 between the current vehicle lane LNm and the destination lane (i.e., the destination lane LN) does not indicate that a lane change is impossible. Condition C22-7 = The vehicle speed of the current vehicle is within a predetermined lane change assistance control range. Condition C22-8 = There are no three-dimensional objects (typically, another vehicle B2) within a predetermined range in the destination lane that may obstruct the lane change.

[0103] As is clear from the above description of the execution conditions, lane change assist control can also be performed during lane departure prevention control. FIG. 30 schematically illustrates a situation in which a host vehicle, whose current lane LNm is the straight lane LNa, the outermost driving lane, while lane departure prevention control is being performed, is attempting to change lanes to a right-turn lane LNr adjacent to the current lane LNm. In the figure, the dashed-dotted arrow indicates a planned travel route RT1 of the host vehicle when changing lanes from the current lane LNm to the right-turn lane LNr. This planned travel route RT1 can be calculated based on the current position and attitude of the host vehicle. Specifically, for example, there is a case in which the in-vehicle system 1 is currently executing a vehicle motion control subtask for at least the lateral direction of the host vehicle, and it has already been determined that the host vehicle will change lanes from the straight lane LNa, which is the current lane LNm, to the right-turn lane LNr. Such a case may occur, for example, when a destination is set by the navigation device 6. In such a case, the planned driving route RT1 may be calculated based on the current position and attitude of the vehicle and route information from the current position to the destination. On the other hand, for example, if the driver D operates the blinker switch 382, ​​the planned driving route RT1 may be calculated based on the current position and attitude of the vehicle and distance information from the current position to the intersection Xr located at the end of the right-turn lane LNr.

[0104] Lane departure prevention control is typically a control in which the in-vehicle system 1 intervenes in at least lateral vehicle motion control of the host vehicle to prevent the host vehicle from deviating from the host lane LNm, i.e., preventing the host vehicle from deviating from the first host lane boundary Lbm1 and the second host lane boundary Lbm2. The first host lane boundary Lbm1 is a virtual line extending along the road so as to define the left edge of the host lane LNm. Specifically, in the example of FIG. 30 , the first host lane boundary Lbm1 is the edge of the outer lane boundary B431 located on the left side of the host lane LNm, on the host lane LNm side. Similarly, the second host lane boundary Lbm2 is a virtual line extending along the road so as to define the right edge of the host lane LNm.

[0105] However, lane departure prevention control during lane change assist control may include control to prevent unnecessary "straying" outside the expected driving area Zr when changing lanes. As shown in FIG. 30 , the expected driving area Zr is a sigmoid curved band-like area formed by smoothly connecting the current lane LNm and the destination lane (i.e., the right-turn lane LNr in the example of FIG. 30 ). Specifically, the expected driving area Zr is the area between the first boundary line BL1 and the second boundary line BL2. The first boundary line BL1 is a sigmoid curve formed by smoothly connecting the first right-turn lane boundary Lbr1 and the first current lane boundary Lbm1 in the parallel running section Zs. The parallel running section Zs refers to the running section where the current lane LNm and the guidance strip marking B461 run parallel to each other, excluding the branching section Zt. The branch section Zt is a driving section where the width of the guidance strip marking B461 gradually decreases in the road width direction. That is, the branch section Zt is the section from the intersection or tangent point of the second current lane boundary Lbm2 and the first zebra zone boundary Lbz1 to the end point of the guidance strip marking B461. The second boundary line BL2 is a sigmoid curve formed by smoothly connecting the second right-turn lane boundary Lbr2 and the first zebra zone boundary Lbz1. Note that, for simplicity of explanation and illustration, in FIG. 30 , the straight-through lane LNa and the adjacent right-turn lane LNr are assumed to have approximately the same width. That is, the distance between the first current lane boundary Lbm1 and the second current lane boundary Lbm2 is assumed to be approximately the same as the distance between the first right-turn lane boundary Lbr1 and the second right-turn lane boundary Lbr2. The first right-turn lane boundary Lbr1 is the edge of the lane boundary line B433 located on the left side of the right-turn lane LNr, on the right-turn lane LNr side. The second right-turn lane boundary Lbr2 is the edge of the center line B432 located on the right side of the right-turn lane LNr, on the right-turn lane LNr side.

[0106] As shown in FIG. 30 , a guidance strip marking B461 is often provided on the near side of the right-turn lane LNr. In addition to the guidance strip marking B461, area markings formed with stripes, such as the no entry sign B463 shown in FIG. 25 , are generally collectively referred to as "zebra zones." However, among such zebra zones, the ones that are subject to lane departure prevention control are primarily the guidance strip marking B461. Furthermore, as will be described later, depending on the type of boundary line, the guidance strip marking B461 may be a type of zebra zone where entry is not prohibited. Therefore, in the following description, unless otherwise noted, the term "zebra zone" refers to the guidance strip marking B461 and the guidance strip indicated by it.

[0107] If the guidance strip marking B461 is painted entirely in white, entry of the host vehicle into the interior of the marking is not prohibited by law, but it is preferable to avoid entry whenever possible. Therefore, in the example of FIG. 30 , lane departure prevention control is executed to prevent the host vehicle from protruding to the right of the first zebra zone boundary Lbz1, i.e., toward the guidance strip marking B461. The first zebra zone boundary Lbz1 is the boundary of the guidance strip marking B461 on the side closer to the host vehicle's lane LNm in the road width direction. The first zebra zone boundary Lbz1 constitutes the boundary on the guidance strip marking B461 side of the expected driving area Zr and smoothly connects to the second right-turn lane boundary Lbr2 in a sigmoid curve. On the other hand, the second zebra zone boundary Lbz2 is the boundary of the guidance strip marking B461 on the opposite side of the first zebra zone boundary Lbz1 in the road width direction. 30, in the parallel running section Zs, the first zebra zone boundary Lbz1 coincides with the second own lane boundary Lbm2, and the second zebra zone boundary Lbz2 coincides with the oncoming lane boundary Lbc, which is the edge of the center line B432 on the oncoming lane LNc side.

[0108] Figure 31 illustrates the example of Figure 30 in which there are multiple vehicles B2 ahead of the host vehicle, with the rearmost vehicle B2 blocking the branch point from the host lane LNm to the right-turn lane LNr. In this case, the host vehicle is not prohibited by law from crossing the second host lane boundary Lbm2 and entering the right-turn lane LNr. Figure 32 illustrates the example of Figure 31 in which the line of vehicles B2 extends to the parallel running section Zs. In this case, the host vehicle is not prohibited by law from traveling within the guidance strip marking B461. In particular, crossing the zebra zone shown in Figure 31 and driving within the zebra zone shown in Figure 32 may be recommended or actively used in emergencies or when the host vehicle is an emergency vehicle.

[0109] If such zebra zone crossing or zebra zone driving is attempted "intentionally" by vehicle motion control or driver D's driving operation, executing normal lane departure prevention control may cause the vehicle motion control to jerk or cause discomfort to the occupants. Therefore, in such cases, this embodiment suppresses the execution of lane departure prevention control. In contrast, this embodiment prohibits the suppression of the execution of lane departure prevention control when it is determined that the host vehicle is not in autonomous driving but is undergoing driving assistance and the driver D is not awake, in order to prevent unintentional "straying" into the zebra zone. Note that "suppressing" the execution of lane departure prevention control includes "prohibiting" the execution, "reducing" the execution mode, and "offsetting" or delaying the timing or position of the execution. Specifically, this embodiment prohibits or temporarily suspends lane departure prevention control during autonomous driving. On the other hand, this embodiment determines the driver D's intention and the safety of passing through the zebra zone during driving assistance, and appropriately determines whether to suppress the execution of lane departure prevention control.

[0110] However, even when the host vehicle is permitted to enter the zebra zone, vehicle control that allows the host vehicle to drift into the oncoming lane LNc is generally inappropriate, except in special cases such as when the host vehicle is an emergency vehicle and safety can be assured. Therefore, in this embodiment, lane departure prevention control is executed to prevent the host vehicle from drifting beyond the second zebra zone boundary Lbz2, which is the boundary of the zebra zone on the oncoming lane LNc side.

[0111] In this embodiment, whether or not to suppress execution of lane departure prevention control to prevent entry into a zebra zone is determined based on the type of zebra zone or the type of road dividing line B43 that constitutes the zebra zone on the vehicle's own lane LNm side. For example, in this embodiment, whether or not to allow entry into a guidance strip marking B461 shown in Figure 30 etc. is determined based on the line type, while entry into a no entry sign B463 shown in Figure 25 is not permitted.

[0112] Specifically, for example, as shown in Figure 33, there is a case where the lane boundary line B433 on the host lane LNm side of the zebra zone is a straight yellow line, prohibiting the host vehicle from leaving the host lane LNm. In this case, the present embodiment does not suppress the execution of lane departure prevention control. Therefore, in this case, normal lane departure prevention control, that is, lane departure prevention control that prevents the host vehicle from leaving the first zebra zone boundary Lbz1 into the zebra zone, is executed.

[0113] As shown in FIG. 34 , a solid center line B432 may be provided through the zebra zone. In this case, this embodiment executes lane departure prevention control to allow the host vehicle to deviate from the first zebra zone boundary Lbz1 toward the zebra zone but not from the center line B432. Alternatively, this embodiment may execute lane departure prevention control to prevent the host vehicle from deviating from the first zebra zone boundary Lbz1 toward the zebra zone without suppressing the execution of lane departure prevention control. Note that the center line B432 in this case may be a yellow line as shown in FIG. 34 or a white line. Furthermore, if the host vehicle travels further forward from the state shown in FIG. 34 and approaches the right-turn lane LNr as shown in FIG. 30 , the execution of lane departure prevention control may be suppressed.

[0114] As shown in FIG. 35 , another vehicle B2 may be in an emergency stop within the zebra zone. Alternatively, as shown in FIG. 36 , a three-dimensional object such as a pole B15 may be installed within the zebra zone. When it is determined that a three-dimensional object that may be an obstacle exists within the zebra zone, this embodiment prohibits the suppression of the execution of "lane departure prevention control to prevent entry into the zebra zone." Therefore, in this case, normal lane departure prevention control, i.e., lane departure prevention control that prevents the host vehicle from moving beyond the first zebra zone boundary Lbz1 into the zebra zone, is executed. Note that if the possibility of a collision with such a three-dimensional object disappears, for example, if the host vehicle passes beside the other vehicle B2 shown in FIG. 35 , the execution of lane departure prevention control may be suppressed. Furthermore, depending on the position of the three-dimensional object, such as the pole B15 or a cone, in the road width direction, the same processing as in FIG. 34 may be executed. That is, for example, lane departure prevention control can be executed to prevent the vehicle from going beyond a virtual boundary line established between the first zebra zone boundary Lbz1 and a solid object.

[0115] In this embodiment, the system recognizes a zebra zone adjacent to the host vehicle's lane LNm in the road width direction as road conditions ahead of the host vehicle, and estimates an entry probability corresponding to the possibility that the host vehicle will enter the zebra zone based on the recognition result of the zebra zone. The higher the entry probability, the higher the possibility that the host vehicle will enter the zebra zone. Factors used to estimate the entry probability include, for example, the vehicle's driving environment and the estimated results of the driver D's intention. The driving environment may be, for example, the host vehicle's approach to an intersection Xr, a branch, or a merge. Therefore, the entry probability can be estimated based on, for example, map information or the driving scene of the host vehicle. Examples of "driving scenes" include right turns, left turns, merges, branching, entering or exiting a predetermined type of road, etc. A "predetermined type of road" is a type of road that differs from a general road or a residential road and that requires special consideration for speed limits, entry and exit methods, etc., and includes, for example, expressways or expressways in Japan and freeways in the United States. The intention of the driver D can be estimated using the detection result of the driver's state by the driver state monitor 37 and the steering torque.

[0116] In this embodiment, if the zebra zone is one in which entry is not prohibited, the execution of lane departure prevention control is suppressed. Here, in this embodiment, the higher the entry probability, the more the execution of lane departure prevention control is suppressed. For example, in this embodiment, the notification mode for occupants such as the driver D is changed depending on the entry probability. Specifically, if the entry probability is high, no warning or notification is given to the occupants. A "warning" is a strong form of information presentation, and includes a relatively loud warning voice and a conspicuous display. A "notification" is a less strong form of information presentation than a "warning" and includes at least one of normal voice guidance and display. Alternatively, if the entry probability is high, the offset amount for notifying the occupants or for initiating intervention in vehicle motion control is increased. If the entry probability is low, the occupants are warned and / or the offset amount is reduced. If the entry probability is medium, the occupants are notified and / or the offset amount is set to a medium level.

[0117] 37A and 37B show an example of an operation flow corresponding to this embodiment, that is, an example of a processing flow realized by the processor 21 executing program instructions stored in the memory 22. In the example shown in FIGS. 37A and 37B, it is assumed that the host vehicle is currently performing SAE Level 2 advanced driving assistance (e.g., hands-off driving). Note that in the flowcharts shown in FIGS. 37A and 37B, "S" is an abbreviation for "step."

[0118] First, in step 101, the processor 21 recognizes a zebra zone. Next, in step 102, the processor 21 calculates a departure point PD shown in FIG. 38. The departure point PD is a point where the host vehicle is estimated to deviate from the host lane LNm. The departure point PD is a point where the planned travel trajectory RT2 intersects with the boundary of the departure destination on the host lane LNm. The planned travel trajectory RT2 is obtained by offsetting the planned travel route RT1, which is the trajectory of the vehicle center point VC as the host vehicle travels, by half the vehicle width of the host vehicle. In the example of FIG. 38, the boundary of the departure destination on the host lane LNm is the second host lane boundary Lbm2.

[0119] The departure point PD can be calculated based on the speed and acceleration of the host vehicle. Specifically, the current running state of the host vehicle is assumed to be uniformly accelerated. Here, if the lateral speed of the host vehicle is Vx, the lateral acceleration of the host vehicle is αx, and the time is t, then X = Vx t + (½) αx t ２ The host vehicle departs from the host lane LNm at a time t that satisfies the following equation. Referring to Figure 38, X in the above equation is the lateral position of the host vehicle, which is the shortest distance in the road width direction between the host vehicle and the boundary of the host lane LNm, the departure point. The distance Y in the road extension direction from the host vehicle to the departure point PD is calculated as Y = vm t using the vehicle speed, i.e., the host vehicle speed vm.

[0120] Next, in step 103, processor 21 determines whether departure point PD calculated in step 102 is within a zebra zone. This determination is a determination of whether departure point PD is included in the boundary of the zebra zone on the side of the host vehicle's lane LNm. In the example of FIG. 39 , the boundary of the zebra zone on the side of the host vehicle's lane LNm is first zebra zone boundary Lbz1. Specifically, processor 21 determines whether distance Y calculated in step 102 is included within the installation distance range of the zebra zone in the direction of extension of the road from the host vehicle. If departure point PD is within a zebra zone (i.e., step 103 = YES), processor 21 proceeds to step 104.

[0121] In step 104, the processor 21 determines whether the boundary line of the zebra zone on the own lane LNm side is yellow. If the boundary line of the zebra zone on the own lane LNm side is white (i.e., step 104 = NO), the processor 21 proceeds to step 105. In step 105, the processor 21 determines whether an obstacle is present in the zebra zone. If no obstacle is present in the zebra zone (i.e., step 105 = NO), the processor 21 proceeds to step 106. In step 106, the processor 21 determines whether the driver D is awake. If the driver D is awake (i.e., step 106 = YES), the processor 21 proceeds to step 107.

[0122] In step 107, the processor 21 estimates the movement accuracy. In this specific example, the movement accuracy is estimated as one of three levels: "high," "medium," or "low." However, the present disclosure is not limited to this example, and the number of levels may be two or four or more. The movement accuracy may be a continuous value or a discontinuous value. In this specific example, the following factors are used to estimate the movement accuracy. First factor: Based on map information and the determined planned driving route RT1, the host vehicle is approaching a point where it plans to change lanes or turn right or left (e.g., a merging point, a branch point, an intersection Xr, etc.). Second factor: Based on recognition information obtained using the ADAS sensor, it is possible to predict that the host vehicle is approaching a point where it may change lanes or turn right or left. Third factor: Based on the driver state detection results and steering torque obtained by the driver state monitor 37, it is possible to confirm the driver D's intention to change lanes or turn right or left with the host vehicle. - Fourth element: an obstacle to be avoided has been confirmed within a predetermined range ahead of the host vehicle based on recognition information obtained using the ADAS sensor. The "obstacle to be avoided" referred to here includes, for example, fallen objects B31 and pedestrians B32, as well as other vehicles B2 parked on the road, other vehicles B2 as cars on the side of the road that are visible because their turn signals are on and are planning to turn right or left, cyclists B33, motorcycles, and other vehicles in general that are ahead of the host vehicle, such as other vehicles B2 parked on the road and are visible because their turn signals are on and are planning to turn right or left, cyclists B33, motorcycles, etc.

[0123] In this specific example, when the first element is established, the processor 21 estimates the movement accuracy to be "high," regardless of whether the other elements are established. It should be noted that a difference in movement accuracy may be established between when only the first element is established and when the other elements are established. Furthermore, even if the first element is not established, when the second and third elements are established, the processor 21 estimates the movement accuracy to be "high," regardless of whether the fourth element is established. On the other hand, when the first and third elements are not established and the second and fourth elements are established, the processor 21 estimates the movement accuracy to be "medium." Furthermore, when the first and second elements are not established and the third or fourth element is established, the processor 21 estimates the movement accuracy to be "medium." Furthermore, when only the second element is established, the processor 21 estimates the movement accuracy to be "low."

[0124] If the departure point PD is not within a zebra zone (i.e., step 103 = NO), the processor 21 proceeds to step 108. If the boundary line of the zebra zone on the host lane LNm side is yellow (i.e., step 104 = YES), the processor 21 proceeds to step 108. If an obstacle is present in the zebra zone (i.e., step 105 = YES), the processor 21 proceeds to step 108. If the driver D is not awake (i.e., step 106 = NO), the processor 21 proceeds to step 108. In step 108, the processor 21 executes the lane departure prevention control as usual without suppressing it. That is, in these cases, suppression of "execution of lane departure prevention control in a manner that prevents entry into the zebra zone" is prohibited. Note that in step 108, "LDA" is written in the figure, and LDA is an abbreviation for lane departure alert, but this specific example and this disclosure are not limited to lane departure alert. In other words, the processing in step 108 may include not only warnings and notifications to the occupants but also vehicle motion control.

[0125] After executing the processing of step 107, the processor 21 advances the processing to step 201 shown in FIG. 37B. In step 201, the processor 21 determines whether the movement probability estimated in step 107 is "high." If the movement probability is high (i.e., step 201 = YES), the processor 21 advances the processing to step 202. In step 202, the processor 21 executes lane departure prevention control in a suppression manner when the movement probability is high. In this case, the lane departure prevention control is suppressed more strongly than in other cases. If the movement probability is not "high" (i.e., step 201 = NO), the processor 21 advances the processing to step 203.

[0126] In step 203, the processor 21 determines whether the movement probability estimated in step 107 is "medium." If the movement probability is medium (i.e., step 203 = YES), the processor 21 proceeds with the process to step 204. On the other hand, if the movement probability is "low," step 201 = NO → step 203 = NO, and the processor 21 proceeds with the process to step 205. In step 204, the processor 21 executes lane departure prevention control in a medium suppression mode when the movement probability is medium. In step 205, the processor 21 executes lane departure prevention control in the weakest suppression mode when the movement probability is low.

[0127] After performing one of the processes of step 202, step 204, and step 205 depending on the movement probability, the processor 21 proceeds to step 206. In step 206, the processor 21 determines whether the host vehicle is about to stray into the oncoming lane LNc. If the host vehicle is about to stray into the oncoming lane LNc (i.e., step 206 = YES), the processor 21 proceeds to step 108 shown in FIG. 37A. In this case, in step 108, the processor 21 executes the lane departure prevention control normally without suppressing it. That is, in this case, suppression of the execution of the lane departure prevention control that prevents the host vehicle from straying into the oncoming lane LNc is prohibited. In contrast, if the scene is not a stray scene (i.e., step 206 = NO), the processor 21 proceeds to step 207. In step 207, the processor 21 continues the state in which the execution of the lane departure prevention control is suppressed.

[0128] (Modifications) The present disclosure is not limited to the above-described embodiment. Therefore, the above-described embodiment can be modified as appropriate. Representative modifications will be described below. In the following description of the modifications, differences from the above-described embodiment will be mainly described. Furthermore, identical or equivalent parts between the above-described embodiment and the modifications are given the same reference numerals. Therefore, in the following description of the modifications, the description of the above-described embodiment can be used as appropriate for components having the same reference numerals as those in the above-described embodiment, unless there is a technical contradiction or special additional explanation.

[0129] The present disclosure is not limited in any way to the specific device configurations shown in the above embodiments. That is, for example, the shape and structure of the vehicle body V1 of the system-equipped vehicle V are not limited to a box shape, i.e., a substantially rectangular shape in a plan view. Furthermore, the vehicle body panel V15 does not need to be provided on the upper side of the passenger compartment V2. Alternatively, the vehicle body panel V15 covering the upper side of the passenger compartment V2 may be detachable. There are also no particular limitations on the use of the system-equipped vehicle V, the positions of the driver's seat V23 and steering wheel V24, the number of occupants, etc.

[0130] The definitions, levels, and categories of driving automation are not limited to those specified in "SAE J3016." Specifically, "autonomous driving" in the above embodiments refers to a driving automation level corresponding to levels 3 to 5 in "SAE J3016," where the driving automation system is responsible for, i.e., performs, all dynamic driving tasks. Therefore, the definition of "autonomous driving" in the above embodiments naturally includes the situation where the driver D is not required to monitor the surroundings. However, the present disclosure is not limited to such an aspect. That is, for example, depending on the definition of "autonomous driving," it can be conceptualized as not only "autonomous driving without the obligation to monitor the surroundings" but also "autonomous driving with the obligation to monitor the surroundings." Specifically, for example, hands-off driving in the above embodiments can also be interpreted as "autonomous driving with the obligation to monitor the surroundings." In this case, "autonomous driving" also includes so-called "partially automated driving," in which the driver D is responsible for, i.e., performs, some dynamic driving tasks, such as the obligation to monitor the surroundings. "Partially automated driving" can be considered to be substantially synonymous with "advanced driving assistance." Furthermore, in the road traffic systems of each country, the types and conditions of automated driving (for example, feasible roads, driving speed ranges, whether lane changes are possible, etc.) may be appropriately considered according to domestic circumstances, etc. Therefore, the present disclosure may be implemented with specifications that are compatible with the road traffic systems of each country.

[0131] The configuration of the in-vehicle system 1 is not limited to the above example. For example, there are no particular limitations on the number or locations of so-called ADAS sensors, such as the sonar sensor 31, radar sensor 32, and camera 34. That is, for example, by minimizing the number of radar sensors 32 and laser radar sensors 33, which are relatively expensive at the time of filing this application, it is possible to promote the early widespread use of autonomously driven vehicles. Therefore, for example, the radar sensor 32 or the laser radar sensor 33 may be omitted.

[0132] The locator 39 is not limited to a configuration incorporating the inertial acquisition unit 392. Specifically, the inertial acquisition unit 392 may be configured to receive output information from an angular velocity sensor and an acceleration sensor provided outside the locator 39 as the behavior sensor 36. The locator 39 may also be integrated with the HD map 5. The locator 39 is not limited to a commercially available product such as "POSLV" manufactured by Applanix.

[0133] The navigation device 6 may be connected to the HMI device 7 so as to be able to communicate information via a sub-communication line different from the in-vehicle communication line 10. Alternatively, the navigation device 6 may have a display screen dedicated to displaying the navigation screen, separate from the HMI device 7. Alternatively, the navigation device 6 may be provided as a part of the HMI device 7. Specifically, for example, the navigation device 6 may be integrated with the main display device 703.

[0134] The HMI device 7 is not limited to a configuration including the meter panel 702, the main display device 703, and the head-up display 704. That is, for example, the meter panel 702 and the main display device 703 may be integrated into a single display device. In this case, the meter panel 702 may be provided as part of a display area of ​​a single display device that is a liquid crystal or organic EL display. That is, the meter panel 702 may be realized by displaying images of bezels, pointers, scales, etc. corresponding to a tachometer, speedometer, water temperature gauge, etc. Furthermore, the HMI device 7 does not necessarily have to include the head-up display 704.

[0135] In the above embodiments, each ECU has a configuration as a so-called in-vehicle microcomputer, including a CPU, an MPU, and the like. However, the present disclosure is not limited to such a configuration. That is, all or part of the ECU may be configured to include a digital circuit, such as an ASIC or FPGA, configured to enable the above-described operations. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field Programmable Gate Array. That is, an ECU may have both an in-vehicle microcomputer portion and a digital circuit portion.

[0136] The program according to the present disclosure, which enables the execution of the various operations, procedures, or processes described in the above embodiments, can be downloaded or upgraded via V2X communication by the in-vehicle communication device 4. Alternatively, the program can be downloaded or upgraded via a terminal device provided at a vehicle manufacturing plant, a repair shop, a dealer, etc. The program can be stored on a memory card, an optical disk, a magnetic disk, etc.

[0137] In this way, each of the above functional configurations and methods may be implemented by a special-purpose computer configured by configuring a processor and memory programmed to execute one or more functions embodied in a computer program. Alternatively, each of the above functional configurations and methods may be implemented by a special-purpose computer configured by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, each of the above functional configurations and methods may be implemented by one or more special-purpose computers configured by combining a processor and memory programmed to execute one or more functions with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored on a computer-readable, non-transitory storage medium as instructions to be executed by a computer. In other words, each of the above functional configurations and methods may be expressed as a computer program including procedures for implementing the same, or as a non-transitory storage medium storing the program.

[0138] The present disclosure is not limited to the specific operational and processing aspects described in the above embodiment. For example, while this specification is based on the premise of "vehicles driving on the left side of the road" in accordance with the road traffic laws of Japan, the present disclosure is not limited to this. For example, in the case of "vehicles driving on the right side of the road" in accordance with the road traffic laws of the United States and other countries, the drawings such as FIG. 30 may be reversed as appropriate, and the description in the specification may be modified accordingly. The same applies to the definition of the zebra zone and the meanings corresponding to the colors and line types of the road dividing lines B43. Furthermore, if the host vehicle is in an autonomous driving state at SAE Level 3 or higher, step 106 in FIG. 37A is omitted. The conditions for executing lane change assist control are not limited to the above specific examples. For example, conditions related to acceleration G and deceleration G may be set in relation to condition C22-3. Furthermore, in condition C22-4, execution of lane change assist control may be canceled if the vehicle is in a completely hands-free state.

[0139] Similar expressions such as "obtain," "calculate," "estimate," "detect," "sensing," and "determine" may be substituted for each other as appropriate within the scope of technical compatibility. Furthermore, "information" and "signal" may be substituted for each other as appropriate within the scope of technical compatibility. Furthermore, "above the threshold" and "exceed the threshold" may be substituted for each other as appropriate within the scope of technical compatibility. Similarly, "below the threshold" and "below the threshold" may be substituted for each other as appropriate within the scope of technical compatibility.

[0140] It goes without saying that the elements constituting the above-described embodiments are not necessarily essential unless expressly stated as essential or clearly considered essential in principle. Furthermore, when numerical values ​​such as the number, amount, and range of components are mentioned, the present disclosure is not limited to those specific numerical values ​​unless expressly stated as essential or clearly limited to specific numerical values ​​in principle. Similarly, when the shape, direction, positional relationship, etc. of components are mentioned, the present disclosure is not limited to those shapes, directions, positional relationships, etc., unless expressly stated as essential or clearly limited to specific shapes, directions, positional relationships, etc. in principle.

[0141] The modified examples are not limited to the above examples. For example, multiple specific examples may be combined with each other as long as there is no technical contradiction. Also, multiple modified examples may be combined with each other as long as there is no technical contradiction. Furthermore, all or part of the above embodiment and all or part of the modified examples may be combined with each other as long as there is no technical contradiction.

[0142] (Viewpoint of the Disclosure) As is clear from the above description of the embodiments and modifications, the present specification discloses at least the following matters.

[0143] [Aspect 1-1] A control device (2) mounted on a vehicle (V), comprising: a processor (21); and a memory (22) that is a non-transient tangible recording medium that stores program instructions used by the processor, wherein the processor executes the program instructions stored in the memory to perform lane departure prevention control based on a recognition result of road conditions ahead of the vehicle, recognize a zebra zone (B461) adjacent to a host lane (LNm) that is the lane in which the vehicle is currently traveling in a width direction of the road as the road conditions, estimate an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition result of the zebra zone, and suppress the execution of the lane departure prevention control as the entry probability becomes higher. [Aspect 1-2] The control device according to Aspect 1-1, wherein the processor changes a notification mode for an occupant (D) of the vehicle according to the entry probability. [Aspect 1-3] The control device according to Aspect 1-1 or Aspect 1-2, wherein the processor estimates the entry probability using map information. [Aspect 1-4] The control device according to any one of Aspects 1-1 to 1-3, wherein the processor estimates the entry probability based on a driving scene of the vehicle. [Aspect 1-5] The control device according to any one of Aspects 1-1 to 1-4, wherein the processor calculates a departure point (PD) at which the vehicle is estimated to deviate from the host lane, and suppresses execution of the lane departure prevention control if the calculated departure point is included in a boundary (Lbz1) of the zebra zone on the host lane side. [Aspect 1-6] The control device according to Aspect 1-5, wherein the processor calculates a distance from the vehicle to the departure point in a road extension direction based on a speed and acceleration of the vehicle, and suppresses execution of the lane departure prevention control if the calculated distance is included in an installation distance range of the zebra zone in the road extension direction from the vehicle. [Aspect 1-7] The control device according to any one of Aspects 1-1 to 1-6, wherein the processor executes the lane departure prevention control so that the vehicle does not deviate from a boundary (Lbz2) on the oncoming lane (LNc) side of the zebra zone.[Aspect 1-8] The control device according to any one of Aspects 1-1 to 1-7, wherein the processor selects whether to suppress the execution of the lane departure prevention control depending on the type of the zebra zone or the type of road dividing line (B433) on the host lane side that constitutes the zebra zone. [Aspect 1-9] The control device according to Aspect 1-8, wherein the processor suppresses the execution of the lane departure prevention control if the zebra zone is a type from which entry is not prohibited. [Aspect 1-10] The control device according to any one of Aspects 1-1 to 1-9, wherein the processor prohibits the suppression of the execution of the lane departure prevention control if it determines that an obstacle (B2, B15) is present in the zebra zone. [Aspect 1-11] The control device according to any one of Aspects 1-1 to 1-10, wherein the processor prohibits the suppression of the execution of the lane departure prevention control if it determines that a driver (D) of the vehicle is in a waking state.

[0144] [Aspect 2-1] A control method executed by a processor (21) provided in a control device (2) mounted on a vehicle (V), the control method executing lane departure prevention control based on a recognition result of road conditions ahead of the vehicle, recognizing, as the road conditions, a zebra zone (B461) adjacent to a host lane (LNm) in which the vehicle is currently traveling in the width direction of the road, estimating an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition result of the zebra zone, and suppressing the execution of the lane departure prevention control as the entry probability increases. [Aspect 2-2] The control method according to Aspect 2-1, wherein a notification mode for an occupant (D) of the vehicle is changed according to the entry probability. [Aspect 2-3] The control method according to Aspect 2-1 or Aspect 2-2, wherein the entry probability is estimated using map information. [Aspect 2-4] The control method according to any one of Aspects 2-1 to 2-3, which estimates the entry probability based on a driving scene of the vehicle. [Aspect 2-5] The control method according to any one of Aspects 2-1 to 2-4, which calculates a departure point (PD) that is a point where the vehicle is estimated to deviate from the own lane, and suppresses execution of the lane departure prevention control if the calculated departure point is included in a boundary (Lbz1) of the zebra zone on the own lane side. [Aspect 2-6] The control method according to Aspect 2-5, which calculates a distance from the vehicle to the departure point in a road extension direction based on a speed and acceleration of the vehicle, and suppresses execution of the lane departure prevention control if the calculated distance is included in an installation distance range of the zebra zone in the road extension direction from the vehicle. [Aspect 2-7] The control method according to any one of Aspects 2-1 to 2-6, wherein the lane departure prevention control is executed so that the vehicle does not deviate from the boundary (Lbz2) on the oncoming lane (LNc) side of the zebra zone. [Aspect 2-8] The control method according to any one of Aspects 2-1 to 2-7, wherein whether to suppress the execution of the lane departure prevention control is selected depending on the type of the zebra zone or the type of road dividing line (B433) on the own lane side that constitutes the zebra zone.[Aspect 2-9] The control method according to Aspect 1-8, wherein execution of the lane departure prevention control is suppressed if the zebra zone is one for which entry is not prohibited. [Aspect 2-10] The control method according to any one of Aspects 2-1 to 2-9, wherein suppression of execution of the lane departure prevention control is prohibited if it is determined that an obstacle (B2, B15) is present in the zebra zone. [Aspect 2-11] The control method according to any one of Aspects 2-1 to 2-10, wherein suppression of execution of the lane departure prevention control is prohibited if it is determined that the driver (D) of the vehicle is in a waking state.

[0145] [Aspect 3-1] A control program executed by a processor (21) provided in a control device (2) mounted on a vehicle (V), wherein the processing executed by the processor: executes lane departure prevention control based on a recognition result of road conditions ahead of the vehicle; recognizes, as the road conditions, a zebra zone (B461) adjacent to a host lane (LNm) in which the vehicle is currently traveling in the width direction of the road; estimates an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition result of the zebra zone; and suppresses the execution of the lane departure prevention control as the entry probability increases. [Aspect 3-2] The control program according to Aspect 3-1, wherein a notification mode for a vehicle occupant (D) is changed according to the entry probability. [Aspect 3-3] The control program according to Aspect 3-1 or Aspect 3-2, wherein the entry probability is estimated using map information. [Aspect 3-4] The control program according to any one of Aspects 3-1 to 3-3, which estimates the entry probability based on a driving scene of the vehicle. [Aspect 3-5] The control program according to any one of Aspects 3-1 to 3-4, which calculates a departure point (PD) that is a point where the vehicle is estimated to deviate from the own lane, and suppresses execution of the lane departure prevention control if the calculated departure point is included in a boundary (Lbz1) of the zebra zone on the own lane side. [Aspect 3-6] The control program according to Aspect 3-5, which calculates a distance from the vehicle to the departure point in a road extension direction based on a speed and acceleration of the vehicle, and suppresses execution of the lane departure prevention control if the calculated distance is included in an installation distance range of the zebra zone in the road extension direction from the vehicle. [Aspect 3-7] The control program according to any one of Aspects 3-1 to 3-6, wherein the lane departure prevention control is executed so that the vehicle does not deviate from the boundary (Lbz2) on the oncoming lane (LNc) side of the zebra zone.[Aspect 3-8] The control program according to any one of Aspects 3-1 to 3-7, which selects whether to inhibit the execution of the lane departure prevention control depending on the type of the zebra zone or the type of road dividing line (B433) on the host lane side that constitutes the zebra zone. [Aspect 3-9] The control program according to Aspect 3-8, which inhibits the execution of the lane departure prevention control if the zebra zone is a type from which entry is not prohibited. [Aspect 3-10] The control program according to any one of Aspects 3-1 to 3-9, which prohibits the inhibition of the execution of the lane departure prevention control if it is determined that an obstacle (B2, B15) is present in the zebra zone. [Aspect 3-11] The control program according to any one of Aspects 3-1 to 3-10, which prohibits the inhibition of the execution of the lane departure prevention control if it is determined that the driver (D) of the vehicle is in a waking state.

[0146] [Aspect 4-1] A computer-readable, non-transient, tangible recording medium (22) having recorded thereon a control program executed by a processor (21) provided in a control device (2) mounted on a vehicle (V), wherein the processing executed by the processor: executes lane departure prevention control based on a recognition result of road conditions ahead of the vehicle; recognizes, as the road conditions, a zebra zone (B461) adjacent to a host lane (LNm) in which the vehicle is currently traveling in the width direction of the road; estimates an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition result of the zebra zone; and suppresses the execution of the lane departure prevention control as the entry probability increases. [Aspect 4-2] The recording medium according to Aspect 4-1, wherein a notification mode for a vehicle occupant (D) is changed according to the entry probability. [Aspect 4-3] The recording medium according to Aspect 4-1 or Aspect 4-2, wherein the entry probability is estimated using map information. [Aspect 4-4] The recording medium according to any one of Aspects 4-1 to 4-3, wherein the entry probability is estimated based on a driving scene of the vehicle. [Aspect 4-5] The recording medium according to any one of Aspects 4-1 to 4-4, wherein a departure point (PD) is calculated as a point where the vehicle is estimated to deviate from the own lane, and when the calculated departure point is included in a boundary (Lbz1) of the zebra zone on the own lane side, execution of the lane departure prevention control is suppressed. [Aspect 4-6] The recording medium according to Aspect 4-5, wherein a distance from the departure point in a road extension direction from the vehicle is calculated based on a speed and acceleration of the vehicle, and when the calculated distance is included in an installation distance range of the zebra zone in the road extension direction from the vehicle, execution of the lane departure prevention control is suppressed. [Aspect 4-7] The recording medium according to any one of Aspects 4-1 to 4-6, wherein the lane departure prevention control is executed so that the vehicle does not deviate from the boundary (Lbz2) on the oncoming lane (LNc) side of the zebra zone.[Aspect 4-8] The recording medium according to any one of Aspects 4-1 to 4-7, wherein whether or not to inhibit execution of the lane departure prevention control is selected depending on the type of the zebra zone or the type of road dividing line (B433) on the host lane side that constitutes the zebra zone. [Aspect 4-9] The recording medium according to Aspect 4-8, wherein execution of the lane departure prevention control is inhibited if the zebra zone is of a type that does not prohibit entry. [Aspect 4-10] The recording medium according to any one of Aspects 4-1 to 4-9, wherein inhibition of execution of the lane departure prevention control is prohibited if it is determined that an obstacle (B2, B15) is present in the zebra zone. [Aspect 4-11] The recording medium according to any one of Aspects 4-1 to 4-10, wherein inhibition of execution of the lane departure prevention control is prohibited if it is determined that the driver (D) of the vehicle is in a waking state.

[0147] [Viewpoint 5-1] A control method executed by a processor (21) provided in a control device (2) mounted on a host vehicle (V), wherein the processor: calculates a departure point (PD) which is a point where the host vehicle is estimated to deviate from the host lane; determines whether the departure point is within a zebra zone, and if it is determined that the departure point is not within the zebra zone, activates the lane departure prevention control in a normal manner; if it is determined that the departure point is within the zebra zone, determines whether a boundary line of the zebra zone on the host lane side is yellow, and if the boundary line on the host lane side is yellow, activates the lane departure prevention control in a normal manner; if the boundary line on the host lane side is white, determines whether an obstacle is present within the zebra zone, and if it is determined that an obstacle is present within the zebra zone, activates the lane departure prevention control in a normal manner; if it is determined that no obstacle is present within the zebra zone, determines whether the driver is awake or not, and if it is determined that the driver is not awake, activates the lane departure prevention control in a normal manner; A method for estimating an entry probability corresponding to the possibility that the host vehicle will enter the zebra zone when it is determined that the driver is in an awake state, and suppressing the execution of the lane departure prevention control as the entry probability increases. [Aspect 5-2] The method according to Aspect 5-1, in which the processor changes a notification mode for an occupant (D) of the host vehicle according to the entry probability. [Aspect 5-3] The method according to Aspect 5-1 or Aspect 5-2, in which the processor estimates the entry probability using map information. [Aspect 5-4] The method according to any one of Aspects 5-1 to 5-3, in which the processor estimates the entry probability based on a driving scene of the host vehicle. [Viewpoint 5-5] The method according to any one of Viewpoints 5-1 to 5-4, wherein the processor calculates a departure point (PD) that is a point where the host vehicle is estimated to deviate from the host lane, and suppresses execution of the lane departure prevention control if the calculated departure point is included in a boundary (Lbz1) on the host lane side of the zebra zone.[Aspect 5-6] The method of Aspect 5-5, wherein the processor calculates the distance from the host vehicle to the departure point in the road extension direction based on the speed and acceleration of the host vehicle, and suppresses the execution of the lane departure prevention control if the calculated distance is within an installation distance range of the zebra zone from the host vehicle in the road extension direction. [Aspect 5-7] The method of any one of Aspects 5-1 to 5-6, wherein the processor executes the lane departure prevention control so that the host vehicle does not deviate from a boundary (Lbz2) on the oncoming lane (LNc) side of the zebra zone. [Aspect 5-8] The method of any one of Aspects 5-1 to 5-7, wherein the processor selects whether to suppress the execution of the lane departure prevention control depending on the type of the zebra zone or the type of road dividing line (B433) on the host lane side that constitutes the zebra zone. [Aspect 5-9] The method according to Aspect 5-8, wherein the processor inhibits execution of the lane departure prevention control if the zebra zone is one for which entry is not prohibited. [Aspect 5-10] The method according to any one of Aspects 5-1 to 5-9, wherein the processor inhibits inhibition of execution of the lane departure prevention control if it determines that an obstacle (B2, B15) is present in the zebra zone. [Aspect 5-11] The method according to any one of Aspects 5-1 to 5-10, wherein the processor inhibits inhibition of execution of the lane departure prevention control if it determines that the driver (D) of the host vehicle is in a waking state.

Claims

1. A control device (2) mounted on a vehicle (V), comprising: a processor (21); and a memory (22) which is a non-transient tangible recording medium that stores program instructions used by the processor, wherein the processor executes the program instructions stored in the memory to perform lane departure prevention control based on the recognition results of the road conditions ahead of the vehicle, recognizes a zebra zone (B461) adjacent to the vehicle's own lane (LNm), which is the lane in which the vehicle is currently traveling, in the road width direction as the road conditions, and estimates an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition results of the zebra zone, and suppresses the execution of the lane departure prevention control the higher the entry probability.

2. The control device according to claim 1, wherein the processor changes a notification mode for a vehicle occupant (D) according to the entry probability.

3. The control device according to claim 1, wherein the processor estimates the approach probability using map information.

4. The control device according to claim 1, wherein the processor estimates the entry probability based on a driving scene of the vehicle.

5. The control device according to claim 1, wherein the processor calculates a departure point (PD), which is a point where the vehicle is estimated to deviate from the own lane, and suppresses execution of the lane departure prevention control if the calculated departure point is included in the boundary (Lbz1) on the own lane side of the zebra zone.

6. The control device described in claim 5, wherein the processor calculates the distance of the departure point in the direction of extension of the road from the vehicle based on the speed and acceleration of the vehicle, and suppresses execution of the lane departure prevention control if the calculated distance is within an installation distance range of the zebra zone in the direction of extension of the road from the vehicle.

7. The control device according to claim 1, wherein the processor executes the lane departure prevention control so that the vehicle does not deviate from a boundary (Lbz2) on the oncoming lane (LNc) side of the zebra zone.

8. The control device described in claim 1, wherein the processor selects whether or not to suppress execution of the lane departure prevention control depending on the type of the zebra zone or the type of road dividing line (B433) on the vehicle's own lane side that constitutes the zebra zone.

9. The control device according to claim 8, wherein the processor inhibits execution of the lane departure prevention control when the zebra zone is of a type to which entry is not prohibited.

10. The control device according to claim 1, wherein the processor prohibits suppression of execution of the lane departure prevention control when it determines that an obstacle (B2, B15) is present within the zebra zone.

11. The control device according to claim 1, wherein the processor prohibits suppression of execution of the lane departure prevention control when it is determined that the driver (D) of the vehicle is not awake.

12. A control method executed by a processor (21) provided in a control device (2) mounted on a vehicle (V), comprising: executing lane departure prevention control based on the recognition results of road conditions ahead of the vehicle; recognizing, as the road conditions, a zebra zone (B461) adjacent to the vehicle's own lane (LNm), which is the lane in which the vehicle is currently traveling in the road width direction; estimating an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition results of the zebra zone; and suppressing the execution of the lane departure prevention control the higher the entry probability.

13. A control program executed by a processor (21) provided in a control device (2) mounted on a vehicle (V), the processing executed by the processor including: a process of executing lane departure prevention control based on the recognition results of road conditions ahead of the vehicle; a process of recognizing, as the road conditions, a zebra zone (B461) adjacent to the vehicle's own lane (LNm), which is the lane in which the vehicle is currently traveling, in the road width direction, and estimating an entry probability corresponding to the possibility that the vehicle will enter the zebra zone based on the recognition results of the zebra zone; and a process of suppressing the execution of the lane departure prevention control the higher the entry probability.