Vehicle control device and program

Abstract

This vehicle control device includes a processor and a communication circuit. The processor is a module that performs processing relating to detection of another vehicle on the basis of data input from at least one sensor mounted on the vehicle. The communication circuit is an interface for the processor to communicate with a camera that captures images in front of the vehicle. The processor acquires information relating to a merging section on the basis of map data or sensor data. Next, when the vehicle is traveling through a preparatory section located before the merging section, information relating to shielding objects in the direction of the main carriageway is acquired on the basis of the map data or the sensor data. The processor then determines the behavior of the vehicle while traveling through the preparatory section on the basis of the information relating to the shielding objects. For example, if a shielding object exists, the behavior is changed in accordance with the height of the shielding object.

Description

Vehicle control apparatus, program Cross-reference to Related Application

[0001] This application is based on Japanese Patent Application No. 2025-45859 filed in Japan on March 19, 2025, the entire content of which is incorporated herein by reference.

[0002] The present disclosure relates to a technology for detecting another vehicle hidden by a shielding object such as a wall.

[0003] Patent Document 1 discloses a technology in which a vehicle wirelessly communicates with another vehicle or a server to acquire information (for example, position and speed) about other vehicles present around the vehicle.

[0004] US Patent No. 11242051 Specification

[0005] A shielding object such as a fence or vegetation may be provided between a merging lane and a main road. When such a shielding object exists, the apparatus used in a main road vehicle is less likely to detect a potential merging vehicle traveling in a merging preparation section. According to the technology of acquiring dynamic map information from a server, the in-vehicle apparatus can recognize the presence of another vehicle located behind a shielding object such as a wall. However, there is a delay in communication via the server. The communication delay may delay the detection of another vehicle hidden by the shielding object.

[0006] Furthermore, a hidden other vehicle does not necessarily cooperate with a dynamic map distribution server. Regarding another vehicle that does not support communication, the other vehicle can only be detected after it exits the shielding object, that is, enters the field of view of the host vehicle's sensor. The shielding object herein is a three-dimensional object that blocks the field of view of a driver or a sensor, and may be interpreted as, for example, an object having a height of 1 m or more from the road surface.

[0007] When a shielding object is provided between the merging section and the main road, the driver / sensor of the merging vehicle has difficulty recognizing vehicles on the main road side, which may delay the detection of their presence. The driver / sensor of a vehicle traveling on the main road also has difficulty recognizing the merging vehicle, which may delay the detection of its presence. A delay in detecting another vehicle with competing right of way (also referred to as a conflicting vehicle) can cause sudden braking or sudden steering.

[0008] One of the purposes of this disclosure is to provide technology for safer passage through merging sections.

[0009] The vehicle control device disclosed herein comprises a processing unit for performing processing for controlling a vehicle, and a communication circuit for the processing unit to receive sensor data indicating the detection result of at least one sensor for detecting objects present around the vehicle, wherein the processing unit is configured to acquire information about a merging section, which is a section where the vehicle's lane merges with another lane, based on sensor data or map data, and, when the vehicle is traveling in a preparation section located before the merging section, to acquire information about an obstruction present between the vehicle's lane and the other lane based on sensor data or map data, and to determine the vehicle's behavior while traveling in the preparation section based on the information about the obstruction.

[0010] The programs included in this disclosure include instructions to cause at least one processor to perform the following actions: receive sensor data using a communication circuit indicating the detection result of at least one sensor for detecting objects present around a vehicle; acquire information about a merging section, which is a section where the vehicle's lane merges with another lane, based on the sensor data or map data; acquire information about an obstruction present between the vehicle's lane and the other lane, based on the sensor data or map data, when the vehicle is traveling in a preparation section located before the merging section; and determine the behavior of the vehicle while traveling in the preparation section, based on the information about the obstruction.

[0011] According to the above technology, the vehicle travels through the preparation section toward the merging section in a manner that corresponds to the information about the obstacles. This makes it possible to change the state of the vehicle upon entering the merging section depending on the presence or absence of obstacles. In other words, the state of the vehicle traveling through the merging section can be optimized to take into account the road structure immediately preceding it. As a result, the vehicle may be able to travel through the merging section more safely.

[0012] The symbols in parentheses in the claims indicate a correspondence with the specific means described later in the embodiments, and do not limit the technical scope of this disclosure.

[0013] This is a block diagram illustrating an example of a vehicle system. This is a block diagram illustrating the function of a vehicle control device. This is a diagram illustrating the preparation section for a merging section. This is a flowchart illustrating the operation of the processor related to passing through a merging section. This is a diagram illustrating the effect of FOV improvement by separation reduction control. This is a diagram illustrating an example of an image area used for image analysis to detect merging vehicles hidden by obstacles.

[0014] Embodiments of this disclosure will be described below with reference to the drawings. This disclosure is not limited to the embodiments described below. The configurations disclosed below may be implemented with various modifications without departing from the gist of the invention. Various modifications may be combined as appropriate, without causing any technical inconsistencies. This disclosure also includes configurations that are not explicitly stated, which are combinations of multiple modifications. In the following description, components having the same function may be denoted by the same reference numeral and their specific description may be omitted. Also, components having the same function may be denoted by the same or similar names and their specific description may be omitted. If only a part of a configuration is referred to, the description of the other parts may be applied elsewhere.

[0015] <Introduction> Figure 1 is a diagram illustrating an example of the schematic configuration of the vehicle system VS related to this disclosure. The vehicle system VS is a system installed in a vehicle Hv. Hereafter, the vehicle Hv on which the vehicle system VS is installed may also be referred to as the vehicle itself. In this disclosure, the term "ego lane" refers to the lane in which the vehicle is traveling, among the multiple lanes provided on the road. The ego lane can also be called the vehicle's lane. An adjacent lane is a lane adjacent to the ego lane.

[0016] In this disclosure, "leading vehicle" basically means a vehicle that is in front of vehicle Hv, traveling in the same lane as vehicle Hv, and is the closest vehicle to vehicle Hv. "Following vehicle" means another vehicle traveling behind vehicle Hv in the same lane. The term "forward vehicle" means another vehicle traveling in front of the vehicle itself. Forward vehicles include not only the leading vehicle but also vehicles traveling diagonally in front. Similarly, "rear vehicle" includes not only the following vehicle but also vehicles traveling diagonally behind vehicle Hv. "Parallel vehicle" is another vehicle traveling alongside vehicle Hv. A parallel vehicle may be another vehicle located to the side (right or left) of vehicle Hv, with a speed difference of within a predetermined value (e.g., 5 km / h) of vehicle Hv. There may be a wall, block, median strip, etc. between vehicle Hv and the parallel vehicle.

[0017] The term "merging vehicle" refers to a vehicle traveling on a merging lane. A merging lane is a lane that merges with the main lane of an expressway. An expressway, in this context, is a road where pedestrians and bicycles are prohibited from entering, such as toll roads like expressways. A merging lane can generally be understood as a lane that disappears after the merging section.

[0018] A competing vehicle refers to any other vehicle that could potentially influence the behavior of vehicle Hv. A competing vehicle is one that could potentially cause contact or an excessively close approach (near-crash). When vehicle Hv is traveling on the main lane of an expressway, any other vehicle that is positioned and traveling at a speed that could potentially enter the merging section in front of or behind vehicle Hv may be considered a competing vehicle. A competing vehicle could be a vehicle positioned to the side of vehicle Hv on the merging lane. Furthermore, when vehicle Hv is traveling on a merging lane, any other vehicle traveling on the main lane to which the merging lane connects (hereinafter referred to as the merging lane) may also be considered a competing vehicle. A competing vehicle on the main lane is any other vehicle that reaches the merging section at the same time as vehicle Hv. For example, a vehicle traveling alongside vehicle Hv on the merging lane could be considered a competing vehicle.

[0019] A non-competing vehicle refers to another vehicle that is unlikely to affect the behavior of vehicle Hv, or has a very low probability of doing so. In other words, a non-competing vehicle is any other vehicle that is unlikely to come into contact with or excessively close to vehicle Hv. When vehicle Hv is traveling on the main lane of an expressway, a merging vehicle that is in a position and at a speed that does not create a conflict of right of way with vehicle Hv may be a non-competing vehicle. Also, when vehicle Hv is traveling in a merging lane, other vehicles traveling a predetermined distance ahead or behind vehicle Hv in the merging lane, and other vehicles traveling in main lanes other than the merging lane, may be non-competing vehicles. That is, other merging vehicles traveling a predetermined distance or more ahead or behind a competing vehicle may be non-competing vehicles.

[0020] In this disclosure, "driver" means a person seated in the driver's seat, i.e., a driver's seat occupant, regardless of whether they are actually performing driving operations. In one scenario, the driver may be understood as the person who should receive the authority and responsibility for driving operations from the vehicle system VS at the end of autonomous driving. The term "driver" may be replaced with "driver's seat occupant" or "vehicle user." The vehicle Hv may be a remotely operated vehicle remotely controlled by an operator located outside the vehicle. The person who takes over driving operations from the vehicle system VS may be an operator located outside the vehicle. The operator is a person who has the authority to control the vehicle remotely from outside the vehicle. The operator may also be included in the concept of a driver.

[0021] Vehicle Hv may be a vehicle equipped with so-called autonomous driving functions. Autonomous driving functions are functions that allow the vehicle to autonomously drive along a predetermined route. The degree of automation of driving operations (hereinafter referred to as the automation level) can be divided into multiple levels, as defined by the Society of Automotive Engineers (SAE International). The automation level can be divided into six stages, for example, levels 0 to 5. In the explanation of the automation level, the system mainly refers to the vehicle control device 30.

[0022] Level 0 is equivalent to manual driving, where the driver performs all driving tasks without system intervention. Driving tasks include steering, acceleration / deceleration, and surrounding area monitoring. Level 1 is a level where the system supports either steering or acceleration / deceleration. Level 2 is a level where the system performs subtasks of both longitudinal and lateral vehicle motion control in a limited area. Level 2 may also include system-performed speed control and steering control.

[0023] Level 3 means that the system performs all driving tasks within the Operational Design Domain (ODD), but in emergencies, control is transferred from the system to the driver. The ODD defines the conditions under which autonomous driving is possible. Level 4 is the level in which the system performs all driving tasks except in specific situations such as certain roads that are unsuitable or extreme environments. Level 5 is the level in which the system performs all driving tasks in all environments.

[0024] Automation levels 3 through 5 are categorized as autonomous driving. The longitudinal and lateral control of dynamic driving tasks by the system at automation levels 2 and above will also be referred to as autonomous driving control below. Autonomous driving control includes vehicle control and autonomous driving equivalent to level 2.

[0025] The vehicle Hv (in other words, the vehicle system VS) described below is equipped with, as an example, Level 3 equivalent autonomous driving functionality. Of course, the vehicle Hv may also be equipped with vehicle control functions up to Level 2. The vehicle Hv may also be equipped with Level 4 or higher autonomous driving functionality. The configuration of the vehicle system VS disclosed below may be modified as appropriate to conform to the laws and customs of the region in which the vehicle system VS is used, the characteristics of the vehicle on which it is installed / the installed equipment, etc.

[0026] <Overall Configuration of the Vehicle System> The vehicle system VS includes, as an example, multiple devices as shown in Figure 1. Specifically, the vehicle system VS includes an environmental sensor 11, a vehicle state sensor 12, a locator 13, a map storage unit 14, a wireless communication device 15, an occupant state sensor 16, a display 17, and a speaker 18. The vehicle system VS also includes an operating member 21, a motion actuator 22, an auxiliary actuator 23, and a vehicle control device 30. The term "device" may include sensors and circuits. Some devices may be configured as subsystems; that is, devices may be subsystems.

[0027] The vehicle control device 30 is interconnected with some of the above-mentioned devices via the in-vehicle network 10, enabling mutual communication. The in-vehicle network 10 is a communication network established within the vehicle. The in-vehicle network 10 includes multiple communication lines. Some devices may be directly connected to the vehicle control device 30 by cables (e.g., wire harnesses). Figure 1 illustrates a case where the operating member 21, motion actuator 22, and auxiliary actuator 23 are connected to the vehicle control device 30 by cables, while other devices are connected to the vehicle control device 30 via the in-vehicle network. The connection configurations between devices may be modified as appropriate.

[0028] The environmental sensor 11 is a device that senses the surrounding environment of the vehicle Hv. The environmental sensor 11 detects targets present in the external environment of the vehicle Hv. The environmental sensor 11 includes multiple sensors. The environmental sensor 11 includes multiple cameras. Cameras may be referred to as image sensors. The multiple cameras include a forward camera 111 that photographs the area in front of the vehicle. There may be only one forward camera 111, but here we assume that there are multiple. At least one of the multiple forward cameras 111 is a visible light camera. One of the multiple forward cameras 111 may be a Time of Flight (ToF) camera or an infrared camera. A ToF camera is a camera that generates distance images.

[0029] The environmental sensor 11 may include, in addition to a camera, at least one, preferably more than one, of LiDAR, radar, ultrasonic sonar, and acoustic sensors. LiDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging. Multiple types of environmental sensors 24 may be combined and implemented on the vehicle Hv to monitor the front, right, left, and rear directions of the vehicle. The environmental sensor 11 in this embodiment includes a left sensor that detects objects on the left side (such as other vehicles) and a right sensor that detects objects on the right side (such as other vehicles). The left sensor and the right sensor together are also referred to as the lateral sensor 112. In this embodiment, the lateral sensor 112 is a camera. In other embodiments, the lateral sensor 112 may be radar or LiDAR.

[0030] The objects detected by the environmental sensor 11 may include both dynamic and static objects. Dynamic objects refer to so-called moving objects such as other vehicles, pedestrians, and cyclists. The terms dynamic objects, moving objects, road users, and traffic participants may be interchangeable. Static objects are features such as road edges, road markings, and structures installed along the road. Road markings may include at least one of lane marks indicating lane boundaries, pedestrian crossings, stop lines, traffic guides, and regulatory arrows. Structures installed along the road may include at least one of road signs, guardrails, traffic lights, utility poles, curbs, walls, fences, and commercial signs. Trees may also be detected as static objects.

[0031] The front camera 111 may also be configured to detect the illumination status of other vehicles' lighting devices, such as hazard lights, brake lights, and turn signals. The front camera 111 may also be configured to detect the illumination status of traffic lights. The environmental sensor 11, including the front camera 111, inputs a data signal indicating the detection result to the vehicle control device 30. The output signal of the environmental sensor 11 may be any signal that allows the vehicle control device 30 to identify the driving environment, and may be a data signal before object recognition processing, such as a video signal. The object recognition function may be provided in the vehicle control device 30 instead of the sensor.

[0032] The vehicle state sensor 12 is a sensor that detects the vehicle state. The vehicle state here may be the internal state of the vehicle Hv. The vehicle state sensor 12 includes multiple types of sensors in the vehicle Hv. The vehicle state sensor 12 may include at least one, more preferably more, of the following: an acceleration sensor, a jerk sensor, a vehicle speed sensor, and a yaw rate sensor. The vehicle state sensor 12 may also include at least one, more preferably more, sensors that detect the operation state of the control devices by the driver, such as a steering angle sensor, a brake sensor, and an accelerator sensor. The accelerator sensor is a sensor that detects the amount / force of depression of the accelerator pedal. The brake sensor is a sensor that detects the amount / force of depression of the brake pedal. The vehicle state sensor 12 inputs a data signal indicating the detection result to the vehicle control device 30 via the in-vehicle network 10.

[0033] The locator 13 is a device that generates and outputs position information of the vehicle Hv (more precisely, the locator 13) using navigation signals transmitted from positioning satellites that constitute the GNSS (Global Navigation Satellite System). The locator 13 may include an inertial sensor in addition to the GNSS receiver. The locator 13 may determine the position and direction of travel of the vehicle by combining the navigation signals received by the GNSS receiver, the measurement results of the inertial sensor, and the vehicle speed information flowing through the in-vehicle network 10. In this disclosure, the data indicating the position of the vehicle generated and output by the locator 13 is also referred to as vehicle position data. Vehicle position data may also be understood as an example of sensor data. The locator 13 outputs the vehicle position data to the vehicle control device 30.

[0034] The map storage unit 14 is a storage device in which map data is stored. The map data held by the map storage unit 14 may be so-called HD (High Definition) map data. The map data stored in the map storage unit 14 includes data such as the three-dimensional shape of roads, the installation locations of road markings (e.g., lane marks), and the installation locations of traffic signs, with the accuracy necessary for autonomous driving. The map storage unit 14 may also store a navigation map showing the road connections. The navigation map may be used to search for a route from the current location to a destination. The map data stored in the map storage unit 14 may be updated by data received by the wireless communication device 15 from a map server or the like. The map storage unit 14 may also be a storage device for temporarily holding map data received by the wireless communication device 15 from a map server until the data expires. The map storage unit 14 may be built into the vehicle control device 30.

[0035] The wireless communication device 15 is a device for the vehicle to communicate wirelessly with an external device. The external device may include at least one of the following: another vehicle, a server, a traffic information center, a roadside unit, and a portable device. The portable device may be, for example, a smartphone. The wireless communication device 15 is configured to perform cellular communication. Cellular communication refers to wireless communication compliant with LTE (Long Term Evolution), 4G, or 5G, etc.

[0036] The occupant status sensor 16 is a sensor that detects the driver's status. The occupant status sensor 16 may be, for example, a driver status monitor (hereinafter referred to as DSM). The DSM is a sensor that detects the driver's face orientation, eye position, gaze direction, and degree of eyelid opening, etc., based on the driver's face image. Eye position is the position of the eyes and may be expressed as height from the road surface, etc. The occupant status sensor 16 transmits driver status data indicating the driver's face orientation, eye position, gaze direction, degree of eyelid opening, etc., to the vehicle control device 30. Driver status data may also be understood as a type of sensor data.

[0037] The display 17 is a device that displays images. The display 17 may be a liquid crystal display or an organic EL display. The display 17 is located on the instrument panel. The display 17 displays images corresponding to signals input from the vehicle control device 30. Images may be referred to as visuals. The display 17 may also be a so-called head-up display that displays images by projecting image light onto a predetermined area of ​​the windshield.

[0038] The speaker 18 is a device that outputs sound corresponding to a signal input from the vehicle control device 30. The term "sound" in this disclosure includes notification sounds, voices, music, etc. The display 17 and speaker 18 are devices for informing the driver of information (hereinafter also referred to as notification devices). The vehicle system VS may also include a vibrator, ambient light, etc., as notification devices.

[0039] The operating member 21 is a component for the driver to input instructions to the vehicle system VS. The operating member 21 includes components for acceleration, deceleration, and steering, namely the accelerator pedal, brake pedal, steering wheel, and shift lever. The operating member 21 may also include a turn signal switch, a light switch, and a mode switch.

[0040] The turn signal switch is a switch for illuminating the turn signal lamps as direction indicators. The turn signal switch also functions as an approval switch for the vehicle's user (e.g., the driver) to approve a lane change. The light switch is a switch for switching the headlight illumination state, i.e., automatic, off, on, high beam, and low beam. The mode switch is a switch for switching the operation mode (in other words, the automation level), which will be described later. Some of the switches may be switches located on the spokes of the steering wheel (so-called steering switches). In addition, the operating member 21 may include a touch panel stacked on the display 17.

[0041] The operating member 21 outputs an operating signal, which is an electrical signal corresponding to the driver's operation, to the vehicle control device 30. The operating signal includes information indicating the driver's operation. Some of the operating members 21 may be connected to the vehicle control device 30 via the in-vehicle network 10 or other devices.

[0042] The motion actuator 22 is an actuator that generates power corresponding to any one of acceleration, deceleration, and steering of the host vehicle. The term actuator may also be rephrased as equipment, system, subsystem, or mechanism. The motion actuator 22 includes a drive mechanism related to propulsion of the vehicle. The drive mechanism may be a powertrain, and includes at least one of an engine and a drive motor. In other words, the drive mechanism may be an engine, an EV system, or a hybrid system. Further, the motion actuator 22 includes a brake actuator as a braking mechanism and a steering actuator as a steering mechanism. The brake actuator may include at least one of a hydraulic brake and a regenerative brake. The steering actuator may be an EPS (Electric Power Steering) motor.

[0043] The motion actuator 22 operates based on a control signal input from a vehicle control device 30, and controls the motion of the vehicle Hv. Another ECU such as a steering ECU that performs steering control, a power unit control ECU that controls a drive source, or a brake ECU may be interposed between the vehicle control device 30 and the motion actuator 22. Since the motion actuator 22 is an actuator that directly affects the movement of the vehicle Hv, it is also referred to as a primary actuator in the present disclosure.

[0044] The auxiliary actuator 23 is an on-vehicle equipment that does not directly affect the movement of the vehicle Hv, but enables safe and legal driving. The auxiliary actuator 23 includes a plurality of lighting devices, a horn, a defogging device for a windshield / camera, a cleaning device for the environmental sensor 11, a wiper motor for a windshield, a wiper motor for a rear glass, and the like. The lighting devices include a headlight, a fog lamp, a brake lamp, a turn signal lamp, a clearance lamp, a hazard lamp, a backup lamp, and the like.

[0045] The vehicle control device 30 is a device that executes part or all of a driving operation on behalf of the driver by controlling the motion actuator 22 and the auxiliary actuator 23 based on the detection results of the environmental sensor 11. The vehicle control device 30 may be an automated driving system (ADS). The vehicle control device 30 may be implemented in the form of a device (that is, an automatic driving device). Descriptions of system and device may be mutually replaced as appropriate.

[0046] The vehicle control device 30 may be implemented using one or more computers. The one or more computers constituting the vehicle control device 30 correspond to the vehicle control device. The vehicle control device 30 includes a processor 31, a memory 32, a storage 33, a communication circuit 34, and a bus connecting these components. The processor 31 may be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The vehicle control device 30 may include a plurality of processors 31, and one of the plurality of processors 31 may be an arithmetic core optimized for processing using a neural network (a so-called NPU: Neural Processing Unit). The processor 31 corresponds to a processing unit.

[0047] The memory 32 is a rewritable volatile storage medium. The memory 32 is, for example, a RAM (Random Access Memory). The storage 33 is a rewritable non-volatile storage medium such as a flash memory. A vehicle control program, which is a program executed by the processor 31, is stored in the storage 33. Execution of the vehicle control program by the processor 31 corresponds to execution of a vehicle control method.

[0048] Furthermore, the storage 33 stores vehicle setting data indicating the mounting positions of the front camera 111 and the side sensors 112 in the vehicle Hv. When the vehicle control device 30 is started up, the vehicle setting data may be read from the storage 33 and stored in the memory 32. The memory area where the vehicle setting data is stored corresponds to the viewpoint information storage unit.

[0049] The communication circuit 34 is hardware for the processor 31 to communicate with other devices that constitute the vehicle system VS, such as the environmental sensor 11. The communication circuit 34 may include a circuit that is compatible with the communication method with the other devices. The communication circuit 34 may also be an input / output circuit or an input / output port. The communication circuit 34 corresponds to the communication circuit. The communication circuit 34 may support any type of wired or wireless communication. Part or all of the wireless communication device 15 may be included in the communication circuit 34. Digital data corresponding to signals received by the communication circuit 34 may be temporarily stored in the memory 32. The data stored in the memory 32 is referenced by the processor 31 as appropriate.

[0050] The communication circuit 34 receives information necessary for implementing vehicle control, such as autonomous driving or driver assistance. Reception can be rephrased as acquisition. The communication circuit 34 acquires sensor data (i.e., detection results) from the environmental sensor 11. The sensor data includes data about objects present around the vehicle, such as moving objects, features, and obstacles. The data on detected objects may include the position, speed of movement, and type or size of the detected object.

[0051] Sensor data related to features may include lane mark data and road edge data. Lane mark data may include not only position data but also line type data. Line type can be represented as a continuous line (solid line) or a dashed line. The communication circuit 34 may include camera images in the sensor data. The communication circuit 34 also receives video data from the front camera 111.

[0052] Furthermore, the communication circuit 34 acquires sensor data related to the vehicle's state, such as the vehicle Hv's speed, acceleration, yaw rate, and external illumination, from the vehicle state sensor 12. In addition, the communication circuit 34 acquires its own vehicle position data from the locator 13. The communication circuit 34 acquires map data of the road section related to the vehicle Hv's travel by referring to the map storage unit 14. The communication circuit 34 may acquire data transmitted from external devices in cooperation with the wireless communication device 15. For example, the communication circuit 34 may acquire vehicle data transmitted from a vehicle ahead via vehicle-to-vehicle communication. The vehicle data may be a dataset including speed and the operating status of brakes and turn signals.

[0053] The communication circuit 34 also acquires information indicating the driver's operation of the vehicle system VS based on the signal from the operating member 21. The processor 31 accepts brake and accelerator operations and driving operations based on the signal from the operating member 21 received via the communication circuit 34. The processor 31 also accepts instructions regarding changes in the operating mode via the operating member 21. When the operating mode is the Level 2 mode described later, the vehicle control device 30 may acquire the driver's instructions regarding acceptance or rejection of automatic lane changes from the operating member 21 (e.g., a turn signal switch) via the operating member 21. The vehicle control device 30 may also be configured to acquire various driver instructions by voice recognition. A device related to voice input, such as a microphone, can also be included in the operating member 21.

[0054] The communication circuit 34 may acquire information indicating the operating status of devices connected to the vehicle control device 30, such as whether the environmental sensor 11 is functioning correctly. The communication circuit 34 acquires driver status data from the occupant status sensor 16, indicating eye position and gaze direction.

[0055] The various data acquired sequentially by the communication circuit 34 are stored in a temporary storage medium such as memory 32 and used by the environment recognition unit F1, mode management unit F2, etc. Data acquired after a certain period of time may be discarded. Various data (also called information) may be acquired by generation, conversion, determination, or calculation based on signals received from other devices. The communication circuit 34 or processor 31 may have a function to generate other data (also called secondary data) based on raw data (also called secondary data) received from other devices.

[0056] <Operating Modes of the Vehicle Control Device> The vehicle control device 30 has multiple operating modes with different levels of automation. Each operating mode has a different range of driving tasks that the driver is responsible for, in other words, a different range of driving tasks in which the vehicle control device 30 intervenes. The term "operating mode" can be rephrased as "driving mode." Here, as an example, the vehicle control device 30 is configured to be switchable between multiple operating modes, including manual driving mode, level 2 mode, and level 3 mode.

[0057] The manual driving mode is an operating mode in which the driver performs all driving tasks. The Level 2 mode is an operating mode corresponding to automation level 2.0, in which the vehicle control device 30 substantially controls the behavior of the vehicle Hv. Here, the Level 2 mode may be an operating mode in which the driver's hands-off is permitted, or it may be an operating mode in which the driver's hands-on is required. In this disclosure, hands-on means holding the steering wheel. Hands-off means taking one's hands off the steering wheel. In Level 2 mode, the driver's eyes-on is required. Eyes-on means monitoring the area outside the vehicle (primarily forward) related to the direction of movement of the vehicle Hv.

[0058] Level 3 mode is an operating mode that performs vehicle control equivalent to automation level 3. Level 3 mode is an operating mode in which eyes-off is permitted to the extent that the driver can quickly return to driving operation in response to a request from the system. Eyes-off means the act of taking one's eyes off the area outside the vehicle that is related to the direction of movement of the vehicle Hv. In this disclosure, unless otherwise noted (basically), the term automated driving means automated driving control of level 3 or higher. Hereinafter, the term automated / autonomous driving may be abbreviated as AD. In other embodiments, the vehicle control device 30 may have an operating mode equivalent to level 4 or 5. In this disclosure, operating modes of level 3 or higher may also be referred to as automated driving mode, i.e., AD mode. Modes that perform control of level 2 or higher are referred to as automatic operation modes. Automatic operation modes include automated driving modes.

[0059] While in AD mode, the processor 31 automatically performs steering, acceleration, deceleration (in other words, braking) of the vehicle so that the vehicle Hv travels along the planned route set by the driver toward the destination. Even if no destination is set, the vehicle control device 30 may select a route to continue driving / circling within the range that satisfies the ODD and continue autonomous driving. The conditions for determining whether autonomous driving is possible or not, in other words, the detailed conditions that define the ODD, can be changed as appropriate. AD mode is terminated due to steering / pedal operation by the driver (so-called override), system limits, exit from the ODD, etc. The vehicle control device 30 may have a function to determine whether or not the vehicle Hv is within the ODD.

[0060] The processor 31 performs control to enable the vehicle to operate autonomously, even while in Level 2 mode. That is, the vehicle control device 30 performs recognition of the driving environment, planning of the driving trajectory, and motion control, even while in Level 2 mode. Motion control includes speed adjustment through acceleration and deceleration, and steering control. In order to promptly initiate autonomous driving or driver assistance in response to a request from the driver, the vehicle control device 30 may continue to perform recognition processing of the driving environment in the background (in other words, potentially) even in manual driving mode.

[0061] <Functions of the Vehicle Control Device> The vehicle control device 30 includes the functional units shown in Figure 2, which are realized by executing an automatic driving program. Specifically, the vehicle control device 30 has an environment recognition unit F1, a mode management unit F2, a planning unit F3, a vehicle control unit F4, and a vehicle control unit F4.

[0062] The environmental recognition unit F1 recognizes the driving environment of the vehicle Hv based on sensor data and map data acquired by the communication circuit 34. The sensor data may include at least one of the detection results of the environmental sensor 11 and the data received by the wireless communication device 15. The environmental recognition unit F1 may also recognize the driving environment of the vehicle Hv by performing sensor fusion processing that integrates the detection results of multiple environmental sensors 11.

[0063] The driving environment may include at least one of weather and road surface conditions. The weather and road surface conditions may be determined by combining the recognition results of the front camera 111 with weather information acquired by the communication circuit 34. The driving environment includes the positions and types of objects present around the vehicle Hv. The environment recognition unit F1 may acquire the speed and direction of movement of detected moving objects. Based on the various data acquired by the communication circuit 34, the environment recognition unit F1 recognizes the position and behavior of other vehicles.

[0064] The environmental recognition unit F1 acquires information regarding the structure (in other words, configuration) of the road within a predetermined distance in front of the vehicle Hv, based on at least one of sensor data and map data. The road structure may include the location of three-dimensional objects, the number of lanes, the location of lane markers, the location of the road edge, the road width, and the curvature of the road. The location of three-dimensional objects refers to the locations of walls, poles, blocks, guardrails, traffic signs, billboards, etc. Road markings may also be included in the road structure. The location of merging sections is also included in the road structure.

[0065] The environmental recognition unit F1 identifies the attributes (also called classifications) of the road on which the vehicle Hv is traveling, based on sensor data or map data acquired by the communication circuit 34. For example, the environmental recognition unit F1 acquires whether the road is a general road, a main line of an expressway, or an auxiliary road. The road is the road on which the vehicle Hv is traveling. An auxiliary road refers to roads connecting general roads and main lines, roads connecting two expressways, roads connecting main lines and parking areas, etc. An auxiliary road also includes merging roads that lead to the main line. The driving environment may include the attributes of these roads.

[0066] Furthermore, the driving environment may include the ego lane number. The ego lane number is a number indicating the location of the ego lane on the road, and is determined relative to the left edge of the road. The ego lane number directly or indirectly represents the number of lanes to the left or right of the ego lane. The environment recognition unit F1 may identify the ego lane number using the distance from the edge of the road to the vehicle Hv, the number of lane marks detected on the left and right, and at least one of the map data. The ego lane number may also be identified using map data and vehicle position data.

[0067] The environmental recognition unit F1 also acquires information about the merging section that vehicle Hv is scheduled to pass through next, if vehicle Hv is traveling on the main road. Information about the merging section may be acquired based on map data or identified based on sensor data from the environmental sensor 11. Information about the merging section may include at least one of the merging start point, the merging end point, and the start point of the warning section. The merging start point is the start point of the merging section. The merging end point is the end point of the merging section. Information about the merging section may include information about the lane being merged into, which is a lane that flows in the same direction as the merging lane. Information about the lane being merged indicates whether the lane being merged into is the rightmost lane or the leftmost lane. The software / hardware module responsible for processing the recognition of the merging section corresponds to the merging section recognition unit F11. The merging section will be described separately later.

[0068] The environmental recognition unit F1 may acquire data indicating the external environment related to ODD, in addition to the information described above. The environmental recognition unit F1 may also acquire traffic rules around the vehicle Hv based on sensor data or map data. Traffic rules may include speed limits and restrictions on lane changes. The environmental recognition unit F1 may generate an environmental model, which is a three-dimensional model that reproduces (represents) the driving environment of the vehicle Hv, as data indicating the external environment. The environmental model may also be called a world model. The environmental model may be a model in which objects detected by the environmental sensor 11, such as moving objects such as other vehicles, lane markers, road edges, and traffic lights, are placed in a three-dimensional space based on the vehicle Hv. The environmental recognition unit F1 may be understood as having a configuration that manages data related to the driving environment. Data management here may include data acquisition (generation) and updating.

[0069] In addition, the environmental recognition unit F1 may acquire information indicating the in-vehicle environment, such as in-vehicle temperature and driver status data. Specifically, it may acquire the driver's eye position, etc. Furthermore, the environmental recognition unit F1 acquires information on the mounting position of the front camera 111 in the vehicle Hv. The mounting position information of the front camera 111 may be registered in the storage 33.

[0070] The mode management unit F2 manages the operating mode of the vehicle control device 30 based on the information acquired by the communication circuit 34. The management of the operating mode may include the switching between manual driving and automatic driving, that is, the transfer of authority between the user and the vehicle control device 30, in other words, the management of the handover of driving (takeover). Furthermore, the management of the operating mode may include the transition from manual driving mode to Level 2 mode and vice versa, and the transition from Level 2 mode to AD mode and vice versa.

[0071] Mode management corresponds to automation level management. The mode management unit F2 estimates the operating mode / automation level intended by the driver based on the operation signal input from the operating member 21. Based on the estimation result, the mode management unit F2 switches the operating mode. For convenience in this disclosure, the currently applied operating mode is also referred to as the current mode.

[0072] The planning unit F3 is configured to create a driving plan based on data of the driving environment (e.g., an environmental model) managed by the environment recognition unit F1. While in AD mode or Level 2 mode, the planning unit F3 generates driving plan data for autonomous driving based on the results of the environment recognition unit F1's recognition of the driving environment. The driving plan may be called a control plan or a driving plan.

[0073] The driving plan data may include route data, track data, and motion plan data. Route data is data that shows the overall (long-term) driving plan, such as the route to the destination. The planning unit F3 may generate route data based on map data that shows the road connections, such as navigation map data.

[0074] Track data is data that shows the relatively local (short-term) driving trajectory. The track plan may include data such as the lane in which the vehicle Hv is traveling, the position of the vehicle Hv within the lane (so-called lateral position), and lane change points. The processor 31 is configured to perform lateral position adjustments according to the situation. Generating track data may also be called track planning or path planning.

[0075] The planning unit F3 may generate track data based on route data and driving environment data. Motion plan data is data that shows the target speed, steering angle, acceleration, etc. for each time point. Motion plan data may be generated based on track data. Thus, the driving plan data may include schedule information for acceleration and deceleration for speed adjustment along the set route / track, and schedule information for steering amount. The driving plan created by the planning unit F3 is input to the vehicle control unit F4.

[0076] The above driving plan may be created based on map data. Automatic driving control technology based on map data may also be called NOA (Navigate on Autopilot). The planning unit F3 may be an implementation of NOA. The planning unit F3 may create or modify the control plan using the detection results of the environmental sensor 11 instead of or in addition to the map data.

[0077] The planning unit F3 generates not only control plans directly related to the vehicle's operation, but also plans related to notifications to the driver using notification devices such as the display 17. For example, the planning unit F3 plans the timing for issuing warnings / requests to the driver, such as behavior warnings, mode change notifications, and TOR (takeover request). A behavior warning is a process that warns of planned vehicle behaviors such as lane changes, overtaking, and deceleration. A mode change notification is a process that notifies the driver that the operating mode will be changed, or that the operating mode will be changed. A TOR is a request made by the vehicle control device 30 to the driver to take over driving operations. A TOR can be rephrased as an intervention request or a takeover request.

[0078] Various notifications, including announcements, proposals, reports, and requests, may include displaying an icon image or text on the display 17, depending on their content. Depending on their importance and urgency, various notifications may also include some or all of the following: outputting a notification sound, outputting a voice message, flashing ambient lights, and vibrating a vibrator. The process for informing the driver of the information may involve generating a signal to drive the notification device and outputting it to the notification device. The planning unit F3 creates notification plan data indicating the content of the notification and the timing of the notification, and provides it to the vehicle control unit F4.

[0079] The vehicle control unit F4 generates control commands for the motion actuator 22 based on the control plan formulated by the planning unit F3. The vehicle control unit F4 then outputs the generated control commands to the motion actuator 22. In addition, the vehicle control unit F4 controls the illumination status of turn signals, headlights, hazard lights, etc., according to the driving plan and driving environment, based on the plan from the planning unit F3 and the external environment. In other words, the vehicle control unit F4 also controls the drive state of the auxiliary actuator 23.

[0080] The vehicle control unit F4 also provides notifications to the driver using notification devices such as the display 17 and the speaker 18. Various notifications are provided by displaying an image on the display 17 and at least one of the following: an audio message or notification sound output from the speaker 18. The notification sound may be a warning sound. Notifications to the driver may also be accompanied by the illumination of ambient lights or the activation of a vibrator. The vehicle control unit F4 executes various notifications based on the plan of the planning unit F3. In other words, the vehicle control unit F4 executes notifications to the driver based on requests from the planning unit F3.

[0081] <Merging Section> Figure 3 shows an example of a road configuration near a merging section. The example shown in Figure 3 has a road structure in which a merging lane connects from the left to a main road with two lanes. L11 refers to the first lane of the main road, and L12 refers to the second lane. L2 is the merging lane provided by the merging route. In Figure 3, the first lane L11 corresponds to the lane being merged. The lane being merged is a lane that is directly connected to the merging lane among the lanes that make up the main road. The lane being merged is a lane that continues to exist beyond the merging section, while the merging lane is a lane that disappears beyond the merging section. In this disclosure, among the lanes provided by the main road, lanes other than the lanes being merged are also called non-merging lanes. In the example shown in Figure 3, the second lane L12 is a non-merging lane. The term "merging lane" may be replaced with "merging route".

[0082] Figure 3 shows vehicle Hv traveling in merging lane L2. In Figure 3, Cv is a competing vehicle on the main lane. Competing vehicle Cv is not limited to being directly beside vehicle Hv; it may be located diagonally in front of or behind vehicle Hv.

[0083] In the diagram, 301 refers to the lane mark defining the left boundary of the first lane L11. 302 represents the lane mark defining the right boundary of the first lane L11. The lane mark indicated by 302 is also the left boundary of the second lane L12. 303 represents the lane mark indicating the right boundary of the second lane L12. 304 is the median strip, and an oncoming lane may exist to its right (not shown). 311 represents the lane mark indicating the right boundary of the merging lane. 312 represents the lane mark indicating the left boundary of the merging lane. 313 represents a structure located behind the merging lane as viewed from the main road (hereinafter referred to as the "rear structure"). The rear structure 313 may be a fence or wall provided along the merging lane.

[0084] 305 represents a transition restrictor. Transition restrictor 305 is a low-profile object having a height that does not obstruct the driver's view. Transition restrictor 305 may be, for example, a pole, a block, or a plant. Transition restrictor 305 may be configured to physically restrict merging vehicles from moving from the merging lane L2 to the main road, while still allowing drivers of vehicles traveling on the main road to recognize the presence of merging vehicles.

[0085] 306 is a barrier. The barrier 306 is a three-dimensional structure with a height of 1 m or more. The barrier 306 is, for example, a wall or a fence. The barrier 306 may be partially provided along the main line for purposes such as noise reduction in residential areas, preventing vehicles from veering off the road, prohibiting pedestrians or animals from entering, or other purposes. The barrier 306 is not provided in the merging section, but may be partially formed along the warning section. The barrier 306 provided along the warning section partially or completely obstructs the view from the driver of a vehicle traveling in the merging lane L2 to the first lane L11, which is the lane being merged. That is, the barrier 306 acts to partially or completely conceal the competing vehicle Cv from the driver of vehicle Hv or the environmental sensor 11. From another perspective, the barrier 306 provided along the warning section largely or completely obstructs the view from the driver of a main line vehicle to the merging lane L2.

[0086] The merging section 101 is a section of road where vehicles can enter the main road from the merging lane L2. The merging section 101 may be a section where, for example, the lane markings 310 of the first lane L11 in the merging direction are in a pattern that allows for vehicle transitions (for example, a dashed line pattern). The merging start point 102 is a point where merging vehicles can move, both physically and from the standpoint of regulatory markings. The merging start point 102 may be a point where there are no three-dimensional objects separating the merging lane L2 and the first lane L11, and where the prohibition of transition / crossing by road markings is lifted. In one phase, the merging start point 102 may be understood as the point where the pattern of the lane markings on the left side of the first lane L11 switches from a continuous line to a dashed line. The merging start point may also be called the merging start line. The merging start point 102 corresponds to the end point of the caution section 104 and the preparation section 201 described below.

[0087] The merging end point 103 is the point where the merging lane L2 itself disappears. The merging end point 103 may be the point in a single phase where the left boundary line of the first lane L11 changes from a dashed line to a continuous line. The merging end point 103 may also be the point where the width of the merging lane defined by lane marks 310 and 311 becomes less than a predetermined value (for example, 1.5 m). The merging end point 103 may also be referred to as the merging end line. The road section from the merging start point 102 to the merging end point 103 corresponds to the merging section 101.

[0088] The warning section 104 is a section of road located before the merging section on the main line (especially the lane being merged). The warning section 104 may be a section of road on the main line where the remaining distance to the merging start point 102 is less than a predetermined value. The predetermined value may be, for example, 200m, 400m, or 600m. This predetermined value can define the warning start point 105, which is the starting point of the warning section 104. The warning start point 105 may be variable depending on the speed, and may be defined, for example, by the remaining time until vehicle Hv reaches the merging start point 102 (hereinafter referred to as the remaining time before merging). The warning start point 105 may be a point where the remaining time before merging is a predetermined value (for example, 8 seconds or 10 seconds). The remaining time before merging can be calculated based on the predicted time of arrival at the merging start point. The warning section is a section of road on the main line located before the merging section, and may be called the preceding section or the main line preparation section.

[0089] In this disclosure, in the merging lane L2, the road section located before the merging start point, as indicated by 201, is also referred to as the preparation section. The caution section 104 described earlier refers to a predetermined section on the main road side, whereas the preparation section 201 refers to a specific section on the merging lane side. In addition, in this disclosure, in the merging lane L2, the section 202 on the direction of travel side of the merging start point is referred to as the transition section.

[0090] The preparation section 201 may be a section on the merging lane where the remaining time to start merging is less than or equal to a predetermined value. In another embodiment, the preparation section 201 may be a section of road on the merging lane where the remaining distance to the merging start point is less than a predetermined value. In another embodiment, the preparation section 201 may be a section on the merging lane from a point where a predetermined sign related to merging is installed to the merging start point. The preparation section 201 may be a section adjacent to the caution section 104. The preparation section 201 may also be an acceleration section in one phase. The preparation section 201 is located in the reverse direction from the merging section. The reverse direction here means the direction opposite to the direction of travel set on the road. The reverse direction corresponds to the rear for a vehicle Hv traveling on the merging lane or main lane. Arrow A1 shown in the upper left of Figure 3 indicates the direction of travel on the road. Arrow A2 indicates the reverse direction.

[0091] Figure 3 illustrates a road structure in which a barrier 306, such as a wall, exists between the preparation section 201 and the caution section 104. The barrier 306 is not present until the merging start point 102, and is replaced by a transition control 305 midway through the caution section 104. From the end of the barrier 306 to the merging section 101, poles serving as the transition control 305 are placed at predetermined intervals.

[0092] The processor 31, acting as the environmental recognition unit F1, obtains the remaining distance to the merging section based on map data or sensor data. Information regarding the merging section may include the remaining distance to the merging section. The remaining distance to the merging section is the distance from the vehicle Hv's current position to the merging start point. The remaining distance to the merging section may be obtained based on map data or determined based on data of guide signs detected by the front camera 111. The environmental recognition unit F1 can calculate the remaining time until the merging start, which is the remaining time until the vehicle Hv reaches the merging start point, from the remaining distance to the merging section and the control plan for the vehicle Hv's driving speed.

[0093] The processor 31 also determines the attributes of the road based on map data or sensor data. If the vehicle Hv is traveling on an auxiliary road (i.e., a merging lane) leading to the main road and is traveling in a preparation section, the processor 31 determines whether or not there is an obstruction in the direction of the main road. The direction of the main road is the direction in which the main road exists when viewed from the merging lane. Arrow A3 shown in Figure 3 indicates the direction of the main road. The direction of the main road can be rephrased as the transition direction.

[0094] The presence or absence of obstructions may be determined based on map data (e.g., a high-precision map), or based on the detection results of the front camera 111 or the side sensor 112. Furthermore, while traveling in the preparation section, the processor 31 attempts to detect other vehicles on the main road based on the sensor data of the side sensor 112. This is because other vehicles on the main road can be considered competitors for a vehicle Hv traveling in the merging lane. Specifically, when vehicle Hv is traveling in the merging lane, other vehicles traveling at a position corresponding to the side of its vehicle on the merging lane of the main road (in other words, vehicles traveling alongside) are considered competitors.

[0095] When the processor 31 detects a vehicle traveling alongside it on the merging lane while driving in the merging lane, it performs preparatory control for merging, such as adjusting the longitudinal position. Here, longitudinal position refers to the relative driving position of the vehicle Hv with respect to surrounding vehicles in the direction of travel on the road. Adjusting the longitudinal position as preparatory control means adjusting the distance between the vehicle and the preceding vehicle, or adjusting the longitudinal position of the vehicle with respect to the vehicle traveling alongside it. Adjusting the longitudinal position as preparatory control may involve accelerating or decelerating so that the driving position does not overlap with that of other vehicles on the main lane in the lateral direction.

[0096] In contrast to the longitudinal position, the lateral position refers to the vehicle Hv's position within the lane. The basic lateral position can be the center of the lane. However, the processor 31 performs lateral position adjustments depending on the situation. Lateral position adjustment is a control that shifts the vehicle Hv's lateral position away from the center of the lane. Here, lateral position refers to the vehicle Hv's position in the road width direction and the vehicle width direction. Lateral position adjustment is also called VLO (Vehicle Lateral Offset). The specific operation of the processor 31 related to lateral position adjustment will be described separately.

[0097] <Example of Vehicle Control Device Operation> Here, the operation of the vehicle control device 30 in the situation where vehicle Hv is traveling in a merging lane in AD mode will be explained using the flowchart shown in Figure 4. The flowchart shown in Figure 4 is an example and will be executed from step S101. The description of the processor 31 as the executing entity in the following steps may be replaced with the vehicle control device 30. Also, the description of the processor 31 may be replaced with the environment recognition unit F1, mode management unit F2, planning unit F3, or vehicle control unit F4 depending on the context.

[0098] In the following explanation, we will use the scenario shown in Figure 3, where the merging lane is located to the left of the main line. From the perspective of vehicle Hv traveling in the merging lane, the main line is to the right. That is, in the following flowchart, "right" for vehicle Hv means the direction of the main line. In the flowchart, the first lane can be replaced with the lane being merged into. The term "second lane" can be replaced with another lane adjacent to the lane being merged into, i.e., a non-merging lane.

[0099] Step S101 is a step to determine whether vehicle Hv has entered the preparation section of the merging lane (201 in Figure 3). Whether vehicle Hv has entered the preparation section may be determined based on vehicle position data and map data. For example, it may be determined that vehicle Hv has entered the preparation section when the remaining time until the start of merging falls below a predetermined first threshold. The first threshold may be 10 seconds, for example. The processor 31 may also determine whether vehicle Hv has entered the preparation section based on information about the merging section included in the sensor data. The processor 31 may also determine that vehicle Hv has entered the preparation section after detecting a sign related to merging with the front camera 111. S101 may be executed periodically while vehicle Hv is traveling on the merging lane. If the processor 31 determines that vehicle Hv has entered the preparation section (S101 YES), it executes the processing from S102 onwards.

[0100] Step S102 is a step in which it is determined whether there is an obstruction such as a wall (306 in Figure 3) in the direction of the main line. In S102, the processor 31 verifies whether there is an obstruction in the direction of the main line based on sensor data or map data input from the side sensor 112 corresponding to the direction of the main line. The processor 31 may also determine whether there is an obstruction in the direction of the main line based on video data from the front camera 111. Step S102 includes obtaining the height of the obstruction. The height of the obstruction may be determined based on sensor data or map data.

[0101] If no obstructions are detected along the preparation section (S102 NO), the processor 31 performs normal behavior in S110. The phrase "along the preparation section" can be interpreted as the direction of the main line in the preparation section. Normal behavior here basically means continuing to drive in the merging lane as before entering the preparation section. If a competing vehicle Cv (in other words, a parallel vehicle) is detected on the main line while driving in the preparation section as normal behavior, the processor 31 may perform a response according to the position and speed of the competing vehicle Cv. Based on the position and speed of the competing vehicle Cv, the processor 31 predicts which vehicle, Hv or the competing vehicle Cv, is likely to reach the merging start point first. If the relative acceleration of the competing vehicle Cv with respect to vehicle Hv is positive, it may be determined that the competing vehicle Cv will reach the merging start point before vehicle Hv.

[0102] If the processor 31 predicts that competing vehicle Cv will reach the merging point before vehicle Hv, it may create a plan to move behind competing vehicle Cv and execute it after entering the merging section. If the processor 31 predicts that vehicle Hv will reach the merging point before competing vehicle Cv, it may create a plan to move in front of competing vehicle Cv and execute it after entering the merging section. If the processor 31 predicts that vehicle Hv will reach the merging point at the same time as competing vehicle Cv, vehicle Hv may create and execute a plan to decelerate and move behind competing vehicle Cv. This is because priority is given to the vehicle on the main road. Of course, this plan may be dynamically adjusted in accordance with the movement of competing vehicle Cv. If behavior is observed in competing vehicle Cv that yields the right of way to vehicle Hv, such as deceleration, the processor 31 may accelerate to move in front of competing vehicle Cv. Note that "main road vehicle" means a vehicle traveling on the main road.

[0103] Furthermore, if there is free space for an LC in front of the competing vehicle Cv, but no free space for an LC behind the competing vehicle Cv, the processor 31 may create and execute a plan to accelerate within an acceptable range of acceleration to get in front of the competing vehicle Cv. The free space for an LC means the space necessary for changing lanes, and will be referred to as the LC space below. LC stands for Lane Change. The length of the LC space in real space may be changed depending on the vehicle speed. The LC space may be at least 1.5 times the length of the vehicle Hv in the longitudinal direction.

[0104] As a normal operation, the vehicle travels through the preparation section, and when it enters the merging section, if there is no competing vehicle Cv (in other words, a vehicle traveling alongside), the processor 31 causes vehicle Hv to change lanes from the merging lane to the merging lane, provided that there is space for LC in the lane being merged into. Furthermore, if the lane being merged into is congested, a plan may be created to implement cooperative LC at a predetermined distance before the end of the merge, or at the point where the preceding vehicle has changed lanes.

[0105] Cooperative lane change (LC) conceptually involves changing lanes while communicating with other vehicles traveling in the merging lane. Cooperative LC may include having other vehicles traveling in the merging lane cooperate (e.g., by slowing down) to create space for vehicle Hv in the merging lane. Cooperative LC may first activate its turn signal and then monitor the movement of other vehicles in the merging lane. The other vehicles to be monitored are those that contribute to creating space for vehicle Hv to enter the merging lane. Specifically, these are the vehicles closest to vehicle Hv at the right rear of vehicle Hv when the vehicle reaches the planned lane change point. If, by chance, another vehicle is directly beside vehicle Hv, cooperative LC may include slowing down or stopping to move behind that other vehicle.

[0106] On the other hand, if an obstruction is detected along the preparation section in S102 (S102 YES), the processor 31 determines in S103 whether the obstruction is higher than the viewpoint position of the vehicle Hv. The viewpoint position of the vehicle Hv may be the mounting position of the front camera 111. The viewpoint position of the vehicle Hv may also be the mounting position of the side camera 112. The processor 31 obtains the viewpoint position of the vehicle Hv by accessing the storage 33. If the obstruction is higher than the viewpoint position of the vehicle Hv (S103 YES), the processor 31 determines in S104 whether a vehicle traveling alongside on the main line is detected.

[0107] The presence or absence of a parallel vehicle may be determined based on sensor data input from the side sensor 112. Note that part or most of the parallel vehicle may be hidden by an obstruction. S104 may be a step of determining whether part of an object resembling a parallel vehicle (for example, a roof) is visible in the image from the camera acting as the side sensor 112. A parallel vehicle being detected beyond an obstruction higher than the viewpoint of vehicle Hv may occur when the parallel vehicle is sufficiently taller than the obstruction. Obstructions higher than the viewpoint of vehicle Hv will also be referred to as tall obstructions below.

[0108] If a vehicle traveling alongside is detected by the side sensor 112 behind a tall obstruction (S104 YES), the processor 31 performs separation increase control, a form of lateral position adjustment, in S105. Separation increase control is a control that shifts the vehicle Hv's travel position away from the lane center in the opposite direction. The opposite direction here is the opposite direction of the main lane. The opposite direction corresponds to the direction away from the obstruction. In the example shown in Figure 3, since the obstruction 306 is to the right of the vehicle Hv, the left side corresponds to the opposite direction for the vehicle Hv. Separation increase control is a lateral position adjustment that increases the distance (also called clearance) from the obstruction.

[0109] In lateral position adjustment, the amount by which the driving position is shifted from the lane center is also called the offset amount. The offset amount may be a fixed value such as 0.5 m. Lateral position adjustment may be performed within the range where the vehicle Hv does not protrude into the adjacent lane / road. The offset amount may be dynamically determined according to the lane width, for example, it may be a value such that the distance between the lane mark and the vehicle Hv is a predetermined value. The driving position shifted from the lane center by lateral position adjustment is also called the offset position. Driving at the offset position is maintained, for example, until the merging start point is passed.

[0110] Given the height of the detected obstruction, the case where S104 is YES corresponds to when the adjacent vehicle is a large vehicle such as a truck. If the distance between the large vehicle and vehicle Hv is too close, the front or rear end of the large vehicle may be outside the FOV of the side sensor 112, making its range of presence unclear. The separation increase control increases the distance between vehicle Hv and the obstruction. This makes it easier for the side sensor 112 or the driver to recognize the total length (or range of presence) of the large adjacent vehicle.

[0111] On the other hand, if S104 is NO, the processor 31 increases the set value of the distance between the vehicle and the preceding vehicle by a predetermined amount in S106. S104 being NO corresponds to a situation where it is unclear whether or not there is an adjacent vehicle. This is because there remains a possibility that a vehicle shorter than the obstruction, such as a small car or a sports car, may be hidden in the shadow of the obstruction. In such a case, the parallel vehicle may be detected for the first time near the merging start point. If the parallel vehicle is detected for the first time near the merging start point, the processor 31 will need to quickly adjust its longitudinal position by decelerating or accelerating. At this time, if the distance to the preceding vehicle is short, it is difficult to adjust the longitudinal position by accelerating.

[0112] For these reasons, the processor 31 of this embodiment increases the degree of freedom of the vehicle Hv's behavior after the merging point by increasing the distance to the preceding vehicle when the conditions of the main lane are unclear due to obstructions. The set value of the distance between vehicles before the change may be the distance specified by the driver or the like. The amount of increase in the distance between vehicles may be a predetermined value. The distance between vehicles and the amount of increase may also be defined by a so-called inter-vehicle time that varies according to the speed. The inter-vehicle time is a parameter that represents the time from when the preceding vehicle passes a certain point until the vehicle Hv passes the same point. When the distance between vehicles before the change is equivalent to 2 seconds, the distance between vehicles after the change may be equivalent to 2.5 seconds or 3.0 seconds.

[0113] Furthermore, the processor 31 may also perform separation reduction control in S106. Separation reduction control is a control that shifts the vehicle Hv's travel position away from the lane center in the direction of the main line. Separation reduction control is a lateral position adjustment that reduces the distance (also called clearance) from the obstruction. The offset amount in separation reduction control may be determined within a range in which the distance between the vehicle Hv and the obstruction is greater than or equal to a predetermined value. The processor 31 may maintain the lateral position changed by separation reduction control for a predetermined time even after the obstruction is removed (for example, in a merging section).

[0114] The separation reduction control in S106 corresponds to a control that moves the vehicle Hv's position toward the main line in advance, even before entering the merging section. By having vehicle Hv travel toward the obstacle / main line from the preparation section, other vehicles on the main line can detect vehicle Hv earlier than if it were maintained in the center of the lane. The separation reduction control in S106 has the effect of signaling to other vehicles on the main line that vehicle Hv is about to merge. This makes it easier for vehicle Hv to smoothly enter the main line.

[0115] If, in S103, the processor 31 determines that the obstruction is not higher than the viewpoint position of the vehicle Hv (S103 NO), then in S120, it determines whether a parallel vehicle has been detected on the main road. S120 may be the same process as S104. If a parallel vehicle has been detected (S120 YES), then in S121, the processor 31 determines, based on sensor data, whether the parallel vehicle is a large vehicle. A large vehicle here means a vehicle that is sufficiently large compared to a typical passenger car. Examples of large vehicles include trailers and trucks. If the parallel vehicle is a large vehicle (S121 YES), then in S122, the processor 31 performs a lateral position adjustment so that the front and rear ends of the roof of the parallel vehicle are visible. If the vehicle Hv is too close to the large vehicle, it becomes difficult to see the entire vehicle. The lateral position adjustment in S122 may be a control that moves away from the obstruction (and thus the adjacent vehicle) so that the front and rear ends of the large parallel vehicle enter the FOV (Field of View) of the lateral sensor 112. In other words, separation increase control may be implemented in S122.

[0116] On the other hand, if no parallel vehicle is detected (S120NO), or if the detected parallel vehicle is not a large vehicle (S121NO), the processor 31 performs separation reduction control in S123. This separation reduction control causes the side sensor 112 or driver to approach the obstacle and peer into the main road. The separation reduction control in S123 may also be called peering control.

[0117] As vehicle Hv approaches an obstacle, the viewpoint of vehicle Hv also approaches the obstacle. As a result, when the viewpoint of vehicle Hv is higher than the obstacle, the line of sight to the main road improves, as shown in Figure 5. The separation reduction control in S123 makes it easier for the processor 31 to detect low-profile vehicles (such as sports cars) traveling in the first lane. In addition, as the viewpoint approaches the obstacle, the processor 31 can more easily recognize the lane mark 302 that separates the first and second lanes. When the lane mark 302 is included in the sensor / driver's FOV, the area where the first lane exists becomes clear. With the area corresponding to the first lane clearly defined, the processor 31 can more easily determine whether other vehicles detected on the main road are traveling in the first lane or not. In Figure 5, 400 represents the viewpoint position before the separation reduction control, and 401 represents the lower end of the FOV in the direction of the main road corresponding to viewpoint 400. 410 represents the viewpoint position after separation reduction control, and 411 represents the lower end of the FOV in the direction of the main line corresponding to viewpoint 410.

[0118] <Supplement to Operation> The above describes an example of operation in which the vehicle behavior is changed, such as moving closer to or away from an obstruction, depending on whether or not part of an adjacent vehicle is visible when the obstruction is higher than the viewpoint of the vehicle Hv. However, the operation of the processor 31 is not limited to this. S104 may be omitted. The processor 31 may be configured to perform distance increase control in the preparation section when the obstruction is higher than the viewpoint of the vehicle Hv, regardless of whether or not an adjacent vehicle is visible. The processor 31 may be configured to maintain driving in the center of the lane when the obstruction is higher than the viewpoint of the vehicle Hv, regardless of whether or not an adjacent vehicle is visible. In that case, the processor 31 may perform a lateral position adjustment to shift the driving position toward the main line when it reaches the merging start point.

[0119] After performing the separation reduction process in S123, the processor 31 may perform a first longitudinal position adjustment if it has detected a lane mark defining the first lane of the main line using the forward camera 111 or the side sensor 112. The first longitudinal position adjustment is to adjust the vehicle's speed so that the longitudinal positions of the vehicle in the first lane and vehicle Hv do not overlap. The vehicle in the first lane is another vehicle traveling in the first lane. The state in which the longitudinal positions of two vehicles overlap means that the two vehicles are traveling side by side. The state in which the longitudinal positions do not overlap means that vehicle Hv is not positioned directly beside the other vehicle. Since the vehicle in the second lane does not compete with vehicle Hv, the longitudinal positions of vehicle Hv and the vehicle in the second lane may overlap. The vehicle in the second lane is another vehicle traveling in the second lane. The first longitudinal position adjustment may be performed while driving in the preparation section. Prior longitudinal position adjustment facilitates the merging of vehicle Hv.

[0120] Furthermore, if the processor 31 has not been able to detect the lane mark defining the first lane of the main road using the forward camera 111 or the side sensor 112 after performing the separation reduction process in S123, it may perform a second vertical position adjustment. The second vertical position adjustment involves accelerating / decelerating so that the vertical position does not overlap with any other vehicles detected on the main road. If the lane mark defining the first lane is not detected, the accuracy of distinguishing between the first lane vehicle and the second lane vehicle decreases. If the first lane vehicle is mistakenly identified as the second lane vehicle, the possibility of the vehicle Hv approaching the other vehicle too closely at the merging point increases. By treating the second lane vehicle as a first lane vehicle and performing vertical position adjustment, the safety of vehicle Hv merging can be improved.

[0121] Steps S104 and S121 may also be steps to actually compare the height of other vehicles on the main line with the obstruction. If an obstruction is detected along the preparation section, the processor 31 may obtain information about other vehicles on the other side of the obstruction (i.e., on the main line) from the side sensor 112 and compare the height of the obstruction with that other vehicle. If the vehicle of interest, which is the other vehicle on the main line of interest, is higher than the obstruction, the processor 31 may perform separation increase control. This will allow the entire upper part of the vehicle of interest to fit within the FOV of the side sensor 112. The vehicle of interest may be a vehicle running alongside. The above description concerns the case when the vehicle control device 30 is operating in AD mode, but the above example of operation may also be applied in Level 2 mode.

[0122] <Summary> With the above configuration, the processor 31 determines the behavior of the vehicle Hv while traveling in the preparation section based on information about obstacles between the merging lane and the main lane. In one phase, the processor 31 changes its behavior depending on whether or not an obstacle is detected. In one mode, the processor 31 maintains driving in the center of the lane if no obstacle is detected, while performing lateral position adjustment if an obstacle is detected.

[0123] Furthermore, the processor 31 changes its behavior according to the height of the obstruction relative to the vehicle Hv's viewpoint. If the obstruction is higher than the vehicle Hv's viewpoint in one phase, the processor 31 does not perform distance reduction control during the preparation section. If the side sensor 112 detects a parallel vehicle beyond a tall obstruction, the processor 31 performs distance increase control. This improves the visibility of the parallel vehicle to the environmental sensor 11, and increases the amount of information about the parallel vehicle that the processor 31 can acquire.

[0124] Furthermore, if the side sensor 112 does not detect a parallel vehicle beyond a tall obstruction, the processor 31 suppresses the speed and increases the distance to the preceding vehicle. This can increase the degree of freedom in adjusting the longitudinal position in the merging section. As a result, a plan to accelerate to get ahead of competing vehicles can also be created as one of the selectable control plans. In addition, if the vehicle is traveling through a preparation section with a tall obstruction, the processor 31 moves its driving position closer to the main line even before reaching the merging start point. This allows the vehicle Hv to announce its presence to vehicles on the main line from the moment it reaches the merging start point.

[0125] Furthermore, the processor 31 performs distance reduction control based on the fact that the obstruction is lower than the vehicle Hv's viewpoint, and peers over the main road. This can improve the recognition of other vehicles and lane markings traveling on the main road. It also reduces the risk of failing to detect competing vehicles. Moreover, by knowing the position of the lane markings on the main road, it becomes easier to determine whether the detected other vehicle is traveling in the first lane or the second lane. In other words, it becomes easier to determine whether the detected other vehicle is truly a competing vehicle for vehicle Hv. For example, it can reduce the risk of misidentifying another vehicle traveling alongside vehicle Hv in the second lane as a competing vehicle.

[0126] <Switching viewpoint information> When the processor 31 is driving in the preparation section in manual driving mode, it may propose a change in driving position based on information about obstacles. Specifically, when the processor 31 is driving in the preparation section in manual driving mode, it may perform a process in S105, S106, S122, and S123 to propose a change in lateral or vertical position.

[0127] In manual driving mode, the processor 31 may use the eye position, which has been registered in advance or detected by the occupant state sensor 16, to compare the height with that of an obstacle. The viewpoint information in manual driving mode may be information indicating the driver's own viewpoint position. Furthermore, if the operating mode is a Level 2 or higher mode (i.e., an autopilot mode), the processor 31 may switch the viewpoint information to the mounting position of a sensor. The sensor here may be a front camera 111 or a side sensor 112. The mounting position of the sensor used as viewpoint information for comparing the height with that of an obstacle may be the lower or higher of the mounting position of the front camera 111 and the mounting position of the side sensor 112. In operating modes of Level 2 or higher, the viewpoint information used may be a value that has been registered in advance as a vehicle setting. As described above, the processor 31 may switch the viewpoint information used for comparing the height with that of an obstacle depending on whether or not it is in manual driving mode.

[0128] <Supplement to Level 2 Mode> In Level 2 mode, the processor 31 may be configured to initiate an automatic lane change based on the input of a driver instruction operation related to the execution of a lane change. The processor 31 sets a planned lane change point based on map data and, depending on the traffic conditions when the vehicle Hv approaches the planned lane change point, performs an intention confirmation process to request the driver's approval for the lane change. The intention confirmation process related to LC corresponds to the process of outputting a request to the driver to input an instruction for a lane change.

[0129] The intention confirmation process may include displaying a message on the display 17 asking whether it is OK to change lanes. The intention confirmation process may also include outputting a predetermined notification sound or an audio message from the speaker 18 asking whether it is OK to change lanes. These actions correspond to a proposal to change lanes. The processor 31 may obtain the driver's response through the driver's operation of the turn signal lever or voice input. If the processor 31 has made a proposal for a lane change and no positive response has been received after a predetermined response waiting time has elapsed, it may cancel the lane change. Conversely, if the processor 31 has made a proposal for a lane change and no negative response has been received after a predetermined response waiting time has elapsed, it may determine that the lane change has been accepted and start the lane change.

[0130] In a configuration where the processor 31 initiates a lane change after receiving approval from the driver, the timing of the intention confirmation process may be changed depending on whether or not there is an obstruction in the direction of the main road in the preparation section. That is, if there is an obstruction in the direction of the main road in the preparation section and a lane change is planned in the merging section, the intention confirmation process may be performed at an earlier timing than when there is no obstruction.

[0131] For convenience, the timing at which the intention confirmation process is carried out will also be referred to as the intention confirmation timing. Furthermore, the intention confirmation timing when there are no obstructions in the direction of the main line in the preparation section (i.e., the normal timing) will be referred to as the normal timing, and the intention confirmation timing when there are obstructions in the direction of the main line in the preparation section will be referred to as the temporary timing.

[0132] The normal timing can be when the vehicle reaches the merging start point or when the remaining time to the merging end point reaches a predetermined value. The temporary timing is set earlier than the normal timing. For example, the temporary timing can be when the remaining time to the merging start reaches a predetermined value. By obtaining driver approval earlier, lane changes (in other words, merging) become possible immediately after entering the merging section.

[0133] <Utilization of the behavior of the preceding vehicle> The processor 31 acquires sensor data indicating the behavior of the preceding vehicle from the environmental sensor 11 using the communication circuit 34. The behavior of the preceding vehicle here refers to acceleration, deceleration, or lane changes of the preceding vehicle. The processor 31 may be configured to estimate whether there is another vehicle in the first lane from the behavior of the preceding vehicle that has entered the merging section when vehicle Hv is traveling in the preparation section.

[0134] For example, when vehicle Hv is traveling through the preparation section, processor 31 may determine that there may be a competing vehicle in the first lane if the preceding vehicle slows down as it approaches the merging point. The fact that the preceding vehicle slows down means that there was a competing vehicle for the preceding vehicle. In that case, it suggests that the first lane is somewhat congested. The slowing down of the preceding vehicle may be detected by a reduction in the distance between vehicles detected by radar, etc., or by the illumination of the brake lights.

[0135] Furthermore, when vehicle Hv is traveling through the preparation section, processor 31 may determine that there may be a competing vehicle in the first lane after the preceding vehicle accelerates as it approaches the merging point. The sudden acceleration of the preceding vehicle suggests that there was another vehicle diagonally behind it. Moreover, the other vehicle diagonally behind the preceding vehicle could also be a competing vehicle for vehicle Hv. The acceleration of the preceding vehicle may be detected by an increase in the distance between vehicles detected by radar or the like.

[0136] Thus, the processor 31 may use the movement of the preceding vehicle near the merging start point as a clue to determine the possibility of the presence of a competing vehicle. If the processor 31 determines that the first lane is not visible due to a tall obstruction and that there is a possibility of a competing vehicle being present based on the behavior of the preceding vehicle, it may increase the following distance as in S106. On the other hand, if no special behavior is observed in the preceding vehicle at the merging start point, vehicle Hv may maintain its normal behavior.

[0137] <Vehicle control considering following vehicles> The processor 31 acquires sensor data indicating the presence or absence of following vehicles from the environmental sensor 11 using the communication circuit 34. The processor 31 may change the vehicle behavior when traveling through a preparation section where an obstruction is provided in the direction of the main line, depending on whether or not a following vehicle is present. Extreme offset driving when there is a following vehicle may confuse the driver of the following vehicle. Offset driving is driving in a state where the vehicle is offset from the center of the lane by lateral position adjustment. Extreme offset driving refers to a large amount of offset. For example, it can be understood as a state where the distance between the lane mark and the wheel is 0.1 m or less, such as when the wheel is on the lane mark.

[0138] Based on the above idea, the processor 31 may change the maximum value of the offset amount depending on whether or not there is a following vehicle. If there is no following vehicle, the processor 31 sets the maximum applicable offset amount to the first offset amount, and if there is a following vehicle, it sets the maximum applicable offset amount to the second offset amount. The second offset amount may be half or less of the first offset amount. For example, if the first offset amount is 1.0 m, the second offset amount may be 0.5 m, etc.

[0139] Furthermore, if there is a following vehicle, an extreme increase in the distance between the preceding vehicle and the following vehicle can confuse the driver of the following vehicle or disrupt the flow of traffic. Therefore, the processor 31 may change the amount of extension of the following distance in S106 depending on whether or not there is a following vehicle. For example, if there is no following vehicle, the processor 31 sets the amount of extension of the following distance to a first extension amount, and if there is a following vehicle, it sets the amount of extension of the following distance to a second extension amount. The second extension amount may be 40%, 50%, or 60% of the first extension amount, etc. The extension amount may be defined in terms of following time. For example, if the first extension amount is equivalent to 1.2 seconds, the second offset amount may be equivalent to 0.6 seconds.

[0140] <Indirect Search Processing> If an obstruction is erected in the direction of the main line in the preparation section, the sensor field of view of vehicle Hv is narrowed by the obstruction. The sensor field of view is the range in which objects can be detected by the environmental sensor 11, and is also called FOV. The FOV of vehicle Hv may be the range in which the front camera 111 can capture images. The FOV of vehicle Hv may also be a combination of the detection ranges of the rear camera, side cameras, radar, and LiDAR. If an obstruction causes a competing vehicle Cv on the main line (especially the merging lane in the warning section) to be outside the FOV, the detection timing of the competing vehicle will be delayed.

[0141] To address these challenges, the processor 31 may be configured to estimate (in other words, predict or detect) the presence of a potential competitor vehicle Cv based on the brightness characteristics of road elements related to the merging lane included in the image from the forward camera 111. A potential competitor vehicle is a competitor vehicle hidden by an obstruction. Road elements related to the merging lane include road elements that constitute the merging lane. For example, the road surface of the merging lane and lane markings 301 and 302 are road elements related to the merging lane. Road elements related to the merging lane may also include landmarks (such as signs) associated with the merging lane.

[0142] At night, the competing vehicle Cv turns on its headlights. The light from the competing vehicle Cv's headlights illuminates road elements in front of the competing vehicle Cv. Even if the competing vehicle Cv itself is not visible in the image of the front camera 111, the road elements illuminated by the competing vehicle Cv's headlights may be captured. For example, as shown in Figure 6, the road surface of the merging lane and lane markings 301 and 302, which are illuminated by the competing vehicle Cv's headlights, appear in the camera image before the actual image of the competing vehicle Cv. Furthermore, the road surface of the merging lane and lane markings illuminated by the competing vehicle Cv's headlights may have different brightness values ​​than the surrounding road elements. In particular, lane markings often have retroreflective properties and may have relatively high brightness values ​​when illuminated by the headlights.

[0143] Here, since the area illuminated by the headlights is finite, the areas of the road surface and lane markings that have high brightness are limited, and the brightness of the areas not reached by the headlights is relatively low. In other words, even with continuously formed road elements, a difference in brightness can occur between the areas within and outside the range of the light's illumination. The processor 31 may estimate the presence of a hidden competing vehicle Cv based on the fact that the road surface and lane markings of the merging lane are partially illuminated, specifically that the brightness of the relatively closer areas is higher than or equal to a predetermined value compared to the brightness of the relatively distant areas.

[0144] As explained above, if the competing vehicle Cv has its headlights on, the processor 31 can detect the light emitted from the competing vehicle Cv's headlights, even if the competing vehicle Cv itself is outside the FOV. The presence of light emitted from the competing vehicle Cv's headlights means that the competing vehicle Cv is present. Based on the above idea, the processor 31 may analyze the image from the front camera 111 and detect the competing vehicle Cv based on the presence of road elements illuminated by the competing vehicle Cv's headlights. Specifically, the processor 31 analyzes the image from the front camera 111 taken at a specific timing while driving through the preparation section and obtains the luminance distribution as the luminance features of road elements related to the merging lane. The processor 31 then detects potential merging vehicles from the luminance features of road elements related to the merging lane.

[0145] For example, the processor 31 may detect a competing vehicle Cv based on the fact that pixels corresponding to lane marks or road surface of the merging lane have a brightness value of a certain value or higher. The processor 31 may also determine whether the road elements related to the merging lane are illuminated by headlights by comparing a pre-prepared reference image of the road elements with the actual reference image. Determining that the road elements related to the merging lane are illuminated by headlights corresponds to determining that there is a merging vehicle. The reference image may be an image of a road element that is not illuminated by headlights at night. The reference image may be prepared in advance for each road element or for each merging section and stored in the storage 33. The processor 31 may detect a merging vehicle by comparing the brightness characteristics of the reference image with the brightness characteristics of the road elements that have actually been photographed.

[0146] Furthermore, the processor 31 may detect hidden merging vehicles by inputting the video from the front camera 111 to a predetermined trained detection model. The detection model here may be a machine learning model trained using video of the merging lane when no hidden merging vehicles exist and video of the merging lane when hidden merging vehicles exist. The detection model may be configured to output data indicating the probability of the presence of hidden merging vehicles from the input merging lane video. The probability may be expressed in binary form or as a probability value. The merging lane video may be video of the merging lane in the merging section. The merging lane video may be video showing a portion of the transition section, or it may be video from the front camera 111 while driving in the preparation section.

[0147] Furthermore, as illustrated in Figure 6, if a vehicle Pv is present on the merging lane, the rear of the vehicle Cv2 will be illuminated by the headlights of the competing vehicle Cv. Therefore, the processor 31 may determine that a competing vehicle Cv is present based on the fact that the rear of the vehicle Cv2 is bright. A vehicle ahead on the main road is another vehicle traveling at a predetermined distance or more ahead of vehicle Hv on the main road (especially on the merging lane). The state in which the rear of the vehicle Cv2 is bright means that the rear of the vehicle Cv2 is illuminated by lights, and specifically, the brightness of the rear of the vehicle Cv2, especially the brightness of the parts other than the taillights, may be above a predetermined value. Whether or not the rear of the vehicle Cv2 is illuminated by headlights may also be determined using a trained model.

[0148] The processor 31, acting as the environmental recognition unit F1, may detect potential (in other words, hidden) merging vehicles that could be competing vehicles using one or more of the methods described above. The processor 31 may also be configured to detect competing vehicles Cv whose physical form is partially or completely hidden by a method that partially modifies or improves upon the above methods. The process of indirectly searching for merging vehicles that could be competing vehicles from the brightness features of images of features constituting the merging lane and / or images of preceding merging vehicles, as described above, is also called the indirect search process. The indirect search process can be understood as a process that estimates the existence of hidden merging vehicles based on the fact that road structures and other vehicles are illuminated by potential merging vehicles.

[0149] The luminance features of a road element may be the average or maximum luminance values ​​of multiple pixels corresponding to that road element. The luminance features of a road element may be feature quantities derived from the luminance information of the image region representing the road element. The luminance features of a road element may be the average, median, or maximum luminance values ​​for each color element. The luminance features of the road surface may be the amount of change in luminance value according to distance. The luminance features may also be the difference in luminance relative to a reference image.

[0150] As described above, the processor 31 searches for potential merging vehicles based on the brightness features of images of road elements related to the merging lane. This allows the processor 31 to detect merging vehicles whose actual location is outside the FOV of vehicle Hv. In other words, the processor 31 can detect merging vehicles earlier. The processor 31 also estimates the presence of potential merging vehicles by using the brightness features of the rear of other vehicles traveling at a predetermined distance or more ahead of vehicle Hv in the merging lane as a clue. This also allows the processor 31 to detect merging vehicles earlier.

[0151] <Supplement to Indirect Search Processing> Vehicle Hv has a left headlight and a right headlight as its headlights. When the processor 31 performs indirect search processing while driving in the preparation section, if the headlights of vehicle Hv on the main road are in high beam mode, the accuracy of detecting hidden competing vehicles may deteriorate. This is because it is not possible to distinguish whether the light received by the road elements constituting the merging lane is from a competing vehicle on the main road or from vehicle Hv. For this reason, when the processor 31 performs indirect search processing while driving in the preparation section, it may temporarily switch the illumination state of the headlights on the main road from high beam to low beam. Alternatively, the processor 31 may temporarily turn off the headlights on the main road. In other words, the processor 31 reduces the amount of light emitted by the headlights on the main road for a predetermined period of time required to acquire the video used for indirect search processing. This may improve the accuracy of the indirect search processing. The length of the period for which the output of the headlights on the main road is reduced may correspond to the time required to acquire the image frames necessary for indirect search processing. For example, it could be 400 milliseconds or 600 milliseconds.

[0152] <Appealing to Mainline Vehicles Using Headlights> While the processor 31 is traveling in the preparation section, it may perform the above actions to detect competing vehicles on the mainline, while also using its headlights to appeal to other vehicles on the mainline about the presence of vehicle Hv. For example, while traveling in the preparation section, the processor 31 may flash its headlights at a speed that does not bother surrounding vehicles. The processor 31 may also direct the illumination direction of its headlights toward the mainline. Adjustment of the illumination direction may be achieved using an adaptive front lighting system or variable beam headlamps (adaptive driving beam; ADB). If the processor 31 is configured to allow left-right adjustment of the illumination direction of its headlights, it may appeal to other vehicles on the mainline by swinging the illumination direction of its headlights left and right at a predetermined speed. Appealing with headlights makes it easier for drivers / systems of vehicles on the mainline to recognize vehicle Hv.

[0153] <Utilization of Gap Images> In addition, if the merging lane is partially visible through a gap in an obstruction, the processor 31 may determine whether a competing vehicle Cv exists from the image of that gap. For example, the processor 31 identifies a gap area, which is an image region corresponding to a gap in an obstruction in the image captured by the camera, based on the detection results from the LiDAR / radar. The gap area may be identified by comparison with the color information of the obstruction. If the color of the gap area included in the captured image is not the color corresponding to the road surface, for example, if it is white, red, yellow, or blue, the processor 31 may determine that a competing vehicle Cv exists. Furthermore, if an object that can be recognized as part of a vehicle is captured in the gap area, the processor 31 may determine that a competing vehicle Cv exists. Whether or not an object that can be recognized as part of a vehicle is captured in the gap area may be determined using predetermined features or a trained model. The processor 31 may also determine that a competing vehicle Cv exists if a light source corresponding to a headlight, or a road element illuminated by it, is captured in the gap area.

[0154] <Utilization of Communication Congestion Level> The processor 31 may obtain the communication congestion level from the wireless communication device 15. The congestion level may be evaluated by the frequency of receiving vehicle-to-vehicle communication packets. The communication congestion level may also be the congestion level of cellular communication. The processor 31 may determine that the lower the cellular communication speed, the higher the congestion level. A higher communication congestion level means that there is also a lot of traffic. The processor 31 may be configured to estimate that there is a competing vehicle on the other side of an obstacle if the communication congestion level is above a predetermined value.

[0155] <Response during traffic congestion> The processor 31 may determine whether the merging section is congested based on the sensor data input from the environmental sensor 11. Traffic congestion is a state in which traffic is heavy. For example, the processor 31 may determine that there is traffic congestion if the speed of the preceding vehicle on the merging lane is below a predetermined value and the speed of the vehicles on the main lane is also below a predetermined value.

[0156] In a congested state, the field of view (FOV) of vehicle Hv is obstructed by surrounding vehicles, making it difficult to obtain information about the merging section. If the processor 31 determines that a congested state exists, it may mimic the behavior of the preceding vehicle and, in accordance with the movement of adjacent vehicles, transition vehicle Hv to the merging lane using a cooperative LC.

[0157] For example, the processor 31 sets the point at which the preceding vehicle transitions to the merging lane on the merging lane as the lane change point, and sets a vehicle on the main lane behind the preceding vehicle that has transitioned to the main lane as the target vehicle. The target vehicle corresponds to another vehicle that has yielded the right of way to the preceding vehicle. The processor 31 moves towards the lane change point while activating its turn signal lamps so as to move behind the target vehicle. In this process, upon receiving confirmation that a predetermined size of space has been created to the side of vehicle Hv with the cooperation of the vehicle following the target vehicle, the processor 31 begins the lane change. With this configuration, it is possible to safely merge vehicle Hv even when visibility is poor in congested conditions.

[0158] <Application to other scenarios> The above description primarily concerns the case where vehicle Hv is traveling in a merging lane. However, some or all of the above explanation may also apply when vehicle Hv is traveling in a merging lane on the main road. When the processor 31 is traveling in a warning section (in other words, a preparation section on the main road side), it may be configured to change its behavior depending on whether or not there is an obstruction in the merging direction. Here, the merging direction means the direction from the main road to where the merging lane is located.

[0159] The processor 31 may perform distance reduction control when driving in a merging lane where a low-profile obstruction is provided in the merging direction. When driving in a merging lane where a tall obstruction is provided in the merging direction, the processor 31 may perform distance increase control. If the processor 31 is driving in a merging lane where a tall obstruction is provided in the merging direction and no video footage related to the merging lane can be acquired, the processor 31 may change lanes to an adjacent lane.

[0160] The current lane may be a merging lane, or it may be a lane being merged into. If the current lane is a merging lane, the other lane that is scheduled to merge with it is a lane being merged into (for example, lane 1). If the current lane is a lane being merged into, the other lane that is scheduled to merge with it is a merging lane. The preparation section may be a preparation section on the main line side.

[0161] <Other> The term "remaining time to start merging" as described above may be replaced with "remaining distance to start merging," which is the remaining distance until vehicle Hv reaches the merging start point. The processor 31 may execute the processing from S102 onward when the remaining time or distance until vehicle Hv reaches the starting point of the merging section is less than a predetermined value. The term "main line" as described above refers to the main line of an expressway, for example, the main line of a highway.

[0162] Sensor data or map data may include dynamic map data generated in real time by the server. Dynamic map data can be considered a type of sensor data because it is generated based on sensor data aggregated on the server. Alternatively, dynamic map data can be considered a type of map data, as its name suggests. The processor 31 may be configured to acquire dynamic map data generated by the server via the wireless communication device 15.

[0163] <Addendum (1)> This specification discloses several technical concepts and several combinations thereof, as listed below. Programs, methods, and recording media corresponding to the following technical concepts are also included in this disclosure.

[0164] [Technical Concept 1] A vehicle control device comprising: a processing unit (31) for performing processing for controlling a vehicle; and a communication circuit (34) for the processing unit to receive sensor data indicating the detection result of at least one sensor for detecting objects present around the vehicle, wherein the processing unit is configured to acquire information about a merging section, which is a section in which the vehicle's lane merges with another lane, based on the sensor data or map data; acquire information about an obstruction present between the vehicle's lane and the other lane based on the sensor data or map data when the vehicle is traveling in a preparation section located before the merging section; and determine the behavior of the vehicle while traveling in the preparation section based on the information about the obstruction.

[0165] [Technical Concept 2] The vehicle control device according to Technical Concept 1, further comprising a viewpoint information storage unit (32) that holds viewpoint information indicating the height of the viewpoint for monitoring the surrounding traffic conditions, wherein the processing unit reads the viewpoint information by accessing the viewpoint information storage unit, and, if an obstruction exists along the preparation section, determines the behavior of the vehicle while traveling through the preparation section based on the relationship between the height of the obstruction and the height of the viewpoint.

[0166] [Technical Concept 3] The vehicle control device according to Technical Concept 2, wherein the processing unit is configured to perform separation reduction control, which is a control that sets the driving position to a position closer to the obstacle than the lane center when the vehicle is traveling in the preparation section and an obstacle exists along the preparation section, and when it is determined that the viewpoint is higher than the obstacle, the processing unit is configured to perform separation reduction control.

[0167] [Technical Concept 4] The vehicle control device according to Technical Concept 3, wherein the at least one sensor includes a camera that takes images of the outside of the vehicle, and the processing unit is configured to adjust the vehicle's speed so that, after performing the separation reduction control, if the camera cannot detect lane marks defining other lanes, the vehicle does not end up alongside any other vehicle detected on the other side of the obstruction.

[0168] [Technical Concept 5] A vehicle control device according to any one of Technical Concepts 2 to 4, wherein the processing unit is configured to determine whether the viewpoint is higher than the obstruction when the vehicle is traveling in the preparation section and the obstruction is located along the preparation section; to acquire information about other vehicles located beyond the obstruction from at least one sensor using the communication circuit; and to change the driving position within its own lane depending on whether or not another vehicle is detected on the other side of the obstruction at a position corresponding to the side of the vehicle.

[0169] [Technical Concept 6] The vehicle control device according to Technical Concept 5, wherein the processing unit is configured to perform separation reduction control, which is a control that sets the driving position within the current lane to a position closer to the obstacle than the lane center, when another vehicle is detected on the other side of the obstacle at a position corresponding to the side of the vehicle and the viewpoint is higher than the obstacle, and when another vehicle is detected on the other side of the obstacle at a position corresponding to the side of the vehicle and the viewpoint is lower than the obstacle, and when the driving position within the current lane to a position further away from the obstacle than the lane center.

[0170] [Technical Concept 7] The vehicle control device according to any one of Technical Concepts 2 to 6, wherein the processing unit is configured to determine whether the viewpoint is higher than the obstruction when the vehicle is traveling through the preparation section and the obstruction is present along the preparation section, and based on the determination that the viewpoint is lower than the obstruction, set the driving position to an offset position closer to the obstruction than the lane center, and maintain driving at the offset position until the vehicle reaches the merging section.

[0171] [Technical Concept 8] The vehicle control device according to technical concept 7, wherein the processing unit is configured to increase the distance between the vehicle and the preceding vehicle in the preparation section based on the determination that the viewpoint is lower than the obstruction.

[0172] [Technical Concept 9] A vehicle control device according to any one of Technical Concepts 2 to 8, comprising an automatic driving mode that performs processing for autonomous driving of the vehicle and a manual driving mode that leaves steering operations to the driver, wherein the at least one sensor includes a camera, and the processing unit is configured to switch the viewpoint information according to the operating mode, using the driver's eye position information as the viewpoint information in the manual driving mode, and using the camera's mounting position information as the viewpoint information in the automatic driving mode.

[0173] [Technical Concept 10] A vehicle control device according to any one of Technical Concepts 2 to 9, wherein the processing unit is configured to acquire information of other vehicles located beyond the obstruction when the obstruction is located along the preparation section, using the communication circuit from at least one sensor, compare the height of the obstruction with that of the other vehicle, and, if the other vehicle is taller than the obstruction, set the driving position within the lane to a position further away from the obstruction than the lane center.

[0174] [Technical Concept 11] A vehicle control device according to any one of Technical Concepts 1 to 10, configured to initiate an automatic lane change based on the input of a driver instruction operation related to the execution of a lane change, to acquire a planned lane change point based on map data, and to output a request to the driver to input an instruction for a lane change at an earlier timing than when the obstruction exists along the preparation section and a lane change is planned in the merging section.

[0175] [Technical Concept 12] The vehicle control device according to any one of Technical Concepts 1 to 11, wherein the processing unit is configured to acquire information indicating the behavior of a preceding vehicle from at least one sensor using the communication circuit, and, when traveling in the preparation section, to estimate whether there is a competing vehicle in the other lane based on the behavior of the preceding vehicle that has entered the merging section.

[0176] [Technical Concept 13] A vehicle control device according to any one of Technical Concepts 1 to 12, wherein the at least one sensor includes a camera, the processing unit is configured to acquire an image related to the other lane from the camera using the communication circuit, and to detect a competing vehicle based on the brightness characteristics of road elements related to the other lane included in the acquired image.

[0177] [Technical Concept 14] The vehicle control device according to any one of technical concepts 1 to 13, wherein the processing unit is configured to acquire information about a following vehicle from at least one sensor using the communication circuit, and to change the behavior in the preparation section or the merging section depending on whether or not a following vehicle is present.

[0178] <Addendum (2)> The various flowcharts shown in this disclosure are all examples, and the number of steps constituting the flowchart and the order of execution of processes can be changed as appropriate. The controls shown in each flowchart may be combined and executed in parallel to the extent that they do not contradict each other. Expressions such as acquisition, determination, detection, generation, and calculation may be used interchangeably. When a device acquires certain data, it also includes when the device generates such data based on signals input from other devices / sensors.

[0179] The devices, systems, and methods described in this disclosure may be implemented by a dedicated computer comprising a processor programmed to perform one or more functions embodied by a computer program. The devices and methods described in this disclosure may be implemented using dedicated hardware logic circuits. The devices and methods described in this disclosure may be implemented by one or more dedicated computers comprising a combination of a processor that executes a computer program and one or more hardware logic circuits. The processor may be any arithmetic core, such as a CPU, MPU, GPU, or DFP (Data Flow Processor). Some or all of the functions of the processing unit may be implemented as hardware. Some or all of the functions of the processing unit may be implemented using a system-on-a-chip (SoC), integrated circuit (IC), or field-programmable gate array (FPGA).

[0180] A computer program includes instructions that are executed by a computer. A computer program may be stored on a computer-readable, non-transitory tangible storage medium. The storage medium for a computer program may be a variety of media, such as an HDD (Hard-disk drive), an SSD (Solid State Drive), or flash memory.

Claims

1. A vehicle control device comprising: a processing unit (31) for performing processing for controlling a vehicle; and a communication circuit (34) for the processing unit to receive sensor data indicating the detection result of at least one sensor for detecting objects present around the vehicle, wherein the processing unit is configured to acquire information about a merging section, which is a section where the vehicle's lane merges with another lane, based on the sensor data or map data; acquire information about an obstruction present between the vehicle's lane and the other lane, based on the sensor data or map data, when the vehicle is traveling in a preparation section located before the merging section; and determine the behavior of the vehicle while traveling in the preparation section, based on the information about the obstruction.

2. The vehicle control device according to claim 1, further comprising a viewpoint information storage unit (32) that stores viewpoint information indicating the height of the viewpoint for monitoring the surrounding traffic conditions, wherein the processing unit is configured to read the viewpoint information by accessing the viewpoint information storage unit, and, if an obstruction exists along the preparation section, to determine the behavior of the vehicle while traveling through the preparation section based on the relationship between the height of the obstruction and the height of the viewpoint.

3. The vehicle control device according to claim 2, wherein the processing unit is configured to determine whether the viewpoint is higher than the obstruction when the vehicle is traveling in the preparation section and the obstruction is present along the preparation section, and if it is determined that the viewpoint is higher than the obstruction, it performs separation reduction control, which is control to set the driving position to a position closer to the obstruction than the center of the lane.

4. The vehicle control device according to claim 3, wherein the at least one sensor includes a camera that takes images of the outside of the vehicle, and the processing unit is configured to adjust the vehicle's speed so that, after performing the separation reduction control, if the camera cannot detect lane marks defining other lanes, the vehicle does not run alongside any other vehicle detected on the other side of the obstruction.

5. The vehicle control device according to claim 2, wherein the processing unit is configured to determine whether the viewpoint is higher than the obstruction when the vehicle is traveling in the preparation section and the obstruction is located along the preparation section; to acquire information about other vehicles located beyond the obstruction from at least one sensor using the communication circuit; and to change the driving position within its own lane depending on whether or not another vehicle is detected on the other side of the obstruction at a position corresponding to the side of the vehicle.

6. The vehicle control device according to claim 5, wherein the processing unit is configured to perform separation reduction control, which is a control to set the driving position within its own lane to a position closer to the obstruction than the lane center, when another vehicle is detected on the other side of the obstruction at a position corresponding to the side of the vehicle and the viewpoint is higher than the obstruction, and to perform separation increase control, which is a control to set the driving position within its own lane to a position further away from the obstruction than the lane center, when another vehicle is detected on the other side of the obstruction at a position corresponding to the side of the vehicle and the viewpoint is lower than the obstruction.

7. The vehicle control device according to claim 2, wherein the processing unit is configured to determine whether the viewpoint is higher than the obstruction when the vehicle is traveling through the preparation section and the obstruction is present along the preparation section, and based on the determination that the viewpoint is lower than the obstruction, set the driving position to an offset position closer to the obstruction than the lane center, and maintain driving at the offset position until the vehicle reaches the merging section.

8. The vehicle control device according to claim 7, wherein the processing unit is configured to increase the distance between the vehicle and the preceding vehicle in the preparation section based on the determination that the viewpoint is lower than the obstruction.

9. A vehicle control device according to claim 2, comprising an automatic driving mode for performing processing to make the vehicle drive autonomously and a manual driving mode for leaving steering operations to the driver, wherein the at least one sensor includes a camera, and the processing unit is configured to switch the viewpoint information according to the operating mode, using the driver's eye position information as the viewpoint information in the manual driving mode, and using the camera's mounting position information as the viewpoint information in the automatic driving mode.

10. The vehicle control device according to claim 2, wherein the processing unit is configured to acquire information of other vehicles located beyond the obstruction when the obstruction is located along the preparation section, using the communication circuit from at least one sensor, compare the height of the obstruction with that of the other vehicle, and, if the other vehicle is taller than the obstruction, set the driving position within the vehicle's own lane to a position further from the obstruction than the center of the lane.

11. The vehicle control device according to claim 1, which is configured to initiate an automatic lane change based on the input of a driver instruction operation for the execution of a lane change, acquires a planned lane change point based on map data, and, if the obstruction exists along the preparation section and a lane change is planned in the merging section, outputs a request to the driver to input an instruction for a lane change at an earlier timing than when the obstruction is not provided along the preparation section.

12. The vehicle control device according to claim 1, wherein the processing unit is configured to acquire information indicating the behavior of a preceding vehicle from at least one sensor using the communication circuit, and, when traveling in the preparation section, to estimate whether there is a competing vehicle in the other lane based on the behavior of the preceding vehicle that has entered the merging section.

13. The vehicle control device according to claim 1, wherein the at least one sensor includes a camera, the processing unit is configured to acquire an image related to the other lane from the camera using the communication circuit, and to detect a competing vehicle based on the brightness characteristics of road elements related to the other lane included in the acquired image.

14. The vehicle control device according to claim 1, wherein the processing unit is configured to acquire information about a following vehicle from at least one sensor using the communication circuit, and to change the behavior in the preparation section or the merging section depending on whether or not a following vehicle is present.

15. A program including instructions for causing at least one processor to perform the following actions: receive sensor data using a communication circuit indicating the detection result of at least one sensor for detecting objects present around a vehicle; acquire information about a merging section, which is a section where the vehicle's lane merges with another lane, based on the sensor data or map data; acquire information about an obstruction present between the vehicle's lane and the other lane, based on the sensor data or map data, when the vehicle is traveling through a preparation section located before the merging section; and determine the behavior of the vehicle while traveling through the preparation section, based on the information about the obstruction.

Images