Vehicle position estimation device, travel control device

Abstract

The present application relates to a vehicle position estimation device, a travel control device. A position estimator (20) is provided with a positioning section (F0) and a severe environment determination section (F6). The positioning section (F0) performs positioning processing for estimating a vehicle position based on positions of landmarks estimated from an image of a front camera and positions of landmarks registered on a map. The severe environment determination section (F6) determines that a vehicle surrounding environment is a severe environment for the front camera based on a case where an effective value of a distance at which a landmark can be detected by image recognition, that is, an effective recognition distance, becomes equal to or less than a predetermined threshold value. In a case where it is determined that the environment is a severe environment, the position estimator (20) shortens an execution period of the positioning processing, and cooperates with a drive support ECU (30) to suppress a vehicle speed by a predetermined amount. In addition to this, a main point in the vehicle speed suppression is displayed on a display in cooperation with an HMI system.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to Japanese Patent Application No. 2020-117248, filed on July 7, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to a vehicle position estimation device that uses image frames from a front-facing camera to estimate the position of a vehicle, and a driving control device that uses the vehicle position estimation device. Background Technology

[0004] To enable autonomous driving and other driving control in vehicles such as automobiles, technologies for determining the vehicle's current position with higher accuracy are required. As a technology for determining the vehicle's position with higher accuracy, Patent Document 1 discloses a technique for determining the vehicle's position based on the observed position of a landmark identified from an image captured by a front-facing camera and the position coordinates of the landmark registered in map data. This process of determining the vehicle's position by comparing (i.e., matching) the image recognition results from the front-facing camera with map data is also known as positioning processing.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-8462

[0006] The positioning process disclosed in Patent Document 1 is based on the premise that landmarks can be accurately identified by a forward-facing camera. However, in adverse environments such as rain or dense fog, the camera image becomes unclear, so the success rate of landmark identification may decrease. In particular, landmarks located further away are more difficult to identify.

[0007] If landmarks are not easily recognized in images, the frequency (success rate) of determining the vehicle's position through localization processing also decreases. As a result, the accuracy of the vehicle's position estimation deteriorates. The accuracy of vehicle position estimation greatly contributes to the safety of autonomous driving, therefore, a vehicle position estimation device with high robustness to harsh environments is required. Summary of the Invention

[0008] This disclosure is made based on this situation and its purpose is to provide a vehicle position estimation device and a driving control device that are highly robust to harsh environments.

[0009] The vehicle position estimation device for achieving this purpose is a vehicle position estimation device mounted on a vehicle and estimating the current position of the vehicle. It includes a control unit using at least one processor. The control unit includes: a positioning unit that determines the vehicle's position coordinates on a map based on the position information of a specified landmark detected using an image frame captured by a front-facing camera and the position information of a landmark registered on a map; and a severe environment determination unit that determines whether the vehicle's surrounding environment is equivalent to a specified severe environment that may reduce the accuracy of object recognition using image frames based on at least one of the information from sensors and communication devices mounted on the vehicle. The control unit is configured to perform different processing depending on whether the severe environment determination unit determines that the surrounding environment is equivalent to a severe environment.

[0010] Based on the above configuration, when the surrounding environment is determined to be a harsh environment, different processing is performed compared to the case where the environment is not determined to be harsh (i.e., normal conditions). Therefore, the processing performed in harsh environments can be different from the processing performed in normal conditions based on the harsh environment. As a result, robustness can be improved compared to a comparative configuration that performs constant processing regardless of whether the environment is harsh or not.

[0011] Furthermore, the driving control device for achieving the above-mentioned purpose is a driving control device mounted on a vehicle and automatically performing speed control of the vehicle. It includes a control unit composed of at least one processor. The control unit includes: a speed control unit that outputs a control signal for speed adjustment to the vehicle's driving actuator; a positioning unit that determines the vehicle's position coordinates on a map based on the position information of a predetermined landmark determined using an image frame generated by a front camera and the position information of a landmark registered on a map; and a severe environment determination unit that determines whether the vehicle's surrounding environment is equivalent to a predetermined severe environment that may reduce the accuracy of object recognition using the image frame, based on at least one of the information from sensors and communication devices mounted on the vehicle. The control unit is configured to suppress the driving speed based on the condition determined by the severe environment determination unit that the surrounding environment is equivalent to a severe environment.

[0012] According to the aforementioned driving control device, the driving speed is suppressed in adverse environments compared to normal conditions. Generally, the higher the vehicle speed, the less clear the camera image becomes. In other words, by suppressing the vehicle speed, image clarity can be expected to improve. If image clarity is improved, the accuracy of landmark detection based on image recognition is improved. That is, according to the above configuration, concerns about the degradation of landmark detection accuracy can be reduced even in adverse environments. Consequently, it is expected to improve the success rate of processing (i.e., positioning processing) that determines the vehicle's position coordinates on the map using the location information of specified landmarks determined using images captured by the front camera. Therefore, the above configuration improves robustness to adverse environments.

[0013] Furthermore, the reference numerals in the accompanying drawings show the correspondence between specific units described in the embodiments described later as an example, and do not limit the technical scope of this disclosure. Attached Figure Description

[0014] Figure 1 This is a block diagram showing the structure of a driving support system.

[0015] Figure 2 It is a block diagram showing the configuration of the front camera.

[0016] Figure 3 This is a function block diagram representing the structure of the position estimator.

[0017] Figure 4 This is an example of a slowdown notification image.

[0018] Figure 5 These are other examples of images representing deceleration notifications.

[0019] Figure 6 This is a flowchart of the adverse environment determination process implemented by the location estimator.

[0020] Figure 7 This is a diagram illustrating the operation of the deceleration requirement unit.

[0021] Figure 8 This is a flowchart illustrating an example of a deceleration and rescheduling process.

[0022] Figure 9 This is an example image representing a notification that overtaking has been aborted.

[0023] Figure 10 This is a flowchart illustrating an example of expedited mediation processing.

[0024] Figure 11 This diagram is used to illustrate the operation of the camera in front.

[0025] Figure 12 This diagram is used to illustrate the changes made to the front camera.

[0026] Figure 13 This is a diagram showing a variation of the system configuration.

[0027] Figure 14 This is a diagram showing a variation of the system configuration.

[0028] Figure 15 This is a diagram showing a variation of the system configuration. Detailed Implementation

[0029] Hereinafter, embodiments of the present disclosure will be described using the accompanying drawings. Figure 1This is a diagram illustrating an example of the general configuration of a driving support system 1 using the position estimator disclosed herein.

[0030] <Overview of the overall structure>

[0031] like Figure 1 As shown, the driving support system 1 includes a front camera 11, millimeter-wave radar 12, vehicle status sensor 13, GNSS receiver 14, V2X onboard unit 15, map storage unit 16, HMI system 17, position estimator 20, and driving support ECU 30. Furthermore, ECU stands for Electronic Control Unit, referring to an electronic control device. HMI stands for Human Machine Interface. V2X stands for Vehicle to X (Everything), referring to communication technology that connects vehicles to various devices.

[0032] The various devices or sensors constituting the driving support system 1 described above are connected as nodes to a communication network built within the vehicle, namely the in-vehicle network Nw. Nodes connected to the in-vehicle network Nw can communicate with each other. Furthermore, specific devices can also be configured to communicate directly without going through the in-vehicle network Nw. For example, the position estimator 20 and the driving support ECU 30 can also be directly electrically connected via a dedicated line. Additionally, although in Figure 1 The in-vehicle network (Nw) is configured as a bus topology, but is not limited to this. Network topologies can also be mesh, star, ring, etc. The network shape can be appropriately modified. Standards for in-vehicle networks (Nw) can include Controller Area Network (CAN), Ethernet, FlexRay, and other standards.

[0033] Vehicles equipped with Driver Assistance System 1 will also be recorded as this vehicle, and passengers sitting in the driver's seat of this vehicle (i.e., driver's seat passengers) will also be recorded as users. Furthermore, the following description defines the forward / backward, left / right, and up / down directions based on this vehicle. Specifically, the forward / backward direction corresponds to the long side direction of this vehicle. The left / right direction corresponds to the width direction of this vehicle. The up / down direction corresponds to the height direction of the vehicle. According to other viewpoints, the up / down direction corresponds to the direction perpendicular to a plane parallel to the forward / backward and left / right directions.

[0034] Furthermore, as an example, the operation of each device will be explained here assuming that the driving assistance system 1 is used in a left-hand traffic area. The leftmost lane in a lane with the same direction of travel will be referred to as the first lane. In a right-hand traffic area, the configuration of this disclosure can be implemented by reversing the left and right directions described above. For example, in a right-hand traffic area, the first lane refers to the rightmost lane in a lane with the same direction of travel. The driving assistance system 1 described below can be modified to comply with the traffic regulations and customs of the area where it is used.

[0035] <Summary of each component>

[0036] The front camera 11 is a camera that takes pictures of the front of the vehicle from a prescribed angle. The front camera 11 may be located, for example, at the upper part of the inside of the passenger compartment of the windshield, in the front grille, or on the roof. Figure 2 As shown, the front camera 11 includes a camera body 40 that generates image frames and an ECU, or camera ECU 41, that detects a specified object by performing recognition processing on the image frames. The camera body 40 is configured to include at least an image sensor and a lens. The camera body 40 generates and outputs captured image data at a specified frame rate (e.g., 60fps). The camera ECU 41 is implemented using an image processing circuit including a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The camera ECU 41 includes a recognizer G1 as a functional module. The recognizer G1 is configured to identify the type of object based on the feature vector of the image generated by the camera body 40. The recognizer G1 can utilize, for example, a CNN (Convolutional Neural Network) or a DNN (Deep Neural Network) that applies deep learning.

[0037] The objects detected by the forward-facing camera 11 include, for example, moving objects such as pedestrians and other vehicles. Other vehicles include bicycles, bicycles with prime movers, and motorcycles. Furthermore, the forward-facing camera 11 is configured to also detect designated road features. Road features detected by the forward-facing camera 11 include road ends, road markings, and structures along the road. Road markings refer to painted designs on the road surface used for traffic control and traffic restrictions. Examples include lane markings (so-called lane lines) indicating lane boundaries, pedestrian crossings, stop lines, guide strips, safety zones, and restriction arrows. Lane markings also include those implemented using road studs such as chatter bars or Bott's Dots. Structures along the road include, for example, guardrails, curbs, trees, utility poles, road signs, and traffic lights.

[0038] In addition, the forward camera 11 detects obstacles such as animals, fallen trees, and objects dropped from vehicles. Here, obstacles refer to three-dimensional objects on the road that obstruct vehicle passage. Examples of dropped objects include boxes, ladders, and bags. Obstacles also include tires detached from vehicles, accident vehicles, and debris from accident vehicles. Furthermore, obstacle categories may include lane-limiting materials such as arrow signs, traffic cones, and guide signs, as well as construction sites, parked vehicles, and the end of traffic congestion. In addition to stationary objects obstructing vehicle passage, obstacles may also include quasi-static map elements. For example, the forward camera 11 identifies and outputs the types of obstacles such as dropped objects through image recognition. Preferably, the forward camera 11 is configured to detect not only the lane in which the vehicle is traveling but also obstacles existing in areas equivalent to adjacent lanes. As an example, the forward camera 11 is configured to detect obstacles in the vehicle's lane and in the adjacent lanes to the left and right.

[0039] The camera ECU41 separates and extracts the background and detected objects from the captured image based on image information including color, brightness, and contrast related to color and brightness. Furthermore, some or all of the terrain features detected by the forward camera 11 are used as landmarks in the position estimator 20. Landmarks in this disclosure refer to terrain features that can be used as markers to determine the vehicle's position on a map. Landmarks can include at least one of the following: restriction signs, guide signs, warning signs, instruction signs, etc., equivalent to traffic signs; signs, traffic lights; poles; guide boards, etc. Guide signs include directional signs, signs indicating area names, signs indicating road names, and signs indicating highway entrances / exits, service areas, etc. Landmarks can also include streetlights, mirrors, utility poles, commercial billboards, signs indicating shop names, historical buildings, and other symbolic structures. Pole includes streetlights and utility poles. Landmarks can also include road undulations / collapses, manholes, junctions, etc. The ends / forks of dividing lines can also be used as landmarks. The types of features used as landmarks can be appropriately varied. Preferred landmarks include traffic lights, directional signs, and other features that do not change with time and are large enough to be recognized even from a distance of 100m or more. Furthermore, road ends and dividing lines can also be included in the landmarks.

[0040] Furthermore, features that can be used as markers for longitudinal position estimation (hereinafter referred to as longitudinal position estimation) are also called landmarks for longitudinal position estimation. Here, longitudinal corresponds to the forward and backward direction of a vehicle. In addition, longitudinal in a straight road section, it corresponds to the direction of road extension as observed from the vehicle, that is, the direction of road extension configuration. Landmarks for longitudinal position estimation can be, for example, map elements that are discretely arranged along the road and lack changes with time, such as traffic signs such as directional signs and road markings such as temporary stop lines. In addition, features that can be used as markers for lateral position estimation (hereinafter referred to as lateral position estimation) are also called landmarks for lateral position estimation. Here, lateral corresponds to the width direction of the road. Landmarks for lateral position estimation refer to features that exist continuously along the road, such as road ends and dividing lines. The forward camera 11 can be configured to detect features set as landmarks.

[0041] The camera ECU 41 calculates the relative distance and direction of landmarks and boundary lines from the vehicle based on the image containing SfM (Structure from Motion) information. It can also determine the relative position (distance and direction) of landmarks relative to the vehicle based on their size and orientation (e.g., tilt) within the image. Furthermore, the camera ECU 41 is configured to identify the type of landmark based on its color, size, shape, etc., such as whether it is a directional sign.

[0042] Furthermore, the camera ECU 41 generates driving road data representing the curvature, width, and other shapes of the route based on the position and shape of lane markings and road ends. In addition, the camera ECU 41 calculates the yaw rate based on SfM (Side Flow Meter). The camera ECU 41 sequentially provides detection result data, representing the relative position and type of the detected object, to the position estimator 20 and the driver support ECU 30 via the vehicle network Nw. The following description of the position estimator 20, etc., refers to at least one of the position estimator 20 and the driver support ECU 30.

[0043] In this embodiment, the camera ECU41, as a more preferred embodiment, also outputs data representing the reliability of the image recognition result. For example, the reliability of the recognition result is calculated based on factors such as rainfall, the presence or absence of backlight, and ambient brightness. Furthermore, the reliability of the recognition result can also be a score representing the consistency of feature quantities. For example, the reliability can also be a probability value representing the accuracy of the recognition result output by the recognizer G1. This probability value can correspond to the aforementioned consistency of feature quantities. The reliability of the recognition result can also be the average of the probability values ​​of each detected object generated by the recognizer G1.

[0044] Furthermore, the camera ECU41 can also evaluate the reliability of the recognition results based on the stability of the recognition results for the same tracked object. For example, it can be evaluated as having high reliability when the recognition results for the same object type are stable, and as having low reliability when the category label used for the recognition results for the same object is unstable. A stable recognition result means that the same result is obtained continuously for a constant time (e.g., 10 seconds) or more. An unstable recognition result means that the recognition results are repeated or cannot be obtained continuously.

[0045] The millimeter-wave radar 12 is a device that detects the relative position and speed of an object relative to the vehicle by transmitting millimeter waves or quasi-millimeter waves towards the front of the vehicle and analyzing the received data of the reflected waves after the transmitted waves are reflected from an object. The millimeter-wave radar 12 is installed, for example, in the front grille or front bumper. The millimeter-wave radar 12 has a built-in radar ECU that identifies the type of the detected object based on its size, speed, and reception intensity. The millimeter-wave radar 12 is also configured to detect some or all of the aforementioned obstacles, including those traveling in front. For example, the millimeter-wave radar 12 identifies whether an object is an obstacle based on its position, speed, size, and reflection intensity. For example, it can roughly determine the type of obstacle based on the size of the detected object and the reception intensity of the reflected waves. The radar ECU outputs the type of detected object, its relative position (direction and distance), and data representing the reception intensity as detection results to the position estimator 20, etc. The objects detected by the millimeter-wave radar 12 also include the aforementioned landmarks.

[0046] The forward-facing camera 11 and the millimeter-wave radar 12 can also be configured to provide observation data used for object recognition to the driver support ECU 30 and the like via the in-vehicle network Nw. For example, the observation data for the forward-facing camera 11 refers to image frames. The observation data for the millimeter-wave radar refers to data representing the received intensity and relative velocity for each detection direction and distance, or data representing the relative position of the detected object and the received intensity. The observation data is equivalent to the raw data observed by the sensors, or the data before performing recognition processing.

[0047] Object recognition processing based on observation data can also be performed by an ECU other than the driver support ECU 30 or similar sensors. Alternatively, the relative position calculation of landmarks can be performed by the landmark information acquisition unit F3. The position estimator 20 and driver support ECU 30 can also incorporate the functions of the camera ECU 41 and millimeter-wave radar 12 (primarily object recognition). In this case, the camera 11 (as the forward-facing camera) and the millimeter-wave radar provide observation data such as image data and ranging data as detection result data to the position estimator 20 and driver support ECU 30.

[0048] The vehicle state sensor 13 is a sensor that detects state quantities related to the driving control of the vehicle. The vehicle state sensor 13 may include, for example, inertial sensors such as a three-axis gyroscope and a three-axis accelerometer. The driving support system 1 may also include a magnetic sensor as the vehicle state sensor 13. Additionally, the driving support system 1 may also include an atmospheric pressure sensor and a temperature sensor as the vehicle state sensor 13. The atmospheric pressure sensor and temperature sensor can also be used for correction processing of the output values ​​of other sensors. Various inertial sensors can also be packaged into an inertial measurement unit (IMU).

[0049] Additionally, the vehicle status sensor 13 may also include a gear position sensor, a steering angle sensor, and a vehicle speed sensor. The gear position sensor detects the position of the gear shift lever. The steering angle sensor detects the rotation angle of the steering wheel (the so-called steering angle). The vehicle speed sensor detects the vehicle's speed. The three-axis accelerometer detects the accelerations acting on the vehicle in the forward, backward, left-right, and up-down directions. The gyroscope sensor detects the angular velocity of rotation about a detection axis; a three-axis gyroscope sensor refers to a sensor with three mutually orthogonal detection axes. The inertial sensor is equivalent to a sensor that detects physical state quantities representing the vehicle's movements as a result of the driver's actions or the control of the driving support ECU 30.

[0050] The vehicle state sensor 13 outputs data representing the current value (i.e., the detection result) of the physical state quantity of the detected object to the in-vehicle network Nw. The output data of each vehicle state sensor 13 is acquired by the position estimator 20, etc., via the in-vehicle network Nw. Furthermore, the vehicle state sensor 13 can be designed with appropriate types of sensors used in the driving support system 1; it is not necessary to have all of the aforementioned sensors. Additionally, the vehicle state sensor 13 may include a raindrop sensor for detecting rainfall and an illuminance sensor for detecting external brightness.

[0051] The GNSS receiver 14 is a device that detects its current position by sequentially receiving navigation signals transmitted from the positioning satellites that constitute the GNSS (Global Navigation Satellite System). GNSS types that can be used include GPS (Global Positioning System), GLONASS, Galileo, IRNSS, QZSS, Beidou, etc.

[0052] The V2X vehicle-to-everything (V2X) device 15 is a device for wireless communication between the vehicle and other devices. Furthermore, in V2X, "V" refers to the vehicle itself, and "X" can refer to various entities other than the vehicle, such as pedestrians, other vehicles, road infrastructure, networks, and servers. The V2X vehicle-to-everything (V2X) device 15 includes a wide-area communication unit and a narrow-area communication unit as communication modules. The wide-area communication unit is a communication module used to implement wireless communication according to a specified wide-area wireless communication standard. This wide-area wireless communication standard can be, for example, LTE (Long Term Evolution), 4G, 5G, and other standards. Furthermore, the wide-area communication unit can also be configured to communicate wirelessly with other devices directly, i.e., without using a base station, in addition to communication via a wireless base station, according to the wide-area wireless communication standard. In other words, the wide-area communication unit can also be configured to implement cellular V2X. By incorporating the V2X vehicle-to-everything (V2X) device 15, the vehicle becomes a connected car capable of connecting to the Internet. For example, the location estimator 20 can download the latest high-precision map data from a designated server and update the map data stored in the map storage unit 16 by cooperating with the V2X vehicle-mounted device 15.

[0053] The narrow-range communication unit of the V2X vehicle-to-everything (V2X) device 15 is a communication module used to directly communicate wirelessly with other mobile entities and roadside vehicles located around the vehicle, based on a narrow-range communication standard that limits the communication distance to within several hundred meters. These other mobile entities are not limited to vehicles but can include pedestrians, bicycles, etc. The narrow-range communication standard can be any standard such as DSRC (Dedicated Short Range Communications) or WAVE (Wireless Access in Vehicular Environment). The narrow-range communication unit broadcasts vehicle information to surrounding vehicles at a predetermined transmission period and receives vehicle information from other vehicles. Vehicle information includes vehicle ID, current location, direction of travel, speed, timestamp, etc. The V2X V2X device 15 can also be configured to communicate with other vehicles by sending and receiving position estimation accuracy information indicating the accuracy of the vehicle's position. The position estimation accuracy information can, for example, indicate whether the positioning process was successful. In addition, the position estimation accuracy information can also be the number of positioning satellites captured by the GNSS receiver 14, the identification number of the positioning satellites, etc.

[0054] Map storage unit 16 is a non-volatile memory that stores high-precision map data. This high-precision map data refers to map data that displays road structures and the coordinates of features along the roads with a precision sufficient for autonomous driving. The high-precision map data includes, for example, three-dimensional shape data of roads, lane data, and feature data. The aforementioned three-dimensional shape data package contains node data related to locations where multiple roads intersect, merge, or branch (hereinafter referred to as nodes), and road segment data related to the roads connecting these locations (hereinafter referred to as road segments).

[0055] The road segment data displays information such as the location coordinates of the road ends, the road width, etc. It may also include data indicating whether the road is a dedicated vehicle road or a general road. Here, a dedicated vehicle road refers to a road where pedestrians and bicycles are prohibited, such as a toll road like a highway. The road segment data may also include attribute information indicating whether autonomous driving is permitted.

[0056] Lane data displays the number of lanes, the location of lane dividers, the direction of travel for each lane, and the points where lanes branch off or merge. Lane data may also include information indicating whether the lane dividers are implemented using solid lines, dashed lines, or Bott's Dots. The location information of lane dividers and road ends (hereinafter referred to as lane dividers, etc.) is represented as a set of coordinates (i.e., a set of points) of the locations forming the lane dividers. Alternatively, the location information of lane dividers, etc., can be represented polynomially. The location information of lane dividers, etc., can also be a collection of line segments (i.e., a line group) represented polynomially.

[0057] The feature data includes information on the location and type of road surface displays such as temporary stop lines, as well as the location, shape, and type of landmarks. Landmarks include traffic signs, traffic lights, poles, commercial signs, and other three-dimensional structures set along the road. Furthermore, the map storage unit 16 can also be configured to temporarily store map data within a specified distance from the vehicle. Additionally, the map data held by the map storage unit 16 can be navigation map data, i.e., navigation map data. Navigation map data is map data with lower accuracy compared to high-precision map data, and contains less information about road shapes compared to high-precision map data. When the navigation map data includes feature data such as landmarks, the representation of the high-precision map below can be replaced by a navigation map. Furthermore, as mentioned above, landmarks here refer to features such as traffic signs used for vehicle position estimation, i.e., positioning processing.

[0058] HMI system 17 is a system that provides an input interface for accepting user operations and an output interface for providing prompts to the user. HMI system 17 includes a display 171 and an HCU (HMI Control Unit) 172. In addition to the display 171, other units that provide information prompts to the user may include speakers, vibrators, and lighting devices (such as LEDs).

[0059] Display 171 is a device for displaying images. Display 171 may be, for example, a central display located at the top center of the dashboard in the vehicle width direction. Display 171 is capable of full-color display and can be implemented using liquid crystal displays, OLED (Organic Light Emitting Diode) displays, plasma displays, etc. Furthermore, the HMI system 17 may also include a head-up display that projects a virtual image onto a portion of the windshield in front of the driver's seat as display 171. Alternatively, display 171 may also be an instrument display located in the dashboard in front of the driver's seat.

[0060] HCU172 is a unified control unit for providing information prompts to the user. HCU172 is implemented using processors such as CPUs and GPUs, RAM (Random Access Memory), flash memory, etc. HCU172 controls the display screen of display 171 based on information provided by the driver support ECU 30 and signals from input devices (not shown). For example, HCU172 displays a deceleration notification image on display 171 based on requests from the position estimator 20 or the driver support ECU 30.

[0061] The position estimator 20 is a device for determining the current position of the vehicle. The functions of the position estimator 20 will be described in detail later. The position estimator 20 is primarily composed of a computer including a processing unit 21, RAM 22, a storage device 23, a communication interface 24, and a bus connecting them. The processing unit 21 is hardware integrated with RAM 22 for computational processing. The processing unit 21 is configured to include at least one processing core such as a CPU (Central Processing Unit). The processing unit 21 performs various processes by accessing RAM 22. The storage device 23 is configured to include a non-volatile storage medium such as flash memory. The storage device 23 stores the program executed by the processing unit 21, i.e., the position estimation program. Executing the position estimation program by the processing unit 21 is equivalent to executing the method corresponding to the position estimation program. The communication interface 24 is a circuit for communicating with other devices via the vehicle network Nw. The communication interface 24 can be implemented using analog circuit components, ICs, etc.

[0062] The driver support ECU 30 is an ECU that controls the driving actuator 18 based on detection results from the front camera 11 and millimeter-wave radar 12, thereby performing some or all of the driving operations in place of the driver. Furthermore, the driver support ECU 30 can also be an automatic operating device that enables the vehicle to drive autonomously based on input from the user's autonomous driving instructions. The driver support ECU 30 is equivalent to a vehicle control module.

[0063] The driving actuator 18 refers to an actuator used to accelerate, decelerate, and turn the vehicle. Examples include braking devices, electronic throttle valves, and steering actuators. Braking devices include brake actuators. In the case of an electric or hybrid vehicle, a motor that generates torque to rotate the drive wheels is also included in the driving actuator 18. Other ECUs, such as a braking ECU and a steering ECU, may also be sandwiched between the driving actuator 18 and the driving support ECU 30.

[0064] The driving support ECU 30 includes a speed control unit H1 that provides ACC (Adaptive Cruise Control) as one of its vehicle control functions. The ACC function allows the vehicle to maintain a constant speed at a predetermined target speed and, when a vehicle is within a predetermined distance of another vehicle, to follow it while maintaining a safe distance. The speed control unit H1 automatically adjusts the vehicle's speed based on user input to activate the driving support function. The target speed for maintaining a constant speed during ACC is set by the user. This user-set target speed for ACC is also recorded as the user-defined vehicle speed.

[0065] In the ACC (Adaptive Cruise Control) function, the range of speeds the vehicle can use while driving under ACC, also known as the variable speed zone, is set through user operation or system design. The system can be configured so that the user can change the upper and lower limits of the variable speed zone within the limits of complying with traffic laws and ensuring safety as a system capability. The variable speed zone can also be set independently based on road type, such as general roads or highways. The lower limit of the variable speed zone can also be 0. The user-set speed can be set within this variable speed range. Furthermore, in addition to user settings, the target speed for the ACC function may be temporarily changed based on requirements from the deceleration requirement section F8 described later. For convenience, the upper limit of the variable speed zone is also recorded as the ACC upper limit speed.

[0066] Such a driving support ECU 30 is primarily composed of a computer, including a processing unit 31, RAM 32, storage device 33, communication interface 34, and a bus connecting them. The processing unit 31 is hardware integrated with RAM 32 for computational processing. The processing unit 31 is configured to include at least one processing core such as a CPU. The processing unit 31 executes various processes for implementing ACC functions, etc., by accessing RAM 32. The storage device 33 is configured to include a non-volatile storage medium such as flash memory. The storage device 33 stores the program executed by the processing unit 31, i.e., the driving support program. Executing the driving support program by the processing unit 31 is equivalent to executing the method corresponding to the driving support program.

[0067] <Functions of Position Estimator 20>

[0068] Here we use Figure 3 The function and operation of the position estimator 20 are explained. The position estimator 20 provides a position estimation result by executing a position estimation program stored in the storage device 23. Figure 3 The functions corresponding to the various functional modules shown are as follows: Specifically, the position estimator 20 includes a provisional position estimation unit F1, a map acquisition unit F2, a landmark information acquisition unit F3, a route information acquisition unit F4, a recognition accuracy evaluation unit F5, a severe environment determination unit F6, a detailed position calculation unit F7, a deceleration requirement unit F8, and a notification processing unit F9 as functional modules. The detailed position calculation unit F7 includes a periodic change unit F71.

[0069] The provisional position estimation unit F1 sequentially measures the vehicle's position by combining the positioning results from the GNSS receiver 14 and the detection results from the inertial sensor. For example, in tunnels or similar locations, the provisional position estimation unit F1 uses yaw rate and vehicle speed to perform dead reckoning (autonomous navigation method). The yaw rate used for dead reckoning can be either the yaw rate identified by the camera ECU 41 using SfM technology, or the yaw rate detected by the yaw rate sensor.

[0070] The map acquisition unit F2 reads map data from the map storage unit 16 within a specified range based on the current location. The current location used for map reference can be either the location determined by the provisional location estimation unit F1 or the location determined by the detailed location calculation unit F7. For example, if the detailed location calculation unit F7 can calculate the current location, map data is acquired using this location information. On the other hand, if the detailed location calculation unit F7 cannot calculate the current location, map data is acquired using the location coordinates calculated by the provisional location estimation unit F1. Furthermore, immediately after the ignition switch power is turned on, the map reference range is determined, for example, based on the previous location calculation result stored in the memory. This is because the previous location calculation result stored in the memory corresponds to the end point of the previous trip, i.e., the parking position. Additionally, the map acquisition unit F2 can also be configured to sequentially download map data corresponding to the current location from an external server, etc., via the V2X vehicle-mounted device 15. The map data corresponding to the current location refers to map data concerning a zone / road segment within a specified distance from the current location.

[0071] The landmark information acquisition unit F3 acquires the relative position, type, color, etc. of the landmarks identified by the front camera 11 (essentially camera ECU 41) from the front camera 11. If the front camera 11 is configured to extract strings attached to signs, etc., it is preferable to also acquire character information recorded on signs, etc. This is because, based on the configuration that allows the acquisition of landmark character information, establishing a correspondence between the landmarks observed by the front camera and landmarks on the map becomes easier.

[0072] Furthermore, the landmark information acquisition unit F3 converts the relative position coordinates of the landmarks acquired from the camera ECU 41 into position coordinates in the global coordinate system (hereinafter also referred to as observation coordinates). The observation coordinates of the landmarks can be calculated by combining the vehicle's current position coordinates and the relative position information of the landmarks with respect to the vehicle. If the detailed position calculation unit F7 can calculate the current position, the vehicle's current position coordinates used for calculating the landmark observation coordinates can be obtained using that position information. On the other hand, if the detailed position calculation unit F7 cannot calculate the current position, the position coordinates calculated by the provisional position estimation unit F1 can be used. Alternatively, the camera ECU 41 can also perform the calculation of the landmark observation coordinates using the vehicle's current position coordinates.

[0073] Furthermore, the landmark information acquisition unit F3 can also determine the relative position of a landmark by complementaryly combining the recognition results of the front camera 11 and the detection results of the millimeter-wave radar 12 or LiDAR. For example, the landmark information acquisition unit F3 can also determine the observation position of a landmark by complementaryly combining the recognition results of the front camera 11 and the detection results of the millimeter-wave radar 12. Specifically, the landmark information acquisition unit F3 can also determine the distance between the landmark and the vehicle, as well as the elevation angle (or height) of the landmark, by using both the recognition results of the front camera 11 and the detection results of the millimeter-wave radar 12. Generally speaking, cameras are good at horizontal position estimation, but not good at vertical position estimation and distance estimation. On the other hand, millimeter-wave radar 12 is good at position estimation in both vertical and vertical directions. In addition, millimeter-wave radar 12 is less affected by fog and rain. By using the front camera 11 and millimeter-wave radar 12 complementaryly to estimate the position of a landmark as described above, the relative position of the landmark can be determined with higher accuracy. As a result, the accuracy of the vehicle's position estimation based on positioning processing is also improved.

[0074] The route information acquisition unit F4 acquires driving road data from the forward camera 11 (essentially a camera ECU 41). That is, it acquires the relative positions of the dividing lines and road ends identified by the forward camera 11. The route information acquisition unit F4 may also convert the relative position information of the dividing lines, etc., into position coordinates on a global coordinate system, similar to the landmark information acquisition unit F3.

[0075] The recognition accuracy evaluation unit F5 obtains the reliability of the recognition result from the front camera 11. Furthermore, when the detailed position calculation unit F7 performs positioning processing using a landmark different from the previously used landmark, the recognition accuracy evaluation unit F5 calculates the error between the vehicle's position coordinates calculated by the provisional position estimation unit F1 and the positioning processing result as the position estimation error. The longer the period during which positioning cannot be performed, the larger the position estimation error; a large position error indirectly indicates the length of the period during which positioning cannot be performed. The increased time during which positioning cannot be performed is due to factors beyond the possibility of longer landmark setting intervals, including a decrease in the recognition performance of the front camera 11. Therefore, the position error can also be used as an indicator of the object recognition performance of the front camera 11. In addition, during the period when positioning processing cannot be performed, the provisional position estimation error can be calculated sequentially by multiplying the elapsed time or travel distance since the last positioning processing by a predetermined error estimation coefficient. Alternatively, a separate functional module for calculating the position estimation error can be provided as an error estimation unit, distinct from the recognition accuracy evaluation unit F5.

[0076] In addition, the recognition accuracy evaluation unit F5 can also evaluate recognition performance based on the proportion of failed detections of landmarks registered on the map that should have been detected along the vehicle's trajectory, i.e., the false negative rate. The false negative rate can be calculated based on the total number N of landmarks registered on the map within a constant distance and the number of landmarks detected up to the end of the journey, i.e., the number of successful detections m. For example, the false negative rate can also be calculated as (N-m) / N. Alternatively, the total number N can also be the number of landmarks that should have been observable from the current location within a specified distance (e.g., 35m) ahead. In this case, the number of successful detections m simply represents the number of landmarks that can be detected at the current location.

[0077] Furthermore, the recognition accuracy evaluation unit F5 calculates the limit of the actual range of landmarks that the front camera 11 can recognize, i.e., the effective recognition distance. The effective recognition distance differs from the designed recognition limit distance and is a parameter that varies depending on important external factors such as fog, rain, and afternoon sun. In a configuration assuming a designed recognition limit distance of about 100m, the effective recognition distance may degrade to less than 20m during heavy rain, for example.

[0078] The recognition accuracy evaluation unit F5 calculates the effective recognition distance, for example, based on the furthest recognition distance of at least one landmark detected within a specified time. The furthest recognition distance is the distance from which the same landmark can be detected from the furthest point. The distance to a landmark that has not been detected up to this point is equivalent to the furthest recognition distance of that landmark when a landmark is detected as the movement progresses. When the furthest detection distances of multiple landmarks are obtained, the effective recognition distance can be their average, maximum, or second largest value.

[0079] For example, if the furthest recognition distances of four landmarks observed within the most recent specified time are 50m, 60m, 30m, and 40m, the effective recognition distance can be calculated as 45m. The furthest recognition distance of a landmark is equivalent to the detection distance at the moment when the landmark could first be detected. The effective recognition distance can also be the maximum value of the furthest recognition distance observed within the most recent specified time.

[0080] Furthermore, the effective recognition distance may decrease due to significant factors other than weather conditions, such as obstruction by preceding vehicles. Therefore, if a preceding vehicle is present within a specified distance, the calculation of the effective recognition distance can be omitted. Alternatively, if a preceding vehicle is present, data indicating the presence of a preceding vehicle (e.g., a preceding vehicle sign) can be added to provide the effective recognition distance to the adverse environment determination unit F6. Additionally, if the road ahead of the vehicle is not straight, in other words, if it is a curve, the effective recognition distance may also decrease. Therefore, if the road ahead is a curve, the calculation of the effective recognition distance can be omitted. Alternatively, if the road ahead is a curve, a correspondence can be established between the effective recognition distance and data indicating that the road ahead is a curve (e.g., a curve sign) and provided to the adverse environment determination unit F6. Furthermore, a curve is a road with a curvature exceeding a specified threshold.

[0081] Furthermore, when multiple types of land features are designated as landmarks, the landmarks used by the recognition accuracy evaluation unit F5 for calculating the effective recognition distance can be limited to a subset of these types. For example, the landmarks used for calculating the effective recognition distance can be limited to landmarks such as directional signs located at a specified distance (e.g., 4.5m) above the road surface (hereinafter referred to as high-altitude landmarks). By limiting the landmarks used for calculating the effective recognition distance to high-altitude landmarks, the reduction in effective recognition distance due to obstruction of the view by other vehicles can be suppressed.

[0082] In addition to calculating the effective recognition distance for discrete features such as signs, the recognition accuracy evaluation unit F5 can also calculate the effective recognition distance for features that exist continuously along a road, such as lane markings and road ends. The effective recognition distance for lane markings and road ends represents information about how far the road surface can be recognized. For example, the effective recognition distance for lane markings can be determined based on the distance to the furthest detection point among the detection points for lane markings. The same applies to the effective recognition distance for road ends. The effective recognition distance for various features such as landmarks, lane markings, and road ends also serves as an evaluation value for recognition accuracy.

[0083] The adverse environment determination unit F6 determines whether the surrounding environment of the vehicle constitutes a defined adverse environment that may reduce the performance (i.e., accuracy) of object recognition using the image frame generated by the front camera 11. In other words, the adverse environment determination unit F6 determines whether the environment constitutes an adverse environment for the front camera 11.

[0084] The adverse environment determination unit F6 determines whether an environment is considered adverse based on the effective recognition distance calculated by the recognition accuracy evaluation unit F5. For example, the adverse environment determination unit F6 determines an environment to be adverse if the effective recognition distance is less than a predetermined threshold (hereinafter referred to as the distance threshold). Specific values ​​for the distance threshold can be, for example, 20m or 30m. Here, as an example, the distance threshold is set to 25m. Of course, the distance threshold can also be determined based on the design's recognition limit distance. For example, values ​​equivalent to 10% or 20% of the design's recognition limit distance can be used as the distance threshold. The distance threshold can also be adjusted according to the type of road. The distance threshold used for intercity highways can be larger than that used for general roads and urban highways. Since the speed on intercity highways is higher than on general roads, it is preferable to use a relatively strict threshold for determining an adverse environment. Here, intercity highways refer to dedicated motor vehicle roads connecting multiple cities. On the other hand, urban highways are dedicated motor vehicle roads whose extension range is limited to a single city. Furthermore, urban highways are often configured to connect with other dedicated motor vehicle roads via junctions, allowing them to extend outside the city.

[0085] The adverse environment assessment based on the effective recognition distance can also be cancelled when there is a preceding vehicle or when driving on a curve. This reduces concerns about misjudging adverse environments due to the presence of a preceding vehicle or changes in curvature. Furthermore, when there is a preceding vehicle, simply following it is sufficient, so there is no need to rigorously determine the vehicle's position. Consequently, the necessity for determining whether it is an adverse environment is also low. In other words, scenarios without a preceding vehicle can be considered scenarios where there is a high necessity for accurately estimating the vehicle's position.

[0086] In addition, the adverse weather determination unit F6 can also determine that the environment is adverse when rain or fog occurs. It can determine the presence or absence of rain using the rain sensor's detection value and the wiper's operating status (including its speed). It can also detect the presence of fog based on the fog light's illumination status. Furthermore, it can obtain weather-related information such as fog and rain from an external server via the V2X vehicle-to-everything (V2X) device 15. Additionally, the adverse weather determination unit F6 can also determine that the environment is adverse when exposed to afternoon sun. It can determine whether the environment is exposed to afternoon sun based on the time of day, the vehicle's direction of travel, and the overall color information / brightness distribution of the image frame. Furthermore, afternoon sun refers to sunlight from the sun whose angle (height) relative to the horizon is, for example, less than 25 degrees.

[0087] Furthermore, the adverse environment determination unit F6 can also determine whether the area surrounding its vehicle is equivalent to an adverse environment by obtaining data on areas that meet the criteria for an adverse environment from a map server. For example, the map server is a server that determines and distributes adverse environment areas based on reports from multiple vehicles. With this configuration, the computational load for determining whether an environment is adverse can be reduced. In addition, the adverse environment determination unit F6 can also share the provisional determination result of whether an environment is adverse with other vehicles via the V2X vehicle-to-everything (V2X) device 15, and determine whether an environment is equivalent to an adverse environment through majority decision or other means.

[0088] Furthermore, as a more preferred embodiment, the harsh environment determination unit F6 of this embodiment includes a species identification unit F61 and a harsh environment degree determination unit F62. However, the species identification unit F61 and the harsh environment degree determination unit F62 are not essential elements. The harsh environment determination unit F6 may also lack either or both of the species identification unit F61 and the harsh environment degree determination unit F62.

[0089] The type identification unit F61 is responsible for determining the type of adverse environment. Adverse environments can be broadly categorized into heavy rain, fog, afternoon sun, and others. For example, if the effective recognition distance of a landmark is below a predetermined distance threshold, and the type identification unit F61 can identify dividing lines and high-altitude landmarks within a first distance from the vehicle, then the type of adverse environment is determined to be fog. Furthermore, if the effective recognition distance of a dividing line / landmark is below a predetermined first distance, and the unit can identify dividing lines or similar features within a first distance from the vehicle, then the type of adverse environment is determined to be afternoon sun. Additionally, if dividing lines / landmarks near the vehicle cannot be identified, then it is determined to be heavy rain. Here, "near the vehicle" can be, for example, within 15 meters of the vehicle.

[0090] Furthermore, when determining the type of adverse environment, information such as time, temperature, humidity, and wiper operating status can also be used. For example, a rainstorm can be identified if the vehicle cannot identify distant or nearby road markings / landmarks, and the wiper speed is above a specified threshold. Additionally, conditions such as temperature, humidity, and terrain meeting the criteria for fog formation can be used as criteria to determine whether the adverse environment is fog based on the effective recognition distance. Similarly, conditions such as time and the vehicle's direction of travel being affected by afternoon sun can be used as criteria to determine whether the adverse environment is afternoon sun based on the effective recognition distance.

[0091] Furthermore, regarding rainfall, it's possible to classify it not only as heavy rain but also into multiple stages based on the intensity (i.e., rainfall amount) of the rain, such as light rain, heavy rain, and torrential rain. Rainfall can also be classified as normal if the wiper speed is low. Heavy rain is defined as rain with an hourly rainfall exceeding a specified threshold (e.g., 50 mm). As an example, rainfall up to 10 mm is recorded as light rain, and rainfall between 20 mm and 50 mm is recorded as heavy rain. While the intensity of rain is divided into three stages here, the number of intensity classifications can be adjusted appropriately.

[0092] The environmental severity determination unit F62 evaluates the degree of environmental severity, that is, the degree of performance degradation of object recognition using image frames. For example, it can represent the environmental severity in four stages, from level 0 to 3. The higher the level, the greater the severity of the environmental severity. The environmental severity can be evaluated based on the effective recognition distance. For example, the environmental severity determination unit F62 can determine it as level 1 if the effective recognition distance is less than a predetermined first distance but more than a predetermined second distance, and as level 2 if it is less than the second distance but more than a predetermined third distance. Furthermore, it can determine it as level 3 if the effective recognition distance is less than the third distance. It can also determine it as level 0 if the effective recognition distance is more than the first distance, meaning it is not a severe environment.

[0093] The first distance can be the same as the aforementioned distance threshold (25m). The second distance is set to be shorter than the first distance. For example, the second distance can be set to a value 5m shorter than the first distance (i.e., 20m). The third distance is set to be shorter than the second distance. For example, the third distance can be set to a value 5m shorter than the second distance (here, 15m). The number representing the severity of the environment can be appropriately changed. Of course, the above values ​​are just examples and can be appropriately changed according to the camera's performance, etc. For example, the first distance can also be 60m, 50m, 40m, etc. The second distance can also be 40m, 30m, etc. The third distance can also be 30m, 20m, etc. The difference between the first and second distances, and the difference between the second and third distances, are not limited to 5m respectively, and can also be 10m or 20m.

[0094] Furthermore, the severe weather severity assessment unit F62 can also evaluate the severity of the weather based on rainfall. For example, it sets the severity level to Level 1 when the rainfall is equivalent to light rain, Level 2 when the rainfall is equivalent to heavy rain, and Level 3 when the rainfall is equivalent to torrential rain. Rainfall can be estimated based on the wiper blade speed or determined by obtaining weather information from an external server.

[0095] The detailed location calculation unit F7 performs positioning processing based on the landmark information acquired by the landmark information acquisition unit F3 and the route information acquired by the route information acquisition unit F4. Positioning processing refers to determining the detailed location of the vehicle by comparing the positions of landmarks, etc., determined from images captured by the forward-facing camera 11 with the coordinates of features registered in high-precision map data. Since relative position information of landmarks, etc., is also used in this position determination process, the process of determining the relative positions of landmarks and dividing lines based on image data can also be included in the positioning process. Hereinafter, the configuration including the map acquisition unit F2, the landmark information acquisition unit F3, the route information acquisition unit F4, and the detailed location calculation unit F7 will be described as the positioning unit F0.

[0096] The detailed position calculation unit F7 uses landmarks such as direction signs to estimate the longitudinal position. Based on the observed coordinates of the landmarks, the detailed position calculation unit F7 establishes a correspondence between landmarks registered on the map and landmarks observed by the forward camera 11, as a longitudinal position estimate. For example, the landmark closest to the observed coordinates of the landmark registered on the map is estimated as the same landmark. Furthermore, when comparing landmarks, it is preferable to use features such as shape, size, and color, and to use landmarks with a higher degree of feature consistency. Once the correspondence between the observed landmark and the landmark on the map is established, the longitudinal position of the vehicle on the map is set to a position offset from the location of the landmark on the map corresponding to the observed landmark by the distance between the observed landmark and the vehicle. Here, "near the front" refers to the direction opposite to the vehicle's direction of travel. When moving forward, "near the front" corresponds to the rear of the vehicle.

[0097] For example, if the image analysis results indicate that the distance to the directional sign in front of the vehicle is 100m, then the vehicle is determined to be located 100m forward from the coordinates of that directional sign registered in the map data. Longitudinal position estimation is equivalent to determining the vehicle's position in the direction of road extension. Longitudinal position estimation can also be called longitudinal positioning processing. By performing such longitudinal position estimation, the detailed remaining distance to feature points (POIs) on the road, such as intersections, curve entrances / exits, tunnel entrances / exits, and the very end of traffic congestion, can be determined.

[0098] Furthermore, for example, when the detailed position calculation unit F7 detects multiple landmarks (such as a direction sign) in front of the vehicle, it uses the landmark closest to the vehicle among these multiple landmarks to perform longitudinal position estimation. Regarding the accuracy of object recognition based on image type and distance, the closer the object is to the vehicle, the higher the recognition accuracy. In other words, by using the landmark closest to the vehicle to perform longitudinal position estimation when multiple landmarks are detected, the accuracy of position estimation can be improved.

[0099] Furthermore, the detailed location calculation unit F7 uses the observed coordinates of road features such as lane markings and road edges that exist continuously along the road to perform lateral position estimation. Lateral position estimation refers to the determination of the driving lane and the detailed position of the vehicle within the driving lane. The detailed position of the vehicle within the driving lane refers, for example, the offset from the center of the lane to the left or right. For example, lateral position estimation is performed based on the distance from the left and right road edges / markings identified by the forward camera 11. For example, if, as a result of image analysis, the distance from the left road edge to the center of the vehicle is determined to be 1.75m, it is determined that the vehicle exists at a position offset 1.75m to the right from the coordinates of the left road edge. Lateral position estimation can also be referred to as lateral positioning processing. In addition, as another method, the detailed location calculation unit F7 can also be configured to perform longitudinal and lateral positioning processing using landmarks such as direction signs.

[0100] The vehicle's position can be represented as a result of the positioning process using the same coordinate system as the map data (e.g., latitude, longitude, altitude). For example, the vehicle's position information can be represented using any absolute coordinate system such as WGS84 (World Geodetic System 1984).

[0101] The detailed location calculation unit F7 performs positioning processing sequentially at a predetermined location estimation period as long as it can detect (i.e., capture) a landmark. The default value for the location estimation period is, for example, 100 milliseconds. The default value for the location estimation period can also be 200 milliseconds or 400 milliseconds. The period adjustment unit F71, based on the adverse environment determination unit F6's determination of an adverse environment, sets the location estimation period to a predetermined adverse environment value shorter than the default value. The adverse environment value can be set to, for example, half of the default value, such as 50 milliseconds. Alternatively, the adverse environment value can be 80 milliseconds, 40 milliseconds, etc. It can be set to a value corresponding to the frame rate of the front camera 11 and the processing speed of the sensor G1.

[0102] In addition, the detailed location calculation unit F7 determines the lane identifier (number), i.e., the lane ID, of the lane in which the vehicle is traveling, based on the distance information from the road edge detected by the front camera 11 and the millimeter-wave radar 12. The lane ID, for example, indicates which lane the vehicle Ma is traveling in from the left end of the road. Furthermore, in areas where right-hand traffic is permitted, the lane ID can also be assigned based on the right end of the road. The lane ID can also be determined by the camera ECU 41. This lane can also be referred to as the vehicle's lane or the lane in which the vehicle is traveling.

[0103] The deceleration request unit F8, based on the adverse environment determination unit F6's determination of an adverse environment, outputs a prescribed deceleration request signal to the driver support ECU 30. The deceleration request signal can be, for example, a signal requiring the target speed setpoint to be reduced by a predetermined amount from its current value. The required deceleration amount can be a constant amount such as 3 km / h, 5 km / h, or 10 km / h. Alternatively, the deceleration amount can be a value corresponding to the target speed, such as 10% of the target speed. Or, the deceleration request signal can also be a signal requiring deceleration to a preset degradation speed. The degradation speed can be set according to the road type; for example, it can be 60 km / h on highways and 40 km / h on ordinary roads.

[0104] Furthermore, the deceleration request signal can either lower the current target speed setting as described above, or it can request a reduction in the ACC upper speed limit. When the current target speed is close to the ACC upper speed limit, lowering the ACC upper speed limit can indirectly suppress the vehicle speed.

[0105] The notification processing unit F9 is configured to cooperate with the HMI system 17 to estimate the vehicle's position and provide accompanying information. For example, when the driver support ECU 30 requests speed reduction based on a deceleration request unit F8, the notification processing unit F9 cooperates with the HMI system 17 to notify the driver of speed reduction due to adverse environmental conditions. For instance, the notification processing unit F9 displays a deceleration notification image on the display 171. Furthermore, notifications related to speed reduction can also be implemented via audio message output from the speaker. The timing of implementing speed reduction-related notifications is not limited to the case where the driver support ECU 30 has actually begun speed reduction.

[0106] like Figure 4 As shown, the deceleration notification image can also include the amount of deceleration, the reason for deceleration, and the standard for maintaining the deceleration state. Including the amount of deceleration in the deceleration notification image can reduce concerns about causing users to feel uncomfortable. Furthermore, informing users of reasons for deceleration such as heavy rain or afternoon sun can further improve user acceptance. In addition, the deceleration notification image can also be... Figure 4 An image containing text, as illustrated. A slowdown notification image can be... Figure 5 An icon image that does not contain a string, as shown in (A), can also be... Figure 5 (B) shows an icon image containing text related to the changed speed, such as the amount of deceleration. It can also be found in... Figure 5 The icon image shown is accompanied by additional icon images representing different types of severe environments, such as heavy rain and afternoon sun.

[0107] The notification processing unit F9 can also, before performing speed suppression (e.g., before outputting a deceleration request signal), use a pre-defined image or sound message, i.e., a deceleration confirmation message, to inquire with the user whether the vehicle speed can be suppressed. In this case, if the user permits speed suppression, the deceleration request unit F8 can output a deceleration request signal to the driver support ECU 30. Alternatively, the driver support ECU 30 can also perform the inquiry regarding whether deceleration is permissible.

[0108] Similar to deceleration notification images, deceleration confirmation messages can also include information such as the amount of deceleration and the duration of the deceleration. By including the amount of deceleration in the confirmation message, concerns about decelerating to an undesirable level can be reduced. Furthermore, since the amount of deceleration is announced in advance, concerns that might confuse the user can also be reduced. As a result, concerns about compromising user satisfaction with the vehicle can be minimized.

[0109] <Workflow of Position Estimator 20>

[0110] Next use Figure 6 The flowchart shown illustrates the adverse environment determination processing performed by the position estimator 20. For example, during the period when the vehicle's driving power is on, it is performed at predetermined intervals (e.g., every second). Figure 6 The flowchart shown illustrates this. In engine-powered vehicles, the driving power supply is, for example, the ignition switch power supply. In electric vehicles, the system main relay corresponds to the driving power supply. Furthermore, the detailed position calculation unit F7 and... Figure 6 The harsh environment determination related processes shown are executed independently (i.e., in parallel) according to the default settings or the settings determined by the harsh environment determination related processes, and the positioning processes are performed sequentially. In this embodiment, as an example, the harsh environment determination related processes include steps S1 to S11.

[0111] First, in step S1, the adverse environment determination unit F6 acquires various information such as effective recognition distance, temperature, and humidity to determine whether the current situation constitutes an adverse environment and proceeds to step S2. In step S2, the adverse environment determination unit F6 determines whether it is an adverse environment. For example, if the effective recognition distance is less than a predetermined distance threshold, it is determined to be an adverse environment and proceeds to step S3. Conversely, if it is determined not to be an adverse environment, the process ends. If it is determined not to be an adverse environment, the detailed location calculation unit F7 performs positioning processing according to predetermined default settings.

[0112] In step S3, the deceleration request unit F8 determines whether deceleration is permissible. For example, it can cooperate with the notification processing unit F9 to output a deceleration confirmation message, and determine whether deceleration is permissible based on the user's response to this output. That is, it can be determined that deceleration is permissible if the user permits it, and that deceleration is not permissible if the user denies the deceleration. The user's response can be obtained via a switch, touch panel, voice input device, etc., provided on the steering wheel, etc.

[0113] Furthermore, in step S3, the deceleration request unit F8 can also determine whether deceleration is possible based on whether the current target speed setting value reaches the lower limit of the specified variable speed zone. That is, it can also determine that deceleration is not possible if the target speed setting value reaches the lower limit of the variable speed zone. It can also determine that deceleration is possible if the target speed setting value does not reach the lower limit of the variable speed zone, or if it is a value greater than the lower limit by a specified value. If it is determined that deceleration is possible, step S4 is executed; otherwise, if it is determined that deceleration is not possible, step S6 is executed.

[0114] In step S4, the processing unit F9 is notified to implement a notification to suppress the vehicle speed by a predetermined amount. For example, a deceleration notification image is displayed on the display 171 and the process proceeds to step S5. Alternatively, the deceleration notification image can be continuously displayed during deceleration performed in conjunction with a severe environment determination.

[0115] In step S5, the deceleration request unit F8 cooperates with the driving support ECU 30 to suppress vehicle speed. Specifically, the deceleration request unit F8 outputs a deceleration request signal to the driving support ECU 30, and the driving support ECU 30 performs deceleration based on this signal. As described above, the deceleration can reduce either the target speed or the ACC upper limit speed. Here, as an example, we assume that the deceleration amount for each deceleration request is, for example, 2 km / h. If the processing in step S5 is completed, the process proceeds to step S6. Furthermore, generally speaking, compared to high-speed movement, low-speed movement can be expected to produce clearer images captured by the forward camera 11. Of course, if the image is clear, the accuracy of object recognition using the image frame can also be expected to improve. In other words, by performing the speed reduction process in step S5, the accuracy of object recognition by the forward camera 11 can be expected to recover.

[0116] In step S6, the period change unit F71 changes the position estimation period from the default value to a value suitable for harsh environments. This increases the frequency at which the detailed position calculation unit F7 performs positioning processing, ensuring that even in harsh environments, the number of successful positioning processes per unit time remains at the same level as in favorable environments. If step S6 is completed and a constant time or a predetermined number of positioning processes have been performed, step S7 is executed. Alternatively, step S7 can be executed when passing landmarks.

[0117] In step S7, the recognition accuracy evaluation unit F5 acquires information related to recognition accuracy. For example, the recognition accuracy evaluation unit F5 acquires the reliability of object recognition from the front camera 11 and moves to step S8. In step S8, it is determined whether the evaluation value of the recognition accuracy of the front camera 11 is within a specified allowable range. For example, it is determined whether the reliability acquired in step S7 is above a specified threshold. Reliability is equivalent to the evaluation value of recognition accuracy. If the reliability is above the specified threshold, the recognition accuracy is considered to be at an allowable level and the process moves to step S9. On the other hand, if the reliability is less than the specified threshold, the recognition accuracy is considered to have not reached an allowable level, and the processing after step S3 is executed again.

[0118] By repeatedly executing steps S3 to S8, for example... Figure 7 As shown, the vehicle speed gradually decreases. Furthermore, the deceleration rate during this decrease can be, for example, 0.5 m / s² (≒0.05G). This control method allows for gradual deceleration, reducing concerns about discomfort for the driver and passenger. Moreover, deceleration stops when the object recognition accuracy reaches an acceptable level. This configuration, in one respect, is equivalent to a configuration that decelerates until the recognition accuracy reaches an acceptable level; in other words, it is equivalent to a configuration that stops further deceleration at the stage where the recognition accuracy reaches an acceptable level.

[0119] By repeatedly performing deceleration processing and confirming recognition accuracy, as described above, with the target speed reduction amount set to approximately several km / h each time, excessive deceleration can be prevented. Furthermore, the cycle change in step S6 only needs to be performed once in this process, and can be omitted when repeatedly executing steps S3 to S8. The process of asking the user whether deceleration is permissible is also the same.

[0120] Furthermore, although the reliability of the recognition result is used as an indicator of recognition accuracy in this embodiment, it is not limited to this. The indicator of recognition accuracy can also be the false negative rate. The false negative rate is also equivalent to an evaluation value of recognition accuracy. Additionally, the determination of whether to perform further deceleration in step S8 may not directly represent recognition accuracy. For example, in step S8, it may be configured to determine whether the estimation error is within a specified allowable range. The estimation error can also be included in the evaluation value of recognition accuracy.

[0121] The upper limit of the allowable range of estimation error (hereinafter referred to as the error threshold Pth) can be, for example, 0.5m, 1m, etc. The error threshold Pth can also be applied differently depending on the type of road. For example, the error threshold Pth1 for large-scale roads such as intercity expressways with a speed limit of 70km / h or higher can be set to a larger value than the error threshold Pth2 for general roads, urban expressways, etc. The error threshold Pth1 can be, for example, 1m to 3m. Furthermore, it is preferable that the error threshold Pth2 is less than 1m, for example, it can be 0.5m. By making the error threshold Pth2 for general roads more stringent than the error threshold Pth1, concerns about exceeding the temporary stop line can be reduced.

[0122] In step S9, the position estimation period and target speed settings established through the above processing are maintained, and the process proceeds to step S10. In step S10, it is determined whether the adverse environment continues. If it is still an adverse environment, the same method as in step S2 can be used. If it is determined in step S10 that the adverse environment continues, the processing after step S8 is executed again. On the other hand, if it is determined in step S10 that it is not an adverse environment, the process proceeds to step S11. In step S11, the various settings that were temporarily changed in the processing of steps S1 to S9 are returned to their default values, and the process ends.

[0123] The acceleration required to return the speed to its original value can be, for example, 0.5 m / s². Furthermore, even with the same magnitude of acceleration, passenger discomfort is less likely during acceleration compared to deceleration. This is because the body is supported by the seat back. The magnitude of the acceleration applied during speed recovery can also be set to be larger than the value applied during deceleration. For example, the target value for acceleration can be 0.8 m / s² or 0.9 m / s². From a ride comfort point of view, the jerk, which represents the rate of change of acceleration over time, is preferably controlled within a range from 1.0 m / s³ to 1.5 m / s³, such as 1.3 m / s³.

[0124] Furthermore, the target speed for speed recovery refers to the target speed of the ACC at the time of execution of step S1, i.e., the value set by the user. For convenience, it is also referred to as the original target speed. The driver support ECU 30 can begin recovery towards the original target speed by outputting a speed recovery permission signal to the driver support ECU 30, for example, through the deceleration request unit F8. Based on the input of the speed recovery permission signal, the driver support ECU 30 gradually increases the speed to the target speed with the aforementioned acceleration. Alternatively, speed recovery can also begin by stopping the output of the deceleration request signal.

[0125] Based on the above structure, vehicle speed is suppressed when determining adverse environmental conditions. Lowering the vehicle speed can be expected to improve image clarity, thus improving image recognition accuracy. Improved image recognition accuracy allows for the detection of relatively distant landmarks or reduces the false negative rate of landmarks. This increases the probability of landmark detection, thereby increasing the opportunity to perform localization processing. Consequently, even in adverse environments, the degradation of vehicle position estimation accuracy can be suppressed.

[0126] Furthermore, if the image recognition accuracy of the forward camera 11 improves, the accuracy of the observed coordinates of the landmarks also improves. Therefore, it is also possible to expect an improvement in the accuracy of the vehicle's position estimation based on the positioning processing. Thus, according to the above configuration, the degradation of position estimation accuracy can be suppressed even in harsh environments, thereby improving driving safety in harsh environments.

[0127] Furthermore, by suppressing the driving speed based on the determination of adverse environmental conditions, the time from the start of landmark identification to passing the landmark can be extended compared to the configuration without suppressing the driving speed. According to this configuration, the number of times / opportunities for positioning processing can be increased compared to the configuration without suppressing the driving speed. As a result, the degradation of the vehicle's position estimation accuracy can be further suppressed.

[0128] Furthermore, based on the above configuration, the position estimation cycle, i.e., the execution cycle of the positioning process, is shortened when a harsh environment is identified. With this configuration, the number of positioning processes executed per unit time increases. Therefore, even assuming a decrease in the success rate of positioning processing, it can be expected that the number of successful positioning processes within a constant time remains at the same level as in a favorable environment. As a result, the degradation of the vehicle's position estimation accuracy can be suppressed.

[0129] Furthermore, it is assumed that the deterioration of the vehicle's position estimation accuracy during autonomous driving leads to the interruption of autonomous driving. Of course, if this results in frequent interruptions of the autonomous driving function, user convenience is reduced. To address this concern, based on the above configuration, the vehicle's position estimation accuracy is not easily deteriorated even in adverse environments, thus reducing concerns about the interruption of autonomous driving due to deterioration in vehicle position estimation accuracy. Moreover, the configuration that suppresses driving speed based on the determination of an adverse environment is equivalent to a configuration that degrades functionality based on the determination of an adverse environment. Other examples of configurations that degrade functionality based on the determination of an adverse environment include prohibiting overtaking, reducing the allowable acceleration by a specified amount during control planning, or reducing the allowable value (upper limit) of the yaw rate by a specified amount. The driving support ECU 30 can also be configured to reduce the upper limit of acceleration / deceleration and yaw rate used in the control plan by a specified amount compared to a specified standard upper limit value based on the position estimator 20's determination of an adverse environment. Here, the standard upper limit value refers to the upper limit value applied when the environment is not determined to be adverse.

[0130] The embodiments of this disclosure have been described above, but this disclosure is not limited to the above-described embodiments. Various modifications described below are also included within the technical scope of this disclosure, and various changes can be made without departing from the spirit of the text, in addition to those described below. For example, the various modifications described below can be appropriately combined and implemented without creating technical contradictions. Furthermore, components having the same function as those described in the above embodiments are given the same reference numerals, and their descriptions are omitted. Additionally, when only a part of the configuration is mentioned, the configuration of the previously described embodiments can be applied to the other parts.

[0131] <Supplement to Deceleration Control Methods>

[0132] The deceleration requirement unit F8 can also instruct the driver support ECU 30 to decelerate to a predetermined provisional target speed based on the determination of adverse environmental conditions. The provisional target speed can be a speed limit or a value less than the speed limit. If the recognition accuracy does not reach the permissible level even after reducing the speed to the provisional target speed, the deceleration requirement unit F8 can further reduce the provisional target speed by a predetermined amount.

[0133] Furthermore, even if the recognition accuracy does not reach the acceptable level when the driving speed is suppressed to the limit speed or a value less than the limit speed, the position estimator 20 may not require further deceleration and can instead utilize other methods to improve recognition accuracy. Methods other than deceleration include shortening the estimation cycle, utilizing the millimeter-wave radar 12 (described later), and changing the image frames used in the positioning process. Additionally, in the presence of a preceding vehicle, safety can be ensured by following it without overtaking, even if the preceding vehicle's speed is assumed to be slower than the target speed of the current vehicle.

[0134] Furthermore, while the above describes the conditions under which deceleration should begin in conditions deemed equivalent to adverse weather, in actual driving environments, it is also assumed that, from the perspective of traffic rules and coordination with other vehicles, scenarios where immediate deceleration is not preferable are also considered. Scenarios where immediate deceleration is not preferable include, for example, driving in an overtaking lane or driving in an acceleration lane used for merging onto a main road to a highway.

[0135] Based on this situation, the deceleration request unit F8 can also determine whether the vehicle is traveling in the overtaking lane, etc., when determining in step S3 whether deceleration is possible. The deceleration request unit F8 can also determine that deceleration is not possible based on the vehicle being traveling in the overtaking lane. Furthermore, if the vehicle is traveling in the overtaking lane, the start of deceleration can be maintained until the vehicle moves into the driving lane. The maintenance of the start of deceleration corresponds to the output of the maintenance deceleration request signal.

[0136] use Figure 8 The flowchart shown illustrates an example of the operation of the position estimator 20 corresponding to the technical concept described above. It can be implemented in parallel with, combined with, or partially substituted for the processing following step S3 described above. Figure 8 The processing flow is shown below. For example... Figure 8 The flowchart shown can be used as the implementation for processes after step S2. For convenience, it will be compared with... Figure 8 The process corresponding to the flowchart shown is called deceleration and braking process. As an example, the deceleration and braking process includes steps S101 to S107. For example, it is executed by processing unit 21, which is the deceleration request unit F8. Figure 8 The steps shown can be modified as appropriate.

[0137] First, in step S101, the determination result of the adverse environment determination unit F6 is obtained, and it is determined whether the current driving environment is an adverse environment. If it is determined to be equivalent to an adverse environment, the process proceeds to step S102. On the other hand, if it is determined not to be an adverse environment, the process ends. In step S102, based on the lane ID determined by the detailed position calculation unit F7, it is determined whether the vehicle is driving in the overtaking lane.

[0138] An overtaking lane is a lane used for passing other vehicles from the side. Generally, lanes other than the overtaking lane are called driving lanes. In this disclosure, lanes other than the overtaking lane are also referred to as driving lanes. The setting of the overtaking lane is determined according to the traffic rules of the region where the vehicle equipped with the driving assistance system 1 is used. For example, in Japan, the rightmost lane is equivalent to the overtaking lane, and the lane to its left is equivalent to the driving lane. In addition, in Germany and other countries where right-hand traffic is allowed, only the rightmost lane is the driving lane, and the lane to its left is the overtaking lane. Which lane is equivalent to the overtaking lane can be registered in map data or determined based on usage area information set in the position estimator 20 and the lane information identified by the forward camera 11. The country code or region code can be used as the usage area information.

[0139] For example, in step S102, the lane ID is compared with the number of lanes on the driving route shown in the map data to determine whether the lane is equivalent to an overtaking lane. If the lane is equivalent to an overtaking lane, the process moves to step S103. On the other hand, if the lane is not equivalent to an overtaking lane, the process moves to step S105.

[0140] In step S103, lane changing is performed in cooperation with the driving support ECU 30. For example, the deceleration request unit F8 outputs a signal to the driving support ECU 30 requesting to move to the driving lane. Based on receiving this request signal, the driving support ECU 30 formulates a plan to move to the driving lane and executes it. For example, the driving support ECU 30 becomes the subject of driving operation (in other words, automatically) and performs the lane change. In one aspect, this control method is equivalent to control that cancels a planned overtaking maneuver and returns to the driving lane based on a judgment that the environment is unfavorable to the camera. Alternatively, control can be implemented by returning to the driving lane after overtaking is completed if it is calculated that the overtaking will be completed within a specified time.

[0141] Furthermore, in driving modes where the driver support ECU 30 is primarily responsible for steering, the ECU 30 can either allow the driver to remove their hands from the steering wheel or require them to maintain a light grip. Of course, it can also be used for lane-changing maneuvers primarily by the driver. In this case, the driver support ECU 30 can provide driving support controls such as traffic condition alerts for the lane-changing destination and steering torque assistance based on the lane-changing request.

[0142] Furthermore, the process of moving the driving position to the driving lane may also include asking the driver / passenger if it is permissible to move the driving position to the driving lane. For example, the position estimator 20 or the driver support ECU 30 could also do so. Figure 9 The output notification shows an overtaking abort notification image moved to the driving lane, and the system determines whether to perform a lane change based on the driver's passenger's response to this output. The position estimator 20 can also determine that movement to the driving lane is permitted if no response is received from the driver's passenger upon displaying the overtaking abort image. The overtaking abort notification image may also include a message requesting the driver's passenger to perform driving operations if they wish to continue driving in the overtaking lane. Furthermore, if the position estimator 20 detects driver's passenger actions such as pressing the accelerator pedal after the overtaking abort image is displayed, it can determine that a lane change is rejected.

[0143] When the driver has given permission to move into the driving lane, the deceleration request unit F8 outputs a lane change request signal to the driving support ECU 30. On the other hand, if the driver refuses to move into the driving lane, the unit can also notify the driver that system-based driving support, such as automatic speed adjustment, is difficult to continue or can be terminated.

[0144] If the movement to the driving lane is completed, step S104 is executed. The completion of the movement to the driving lane can be determined based on the recognition result of the forward camera 11, such as the lane ID becoming 1. In step S104, a deceleration request signal is output to the driver support ECU 30, and the process ends. The deceleration request signal can be output repeatedly until the recognition accuracy reaches an acceptable level, or it can be output only once. Furthermore, the deceleration request signal output here can also be a signal indicating deceleration up to a predetermined provisional target speed, such as a speed limit.

[0145] In step S105, the lane ID is compared with the road structure information shown in the map data to determine whether the vehicle is traveling in the acceleration lane. If the vehicle is traveling in the acceleration lane, step S105 is affirmatively determined and the process proceeds to step S106. On the other hand, if the vehicle is not traveling in the acceleration lane, step S107 is executed.

[0146] In step S106, in cooperation with the HMI system 17, an image and sound are output requesting the driver to manually adjust the speed and steering, and the process ends. This configuration reduces concerns about system-based deceleration occurring in situations where acceleration should occur. Furthermore, by quickly transferring operational control to the driver in situations where system-based speed maintenance or acceleration is difficult, safety is ensured. However, step S106 is just one example; the processing during acceleration in the lane is not limited to this. It is also possible to abort deceleration and select a process to accelerate to the prescribed speed for merging when the traffic volume on the main road is below a specified value and it is determined that merging on the main road can be done at a speed specified below the target merging speed used in normal operation. The target merging speed used in normal operation can be, for example, the speed limit of the main road or a user-set vehicle speed.

[0147] In step S107, the same as in step S104, a deceleration request signal is output to the driver support ECU 30 and the process ends. Furthermore, steps S105 to S106 described above can be any process or can be omitted. In this case, if it is determined in step S102 that the lane is not an overtaking lane, the process moves to step S107, and deceleration control begins. Based on the above configuration, concerns can be reduced about automatically initiating deceleration based on adverse environmental conditions in situations where deceleration should not occur.

[0148] <Supplement to Speed ​​Recovery Control>

[0149] Furthermore, the above describes the process of restoring the suppressed speed to its original speed in step S11 when the environment is determined not to be adverse in step S10. However, in actual driving environments, it is also assumed that acceleration is not preferable from the viewpoint of traffic regulations and coordination with other vehicles. Scenarios where acceleration is not preferable include situations where there is a curve or obstacle within a specified distance ahead, or situations where the distance to the preceding vehicle is within a specified distance. Obstacles here refer to lane restrictions caused by construction, fallen objects, parked vehicles on the road, accident vehicles, etc.

[0150] Based on this, the position estimator 20 can also determine whether to accelerate based on the road structure ahead, provided that the detected environment is not adverse. For example, it can maintain acceleration if the remaining distance to the curve is less than a predetermined threshold. The start of maintaining acceleration corresponds, for example, to the output of a speed maintenance recovery permission signal.

[0151] use Figure 10The flowchart shown illustrates an example of the operation of the position estimator 20 corresponding to the technical concept described above. It can be implemented in parallel with, combined with, or partially substituted for step S11 described above. Figure 10 The processing flow is shown below. For example... Figure 10 The flowchart shown can be used as the implementation for processes after step S11. For convenience, it will be compared with... Figure 10 The process corresponding to the flowchart shown is called deceleration and braking process. As an example, the deceleration and braking process includes steps S201 to S209. The deceleration and braking process can be performed at a predetermined period (e.g., every second) during the period when the speed is suppressed due to a severe environmental condition determination. Figure 10 The steps shown are performed, for example, by the processing unit 21.

[0152] First, in step S201, the determination result of the harsh environment determination unit F6 is obtained, and it is determined whether the harsh environment should continue. If it is determined that the environment is still harsh, step S201 is affirmatively determined and the process ends. On the other hand, if it is determined that the environment is not harsh, the process proceeds to step S202.

[0153] In step S202, referring to map data, it is determined whether there is a curve within a predetermined distance ahead. This determination process is equivalent to determining whether the remaining distance to the curve calculated using map data is less than a predetermined threshold. If there is a curve within the predetermined distance ahead, proceed to step S203. On the other hand, if there is no curve within the predetermined distance ahead, proceed to step S204. In step S203, it is decided to maintain the acceleration toward the original target speed until passing the curve ahead, and the process ends. The original target speed, as described above, refers to the user-set vehicle speed or speed limit. Furthermore, even if the acceleration toward the target speed is maintained, if there is a preceding vehicle, acceleration may be performed in accordance with the actions of the preceding vehicle to maintain the predetermined inter-vehicle distance.

[0154] In step S204, the presence of an obstacle is determined based on the signal input from the front camera 11 or the millimeter-wave radar 12. If an obstacle is detected by the front camera 11 or the millimeter-wave radar 12, the process proceeds to step S205. Conversely, if no obstacle is detected by the front camera 11 or the millimeter-wave radar 12, the process proceeds to step S206. In step S205, it is decided to maintain the acceleration towards the original target speed until the obstacle is passed, and the process ends.

[0155] In step S206, the presence of a preceding vehicle is determined based on the signal input from the forward camera 11 or millimeter-wave radar 12. If a preceding vehicle is detected, the process proceeds to step S207. Conversely, if no preceding vehicle is detected, the process proceeds to step S208. Alternatively, even if a preceding vehicle is detected, if the distance to the preceding vehicle is greater than a predetermined following distance, it may be considered that there is no preceding vehicle related to ACC, and the process proceeds to step S208. The following distance can be a constant value such as 60m or 100m, or it can be dynamically determined based on the concept of time intervals. For example, the following distance can be set to the equivalent of 2 seconds or 3 seconds.

[0156] In step S207, acceleration toward the target speed is stopped. In this case, the driving support ECU 30 can also begin speed control for following the preceding vehicle. In step S208, the vehicle is determined to be traveling in a deceleration lane by comparing the lane ID with the road structure information shown in the map data. If the vehicle is traveling in a deceleration lane, proceed to step S207 and stop acceleration. On the other hand, if the vehicle is not traveling in a deceleration lane, proceed to step S209. Furthermore, the determination process in step S208, i.e., determining whether the driving position is in a deceleration lane, can be any element or can be omitted.

[0157] In step S209, the driving support ECU 30 is instructed to begin accelerating toward the target speed, i.e., acceleration for speed recovery. For example, the deceleration request unit F8 outputs a speed recovery permission signal to the driving support ECU 30.

[0158] From the perspectives of passenger comfort, safety, and other considerations, it is preferable to decelerate while driving on or near curves. Additionally, it is preferable to decelerate when passing the side of an obstacle to prevent unexpected sudden movements. If, assuming a curve or obstacle exists ahead of the vehicle, acceleration occurs as the adverse environment condition is resolved, there is a possibility of subsequent deceleration. In other words, unnecessary acceleration or deceleration may occur. Based on the above process, when acceleration is a crucial factor in the presence of curves or obstacles ahead of the vehicle, acceleration for speed recovery is retained, thus preventing unnecessary acceleration.

[0159] <Supplement to Slow-Down-Based Harsh Environment Response>

[0160] The landmark information acquisition unit F3 can also be configured to determine the nearest landmark ahead by referring to map data and calculate the distance between the nearest landmark ahead and the vehicle as the nearest landmark distance. Here, the nearest landmark refers to the landmark registered in the map data that is closest to the vehicle in front of it. Furthermore, the deceleration request unit F8 can also be configured to perform deceleration-related processing if the nearest landmark distance is less than a specified distance. This deceleration-related processing includes inquiring with the driver's seat passenger whether the vehicle speed can be reduced, and outputting a deceleration request signal.

[0161] Furthermore, the deceleration amount can be adjusted based on the type of adverse environment. The type of adverse environment is determined by the type identification unit F61. For example, the deceleration amount for a heavy rainstorm can be set to be greater than the deceleration amount for a western sun or fog. With this configuration, an appropriate deceleration amount corresponding to the type of adverse environment can be applied.

[0162] Furthermore, the deceleration amount can be adjusted based on the severity of the environment, as determined by the severity determination unit F62. For example, it is preferable to set the deceleration amount to be greater as the severity of the environment increases. With this configuration, an appropriate deceleration amount corresponding to the severity of the environment can be applied.

[0163] The adverse environment determination unit F6 can also be configured to determine the scope of influence based on the type of adverse environment when it is determined to be an adverse environment, and the position estimator 20 can change the control measures implemented according to the size of the scope of influence. For example, if the scope of influence is tens or hundreds of meters and can be escaped within a few minutes, deceleration can be used as a response. On the other hand, if the scope of influence is several kilometers and the adverse environment lasts for more than ten minutes, the system can also cooperate with the HMI system 17 to ask the driver whether to stop the ACC function or to continue using the ACC function after deceleration. For example, in the case of heavy rain, the scope of influence can be estimated based on the size of the rain cloud that brings the rain and the predicted movement information of the rain cloud. For example, the size of the rain cloud and the predicted movement information can be obtained from an external server that distributes weather information via the V2X vehicle-mounted device 15. By configuring different responses based on the size of the scope of influence, responses that correspond to the preferences of the driver can be adopted. In addition, it can also be configured so that the driver can set responses corresponding to the size of the scope of influence, the type of adverse environment, and the severity of the adverse environment. With this configuration, it is expected that the driver's satisfaction with the vehicle's operation will be improved.

[0164] <Supplement to Location Processing>

[0165] The positioning unit F0 can also be configured to change the material used in the positioning process based on whether the harsh environment determination unit F6 determines that the surrounding environment is equivalent to a harsh environment.

[0166] For example, the positioning unit F0 may also be configured to switch between using the millimeter-wave radar 12 and other detection results during the positioning process based on whether it is determined to be an adverse environment. As a more specific example, the positioning unit F0 may also switch between using the millimeter-wave radar 12 and other detection results based on whether it is determined to be an adverse environment. Furthermore, since the location information of the landmark is used in the positioning process, changing the material used in calculating the location of the landmark is equivalent to changing the material used in the positioning process.

[0167] Specifically, when the adverse environment determination unit F6 does not determine that the surrounding environment is equivalent to an adverse environment, the positioning unit F0 calculates the observed coordinates of the landmark without using the detection results of the millimeter-wave radar 12 and performs positioning processing. On the other hand, when the adverse environment determination unit F6 determines that the environment is equivalent to an adverse environment, the positioning unit F0 calculates the observed coordinates of the landmark using the recognition results of the front camera 11 and the detection results of the millimeter-wave radar 12, and then performs positioning processing. The above configuration is equivalent to performing positioning processing without using the detection result data of the millimeter-wave radar 12 under normal circumstances, and performing positioning processing using the image data of the front camera 11 and the detection result data of the millimeter-wave radar 12 when an adverse environment is determined.

[0168] Based on the above configuration, the positioning unit F0 performs positioning processing without using the detection results of the millimeter-wave radar 12 when the environment is not harsh (i.e., under normal conditions), thus reducing the load on the processing unit 21. On the other hand, in harsh environments, positioning processing is performed using the observed coordinates of landmarks calculated through sensor fusion, thus suppressing the degradation of position estimation accuracy due to rainfall, etc.

[0169] Furthermore, in the process of combining image recognition results and radar detection results to detect landmarks, the landmark information acquisition unit F3 can also adjust the weight of the radar detection results in the landmark detection process based on whether it is determined to be equivalent to a harsh environment. Here, image recognition results refer to data generated by inputting image data captured by the front camera 11 into a designated recognizer, indicating the position, type, etc., of detected objects within the field of view of the front camera 11. Radar detection results refer to data detected by millimeter-wave radar 12 or LiDAR, indicating the relative position, type, and speed of objects present in front of the vehicle.

[0170] For example, if the adverse environment determination unit F6 determines the environment to be adverse, the weight of the radar detection results in calculating the location of the landmark can be increased compared to normal conditions. The millimeter-wave radar 12 is less susceptible to rain, fog, etc., so the above configuration allows for maintaining a high level of accuracy in estimating the distance to the landmark. Furthermore, once the location of the landmark is calculated through sensor fusion of the front camera 11 and the millimeter-wave radar 12, the weight of the image recognition results can be reduced (e.g., it can be 0). Specifically, from the moment sensor fusion is successful, the location of the landmark can be calculated / updated based on the radar detection results for a specified period, as long as the landmark can be tracked by the millimeter-wave radar 12. With this configuration, the frequency of the inability to determine the location of the landmark can be suppressed during rainfall when the recognition capability of the front camera 11 may deteriorate. As a result, the accuracy of the positioning process can be suppressed.

[0171] The position estimator 20 can also be configured to change the image frame used for positioning processing based on whether the adverse environment determination unit F6 determines that the surrounding environment is equivalent to an adverse environment. This change in image frame used for positioning processing can be achieved through cooperation with the front camera 11. Furthermore, since the positioning processing uses location information of landmarks based on image recognition, changing the image frame used for landmark detection is equivalent to changing the image frame used for positioning processing. Landmark detection also includes determining the location and type of the landmark.

[0172] For example, the front camera 11 Figure 11 As shown, sometimes it is configured to generate multiple types of frames with different exposure times in a prescribed order. Figure 11 The image frames shown are of type A, which are landmark frames captured with an exposure time suitable for landmark detection. Type B image frames are, for example, frames captured with an exposure time suitable for vehicle detection. Type C image frames are, for example, frames captured with an exposure time suitable for dividing line detection. Furthermore, the types of image frames generated by the front camera 11 are not limited to these; for example, they can be configured to generate image frames for pedestrian detection, or image frames for detecting lighting equipment such as headlights and traffic lights. For convenience, image frames used for detecting objects other than landmarks, such as image frames for vehicle detection, are designated as frames for other purposes.

[0173] When the front camera 11 generates multiple types of image frames with different purposes and exposure times, the positioning unit F0 can change the type and combination of image frames used for landmark detection based on whether the environment is determined to be adverse. For example, it is possible to perform positioning processing using only landmark frames under normal conditions, while in adverse environments, positioning processing using other purpose frames is performed in addition to positioning processing using landmark frames. By additionally performing positioning processing using other purpose frames in this way, the frequency of performing positioning processing can be increased by up to two times. In other words, the execution interval of positioning processing can be shortened.

[0174] Furthermore, when determining adverse environment conditions, the positioning unit F0 can also combine landmark frames and other purpose frames to generate a higher resolution image frame (hereinafter referred to as a super-resolution frame), and use the super-resolution frame to perform positioning processing. For example, super-resolution frames can be generated using various super-resolution techniques such as fractal methods and methods utilizing self-similarity. In addition, as a premise of the above configuration, it is assumed that the recognizer G1 is also configured to detect landmarks based on other purpose frames such as vehicle detection image frames and super-resolution frames.

[0175] The driving support system 1 can also have multiple cameras with different viewing angles as a front camera 11. For example, Figure 12 The device can also include three cameras: a mid-range camera 11a, a telephoto camera 11b, and a wide-angle camera 11c. The mid-range camera 11a has a lens configured with an angle of view of approximately 50 degrees, capable of capturing images up to 150 meters away. The telephoto camera 11b has a relatively narrow angle of view, allowing it to capture images at greater distances than the mid-range camera 11a. For example, the telephoto camera 11b has an angle of view of approximately 30 to 40 degrees and can capture images at distances exceeding 250 meters. The wide-angle camera 11c is configured to capture a wide area around the vehicle. For example, the wide-angle camera 11c has an angle of view of approximately 120 to 150 degrees and can capture images within 50 meters in front of the vehicle.

[0176] In the case where the vehicle is equipped with multiple types of front cameras 11 as described above, the positioning unit F0 can switch the source of the image frames used for landmark detection based on whether the environment is deemed to be adverse. For example, it can be configured to use image frames from the mid-range camera 11a for landmark detection under normal conditions, and use image frames from the telephoto camera 11b for landmark detection when exposed to afternoon sun. Since the telephoto camera 11b has a relatively narrow angle, it is less affected by afternoon sun compared to the mid-range camera 11a, which has a wider angle of view. Therefore, by switching the camera used for positioning processing to the relatively narrow-angle telephoto camera 11b in backlight conditions such as afternoon sun, as described above, it is expected to suppress the degradation of landmark recognition accuracy. Furthermore, it is also possible to not completely switch the front cameras 11 used for positioning processing, using only the mid-range camera 11a under normal conditions, and using both the mid-range camera 11a and the telephoto camera 11b in backlight conditions. In other words, when multiple types of front cameras 11 are present, the positioning unit F0 can also be configured to change the combination of front cameras 11 used for positioning processing based on whether the environment is deemed to be adverse.

[0177] The driving support system 1 may also include a LiDAR with a detection range covering the area in front of the vehicle. A LiDAR is a device that generates three-dimensional point data representing the positions of reflection points in each detection direction by illuminating a laser. This three-dimensional point data is equivalent to the observation data for the LiDAR. The detection results of the LiDAR with a detection range covering the area in front of the vehicle are the same as those of the millimeter-wave radar described above, and can be used for positioning processing.

[0178] The above are examples of temporary measures (hereinafter referred to as adverse environment response measures) used when determining adverse environment conditions. These measures include suppressing vehicle speed, shortening the position estimation period, changing the degree of sharing of detection results from the millimeter-wave radar 12, and changing the image frames used for landmark detection. Appropriate measures can be selected as the actual adverse environment response measures, or all of the above measures can be implemented without parallel processing. One or more of the various adverse environment response measures described above can be implemented when determining adverse environment conditions. For example, if the environment is determined to be equivalent to an adverse environment, vehicle speed can be reduced, but the estimation period can be left unchanged.

[0179] Alternatively, it can be configured as a combination of adverse environment response measures implemented according to changes in the severity of the adverse environment. For example, in the case of a severity level 1 (a lower case), the position estimation period can be limited to a certain extent, while in the case of a severity level 2 or higher (a higher case), speed suppression can be additionally implemented. Furthermore, in the case of a severity level 3, the amount of speed suppression (i.e., the amount of deceleration) can be increased compared to the case of level 2. The higher the severity of the adverse environment, the shorter the position estimation period can be.

[0180] Furthermore, it can be configured as a combination of adverse environment response measures that are modified according to the type of adverse environment. For example, in the case of fog, the position estimation period can be shortened to a limited extent, while in the case of heavy rain, vehicle speed suppression can be additionally implemented. In addition, in the case of afternoon sun, the image frames used for positioning can be changed, and the positioning period can be shortened. Changing the image frames used for positioning also includes switching the image frames used for positioning from those captured by the mid-range camera 11a to those generated by the telephoto camera 11b. Generating super-resolution frames and using them for positioning processing also includes the concept of changing the image frames used for positioning processing.

[0181] Furthermore, super-resolution frames can also be generated by appending information from other purpose frames to positioning frames generated by a front-facing camera 11 (e.g., a mid-range camera 11a). Alternatively, super-resolution frames can be generated by combining image frames from the mid-range camera 11a with image frames from the telephoto camera 11b. In other words, the images used for super-resolution processing can be multiple frames generated by the same camera or multiple frames generated by different cameras. Moreover, depending on the method of generating super-resolution frames by combining images from the mid-range camera 11a and the telephoto camera 11b, information about distant areas that the mid-range camera 11a is not adept at using is appended to the image from the telephoto camera 11b. As a result, an improvement in the effective recognition distance for mid- to distant landmarks can be expected.

[0182] In addition, the position estimator 20 can also request the ACC function to be stopped if the position estimation error exceeds a predetermined threshold even after adverse environment response processing is implemented. Furthermore, in configurations capable of implementing Level 3 or higher autonomous driving, if the position estimation error in autonomous driving exceeds a predetermined threshold even after adverse environment response processing is implemented, the process of transferring driving authority to the driver's seat passenger can also be performed in cooperation with the HMI system 17.

[0183] <Supplement to System Composition>

[0184] The above embodiment illustrates a configuration where the position estimator 20 is positioned outside the front camera 11, but the configuration of the position estimator 20 is not limited to this. Figure 13 The functionality of the position estimator 20 shown can also be integrated into the camera ECU 41. Additionally, as... Figure 14 As shown, the function of the position estimator 20 can also be built into the driving support ECU 30. The driving support ECU 30, which includes the function of the position estimator 20, is equivalent to a driving control device. In addition, the driving support ECU 30 can also have the function of the camera ECU 41 (mainly the recognition device G1). That is, it can also be configured such that the front camera 11 outputs image data to the driving support ECU 30, and the driving support ECU 30 performs image recognition and other processing.

[0185] Furthermore, when the driving support ECU 30 has the function of the position estimator 20, such as the function of the adverse environment determination unit F6, the driving support ECU 30 performs processing to suppress vehicle speed based on the determination that the environment is adverse. In addition, the driving support ECU 30 equipped with the adverse environment determination unit F6 can also automatically perform processing to gradually restore the vehicle speed to the original target speed based on the determination that the environment is no longer adverse. Of course, when driving in the overtaking lane with deceleration due to adverse environment determination, it can be maintained until moving to the driving lane. In addition, if the presence of factors that are important for maintaining acceleration is detected, the speed recovery that occurs after leaving the adverse environment can be maintained or stopped.

[0186] In addition, such as Figure 15 As shown, the camera ECU 41 may also include a portion of the position estimator 20, such as a recognition accuracy evaluation unit F5 and a severe environment determination unit F6. In this case, the camera ECU 41 outputs an environment determination signal SgX to the driver support ECU 30, indicating whether the driving environment is equivalent to a severe environment from the perspective of the external camera. The environment determination signal SgX can be a signal that indicates whether it is a severe environment with 0 / 1. Alternatively, the environment determination signal SgX can also be a signal indicating at least one of the degree of severe environment and the type of severe environment. Figure 15 The driving support ECU30 shown in the diagram can perform vehicle speed suppression and recovery based on the environmental determination signal SgX. Figure 15 The driving position control unit H2 shown represents a functional unit that controls the lateral driving position, such as the movement from the overtaking lane to the driving lane. Figure 15 The signal SgY shown represents the result of landmark recognition, and SgZ represents the result of positioning processing.

[0187] <Supplement to Driving Support ECU30>

[0188] The driving support ECU 30 can also be configured to automatically perform all operations involved in vehicle driving. That is, the driving support ECU 30 can also be configured as an automated operating device that implements driving control at Level 3 or higher of the automated driving system as defined by the Society of Automotive Engineers (SAE International). Furthermore, Level 1 of automated driving refers to a level where the system supports either steering or acceleration / deceleration operations; Level 2 of automated driving refers to a level where the system supports multiple steering and acceleration / deceleration operations. Level 3 of automated driving refers to a level where the system performs all driving operations within the Operational Design Domain (ODD) that specifies the conditions for automated driving, and, in emergencies, transfers control from the system to the driver. The ODD corresponds to the conditions for automated driving permission. Level 4 refers to a level where the system performs all driving operations within the ODD. Level 5 refers to a level where the system performs all driving operations without location restrictions. Levels 3 and above are equivalent to autonomous driving levels that automatically perform all controls involved in vehicle driving. When operating at autonomous driving level 3 or above, the driving support ECU30 automatically performs vehicle steering, acceleration, deceleration (i.e., braking), etc., so that the vehicle can travel along the road to the destination set by the driver or operator.

[0189] <Postscript (Part 1)>

[0190] The control unit and method described in this disclosure can also be implemented by a dedicated computer configured with a processor programmed to perform one or more functions embodied in a computer program. Furthermore, the apparatus and method described in this disclosure can also be implemented by dedicated hardware logic circuits. Moreover, the apparatus and method described in this disclosure can also be implemented by one or more dedicated computers configured by a combination of a processor executing a computer program and one or more hardware logic circuits. Additionally, the computer program can be stored as instructions executable by a computer on a non-transitional tangible recording medium readable by a computer. That is, the unit and / or functions provided by the position estimator 20 can be provided by software recorded in a physical memory device and a computer executing that software, by software alone, by hardware alone, or by a combination thereof. Some or all of the functions possessed by the position estimator 20 can also be implemented as hardware. Implementing a function as hardware includes using one or more ICs, etc. The processing unit 21 can also be implemented using an MPU or GPU instead of a CPU. Furthermore, the processing unit 21 can be implemented by combining multiple types of arithmetic processing devices such as CPUs, MPUs, and GPUs. Furthermore, ECUs can also be implemented using FPGAs (field-programmable gate arrays) or ASICs (application-specific integrated circuits). Various programs can be stored on non-transitory tangible storage media. As the program storage medium, various storage media can be used, such as HDDs (hard disk drives), SSDs (solid state drives), EPROMs (erasable programmable ROMs), and flash memory.

[0191] <Postscript (Part Two)>

[0192] This disclosure also includes, for example, the following components.

[0193] [Composition (1)]

[0194] A vehicle position estimation device for estimating the current position of a vehicle when mounted on a vehicle.

[0195] It has a control unit (21) consisting of at least one processor.

[0196] The control unit has:

[0197] The positioning unit (F0) determines the vehicle's coordinates on the map based on the location information of designated landmarks detected using image frames captured by the front-facing camera and the location information of landmarks registered on the map; and

[0198] The environmental assessment unit (F6) determines, based on at least one of the information from sensors and communication devices mounted on the vehicle, whether the vehicle's surrounding environment constitutes a specified harsh environment that may reduce the object recognition performance of the image frames used.

[0199] The control unit is configured to output a signal to the vehicle control module (30), which automatically controls the driving speed within a specified speed range, a signal that requests a temporary reduction in the upper limit of the speed range, based on the condition that the environment determination unit determines that the surrounding environment is equivalent to a harsh environment.

[0200] Based on the above configuration, in adverse environment conditions, the upper limit of the speed range that vehicle control modules providing autonomous driving and ACC functions can use can be reduced. As a result, the vehicle's speed is indirectly suppressed, thus improving the clarity of camera images and increasing the success rate of positioning processing and the accuracy of position estimation.

[0201] [Composition (2)]

[0202] A vehicle position estimation device for estimating the current position of a vehicle.

[0203] It has a control unit (21) consisting of at least one processor.

[0204] The control unit has:

[0205] The positioning unit (F0) determines the vehicle's coordinates on the map based on the location information of designated landmarks detected using image frames captured by the front-facing camera and the location information of landmarks registered on the map; and

[0206] The environmental assessment unit (F6) determines, based on at least one of the information from sensors and communication devices mounted on the vehicle, whether the vehicle's surrounding environment constitutes a specified harsh environment that may reduce the object recognition performance of the image frames used.

[0207] The control unit is configured to output a signal to the vehicle control module (30) that automatically controls the vehicle's speed, based on the condition that the surrounding environment is equivalent to a harsh environment, requiring the target speed setpoint used for speed control to be reduced by a specified amount.

[0208] The above configuration is equivalent to temporarily reducing the target speed set by the driver's passenger for ACC by a predetermined amount based on the determination of adverse environmental conditions. According to this configuration, it is possible to suppress the driving speed when not following a preceding vehicle.

[0209] [Composition (3)]

[0210] A vehicle position estimation device for estimating the current position of a vehicle.

[0211] It has a control unit (21) consisting of at least one processor.

[0212] The control unit has:

[0213] The positioning unit (F0) determines the vehicle's coordinates on the map based on the location information of designated landmarks detected using image frames captured by the front-facing camera and the location information of landmarks registered on the map; and

[0214] The environmental assessment unit (F6) determines, based on at least one of the information from sensors and communication devices mounted on the vehicle, whether the vehicle's surrounding environment constitutes a specified harsh environment that may reduce the object recognition performance of the image frames used.

[0215] The control unit is configured to acquire detection result data from an object detector (12) that uses radio waves or lasers to detect objects present in front of the vehicle.

[0216] The positioning unit combines image frames captured by the front camera and detection result data from the object detector to perform the processing of calculating the vehicle's position, and configures the weights of the object detector detection result data in the process of calculating the vehicle's position to be changed based on whether the environment determination unit determines that the surrounding environment is equivalent to a harsh environment.

[0217] In the above configuration, the weight of the detection result of the millimeter-wave radar 12 during sensor fusion is changed depending on whether the environment is determined to be severe. For example, when a severe environment is determined, the weight of the detection result of the millimeter-wave radar 12 is increased compared to the normal environment. Furthermore, the method of changing the weight ratio also includes a configuration that temporarily sets the weight of the image recognition result to 0. For example, the detection result of the millimeter-wave radar 12 can be increased for a specified time from the last successful fusion, so that tracking can be performed using only the millimeter-wave radar 12. Alternatively, the weight of the detection result of the millimeter-wave radar 12 in a distant area can be increased when a severe environment is determined. The method of changing the weight of the detection result of the millimeter-wave radar 12 in sensor fusion can also be changed according to the type and severity of the severe environment.

[0218] [Composition (4)]

[0219] A vehicle position estimation device for estimating the current position of the vehicle.

[0220] It has a control unit (21) consisting of at least one processor.

[0221] The control unit has:

[0222] The positioning unit (F0) determines the vehicle's coordinates on the map based on the location information of designated landmarks detected using image frames captured by the front-facing camera and the location information of landmarks registered on the map; and

[0223] The environmental assessment unit (F6) determines, based on at least one of the information from sensors and communication devices mounted on the vehicle, whether the vehicle's surrounding environment constitutes a specified harsh environment that may reduce the object recognition performance of the image frames used.

[0224] The control unit is configured to acquire detection result data from an object detector (12) that uses radio waves or lasers to detect objects present in front of the vehicle.

[0225] The positioning unit is configured to perform the process of calculating the vehicle's position without using the detection result data of the object detector when the adverse environment determination unit does not determine that the surrounding environment is equivalent to an adverse environment. On the other hand, when the adverse environment determination unit determines that the surrounding environment is equivalent to an adverse environment, the unit performs the process of calculating the vehicle's position using image frames captured by the front camera and the detection result data of the object detector.

[0226] Based on the above configuration, the detection results of the millimeter-wave radar 12 are not used under normal circumstances, thereby reducing the processing load. On the other hand, the detection results of the millimeter-wave radar 12 are used when determining adverse environments, thereby maintaining the recognition accuracy of the object and the vehicle's position.

[0227] [Composition (5)]

[0228] A vehicle position estimation device for estimating the current position of the vehicle.

[0229] It has a control unit (21) consisting of at least one processor.

[0230] The control unit has:

[0231] The positioning unit (F0) determines the vehicle's coordinates on the map based on the location information of designated landmarks detected using image frames captured by the front-facing camera and the location information of landmarks registered on the map; and

[0232] The environmental assessment unit (F6) determines, based on at least one of the information from sensors and communication devices mounted on the vehicle, whether the vehicle's surrounding environment constitutes a specified harsh environment that may reduce the object recognition performance of the image frames used.

[0233] The severe environment assessment component is responsible for determining the degree of severity of the environment.

[0234] The greater the severity of the adverse environment determined by the adverse environment determination unit, the shorter the execution interval for calculating the vehicle's position or the greater the requirement to reduce the driving speed.

[0235] [Composition (6)]

[0236] A driving control device mounted on a vehicle and automatically performing speed control of the vehicle.

[0237] It has a control unit (21, 31) consisting of at least one processor.

[0238] The control unit has:

[0239] The speed control unit (H1) outputs a control signal for speed adjustment to the vehicle's driving actuator (18) based on the detection results of the camera mounted on the vehicle.

[0240] The positioning unit (F0) determines the vehicle's coordinates on the map based on the location information of designated landmarks determined using images captured by a front-facing camera and the location information of landmarks registered on the map; and

[0241] The Harsh Environment Assessment Unit (F6) determines, based on at least one of the information from sensors and communication devices mounted on the vehicle, whether the vehicle's surrounding environment constitutes a specified harsh environment that may reduce the accuracy of object recognition using image frames.

[0242] The control unit is configured to suppress the driving speed based on the adverse environment determination unit's determination that the surrounding environment is equivalent to an adverse environment, and to notify the vehicle of the purpose of suppressing the speed via a reporting device.

Claims

1. A vehicle position estimation device, which is mounted on a vehicle and estimates the current position of the vehicle, wherein, It has a control unit consisting of at least one processor. The aforementioned control unit includes: The positioning unit determines the vehicle's position coordinates on the map based on the location information of the specified landmarks detected by the image frames captured by the front camera and the location information of the aforementioned landmarks registered on the map. as well as The adverse environment determination unit, based on at least one of the information from sensors and communication devices mounted on the vehicle, determines whether the surrounding environment of the vehicle is equivalent to a specified adverse environment that reduces the accuracy of object recognition using the image frames. The aforementioned control unit is configured such that, when not driving in an overtaking lane or acceleration lane, it outputs a deceleration request signal to the vehicle control module that automatically controls the vehicle's speed, based on the adverse environment determination unit's assessment that the surrounding environment is equivalent to an adverse environment. Conversely, when driving in an overtaking lane or acceleration lane, it does not output the deceleration request signal even if the adverse environment determination unit assesses that the surrounding environment is equivalent to an adverse environment. The deceleration request signal is a signal requesting the vehicle to slow down. The aforementioned severe environment assessment component is used to determine the degree of severe environment. The aforementioned control unit is configured to perform different processing based on the severity of the severe environment determined by the aforementioned severe environment determination unit.

2. The vehicle position estimation device according to claim 1, wherein, The aforementioned control unit includes: The recognition accuracy evaluation unit evaluates the recognition accuracy of the aforementioned landmarks based on the aforementioned image frames. Based on the severe environment determination unit's determination that the surrounding environment is equivalent to the severe environment, the control unit repeatedly outputs a signal to the vehicle control module requesting that the target speed setpoint used for speed control be reduced by a specified amount as a deceleration request signal until the recognition accuracy reaches the specified allowable level.

3. The vehicle position estimation device according to claim 1, wherein, The control unit is configured to output a signal to the vehicle control module requesting the vehicle to move to the driving lane even when the adverse environment determination unit determines that the surrounding environment is equivalent to the adverse environment, and the vehicle is driving in the overtaking lane.

4. The vehicle position estimation device according to claim 1, wherein, The control unit is configured such that, when the vehicle control module suppresses the driving speed based on the condition that the surrounding environment is equivalent to the adverse environment, it outputs a speed recovery permission signal to the vehicle control module when the surrounding environment is determined to no longer be the adverse environment.

5. The vehicle position estimation device according to claim 4, wherein, The control unit is configured to determine whether there is an important factor for maintaining acceleration in front of the vehicle based on the signal input from the forward camera and the information shown in the map indicating the driving environment in front of the vehicle. If there is such an important factor for maintaining acceleration in front of the vehicle, the control unit will not output the speed recovery permission signal.

6. The vehicle position estimation device according to claim 1, wherein, The notification processing unit, when the adverse environment determination unit determines that the surrounding environment is equivalent to the adverse environment and the vehicle speed is suppressed, sends a notification via a reporting device to suppress the vehicle speed.

7. The vehicle position estimation device according to claim 1, wherein, The positioning unit shortens the execution interval for calculating the vehicle's position based on the condition that the surrounding environment is equivalent to the aforementioned adverse environment, as determined by the aforementioned adverse environment determination unit.

8. The vehicle position estimation device according to claim 1, wherein, The material used by the positioning unit to change the calculation of the vehicle's position when the aforementioned adverse environment determination unit determines that the surrounding environment is equivalent to the aforementioned adverse environment.

9. The vehicle position estimation device according to claim 8, wherein, When the aforementioned adverse environment determination unit determines that the surrounding environment is equivalent to the aforementioned adverse environment, the aforementioned positioning unit changes the image frame used in the process of calculating the position of the aforementioned vehicle.

10. The vehicle position estimation device according to claim 9, wherein, Vehicles used to carry multiple types of cameras with different perspectives as the aforementioned front-facing cameras. When the aforementioned adverse environment determination unit determines that the surrounding environment is equivalent to the aforementioned adverse environment, the aforementioned positioning unit changes the combination of the aforementioned front camera, which is used as the source of generating image frames for the process of calculating the position of the aforementioned vehicle.

11. The vehicle position estimation device according to claim 9, wherein, The aforementioned front camera sequentially generates multiple types of image frames, including landmark frames and other-purpose frames. The landmark frames are image frames captured with an exposure time suitable for the detection of the aforementioned landmarks, while the other-purpose frames are image frames captured with an exposure time suitable for other specified purposes. If the aforementioned adverse environment determination unit does not determine that the surrounding environment is equivalent to the aforementioned adverse environment, the aforementioned positioning unit uses the aforementioned landmark frame to perform the process of calculating the position of the vehicle. On the other hand, if the aforementioned adverse environment determination unit determines that the surrounding environment is equivalent to the aforementioned adverse environment, it uses the aforementioned landmark frame and the aforementioned other purpose frame to perform the process of calculating the position of the vehicle.

12. The vehicle position estimation device according to claim 1, wherein, The aforementioned control unit is configured to output a signal requesting speed suppression to the vehicle control module, which automatically controls the vehicle's speed, based on the aforementioned adverse environment determination unit's determination that the surrounding environment is equivalent to the aforementioned adverse environment conditions. The higher the severity of the aforementioned adverse environmental conditions, the greater the required deceleration amount.

13. The vehicle position estimation device according to any one of claims 1 to 12, wherein, The control unit described above is configured to calculate the effective recognition distance, which represents the distance range within which the landmark can be actually identified based on the image frame. The aforementioned severe environment determination unit is configured to determine that a severe environment exists when the effective recognition distance degrades to below a specified value.

14. The vehicle position estimation device according to claim 1, wherein, The aforementioned harsh environment determination section is designed to determine the type of harsh environment. The control unit is configured to perform different processing based on the type of adverse environment determined by the adverse environment determination unit.

15. The vehicle position estimation device according to claim 14, wherein, The aforementioned control unit is configured to output a signal requesting speed suppression to the vehicle control module, which automatically controls the vehicle's speed, based on the aforementioned adverse environment determination unit's determination that the surrounding environment is equivalent to the aforementioned adverse environment conditions. The aforementioned severe environment determination component consists of determining whether it is equivalent to any one of fog, afternoon sun exposure, and heavy rain as the type of severe environment. When the control unit determines that the type of adverse environment is heavy rain, it increases the required deceleration amount compared to when the type of adverse environment is afternoon sun.

Images