Vehicle control device

Abstract

The vehicle control device includes: a camera (1a) that uses an image sensor to capture images of the surrounding environment of the vehicle and obtain position information of a target object; a detector (1b) that obtains position information of a target object based on reflected waves from a detected object within the capture area of ​​the camera (1a); a driving control unit (16) that performs driving control of the vehicle based on the two position information; a setting unit (17a) that sets a threshold value that is compared with an evaluation value calculated based on signal values ​​from the image sensor; a judgment unit (17b) that compares the evaluation value with the threshold value and judges whether the shooting environment in the camera (1a) has deteriorated; and a degradation indication unit (17c) that, when it is judged that the shooting environment has deteriorated, instructs the driving control unit (16) to restrict or prohibit a prescribed driving assistance function or automatic driving function. The setting unit (17a) sets a first threshold value if the shooting area of ​​the camera (1a) may contain the sun; otherwise, it sets a second threshold value that is larger than the first threshold value.

Description

Technical Field

[0001] This invention relates to a vehicle control device that controls a vehicle based on information from sensors that detect external objects. Background Technology

[0002] As a technology, there is a known device that identifies a vehicle traveling in front of the vehicle by using sensor fusion of a camera and radar, and controls the vehicle to follow the vehicle based on the identification result. Such a device is described, for example, in Patent Document 1. The camera's ability to identify vehicles in front can sometimes be reduced due to backlighting and nighttime illumination, thus causing a degradation in the function of limiting driver assistance features for safety reasons.

[0003] However, when a decrease in the camera's recognition capability is detected, there is a problem of functional degradation, which involves over-detecting oncoming vehicles and roadside signs, leading to repeated restrictions on driver assistance functions.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent document 1: Japanese Patent Application Publication No. 2005-145396 (JP2005-145396A). Summary of the Invention

[0007] A vehicle control device according to one embodiment of the present invention includes: a camera that uses an image sensor to capture images of the surrounding environment of the vehicle and obtain position information of a target object; a detector that obtains position information of a target object based on reflected waves from a detected object within the camera's capture area; a driving control unit that performs driving control of the vehicle based on the position information obtained by the camera and the position information obtained by the detector; a setting unit that sets a threshold for comparison with an evaluation value calculated based on signal values ​​from the image sensor; a judgment unit that compares the evaluation value with the threshold and determines whether the shooting environment in the camera has deteriorated; and a degradation indication unit that, when the judgment unit determines that the shooting environment has deteriorated, instructs the driving control unit to restrict or prohibit a prescribed driving assistance function or automatic driving function. The setting unit sets a first threshold as a threshold when the camera's capture area may contain the sun or the sky around the sun, and sets a second threshold greater than the first threshold as a threshold when the camera's capture area may not contain the sun or the sky around the sun. Attached Figure Description

[0008] The objectives, features, and advantages of the present invention are further illustrated by the following description of embodiments in conjunction with the accompanying drawings.

[0009] Figure 1This is a block diagram that schematically illustrates the overall configuration of a vehicle control system for an autonomous vehicle with a vehicle control device having an embodiment.

[0010] Figure 2 This is a block diagram illustrating the main components of a vehicle control device according to an embodiment.

[0011] Figure 3 This is a flowchart illustrating the process of determining whether or not functional degradation needs to be implemented.

[0012] Figure 4 This is a detailed explanation. Figure 3 The flowchart of the S10 process.

[0013] Figure 5A This is a graph showing an example of how evaluation values ​​and thresholds change over time.

[0014] Figure 5B This is a diagram illustrating how the judgment of whether or not functional degradation is necessary changes over time.

[0015] Figure 6A This is a graph showing other examples of how evaluation values ​​and thresholds change over time.

[0016] Figure 6B This is a graph showing other examples of how the judgment of whether or not functional degradation is needed changes over time. Detailed Implementation

[0017] The following is for reference Figures 1 to 6B Embodiments of the present invention will be described. The vehicle control device according to the embodiments of the present invention can be applied to vehicles with autonomous driving functions, that is, both autonomous vehicles and manually driven vehicles without autonomous driving functions. Manually driven vehicles include vehicles equipped with driving assistance functions. Hereinafter, examples of applying the vehicle control device to autonomous vehicles will be described.

[0018] It should be noted that sometimes the vehicle using the vehicle control device in the application implementation method is referred to as "this vehicle" to distinguish it from other vehicles.

[0019] This vehicle can be any one of the following: an engine vehicle with an internal combustion engine as the driving source, an electric vehicle with an electric motor as the driving source, or a hybrid vehicle with both an engine and an electric motor as driving sources. This vehicle (autonomous driving vehicle) can operate not only in an autonomous driving mode that does not require driver operation, but also in a manual driving mode based on driver operation (with the ability to use driving assistance functions).

[0020] <Outline Components of Autonomous Driving>

[0021] First, let's explain the general structure of autonomous driving. Figure 1 This is a block diagram that schematically illustrates the overall configuration of a vehicle control system 100 for an automated vehicle having a vehicle control device with an embodiment. (As shown) Figure 1 As shown, the vehicle control system 100 mainly includes a controller 10 and external sensor group 1, internal sensor group 2, input / output device 3, positioning unit 4, map database 5, navigation device 6, communication unit 7, and driving actuator AC, which are communicatively connected to the controller 10 via CAN communication line, etc.

[0022] External sensor group 1 is a collective term for multiple sensors (external sensors) that detect external conditions serving as information about the vehicle's surroundings. For example, external sensor group 1 includes a LiDAR (Laser Imaging Detection and Ranging) system that detects the position (distance and direction from the vehicle) of objects around the vehicle by illuminating them with a laser and detecting the reflected light; a radar system that detects the position of objects around the vehicle by illuminating them with electromagnetic waves and detecting the reflected waves; and a camera that captures images of the vehicle's surroundings using imaging elements such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensors. The imaging element is also called an image sensor. LiDAR and radar can detect objects within the camera's field of view. LiDAR and radar can also be referred to as detectors.

[0023] Internal sensor group 2 is a collective term for multiple sensors (internal sensors) that detect the driving status of the vehicle. Internal sensor group 2 includes, for example, a vehicle speed sensor to detect the vehicle's speed, an acceleration sensor to detect the vehicle's acceleration in the forward and backward and left and right directions, a gyroscope sensor to detect the vehicle's rotation or change of direction as angular velocity, and a speed sensor to detect the rotational speed of the driving source. Sensors that detect driver operations in manual driving mode, such as operation of the accelerator pedal, brake pedal, and steering wheel, are also included in internal sensor group 2.

[0024] Input / output device 3 is a general term for devices that allow the driver to input commands and output information to the driver. For example, input / output device 3 includes various switches that allow the driver to input various commands by operating the control components, a microphone for the driver to input commands by voice, a display that provides information to the driver by displaying images, and a speaker that provides information to the driver by voice.

[0025] The positioning unit (GNSS (Global Navigation Satellite System) unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. Alternatively, the positioning sensor can be included in the internal sensor group 2. Positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 4 uses the positioning information received by the positioning sensor to determine the vehicle's current position (latitude, longitude, and altitude).

[0026] Map database 5 is a device that stores general map information for navigation device 6, and is composed of, for example, a hard disk or semiconductor components. The map information includes road location information, road shape (curvature, etc.) information, and the location information of intersections or forks in the road. It should be noted that the map information stored in map database 5 is different from the high-precision map information stored in storage unit 12 of controller 10.

[0027] The navigation device 6 is a device that searches for a target path on the road to a destination input by the driver and guides the driver along that path. The destination is input and the driver is guided along the target path via the input / output device 3. The target path is calculated based on the vehicle's current position determined by the positioning unit 4 and map information stored in the map database 5. The vehicle's current position can also be determined using the detection values ​​from the external sensor group 1, and the target path can be calculated based on that current position and high-precision map information stored in the storage unit 12.

[0028] Communication unit 7 communicates with various servers (not shown) via a network including wireless communication networks such as the Internet and mobile phone networks, periodically or at any time obtaining map information, driving history information, and traffic information from the servers. The network includes not only public wireless communication networks but also closed communication networks set up for each designated management area, such as wireless LAN, Wi-Fi (registered trademark), and Bluetooth (registered trademark). The obtained map information is output to map database 5 and storage unit 12, and the map information is updated.

[0029] An actuator (AC) is a driving actuator used to control the movement of the vehicle. When the driving source is an engine, the actuator AC includes a throttle actuator for adjusting the throttle opening (throttle valve opening). When the driving source is a drive motor, the drive motor is included in the actuator AC. Braking actuators that operate the vehicle's braking system and steering actuators that drive the steering system are also included in the actuator AC.

[0030] The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 is configured as a computer including an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as ROM (Read Only Memory) and RAM (Random Access Memory), and other peripheral circuits (not shown) such as I / O interfaces. It should be noted that multiple ECUs with different functions, such as an engine control ECU, a drive motor control ECU, and a braking device ECU, can be set separately. Figure 1 For convenience, controller 10 is shown as a collection of these ECUs.

[0031] The storage unit 12 stores high-precision road map information. This road map information includes road location information, road shape (curvature, etc.), road slope information, location information of intersections or forks in the road, number of lanes, lane width, and the location information of each lane (center position of the lane or information of the lane boundary line), information on landmarks (traffic lights, signs, buildings, etc.) that serve as markers on the map, and road surface features such as unevenness. Landmark information includes the shape (outline), characteristics, and location of the landmarks.

[0032] The computing unit 11 has a functional structure including a vehicle position recognition unit 13, an external recognition unit 14, an action plan generation unit 15, a driving control unit 16, and a sensor group management unit 17.

[0033] The vehicle position recognition unit 13 identifies the vehicle's position on the map (vehicle position) based on the vehicle's position information obtained from the positioning unit 4 and the map information from the map database 5. Alternatively, the vehicle position can be identified using map information stored in the storage unit 12 and surrounding information detected by the external sensor group 1, thereby enabling high-precision vehicle position identification. It should be noted that when the vehicle's position can be determined using external sensors installed on or beside the road, the vehicle position can also be identified by communicating with these sensors via the communication unit 7. The vehicle position recognition unit 13 can also calculate the azimuth angle of the direction captured by the camera constituting the external sensor group 1 based on the output of the gyroscope sensor constituting the internal sensor group 2.

[0034] The external identification unit 14 identifies the external conditions around the vehicle based on signals from the external sensor group 1, such as lidar, radar, and cameras. For example, it identifies the position, speed, and acceleration of surrounding vehicles (vehicles in front and behind) traveling around the vehicle, the position of surrounding vehicles parked or stationary around the vehicle, and the position and state of other objects. Other objects include signs, traffic lights, road markings, stop lines, buildings, guardrails, utility poles, signs, pedestrians, bicycles, tunnel entrances, etc. The state of other objects includes the color of traffic lights (red, green, yellow), and the speed and direction of movement of pedestrians and bicycles.

[0035] The object being detected by the external sensor group 1 is called the target object. Target objects include both people and objects, as well as moving and stationary objects. The external identification unit 14 comprehensively processes (fusion processes) the detection data from the different types of sensors (e.g., cameras and detectors) constituting the external sensor group 1, determines whether the same target object is detected by each sensor, and derives the target object's position data. For example, if the same target object is detected, fusion processing such as coordinate transformation, data supplementation, and averaging of the detection data is performed to derive the target object's position data. Thus, the position of the target object can be identified with high accuracy.

[0036] The action plan generation unit 15 generates a driving trajectory (target trajectory) of the vehicle from the current point in time to a predetermined time elapsed, based on, for example, the target path calculated by the navigation device 6, map information stored in the storage unit 12, the vehicle position identified by the vehicle position recognition unit 13, and external conditions (objects) identified by the external environment recognition unit 14. When multiple candidate trajectories exist on the target path, the action plan generation unit 15 selects the best trajectory that complies with laws and regulations and meets criteria such as efficient and safe driving, and designates the selected trajectory as the target trajectory. Then, the action plan generation unit 15 generates an action plan corresponding to the generated target trajectory. The action plan generation unit 15 generates various action plans corresponding to driving modes such as overtaking, lane changing, following, maintaining lane position, constant speed, deceleration, or acceleration. When generating the target trajectory, the action plan generation unit 15 first determines the driving mode and generates the target trajectory based on the driving mode.

[0037] In autonomous driving mode, the driving control unit 16 controls each actuator AC to make the vehicle travel along the target trajectory generated by the action plan generation unit 15. More specifically, the driving control unit 16 considers the driving resistance determined by road slope, etc., in autonomous driving mode, and calculates the required driving force to obtain the target acceleration per unit time calculated by the action plan generation unit 15. Furthermore, for example, feedback control is performed on the actuator ACs to make the actual acceleration detected by the internal sensor group 2 the target acceleration. That is, the actuator ACs are controlled so that the vehicle travels at the target speed and target acceleration. It should be noted that in manual driving mode, the driving control unit 16 controls each actuator AC based on driving commands (steering operations, etc.) obtained from the driver by the internal sensor group 2.

[0038] The sensor group management unit 17 manages the status of the different types of sensors (cameras and detectors) that constitute the external sensor group 1. For example, it determines whether the camera's shooting environment has deteriorated, and whether functional degradation in autonomous driving and driver assistance functions is needed or not. The determination of the shooting environment deterioration and the need for functional degradation will be described in detail later.

[0039] <Whitening in the Camera>

[0040] However, in this embodiment, the position data of the same target object is determined based on the detection values ​​of different types of external sensor groups 1, such as cameras and radars. Generally, cameras experience backlighting conditions, such as the sun or bright sky around the sun, resulting in strong light entering the image sensor. When the light entering the image sensor is too strong, the grayscale of the image captured by the camera (called the camera image) is lost, resulting in a washed-out image (so-called whitening). When the image appears whitening, the camera's recognition ability is reduced, making it difficult to accurately detect the position data of the target object based on the camera image.

[0041] <Judgment on the deterioration of the filming environment>

[0042] Therefore, in this implementation, the following determination is made regarding the deterioration of the camera's shooting environment, which causes a whitening effect and reduces the camera's recognition capability: First, for a camera image, if the pixel value constituting the image is higher than a predetermined reference value, it is binarized to white; if it is lower than the reference value, it is binarized to black, thus obtaining a binary image. In a predetermined area of ​​this binary image, if the ratio of the number of white pixels to the total number of pixels in the predetermined area (called the evaluation value) exceeds a predetermined threshold for a predetermined period of time (in other words, if the evaluation value calculated from multiple camera images taken at a predetermined period exceeds the threshold a predetermined number of times), it is determined that the camera's shooting environment has deteriorated. The evaluation value is a value between 0 and 100.

[0043] <About threshold>

[0044] In this implementation, two thresholds are used to determine the deterioration of the camera's shooting environment, distinguishing between situations where there is a possibility of backlighting caused by sunlight and situations where there is no possibility of backlighting caused by sunlight. For example, if the camera's shooting area may include the sun or the sky around the sun, a first threshold (e.g., a value equivalent to an evaluation value of 80) is set as the threshold; otherwise, a second threshold (e.g., a value equivalent to an evaluation value of 99) that is larger than the first threshold is set as the threshold.

[0045] The situation where the camera's shooting area may include the sun or the sky around the sun refers to a situation where the azimuth angle of the camera's shooting direction, calculated by the vehicle position recognition unit 13 based on the output of the gyroscope sensor constituting the internal sensor group 2, is approximately consistent with the current position of the vehicle determined by the positioning unit 4 and the solar azimuth angle at the current moment calculated by the sensor group management unit 17 according to a known formula, and the elevation angle of the camera's shooting direction, calculated by the vehicle position recognition unit 13 based on the output of the gyroscope sensor constituting the internal sensor group 2, is approximately consistent with the current position of the vehicle determined by the positioning unit 4 and the solar altitude at the current moment calculated by the sensor group management unit 17 according to a known formula.

[0046] It should be noted that even if the azimuth and elevation angles (sun altitude) are roughly the same, on a cloudy day, the camera's shooting area may not include the sun or the sky around the sun, and backlighting due to sunlight may not occur. Therefore, the above describes the situation where the camera's shooting area "may include" the sun or the sky around the sun.

[0047] Furthermore, the reason for using two thresholds to distinguish between situations where there is a possibility of backlighting caused by sunlight and situations where there is no possibility of backlighting caused by sunlight is to ensure that the processing implemented after judging the deterioration of the camera's shooting environment (the determination of whether or not the function degradation needs to be judged, which will be detailed later) is different in cases where backlighting is caused by sunlight and cases where backlighting is caused by the light from the headlights of oncoming vehicles or the light from road signs at night.

[0048] <Regarding functional degradation>

[0049] The deterioration of the camera's shooting environment, which produces the aforementioned whitening effect, leads to a decrease in the accuracy of the target object's position data used in the fusion process. Therefore, in order to ensure safety, when it is determined that the camera's shooting environment has deteriorated, a function degradation process is performed on the autonomous driving function in autonomous driving mode and the driving assistance function in manual driving mode to determine whether or not function degradation is necessary.

[0050] Regarding driver assistance functions in manual driving mode, here's an example of functional degradation: For instance, automatically implementing both acceleration and braking control based on the recognition of a preceding vehicle while temporarily disabling Adaptive Cruise Control (ACC) to maintain a proper following distance while keeping pace with the preceding vehicle is considered a degradation in terms of suspending (or disabling) the entire ACC function. Similarly, automatically implementing steering control based on lane recognition while temporarily disabling the driver's steering assistance to keep the vehicle near the center of the lane is also considered a degradation in terms of suspending (or disabling) the entire LKAS function. Furthermore, temporarily disabling only one function when both ACC and LKAS are active is considered a degradation in terms of suspending only some driver assistance functions.

[0051] <Determination and handling of whether functional degradation is necessary>

[0052] The process for determining whether functional degradation is necessary is explained in two ways: one is when there is a possibility of backlighting caused by sunlight (when a first threshold is set), and the other is when there is no possibility of backlighting caused by sunlight (when a second threshold is set).

[0053] (1) There is a possibility of backlighting caused by sunlight (the case where a first threshold is set).

[0054] In situations where there is a possibility of backlighting due to sunlight, the threshold is lowered to, for example, an evaluation value of 80. The determination of whether functional degradation is necessary begins at an earlier stage before the evaluation value approaches 100 (when the evaluation value exceeds 80). The determination of whether functional degradation is necessary given the first threshold is based on a comprehensive assessment of whether the camera can detect the target object and the detector's detection status of the target object. Specifically, if the same target object previously detected by both the camera and detector is no longer detected by the camera but is detected by the detector, it is considered that the camera's failure to detect it is due to backlighting, and functional degradation is deemed necessary. If the same target object previously detected by both the camera and detector continues to be detected by both the camera and detector, functional degradation is deemed unnecessary.

[0055] (2) The case where there is no possibility of backlighting due to sunlight (the case where a second threshold is set).

[0056] In situations where there is no possibility of backlighting due to sunlight, the threshold is raised; for example, functional degradation is only considered necessary when the evaluation value reaches 100. In other words, in determining whether functional degradation is necessary under the condition of setting a second threshold, if the evaluation value is below 99, functional degradation is considered unnecessary. With this configuration, backlighting caused by the headlights of oncoming vehicles or the light from road signs during nighttime driving rarely exceeds the evaluation value of 99, and therefore functional degradation is almost never needed.

[0057] <Main Components of Vehicle Control Devices>

[0058] Figure 2 This is a block diagram illustrating the main components of a vehicle control device 50 according to an embodiment. The vehicle control device 50 is configured as follows: Figure 1 It is part of the vehicle control system 100. For example... Figure 2 As shown, the vehicle control device 50 includes a camera 1a, a detector 1b, a controller 10, and an actuator AC.

[0059] Camera 1a is a single-lens camera with the aforementioned imaging elements such as a CCD or CMOS sensor, constituting... Figure 1It is part of the external sensor group 1. Camera 1a can also be a stereo camera. Camera 1a is mounted, for example, at a predetermined position at the front of the vehicle, and continuously captures images of the space in front of the vehicle to obtain images (camera images) of the target objects. Target objects include vehicles traveling in front of the vehicle, people, structures, etc. Based on the camera images, the position and type of the target objects can be identified. That is, when the horizontal direction of the two-dimensional camera image is set as the x-direction and the vertical direction is set as the y-direction, the position of the target object in the vehicle width direction can be determined based on the position in the x-direction of the camera image, and the position of the target object in the height direction and the direction of travel can be determined based on the position in the y-direction. In other words, the position data (position information) of the target objects can be obtained through camera 1a.

[0060] Detector 1b is a detector that detects the distance from the vehicle to a target object based on reflected waves from the target object, including either radar or lidar. Detector 1b can acquire position data (position information) of the target object relative to the vehicle. The position data includes the target object's position and velocity. The detection range of detector 1b is contained within the image area of ​​camera 1a. Therefore, if the target object detected by camera 1a is the same as the target object detected by detector 1b, the position and velocity of the target object can be derived by performing sensor fusion processing.

[0061] Figure 2 The controller 10 has a setting unit 17a, a judgment unit 17b, a degradation indication unit 17c, and a driving control unit 16, which serves as the arithmetic unit 11. Figure 1 The sensor group management unit 17 is a functional structure that performs functions such as setting unit 17a, judgment unit 17b, and degradation indication unit 17c. These are provided within the sensor group management unit 17 to determine whether or not functional degradation is required and to instruct the driving control unit 16.

[0062] <Flowchart Explanation>

[0063] Figure 3 and Figure 4 It is shown by Figure 2 A flowchart illustrating an example of the processing performed by the controller 10, particularly the processing implemented by the sensor group management unit 17. Figure 3 This is a flowchart illustrating the process of determining whether or not functional degradation needs to be implemented. Figure 4 This is a detailed explanation. Figure 3 The flowchart of the S10 (S: processing step) process.

[0064] Figure 3 and Figure 4The process shown in the flowchart is implemented, for example, when the camera 1a starts capturing camera images at a predetermined cycle, and is performed each time a camera image is captured. The sensor group management unit 17 of the controller 10 performs the following actions each time a camera image is captured: determining the possibility of backlight caused by sunlight, setting a threshold for determining the deterioration of the shooting environment, determining the deterioration of the shooting environment, determining whether functional degradation is needed, and instructing the driving control unit 16.

[0065] exist Figure 3 In S10, the sensor group management unit 17 performs a judgment related to the shooting environment and proceeds to S20A. The judgment unit 17b of the sensor group management unit 17 judges the deterioration of the shooting environment as a judgment related to the shooting environment.

[0066] In S20A, the sensor group management unit 17 determines whether the judgment unit 17b has determined that the shooting environment has deteriorated based on a first threshold. When the shooting environment is determined to have deteriorated based on the first threshold, the sensor group management unit 17 affirms S20A (S20A: Yes) and proceeds to S30. When the shooting environment is not determined to have deteriorated based on the first threshold, the sensor group management unit 17 negates S20A (S20A: No) and proceeds to S20B.

[0067] In S20B, the sensor group management unit 17 determines whether the judgment unit 17b has determined that the shooting environment has deteriorated based on the second threshold. When the shooting environment is determined to have deteriorated based on the second threshold, the sensor group management unit 17 makes an affirmative judgment (S20B: Yes) on S20B and proceeds to S90. When the shooting environment is not determined to have deteriorated based on the second threshold, the sensor group management unit 17 makes a negative judgment (S20B: No) on S20B and proceeds to S70.

[0068] In step S30, the sensor group management unit 17 changes the priority of the position information obtained by the external sensor group 1 and proceeds to step S40. Specifically, if the setting prioritizes the x-direction position information of the target object in the camera image obtained by camera 1a as the vehicle width direction position information, the setting is changed to prioritize the target object's position data (position information) obtained by detector 1b. Due to this configuration, even if the recognition capability of camera 1a may decrease due to deterioration of the shooting environment, the target object's position data (position information) obtained by detector 1b can be used as the target object's vehicle width direction position information.

[0069] In S40, the sensor group management unit 17 determines whether the camera 1a has detected a target object. For example, if the camera 1a detects the same target object that has been previously detected by both the camera 1a and the detector 1b, the sensor group management unit 17 makes a positive (S40: Yes) determination in S40 and proceeds to S80. A positive (S40: Yes) determination in S40 means that although the camera 1a's recognition capability may decrease due to deterioration of the shooting environment, the camera 1a can still detect the target object. If the camera 1a does not detect the same target object that has been previously detected by both the camera 1a and the detector 1b, the sensor group management unit 17 makes a negative (S40: No) determination in S40 and proceeds to S50.

[0070] In S50, the sensor group management unit 17 determines whether the detector 1b has detected a target object. For example, if the detector 1b detects the same target object that was previously detected by both the camera 1a and the detector 1b, the sensor group management unit 17 makes a positive (S50: Yes) determination in S50 and proceeds to S60. If the positive (S50: Yes) determination is made in S50, it is highly likely that the target object cannot be detected by the camera 1a due to deterioration of the shooting environment. If the detector 1b does not detect the same target object that was previously detected by both the camera 1a and the detector 1b, the sensor group management unit 17 makes a negative (S50: No) determination in S50 and proceeds to S80. A negative (S50: No) determination in S50 occurs when the target object has moved out of the detection range of both the camera 1a and the detector 1b.

[0071] In S60, the degradation indicator 17c of the sensor group management unit 17 determines that functional degradation is required and terminates the process. Figure 3 The degradation instruction unit 17c, when determining that a function degradation is required, instructs the driving control unit 16 to perform a function degradation.

[0072] In S70, which proceeds after a negative determination of S20B (S20B: No), if the priority of the position information obtained by the external sensor group 1 has changed, the sensor group management unit 17 restores the priority and proceeds to S80. Specifically, if the setting prioritizes the position data (position information) of the target object obtained by the detector 1b, the setting is changed to prioritize the position information in the x-direction of the camera image obtained by the camera 1a as the position information in the vehicle width direction of the target object. Due to this configuration, the position information of the target object based on the camera image obtained by the camera 1a can be prioritized, provided that the shooting environment does not deteriorate and the recognition capability of the camera 1a does not decrease.

[0073] Following S70, in S80, which proceeds after the affirmative determination (S40: Yes) of S40 and the negative determination (S50: No) of S50, the degradation indication unit 17c of the sensor group management unit 17 determines that functional degradation is not required and ends. Figure 3 The degradation instruction unit 17c, when determining that functional degradation is unnecessary, and having previously instructed the driving control unit 16 to perform functional degradation, implements an instruction to restore the functional degradation to normal function.

[0074] In S90, which proceeds after a positive determination (S20B: Yes) is made regarding S20B, similar to S30, the sensor group management unit 17 changes the priority of the position information obtained from the external sensor group 1 and proceeds to S60. Due to this configuration, in situations where the camera 1a's recognition capability may decrease due to deterioration of the shooting environment, the target's position data (position information) obtained from the detector 1b can be used as the target's position information in the vehicle width direction.

[0075] Reference Figure 4 ,right Figure 3 The processing of S10 will be explained in detail.

[0076] In S101, the sensor group management unit 17 determines whether the area captured by the camera 1a may contain the sun or the sky around the sun. As described above, if the azimuth angle of the direction captured by the camera 1a is approximately the same as the azimuth angle of the sun, and the elevation angle of the direction captured by the camera 1a is approximately the same as the altitude of the sun, the sensor group management unit 17 makes a positive determination (S101: Yes) in S101 and proceeds to S102. If the azimuth angle and elevation angle (altitude of the sun) are not approximately the same, the sensor group management unit 17 makes a negative determination (S101: No) in S101 and proceeds to S103.

[0077] In S102, the setting unit 17a of the sensor group management unit 17 sets a first threshold (for example, equivalent to an evaluation value of 80), and proceeds to S104. In S103, the setting unit 17a of the sensor group management unit 17 sets a second threshold (for example, equivalent to an evaluation value of 99), and proceeds to S104.

[0078] In S104, the sensor group management unit 17 determines whether the evaluation value > threshold is true. If the evaluation value > threshold is true, the sensor group management unit 17 makes an affirmative determination (S104: Yes) for S104 and proceeds to S105. If the evaluation value > threshold is false, the sensor group management unit 17 makes a negative determination (S104: No) for S104 and proceeds to S106.

[0079] In S105, the judgment unit 17b of the sensor group management unit 17 determines that the shooting environment in the camera 1a has deteriorated, and ends the process. Figure 4 The processing continues. In S106, the judgment unit 17b of the sensor group management unit 17 determines that the shooting environment in the camera 1a has not deteriorated, and ends the process. Figure 4 The processing.

[0080] <Example of determining whether functional degradation is necessary>

[0081] Reference Figure 5A , 5B and Figure 6A , 6B This section explains an example of determining whether functional degradation is necessary in a vehicle control system 100 with a vehicle control device having an implementation method. Figure 5A , 5B and Figure 6A , 6B It is a graphical representation of information obtained by the vehicle control system 100 of the vehicle traveling east-west on a highway towards the setting sun in the evening. Figure 5A This is a graph showing an example of how evaluation values ​​and thresholds change over time. Figure 5A The horizontal axis represents the elapsed time (in seconds) since camera 1a started taking pictures. Figure 5A The vertical axis represents the evaluation value calculated by the sensor group management unit 17 (represented by a solid line) and the threshold value set by the setting unit 17a (represented by a dashed line). Figure 5A In this process, the initial value of the threshold is 99 (second initial value). After 10 seconds, the shooting area becomes a bright sky, and the evaluation value (represented by a solid line) calculated by the sensor group management unit 17 changes from 0 to 81. At this time, since the evaluation value > threshold is not true, the sensor group management unit 17 determines that the shooting environment of camera 1a has not deteriorated (S106).

[0082] After 25 seconds, the sensor group management unit 17 determines that the shooting area of ​​camera 1a may contain the sun or the sky around the sun (making a positive determination in S101 (S101: Yes)), and therefore the setting unit 17a sets the threshold to 80 (first threshold). The evaluation value (represented by a solid line) calculated by the sensor group management unit 17 is still 81. Therefore, since the evaluation value > threshold, the judgment unit 17b of the sensor group management unit 17 determines that the shooting environment in camera 1a has deteriorated (S105). Afterwards, the sensor group management unit 17 performs... Figure 3 The processing of S30 to S50.

[0083] Figure 5B This is a diagram illustrating how the judgment of whether or not functional degradation is necessary changes over time. Figure 5B The horizontal axis and Figure 5AThe horizontal axis is the same, representing the elapsed time (in seconds) since camera 1a started taking pictures. Figure 5B The vertical axis represents the functional degradation judgment result determined by the degradation indication unit 17c of the sensor group management unit 17. Figure 5B In the middle, it is determined that the function does not need to degrade.

[0084] Figure 6A This is a graph showing other examples of how evaluation values ​​and thresholds change over time. Figure 6A The horizontal axis represents the elapsed time (in seconds) since camera 1a started taking pictures. A selection is shown below. Figure 5A The next 325 seconds to 375 seconds. Figure 6A The vertical axis represents the evaluation value calculated by the sensor group management unit 17 (represented by a solid line) and the threshold value set by the setting unit 17a (represented by a dashed line). Figure 6A In the process, with the threshold set to 80 (first threshold) remaining unchanged, the evaluation value (solid line) calculated by the sensor group management unit 17 also remains unchanged at 81. Therefore, since the evaluation value > threshold, the judgment unit 17b of the sensor group management unit 17 continues to judge the deterioration of the shooting environment in the camera 1a (S105). The sensor group management unit 17 implements... Figure 3 The processing of S30 to S50.

[0085] Figure 6B This is a graph showing how the judgment of whether or not functional degradation is needed changes over time. Figure 6B The horizontal axis and Figure 6A The horizontal axis is the same, representing the elapsed time (in seconds) since the camera 1a started shooting, with 325 seconds to 375 seconds shown as an example. Figure 6B The vertical axis represents the result of the function degradation judgment determined by the degradation indicator 17c of the sensor group management unit 17. After 340 seconds, the camera 1a did not detect the target (a negative judgment was made in S40 (S40: No)), and the detector 1b detected the target (a positive judgment was made in S50 (S50: Yes)). Therefore, the degradation indicator 17c determined that function degradation was required (S60). Thereafter, the determination of function degradation was continued until 370 seconds had passed since the camera 1a detected the target again.

[0086] The implementation method described above achieves the following effects.

[0087] (1) The vehicle control system 100 of the autonomous vehicle having the vehicle control device 50 of the embodiment includes: a camera 1a, which uses an image sensor to capture images of the surrounding environment of the vehicle to obtain position information of a target object; a detector 1b, which obtains position information of a target object based on reflected waves from a detected object within the capture area of ​​the camera 1a; a driving control unit 16, which performs driving control of the vehicle based on the position information obtained by the camera 1a and the position information obtained by the detector 1b; a setting unit 17a, which sets a threshold for comparing with an evaluation value calculated based on signal values ​​from the image sensor; a judgment unit 17b, which compares the evaluation value with the threshold to determine whether the shooting environment in the camera 1a has deteriorated; and a degradation indication unit 17c, which, when the judgment unit 17b determines that the shooting environment has deteriorated, instructs the driving control unit 16 to restrict or prohibit a specified driving assistance function or autonomous driving function. Figure 2 If the shooting area of ​​the camera 1a may contain the sun or the sky around the sun, the setting unit 17a sets a first threshold (equivalent to evaluation value 80) as the threshold; otherwise, it sets a second threshold (equivalent to evaluation value 99) that is greater than the first threshold as the threshold.

[0088] Because of this configuration, when there is a possibility of backlighting due to sunlight, the threshold can be lowered to an evaluation value of 80 (S102). The possibility of backlighting is detected at an earlier stage (when the evaluation value exceeds 80) before the evaluation value approaches 100 (equivalent to strong backlighting from the sun in the shooting area), (a positive determination is made for S20A (S20A: Yes)), and a judgment process is implemented to determine whether functional degradation is necessary (S40, S50). On the other hand, when there is no possibility of backlighting due to sunlight, the threshold is raised to an evaluation value of 99 (S103), and functional degradation is determined only when the evaluation value reaches 100 (a positive determination is made for S20B (S20B: Yes)). Generally, since backlighting caused by the headlights of oncoming vehicles or the light from road signs at night is mostly below an evaluation value of 99, over-detection of backlighting can be suppressed, and functional degradation can be avoided.

[0089] (2) In the vehicle control device 50, the setting unit 17a sets a first threshold or a second threshold based on information indicating the solar altitude, solar azimuth, and the vehicle's direction of travel. Because of this configuration, it is possible to appropriately determine whether the area captured by the camera 1a may contain the sun or the sky around the sun, and appropriately set the first threshold and the second threshold respectively in cases where there is a possibility of backlighting caused by sunlight and in cases where there is no possibility of backlighting caused by sunlight.

[0090] (3) In the vehicle control device 50, when the judgment unit 17b determines that the shooting environment has deteriorated under the state of setting a first threshold, the degradation indication unit 17c instructs the driving control unit 16 to restrict or prohibit the prescribed driving assistance function or automatic driving function if the camera 1a has not obtained the position information of the target object but the detector 1b has obtained the position information of the target object. Due to this configuration, for example, if the same target object that was previously detected by the camera 1a and the detector 1b is not detected by the camera 1a (a negative determination is made in S40 (S40: No)) but is detected by the detector 1b (a positive determination is made in S50 (S50: Yes)), it is considered that the camera did not detect it due to backlighting, and function degradation can be implemented.

[0091] (4) In the vehicle control device 50, when the judgment unit 17b determines that the shooting environment has deteriorated under the state of setting a first threshold, the degradation indication unit 17c does not indicate the restriction or prohibition of the prescribed driving assistance function or automatic driving function to the driving control unit 16 after obtaining the position information of the target object from the camera 1a. Due to this configuration, for example, if the same target object that was previously detected by the camera 1a and the detector 1b is detected by the camera 1a again (a positive determination is made in S40 (S40: Yes)), functional degradation can be avoided.

[0092] (5) In the vehicle control device 50, when the judgment unit 17b determines that the shooting environment has deteriorated under the condition of setting a second threshold, the degradation instruction unit 17c instructs the driving control unit 16 to restrict or prohibit the prescribed driving assistance function or automatic driving function. Because of this configuration, even if there is no possibility of backlighting caused by sunlight, function degradation can be implemented when the evaluation value reaches 100 (a positive determination is made for S20B (S20B: Yes)).

[0093] (6) The vehicle control device 50 includes a sensor group management unit 17. When the judgment unit 17b determines that the shooting environment has deteriorated, the sensor group management unit 17 prioritizes a portion of the position information obtained by the detector 1b over the position information obtained by the camera 1a. Because of this configuration, in situations where the recognition capability of the camera 1a may decrease due to the deterioration of the shooting environment, the position data (position information) of the target object obtained by the detector 1b can be used as the position information of the target object in the vehicle width direction. It should be noted that speed information in the vehicle width direction can also be prioritized.

[0094] The above embodiments can be modified in various ways. Hereinafter, modifications will be described. In the above embodiments, an example of functional degradation of the driving assistance function in manual driving mode was described. However, in automatic driving mode, for example, lowering the automatic driving level will result in functional degradation. That is, functional degradation also includes lowering the so-called automatic driving level from level 5 to level 4 or any one of levels 3 to 1. It also includes lowering the automatic driving level from level 4 to level 3, level 2, or level 1. Similarly, it includes lowering the automatic driving level from level 3 to level 2 or 1, and lowering the automatic driving level from level 2 to level 1. Furthermore, suspending all automatic driving functions is also an example of functional degradation. That is, it includes lowering the automatic driving level by one or more levels.

[0095] It should be noted that the levels of autonomous driving are as follows: Level 5 is a fully automated driving function where the vehicle control system 100 always performs all driving tasks. Level 4 is a fully automated driving function under specific conditions where the vehicle control system 100 performs all driving tasks in specific locations such as highways. Level 3 is a conditional automated driving function where the vehicle control system 100 performs all driving tasks in specific locations such as highways, but the driver needs to appropriately respond to the intervention and requests of the vehicle control system 100.

[0096] Level 2 is a driver assistance function in which the vehicle control system 100 partially supports steering correction and acceleration / deceleration. Level 1 is a driver assistance function in which the vehicle control system 100 supports one of steering correction and acceleration / deceleration, with the other controlled by the driver.

[0097] In the above embodiments, an example of applying the vehicle control device 50 to an autonomous vehicle was described, but the present invention can also be applied to a manually driven vehicle with driving assistance functions.

[0098] The present invention can also be used as a vehicle control method, including: a step of obtaining the position information of a target object by using an image sensor to capture the surrounding environment of the vehicle through a camera 1a; a step of obtaining the position information of the target object by a detector 1b based on the reflected wave of the detected object in the shooting area of ​​the camera 1a; a step of implementing vehicle driving control based on the position information obtained by the camera 1a and the position information obtained by the detector 1b; a step of setting a threshold for comparing with an evaluation value calculated based on the signal value from the image sensor; a step of judging the deterioration of the shooting environment in the camera 1a by comparing the evaluation value with the threshold; and a step of instructing the driving control unit 16 to restrict or prohibit a prescribed driving assistance function or automatic driving function when it is judged that the shooting environment has deteriorated. The setting step includes setting a first threshold as a threshold when the shooting area of ​​the camera 1a may contain the sun or the sky around the sun, and setting a second threshold greater than the first threshold as a threshold when the shooting area of ​​the camera 1a may not contain the sun or the sky around the sun.

[0099] The above description is merely one example. As long as the characteristics of the invention are not altered, the invention is not limited to the above-described embodiments and modifications. Any combination of the above-described embodiments and modifications is permissible.

[0100] By employing this invention, over-detection can be appropriately suppressed when the recognition capability of the detection camera declines.

[0101] The present invention has been described above in conjunction with preferred embodiments, but those skilled in the art should understand that various modifications and alterations can be made without departing from the scope of the following claims.

Claims

1. A control device for a vehicle, characterized by comprising: Possessing: a camera (1a) that photographs the outside of the surroundings of the host vehicle using an image sensor, and acquires position information of a target object; a detector (1b) that acquires position information of a target object based on a reflected wave from a detection object within a photographing region of the camera (1a); a travel control section (16) that performs travel control of the vehicle based on the position information acquired by the camera (1a) and the position information acquired by the detector (1b); a setting section (17a) that sets a threshold value for comparison with an evaluation value calculated from a signal value from the image sensor; a determination section (17b) that compares the evaluation value with the threshold value, and determines deterioration of a photographing environment in the camera (1a); and a deterioration instruction section (17c) that instructs the travel control section (16) of restriction or prohibition of a prescribed driving assist function or automatic driving function in the case where deterioration of the photographing environment is determined by the determination section (17b), the setting section (17a) sets a first threshold value as the threshold value in the case where the photographing region of the camera (1a) is likely to contain the sun or the sky around the sun, and sets a second threshold value that is larger than the first threshold value as the threshold value in the case where the photographing region of the camera (1a) is not likely to contain the sun or the sky around the sun.

2. The control device for a vehicle according to claim 1, characterized in that the setting section (17a) sets the first threshold value or the second threshold value based on information indicating a sun altitude, a sun azimuth, and a traveling direction of the host vehicle.

3. The control device for a vehicle according to claim 1, characterized in that when deterioration of the photographing environment is determined by the determination section (17b) in a state where the first threshold value is set, the deterioration instruction section (17c) instructs the travel control section (16) of restriction or prohibition of the prescribed driving assist function or the automatic driving function in the case where position information of the target object is not acquired by the camera (1a) and position information of the target object is acquired by the detector (1b).

4. The control device for a vehicle according to claim 3, characterized in that when deterioration of the photographing environment is determined by the determination section (17b) in a state where the first threshold value is set, the deterioration instruction section (17c) does not instruct the travel control section (16) of restriction or prohibition of the prescribed driving assist function or the automatic driving function in the case where position information of the target object is acquired by the camera (1a).

5. The control device for a vehicle according to any one of claims 1 to 4, characterized in that when deterioration of the photographing environment is determined by the determination section (17b) in a state where the second threshold value is set, the deterioration instruction section (17c) instructs the travel control section (16) of restriction or prohibition of the prescribed driving assist function or the automatic driving function.

6. The control device for a vehicle according to any one of claims 1 to 4, characterized in that The sensor group management unit (17) prioritizes part of the position information acquired by the detector (1b) over the position information acquired by the camera (1a) when the deterioration of the imaging environment is determined by the determination unit (17b).

7. The control device for vehicle according to any one of claims 1 to 4, characterized in that the prescribed driving assist function includes at least one of a follow-up travel function that controls acceleration and deceleration to keep a vehicle-to-vehicle distance between the host vehicle and a preceding vehicle traveling ahead of the host vehicle as a prescribed vehicle-to-vehicle distance and follow the preceding vehicle, and a lane keeping function that controls steering to make the host vehicle travel in the center of a travel lane.

8. The control device for vehicle according to any one of claims 1 to 4, characterized in that the restriction or prohibition of the automatic driving function includes a reduction of an automatic driving level by one level or more.

9. A control method for a vehicle, characterized by, comprises the steps of: acquiring position information of a target object by a camera (1a) that images an outside of the host vehicle using an image sensor; acquiring position information of a target object by a detector (1b) that acquires position information of a target object based on a reflected wave from a detection object within an imaging area of the camera (1a); implementing travel control of the vehicle based on the position information acquired by the camera (1a) and the position information acquired by the detector (1b); setting a threshold value to be compared with an evaluation value calculated based on a signal value from the image sensor; comparing the evaluation value with the threshold value to determine deterioration of an imaging environment in the camera (1a); and instructing a travel control unit (16) to restrict or prohibit a prescribed driving assist function or an automatic driving function when the deterioration of the imaging environment is determined, the setting step includes setting a first threshold value as the threshold value in a case where the imaging area of the camera (1a) is likely to include the sun or a sky around the sun, and setting a second threshold value larger than the first threshold value as the threshold value in a case where the imaging area of the camera (1a) is unlikely to include the sun or the sky around the sun.

Images