Object recognition device
The object recognition device adjusts confidence thresholds based on headlight illumination areas to enhance detection accuracy, addressing false detections and improving collision avoidance systems.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- TOYOTA JIDOSHA KK
- Filing Date
- 2024-01-11
- Publication Date
- 2026-07-29
AI Technical Summary
Existing object recognition systems in vehicles fail to accurately detect objects in low-beam and high-beam states due to inappropriate correction of pixel value thresholds, leading to false detections or non-detections.
An object recognition device that adjusts the confidence threshold based on the presence of targets within the headlight illumination area, using a smaller threshold when targets are outside this area to prevent false detections and improve recognition accuracy.
Enhances the accuracy of object recognition by preventing false positives and negatives, especially in low-light conditions, thereby improving collision avoidance systems.
Smart Images
Figure 0007896640000001 
Figure 0007896640000002 
Figure 0007896640000003
Abstract
Description
Technical Field
[0001] The present disclosure relates to an object recognition device, and particularly to a technique suitable for recognizing an object in front of a vehicle.
Background Art
[0002] For example, in Patent Document 1, in an in-vehicle image processing device, by changing a correction threshold value of a pixel value used for image recognition processing between a case where a headlight is in a low-beam state and a case where the headlight is in a high-beam state, a technique is disclosed that enables appropriate object recognition even in the low-beam state.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
[0004] For example, in a vehicle equipped with a system that partially switches between a high-beam state and a low-beam state, such as an Adaptive High-beam System (AHS), the irradiation area of the headlight is always variable in each of the high-beam state and the low-beam state. Therefore, if only the correction threshold value of the pixel value is changed based on whether the headlight is in the low-beam state or the high-beam state as in the device described in Patent Document 1, the correction process is not appropriately performed, and there is a possibility of false detection or non-detection of an object.
[0005] The technology of the present disclosure has been made in view of the above circumstances, and an object thereof is to effectively improve the recognition accuracy of an object.
[0006] The object recognition device of the present disclosure is imaging means for imaging a predetermined range in front of the vehicle, A target recognition device comprising: a target recognition means that calculates a confidence index for target recognition of a target in an image captured by the aforementioned imaging means, and recognizes the target as an actual existing target if the calculated confidence index is equal to or greater than a predetermined confidence threshold, Based on information regarding the presence of a target in front of the vehicle, a light distribution control means controls the light distribution of the headlights by blocking or dimming a portion of the light emitted from the headlights of the vehicle, thereby controlling the light distribution of the headlights in a light distribution pattern that includes an illumination area for emitting the light and a dimming area for blocking or dimming the light. The system includes a confidence threshold changing means that, if the target in the image is not present in the irradiation area, changes the confidence threshold to a smaller value than when the target is present in the irradiation area. It is characterized by the following: [Brief explanation of the drawing]
[0007] [Figure 1] This is a schematic diagram showing the hardware configuration of the vehicle according to this embodiment. [Figure 2] This is a schematic diagram showing the software configuration of the control device according to this embodiment. [Figure 3] This is a schematic diagram illustrating an example of the light distribution pattern of high-beam illumination light emitted from the headlight according to this embodiment. [Figure 4] This flowchart illustrates the routines for target recognition processing and PCS control processing according to this embodiment. [Modes for carrying out the invention]
[0008] The target recognition device according to this embodiment will be described below with reference to the drawings.
[0009] [Hardware configuration] Figure 1 is a schematic diagram showing the hardware configuration of vehicle VH according to this embodiment. Hereafter, vehicle VH may be referred to as "our vehicle" when it is necessary to distinguish it from other vehicles, etc.
[0010] Vehicle VH has an ECU (Electronic Control Unit) 10. The ECU 10 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, and an interface device 14, etc. The CPU 11 is a processor that executes various programs stored in the ROM 12. The ROM 12 is a non-volatile memory that stores data necessary for the CPU 11 to execute various programs. The RAM 13 is a volatile memory that provides a work area that is expanded when various programs are executed by the CPU 11. The interface device 14 is a communication device for communicating with external devices.
[0011] The ECU10 is a central device that performs driver assistance control such as collision avoidance control (Pre-Crash Safety Control: hereinafter referred to as PCS control). Driver assistance control is a concept that includes autonomous driving control. The ECU10 is connected to the drive unit 20, steering unit 21, braking unit 22, interior sensor unit 30, exterior sensor unit 40, automatic headlight sensor 50, left headlight 60L, right headlight 60R, HMI (Human Machine Interface) 70, etc., in a communication manner.
[0012] The drive unit 20 generates a driving force that is transmitted to the drive wheels of the vehicle VH. Examples of the drive unit 20 include an electric motor and an engine. In this embodiment, the vehicle VH may be a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), a fuel cell electric vehicle (FCEV), a battery electric vehicle (BEV), or an engine-powered vehicle. The steering unit 21 applies steering force to the wheels of the vehicle VH. The braking unit 22 applies braking force to the wheels of the vehicle VH.
[0013] The internal sensor device 30 consists of sensors that acquire the state of the vehicle's VH. The internal sensor device 30 includes a vehicle speed sensor 31, an accelerator sensor 32, a brake sensor 33, a steering angle sensor 34, a steering torque sensor 35, a yaw rate sensor 36, and the like.
[0014] The vehicle speed sensor 31 detects the vehicle speed (vehicle speed V) of the vehicle VH. The accelerator sensor 32 detects the amount of accelerator pedal operation by the driver (not shown). The brake sensor 33 detects the amount of brake pedal operation by the driver (not shown). The steering angle sensor 34 detects the rotation angle (steering angle) of the steering wheel or steering shaft (not shown). The steering torque sensor 35 detects the rotation torque (steering torque) of the steering wheel or steering shaft (not shown). The yaw rate sensor 36 detects the yaw rate of the vehicle VH. The internal sensor device 30 transmits the state of the vehicle VH detected by each sensor 31 to 36 to the ECU 10 at a predetermined interval.
[0015] The external sensor device 40 is a set of sensors that recognize object information relating to objects around the vehicle VH. Specifically, the external sensor device 40 includes a radar sensor 41, a camera 42, etc. Here, object information can be seen, for example, surrounding vehicles, pedestrians, traffic lights, road markings, signs, etc.
[0016] The radar sensor 41 detects targets present around the vehicle VH. The radar sensor 41 includes a millimeter-wave radar and / or a lidar. The millimeter-wave radar emits millimeter-wave radio waves (millimeter waves) and receives millimeter waves (reflected waves) reflected by targets within its radiation range. The millimeter-wave radar acquires the relative distance between the vehicle VH and the target, the relative speed between the vehicle VH and the target, etc., based on the phase difference between the transmitted millimeter waves and the received reflected waves, the attenuation level of the reflected waves, and the time from the transmission of the millimeter waves to the reception of the reflected waves. The lidar sequentially scans pulsed laser light with a wavelength shorter than millimeter waves in multiple directions and receives reflected light reflected by targets to acquire the shape of targets detected in front of the vehicle VH, the relative distance between the vehicle VH and the target, the relative speed between the vehicle VH and the target, etc. The camera 42 is the imaging means of this disclosure, and acquires target information in front of the vehicle VH by imaging the area in front of the vehicle VH. For example, a digital camera having an image sensor such as a CMOS or CCD can be used as the camera 42. The external sensor device 40 repeatedly transmits target information acquired by the radar sensor 41 and the camera 42 to the ECU 10 at predetermined intervals.
[0017] The illuminance sensor (light sensor) 50 is a sensor that detects the illuminance of light. The illuminance sensor 50 is mounted on the vehicle VH so that it can detect the illuminance around the vehicle VH. The illuminance sensor 50 transmits the detected illuminance information to the ECU 10 at a predetermined interval.
[0018] The left headlight 60L and the right headlight 60R emit light towards the front of the vehicle VH. Here, the front of the vehicle VH includes not only the front but also the left and right diagonal directions. The left headlight 60L is located on the left side of the front of the vehicle VH. The right headlight 60R is located on the right side of the front of the vehicle VH. The left headlight 60L and the right headlight 60R are basically constructed almost identically, with the left and right sides reversed. Therefore, in the following, when there is no need to distinguish between them, the left headlight 60L and the right headlight 60R will simply be referred to as "headlight 60".
[0019] The headlamp 60 includes a low-beam headlamp and a high-beam headlamp. The low-beam headlamp irradiates low-beam irradiation light onto the front area of the vehicle VH. The high-beam headlamp irradiates high-beam irradiation light onto a wider area in front of the vehicle VH than the low-beam irradiation light. The headlamp 60 lights up or goes out according to an instruction signal transmitted from the ECU 10 in response to an operation of an operating device (not shown) by the driver. Also, when the operating device (not shown) is operated to the automatic position, the headlamp 60 lights up or goes out according to an instruction signal transmitted from the ECU 10 based on the illuminance information acquired by the ambient light sensor 50.
[0020] [[ID=^4]] The headlamp 60 is an AHS-compatible headlamp and has a function of blocking a part of the irradiation area by high-beam irradiation according to the position of a dimming target object (for example, a preceding vehicle, an oncoming vehicle, a pedestrian, a sign, etc.) acquired by the external sensor device 40. In the present disclosure, blocking light includes the concept of reducing light. Examples of headlamps having such a function include those having a plurality of LEDs (Light Emitting Diodes) arranged in a matrix, or those having a DMD (Digital Mirror Device) constituted by a plurality of micro mirror elements arranged in a matrix, or those having a MEMS (Micro Electro Mechanical Systems) mirror, etc. Since the configurations of these headlamps are well-known, detailed descriptions thereof are omitted.
[0021] The HMI 70 is an interface for inputting and outputting information between the ECU 10 and the driver, and includes an input device and an output device. Examples of the input device include a touch panel, a switch, a voice pickup microphone, etc. Examples of the output device include a display device 71, a speaker 72, etc. The display device 71 is, for example, a center display, a multi-information display, a head-up display, etc. The speaker 72 is, for example, a speaker of an audio system or a navigation system.
[0022] [Software Configuration] FIG. 2 is a schematic diagram showing the software configuration of the ECU 10 according to the present embodiment. As shown in FIG. 2, the ECU 10 includes, as functional elements, a light distribution control unit 100, a target recognition unit 110, a reliability threshold correction unit 120, a PCS control unit 130, and the like. Each of these functional elements 100 to 130 is realized by the CPU 11 of the ECU 10 reading out a program stored in the ROM 12 and executing it in the RAM 13. Note that all or part of each of the functional elements 100 to 130 can also be provided in another ECU separate from the ECU 10 or in an information processing device of a facility (such as a management center) that can communicate with the vehicle VH.
[0023] The light distribution control unit 100 is the light distribution control means of the present disclosure. It acquires the position of the dimming target object based on the detection result of the external sensor device 40, and performs light distribution control to block or dim a part of the irradiation area by high beam irradiation according to the position of the acquired dimming target object. Examples of the dimming target object include a preceding vehicle, an oncoming vehicle, the face of a pedestrian, a road sign, and the like. By blocking or dimming the position of the preceding vehicle, the oncoming vehicle, or the face of the pedestrian, it becomes possible to prevent the driver of the preceding vehicle, the oncoming vehicle, or the pedestrian from being dazzled by glare. Also, by blocking or dimming the position of a retroreflective object such as a road sign, it becomes possible to prevent the driver of the host vehicle VH from being dazzled by the reflected light from the retroreflective object.
[0024] Figure 3 shows an example of the light distribution pattern of the high beam illumination emitted from the headlight 60. The light distribution pattern shown in Figure 3 is a composite of the illumination from the left headlight 60L and the right headlight 60R. The area enclosed by the dashed line X in the figure indicates the illumination area due to the high beam illumination from the headlight 60. The area enclosed by the dashed line Y in the figure is an example of the shooting range of the camera 42. Based on the detection results of the external sensor device 40, the headlight control unit 100 acquires, for example, an oncoming vehicle VH2 as a target for dimming, and illuminates the area corresponding to the position of the oncoming vehicle VH2 with a high beam illumination pattern that is shielded or dimmed from the headlight 60. Hereinafter, the area illuminated by the high beam from the headlight 60 will be referred to as "headlight illumination area A". The area not illuminated by the high beam from the headlight 60 (including the shielded or dimmed area) will be referred to as "headlight non-illuminated area B".
[0025] The target recognition unit 110 is a target recognition means according to the present disclosure, which recognizes a target located in front of the vehicle VH based on image data transmitted from the camera 42 of the external sensor device 40. Specifically, the target recognition unit 110 calculates the confidence level DR of the target based on the clarity of the target's outline included in the image data captured by the camera 42, the degree of agreement with the feature points of the registered image pattern, etc. The confidence level DR is an index that indicates the probability that the target actually exists. If the confidence level DR is small, the probability that the target actually exists is low, and if the confidence level DR is large, the probability that the target actually exists is high. If the calculated confidence level DR is equal to or greater than a predetermined confidence threshold DRv (DR ≥ DRv), the target recognition unit 110 recognizes the target included in the image data as an actually existing target.
[0026] Incidentally, targets located in the headlight-unlit area B (indicated as VH2 and H in Figure 3) tend to have a lower confidence level (DR) compared to targets located in the headlight-unlit area A. Therefore, if the confidence threshold DRv is set to a uniform fixed value, it may lead to false detections or failures to detect targets that actually exist in the headlight-unlit area B by the target recognition unit 110.
[0027] The reliability threshold correction unit 120 performs threshold correction to correct the reliability threshold DRv in order to prevent false detection or failure to detect targets in the headlight non-illuminated area B. The reliability threshold correction unit 120 is the reliability threshold changing means of this disclosure. Based on the light distribution control information from the light distribution control unit 100, the reliability threshold correction unit 120 identifies the headlight illumination area A by the headlight 60 from the image data of the camera 42. Once the reliability threshold correction unit 120 identifies the headlight illumination area A, it determines whether or not a target in the image data is located within the headlight illumination area A. If the reliability threshold correction unit 120 determines that a target is located within the headlight illumination area A, it does not perform threshold correction.
[0028] On the other hand, the confidence threshold correction unit 120 performs threshold correction when it determines that the target is not in the headlight illumination area A, that is, when the target in the image data is in the headlight non-illumination area B. The confidence threshold correction unit 120 performs threshold correction by multiplying the confidence threshold DRv by a predetermined gain coefficient k. Hereinafter, the corrected confidence threshold (=DRv × k) will be referred to as the "corrected confidence threshold DRv'". The gain coefficient k is a value greater than or equal to 0 and less than 1 (0 ≤ k < 1). That is, the corrected confidence threshold DRv' is a smaller value than the confidence threshold DRv. In this way, when the target in the image data is in the headlight non-illumination area B, the target recognition unit 110 corrects the threshold used for determining target recognition to a corrected confidence threshold DRv' which is smaller than the confidence threshold DRv, thereby effectively preventing the failure to detect or the false detection of targets in the headlight non-illumination area B. The gain coefficient k may be a fixed value, or it may be a variable value corresponding to the illuminance detected by the light sensor 50.
[0029] The PCS control unit 130 is a collision avoidance control means of the present disclosure, which performs PCS control to avoid a collision between the vehicle VH and a target in front of it or to mitigate the damage of a collision. Based on the target information recognized by the target recognition unit 110, the PCS control unit 130 determines whether or not there is a target subject to PCS control (hereinafter referred to as a target target) in front of the vehicle VH. If the PCS control unit 130 determines that a target target exists, it acquires the coordinate information of the target target based on the detection results of the external sensor device 40. The PCS control unit 130 also calculates the turning radius of the vehicle VH based on the detection results of the vehicle speed sensor 31, the steering angle sensor 34, and the yaw rate sensor 36, and calculates the trajectory of the vehicle VH based on this turning radius. The PCS control unit 130 determines whether the target target in front of the vehicle VH is an obstacle that could potentially collide with the vehicle VH. The PCS control unit 130 calculates the trajectory of the target object based on the coordinate information of the target object when the target object is moving, and determines the target object to be an obstacle when the trajectory of the target object intersects with the trajectory of the vehicle VH. In addition, the PCS control unit 130 determines the target object to be an obstacle when the target object is stationary, and the trajectory of the vehicle VH intersects with the target object's current position.
[0030] When the PCS control unit 130 determines that a target object is an obstacle, it calculates the predicted collision time (TTC) until the vehicle VH collides with the obstacle, based on the distance L from the vehicle VH to the obstacle and the relative velocity Vr of the vehicle VH with respect to the obstacle. TTC is an index value indicating the probability that the vehicle VH will collide with the obstacle. TTC can be obtained by dividing the distance L from the vehicle VH to the obstacle by the relative velocity Vr (TTC = L / Vr). If the TTC is less than or equal to a predetermined collision determination threshold Tv, the PCS control unit 130 determines that there is a high probability that the vehicle VH will collide with the obstacle. In this embodiment, when the target object is located in the headlight-unilluminated area B, the target recognition unit 110 recognizes the presence of the object based on a corrected confidence threshold DRv' which is smaller than the confidence threshold DRv. That is, even when the obstacle targeted by PCS control is located in the headlight-unilluminated area B, the system is configured to effectively prevent failure to detect or false detection of the obstacle. This will make it possible to reliably improve the accuracy of collision detection in PCS control.
[0031] When the PCS control unit 130 determines that there is a high probability that the vehicle VH will collide with an obstacle, it issues a warning via the speaker 72 and / or the display device 71, and also performs automatic brake control. Automatic brake control is a control that slows down the vehicle VH by controlling the operation of the braking device 22 and / or the drive device 20 so that the deceleration of the vehicle VH matches a predetermined target deceleration. This allows the vehicle VH to be forcibly decelerated without requiring the driver to operate the brake pedal.
[0032] Next, based on Figure 4, the routines for target recognition processing and PCS control processing by the CPU 11 of the ECU 10 will be explained. This routine starts when vehicle VH is in motion.
[0033] In step S100, the ECU 10 determines, based on the detection results of the external sensor device 40, whether or not a PCS-controlled target exists in front of the vehicle VH. If a target exists (Yes), the ECU 10 proceeds to the process in step S110. On the other hand, if a target does not exist (No), the ECU 10 returns to this routine.
[0034] In step S110, the ECU 10 determines whether the illuminance around the vehicle VH is below a predetermined illuminance based on the illuminance information acquired by the light sensor 50. If the illuminance around the vehicle VH is below the predetermined illuminance (Yes), i.e., it is dark, the ECU 10 proceeds to the process in step S120. On the other hand, if the illuminance around the vehicle VH is not below the predetermined illuminance (No), i.e., it is bright, the ECU 10 proceeds to the process in step S160.
[0035] In step S120, the ECU 10 determines whether or not the light distribution control of the headlight 60 is currently being performed. If the light distribution control is being performed (Yes), the ECU 10 proceeds to the process in step S130. On the other hand, if the light distribution control is not being performed (No), the ECU 10 proceeds to the process in step S160.
[0036] In step S130, the ECU 10 identifies the headlight illumination area A by the headlight 60. Next, in step S140, the ECU 10 determines whether or not the target object is located in the headlight illumination area A. If the target object is located in the headlight illumination area A (Yes), the ECU 10 proceeds to step S160. On the other hand, if the target object is not located in the headlight illumination area A (No), that is, if the target object is located in the headlight non-illuminated area B, the ECU 10 proceeds to step S150.
[0037] In step S150, the ECU 10 corrects the confidence threshold DRv to set the threshold used for determining target recognition as the corrected confidence threshold DRv'. Then, in step S155, targets with a confidence level DR equal to or greater than the corrected confidence threshold DRv' are recognized as targets for PCS control.
[0038] When the process proceeds from step S110, step S120, or step S140 to step S160, the ECU 10 sets the threshold used for determining target recognition to the normal confidence threshold DRv. Then, in step S165, targets with a confidence level DR equal to or greater than the confidence threshold DRv are recognized as targets for PCS control.
[0039] In step S170, the ECU 10 determines whether the object recognized in step S155 and / or step S165 is an obstacle. If the object is moving, the ECU 10 determines the object is an obstacle if the object's trajectory intersects with the trajectory of the vehicle VH. If the object is stationary, the ECU 10 determines the object is an obstacle if the trajectory of the vehicle VH intersects with the object's current position. If the ECU 10 determines the object is an obstacle (Yes), it proceeds to step S180. On the other hand, if the ECU 10 determines that the object in front of the vehicle VH is not an obstacle (No), it returns to this routine.
[0040] In step S180, the ECU 10 calculates TTC (=L / vr) by dividing the distance L from the vehicle VH to the target by the relative velocity Vr. Next, in step S185, the ECU 10 determines whether TTC is less than or equal to the collision detection threshold Tv. If TTC is less than or equal to the collision detection threshold Tv (Yes), the ECU 10 proceeds to the process in step S190. On the other hand, if TTC is greater than the collision detection threshold Tv (No), the ECU 10 returns to this routine.
[0041] In step S190, the ECU 10 issues a warning and performs automatic braking control to decelerate the vehicle VH based on a predetermined target deceleration. After that, the ECU 10 returns to this routine.
[0042] Although the target recognition device according to this embodiment has been described above, this disclosure is not limited to the above embodiment, and various modifications are possible without departing from the purpose of this disclosure.
[0043] For example, in the above embodiment, the object recognized by the object recognition unit 110 was described as being applied to PCS control, but it is also possible to apply it to other driver assistance controls such as Adaptive Cruise Control (ACC) and Lane Trace Assist (LTA). Furthermore, the technology disclosed herein can also be applied to autonomous vehicles that perform some or all of the driving operations automatically.
Claims
1. A photographic means for photographing a predetermined area in front of the vehicle, A target recognition device comprising: a target recognition means that calculates a confidence index for target recognition of a target in an image captured by the aforementioned imaging means, and recognizes the target as an actual existing target if the calculated confidence index is equal to or greater than a predetermined confidence threshold, Based on information regarding the presence of a target in front of the vehicle, a light distribution control means controls the light distribution of the headlights by blocking or dimming a portion of the light emitted from the headlights of the vehicle, thereby controlling the light distribution of the headlights in a light distribution pattern that includes an illumination area for emitting the light and a dimming area for blocking or dimming the light. The system includes a confidence threshold changing means that, if the target in the image is not present in the irradiation area, changes the confidence threshold to a smaller value than when the target is present in the irradiation area. A target recognition device characterized by the following features.
2. A target recognition device according to claim 1, The confidence threshold changing means, when no target in the image is present in the irradiation area, changes the confidence threshold to a smaller value by multiplying it by a gain coefficient that is 0 or greater and less than 1. A target recognition device characterized by the following features.
3. A target recognition device according to claim 1 or 2, The vehicle is equipped with collision avoidance control means that, when an object recognized in front of the vehicle meets predetermined collision conditions, performs collision avoidance control to avoid a collision between the vehicle and the object or to mitigate the damage of the collision. The target recognition means transmits information about the recognized target to the collision avoidance control means, thereby causing the collision avoidance control means to recognize the object. A target recognition device characterized by the following features.