Object recognition device

Abstract

To provide an object recognition device capable of preventing the behavior of an object image from being recognized together with the behavior (vector) of a virtual image even when the behavior (vector) of the object image of a real object is detected and recognized confused with the virtual image.SOLUTION: When reliability of all patterns for an object image (first to third reliability D1 to D3) is high and the third reliability D3 is continuously high for two frames or more, a travel_ECU 14 adds a frequency "1" to a first evaluation value Ve1 indicating that the object image is highly likely to be a real image, and, upon the first evaluation value Ve1 exceeding a comparison value, determines that the object image is the real image.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an object recognition device that recognizes whether an object image detected based on a reflection signal of a transmission signal irradiated into space is a real image or a virtual image.

Background Art

[0002] Conventionally, in vehicles such as automobiles, a driving support device for assisting a driver's driving operation has been put into practical use for the purpose of reducing the burden of the driver's driving operation and improving safety.

[0003] As a technology related to the active safety of this type of driving support device, various technologies for performing emergency collision avoidance control with obstacles (objects) such as a preceding vehicle existing in front of the traveling path of the host vehicle or a pedestrian invading the traveling path of the host vehicle have been proposed.

[0004] In such emergency collision avoidance control, as a sensor for detecting obstacles such as a preceding vehicle or a pedestrian, a radar such as a millimeter-wave radar or a lidar is widely used. Object detection using a radar has the advantages of being able to ensure a sufficient detection distance and being less affected by weather and the like. On the other hand, object detection using a radar tends to erroneously detect a virtual image (ghost) due to the influence of so-called multipath and the like.

[0005] For this reason, many technologies have been proposed for excluding virtual images from the object images detected using a radar and recognizing only real images. For example, Patent Document 1 discloses a technique for determining that an object image having a longer relative distance to the host vehicle among two object images selected as determination targets is more likely to be a virtual image (less reliable) than an object image having a shorter relative distance to the host vehicle. Patent Document 1 also discloses a technique for determining that an object image having a small reception intensity of a reflection signal is more likely to be a virtual image (less reliable) than an object image having a large reception intensity.

[0006] Furthermore, in this type of object recognition, a technique is known in which the evaluation value for an object image is cumulatively updated each time the same object image is detected between frames, and the recognition of whether or not the object image is a real image is performed based on the updated evaluation value. Commonly used evaluation values ​​for an object image include, for example, the cumulative number of times the object image has been judged to have high confidence, and the number of times the object image has been judged to have low confidence. When the cumulative number of times the object image has been judged to have high confidence exceeds the cumulative number of times it has been judged to have low confidence, the object image is recognized as a real image. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2009-186277 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, for example, if a virtual image of an undetected object has already been detected at a close distance to a real, undetected object by a sensor used in a vehicle, the undetected object may be confused with the already detected virtual image. In such cases, the confused object image inherits the behavior (vector) of the previous virtual image. If the confused object image is recognized as a real image before the influence of the virtual image's behavior on the confused object image is resolved, a behavior (vector) different from the actual behavior may be recognized as the behavior (vector) of the real object, and there is a risk that emergency collision avoidance control will be executed in accordance with that behavior (vector).

[0009] The present invention aims to provide an object recognition device that can suppress the recognition of the behavior (vector) of an object image of a real object in accordance with the behavior (vector) of the virtual image, even when the behavior (vector) of the object image of a real object is detected and recognized in confusion with that of a virtual image. [Means for solving the problem]

[0010] An object recognition device according to one aspect of the present invention detects an object image frame by frame based on the reflected signal of the transmitted signal, and the detected object image The confidence levels of multiple patterns are calculated based on a predetermined calculation method, and the calculated confidence levels An object recognition device that performs object recognition based on the above, wherein the reliability of all patterns for the object image is high, multiple The system includes: a first evaluation value calculation unit that adds a frequency of a first evaluation value indicating that the object image is likely to be a real image when a predetermined specific confidence level among the confidence levels of the pattern remains high for a set number of frames or more; and a real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds a comparison value. The aforementioned specific confidence level is the confidence level calculated from among a plurality of confidence levels based on the time-series behavior of the object image.

[0011] An object recognition device according to another aspect of the present invention detects an object image frame by frame based on the reflected signal of the transmitted signal. Multiple patterns The reliability is calculated using a predetermined method. Calculate A reliability calculation unit, and the reliability of all patterns for the object image is high, multiple The system includes: a first evaluation value calculation unit that adds a frequency of a first evaluation value indicating that the object image is likely to be a real image when a predetermined specific confidence level among the confidence levels of the pattern remains high for a set number of frames or more; and a real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds a comparison value. The aforementioned specific confidence level is the confidence level calculated from among a plurality of confidence levels based on the time-series behavior of the object image. Furthermore, the system includes a second evaluation value calculation unit that adds a second evaluation value frequency indicating a high probability that the object image is a virtual image when at least one of the confidence levels of multiple patterns for the object image is low, and the real image determination unit determines that the object image is a real image when a set frame has elapsed since the detection of the object image and the frequency of the first evaluation value exceeds the frequency of the second evaluation value. Furthermore, the object recognition device detects an object image frame by frame based on the reflected signal of the transmitted signal, calculates the reliability of at least one pattern of the detected object image based on a preset calculation method, and performs object recognition based on the calculated reliability, comprising: a first evaluation value calculation unit that adds the frequency of a first evaluation value indicating that the object image is likely to be a real image when the reliability of all patterns for the object image is high and a predetermined specific reliability among the reliability of at least one pattern remains high for a set frame or longer; a real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds a comparison value; and a second evaluation value calculation unit that adds the frequency of a second evaluation value indicating that the object image is likely to be a virtual image when at least one of the reliability of the at least one pattern for the object image is low, wherein the real image determination unit determines that the object image is a real image after a set frame has elapsed since the detection of the object image and the frequency of the first evaluation value exceeds the frequency of the second evaluation value. Furthermore, the system includes a reliability calculation unit that calculates the reliability of at least one pattern of object images detected frame by frame based on the reflected signal of the transmitted signal using a preset calculation method; a first evaluation value calculation unit that adds the frequency of a first evaluation value indicating that the object image is likely to be a real image when the reliability of all patterns for the object image is high and a predetermined specific reliability among the reliability of at least one pattern remains high for a set number of frames or longer; a real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds a comparison value; and a second evaluation value calculation unit that adds the frequency of a second evaluation value indicating that the object image is likely to be a virtual image when at least one of the reliability of the at least one pattern for the object image is low, wherein the real image determination unit determines that the object image is a real image after a set number of frames have elapsed since the detection of the object image and the frequency of the first evaluation value exceeds the frequency of the second evaluation value. [Effects of the Invention]

[0012] According to the object recognition device of the present invention, even when the behavior (vector) of an object image of a real object is detected and recognized in confusion with a virtual image, it is possible to suppress the recognition of the object image's behavior in accordance with the behavior (vector) of the virtual image. [Brief explanation of the drawing]

[0013] [Figure 1]Schematic Configuration Diagram of Driving Support Device [Figure 2] Explanatory Diagram Showing Monitoring Areas of Stereo Camera and Radar [Figure 3] Schematic Configuration Diagram of Radar Unit and Object Recognition Device [Figure 4] Flowchart Showing Object Image Detection Routine [Figure 5] Flowchart Showing Real Image Judgment Routine for Object Image [Figure 6] Time Chart Illustrating Transition of Reliability of Object Image Calculated by Plurality of Pattern Calculation Methods [Figure 7] Chart Showing Transition of Evaluation Value Obtained by Cumulatively Adding Frequencies Based on Respective Reliabilities in FIG. 6 [Figure 8] Explanatory Diagram Showing Virtual Image of Own Vehicle [Figure 9] Explanatory Diagram Showing Virtual Image of Own Vehicle and Image of Pedestrian [Figure 10] Explanatory Diagram Showing Image of Pedestrian Detected in Confusion with Virtual Image of Own Vehicle [Figure 11] Explanatory Diagram Showing Image of Pedestrian Recognized as Real Image [Figure 12] Chart Showing Transition of Evaluation Value Obtained by Cumulatively Adding Frequencies Based on Respective Reliabilities in FIG. 6 for First Comparative Example [Figure 13] Chart Showing Transition of Evaluation Value Obtained by Cumulatively Adding Frequencies Based on Respective Reliabilities in FIG. 6 for Second Comparative Example [Figure 14] Schematic Configuration Diagram of Radar Unit and Object Recognition Device for First Modified Example [Figure 15] Schematic Configuration Diagram of Radar Unit and Object Recognition Device for Second Modified Example

Mode for Carrying Out the Invention

[0014] An embodiment of one aspect of the present invention will be described in detail below with reference to the drawings. In the drawings used in the following description, the scale of each component is different in order to make each component recognizable on the drawing for illustrative purposes. Therefore, the present invention is not limited to the number of components, the shapes of the components, the ratio of the sizes of the components, and the relative positional relationships of each component as shown in these drawings.

[0015] As shown in Figures 1 and 2, the driver assistance device 1 is configured to have, for example, a camera unit 10 fixed to the front and upper center of the interior of the vehicle (own vehicle) M.

[0016] This camera unit 10 is composed of a stereo camera 11, an image processing unit (IPU) 12, an image recognition unit (image recognition ECU) 13, and a driving control unit (driving ECU) 14.

[0017] The stereo camera 11 includes a main camera 11a and a sub-camera 11b. The main camera 11a and sub-camera 11b are made of, for example, a CMOS sensor. These main camera 11a and sub-camera 11b are positioned symmetrically on either side of the center in the vehicle width direction.

[0018] The main camera 11a and the sub-camera 11b stereo-image the driving environment in the area Af (see Figure 2) in front of the vehicle from different viewpoints. The imaging periods of the main camera 11a and the sub-camera 11b are synchronized with each other.

[0019] The IPU 12 processes the driving environment image captured by the stereo camera 11 according to a predetermined method. This allows the IPU 12 to detect the edges of various objects represented in the image, such as three-dimensional objects and road markings. The IPU 12 then calculates distance information from the positional displacement of corresponding edges in the left and right images. Based on this, the IPU 12 generates image information (distance image information) that includes distance information.

[0020] The image recognition ECU13 determines the road curvature [1 / m] of the lane markings that demarcate the left and right sides of the lane in which the vehicle M is traveling (the vehicle's path), and the width between the left and right lane markings (lane width), based on distance image information received from the IPU12. The image recognition ECU13 also determines the road curvature and the width between the left and right lane markings of adjacent lanes to the lane in which the vehicle M is traveling. Various methods are known for determining these road curvatures and lane widths. For example, the image recognition ECU13 performs a luminance-based binarization process on each pixel in the distance image. This allows the image recognition ECU13 to extract candidate points for lane markings on the road. The image recognition ECU13 then performs a curve approximation using the least squares method or the like on the extracted sequence of candidate points for lane markings. This allows the image recognition ECU13 to determine the curvature of the left and right lane markings for each predetermined section. Furthermore, the image recognition ECU13 calculates the lane width from the difference in curvature between the left and right lane markings.

[0021] The image recognition ECU13 then calculates the lane center and the vehicle's lateral position deviation based on the curvature of the left and right lane markings and the lane width. Here, the vehicle's lateral position deviation is the distance from the lane center to the center of the vehicle M in the vehicle width direction.

[0022] Furthermore, the image recognition ECU13 performs predetermined pattern matching on the distance image information. This allows the image recognition ECU13 to recognize three-dimensional objects such as guardrails, curbs, median strips, and surrounding vehicles along the road. In this recognition of three-dimensional objects by the image recognition ECU13, for example, the type of object, the distance to the object, the speed of the object, and the relative speed between the object and the vehicle M are recognized.

[0023] The various pieces of information recognized by the image recognition ECU13 are output to the driving ECU14 as driving environment information.

[0024] Thus, in this embodiment, the image recognition ECU 13, together with the stereo camera 11 and the IPU 12, corresponds to a specific example of a driving environment recognition unit that recognizes information about the driving environment outside the vehicle.

[0025] The ECU14 is a control unit for the overall control of the driver assistance system 1.

[0026] This driving ECU14 is connected to various control units, including the cockpit control unit (CP_ECU)21, the engine control unit (E / G_ECU)22, the transmission control unit (T / M_ECU)23, the brake control unit (BK_ECU)24, and the power steering control unit (PS_ECU)25, via an in-vehicle communication line such as CAN (Controller Area Network).

[0027] Furthermore, the driving ECU14 is connected to various sensors, including a locator unit 36, a left front side sensor unit 37lf, a right front side sensor unit 37rf, a left rear side sensor unit 37lf, and a right rear side sensor unit 37rr.

[0028] The CP_ECU21 is connected to a Human-Machine Interface (HMI)31 located around the driver's seat. The HMI31 includes, for example, operation switches for setting and executing various driver assistance controls, a mode switch for switching between driver assistance modes, a steering touch sensor for detecting the driver's steering state, a turn signal switch, a driver monitoring system (DMS) for driver facial recognition and gaze detection, a touch panel display, a combination meter, and a speaker.

[0029] When CP_ECU21 receives a control signal from Driving_ECU14, it appropriately notifies the driver of various information such as various warnings for the preceding vehicle, the status of driver assistance control implementation, and the driving environment of its own vehicle M, via display or voice through HMI31.

[0030] Furthermore, the CP_ECU25 outputs various input information to the Driving_ECU14, including the on or off operation status of various driver assistance controls input by the driver via the HMI31, the set vehicle speed (set speed) Vs for the vehicle M, the operation status of the turn signal switch, and the ID of the driver recognized by the DMS.

[0031] The output side of the E / G_ECU22 is connected to the throttle actuator 32 of the electronically controlled throttle, etc. Various sensors, such as an accelerator sensor (not shown), are connected to the input side of the E / G_ECU22.

[0032] The E / G_ECU22 controls the throttle actuator 32 based on control signals from the Driving_ECU14 or detection signals from various sensors. This allows the E / G_ECU22 to adjust the amount of intake air for the engine and generate the desired engine output. The E / G_ECU22 also outputs signals such as the accelerator opening angle detected by the various sensors to the Driving_ECU14.

[0033] The output side of T / M_ECU23 is connected to the hydraulic control circuit 33. Various sensors, such as a shift position sensor (not shown), are connected to the input side of T / M_ECU23. Based on the engine torque signal estimated by E / G_ECU22 and the detection signals from the various sensors, T / M_ECU23 performs hydraulic control on the hydraulic control circuit 33. As a result, T / M_ECU23 operates the friction engagement elements and pulleys provided in the automatic transmission to shift the engine output to the desired gear ratio. T / M_ECU23 also outputs signals such as the shift position detected by the various sensors to the driving_ECU14.

[0034] A brake actuator 34 is connected to the output side of the BK_ECU24. The brake actuator 34 adjusts the brake fluid pressure output to the brake wheel cylinders located on each wheel. Various sensors, such as a brake pedal sensor, yaw rate sensor, longitudinal acceleration sensor, and vehicle speed sensor (not shown), are connected to the input side of the BK_ECU24.

[0035] The BK_ECU24 controls the brake actuator 34 based on control signals from the Driving_ECU14 or detection signals from various sensors. This allows the BK_ECU24 to appropriately generate braking force on each wheel for forced braking control and yaw rate control of the vehicle M. The BK_ECU24 also outputs signals such as brake operation status, yaw rate, longitudinal acceleration, and vehicle speed (vehicle speed) detected by various sensors to the Driving_ECU14.

[0036] The output side of the PS_ECU25 is connected to an electric power steering motor 35. The electric power steering motor 35 applies steering torque to the steering mechanism through the rotational force of the motor. Various sensors, such as a steering torque sensor and a steering angle sensor, are connected to the input side of the PS_ECU25.

[0037] The PS_ECU25 controls the electric power steering motor 35 based on control signals from the driving_ECU14 or detection signals from various sensors. This causes the PS_ECU25 to generate steering torque for the steering mechanism. The PS_ECU25 also outputs signals such as the steering torque and steering angle detected by the various sensors to the driving_ECU14.

[0038] The locator unit 36 ​​is comprised of a GNSS sensor 36a and a road map database (road map DB) 36b. Higher accuracy is preferable, and if a high-precision road map database is available, using it can further improve accuracy.

[0039] The GNSS sensor 36a determines the position of the vehicle M (latitude, longitude, altitude, etc.) by receiving positioning signals transmitted from multiple positioning satellites.

[0040] The road map DB36b installed in the vehicle M may be a large-capacity storage medium such as an HDD. This road map DB36b may store high-precision road map information (dynamic map). This road map information includes, for example, lane data necessary for autonomous driving, such as lane width data, lane center position coordinate data, lane direction angle data, and speed limit data. The lane data is stored at intervals of several meters for each lane on the road map. For example, based on a request signal from the driving_ECU14, the road map DB36b outputs road map information for a set range based on the vehicle's position determined by the GNSS sensor 36a as driving environment information to the driving_ECU14. If the communication conditions are good, the road map DB36b may also receive high-precision road map information (dynamic map) with a particularly large amount of information from outside the vehicle M in accordance with the vehicle M's driving conditions.

[0041] Thus, in this embodiment, the road map DB36b, together with the GNSS sensor 36a, constitutes a specific example of a driving environment recognition unit that recognizes information about the driving environment outside the vehicle.

[0042] The left front side sensor unit 37lf and the right front side sensor unit 37rf are configured, for example, to include millimeter-wave radar. These left front side sensor unit 37lf and the right front side sensor unit 37rf are respectively located on the left and right sides of the front bumper of the vehicle M. The left front side sensor unit 37lf and the right front side sensor unit 37rf detect three-dimensional objects in the left and right diagonally forward and lateral regions Alf and Arf (see Figure 2) of the vehicle M, which are difficult to recognize with the image from the stereo camera 11, as driving environment information.

[0043] The left rear side sensor unit 37lr and the right rear side sensor unit 37rr are configured, for example, with millimeter-wave radar. These left rear side sensor unit 37lr and the right rear side sensor unit 37rr are respectively located on the left and right sides of the rear bumper. The left rear side sensor unit 37lr and the right rear side sensor unit 37rr detect three-dimensional objects in the left and right diagonal side and rear areas Alr and Arr (see Figure 2) of the vehicle M, which are difficult to recognize with the left front side sensor unit 37lf and the right front side sensor unit 37rf, as driving environment information.

[0044] Here, the left front side sensor unit 37lf, the right front side sensor unit 37rf, the left rear side sensor unit 37lr, and the right rear side sensor unit 37rr have basically the same configuration, so in the following explanation, they will be collectively referred to as radar unit 37 as appropriate.

[0045] As shown in Figure 3, the radar unit 37 is configured to include, for example, a transmitter 38, a receiver (receiving antenna) 39, and a radar control unit (radar ECU) 40.

[0046] The transmitter 38, for example, emits a millimeter-wave signal of a predetermined wavelength into space as an output signal.

[0047] The receiver 39 receives reflected signals of the transmitted signal that have been reflected by various objects present in space A, which is within the space that the vehicle can perceive. That is, for example, if the radar unit 37 is the left front side sensor unit 37lf, the receiver 39 receives reflected signals of the transmitted signal that have been reflected by various objects present in area Alf.

[0048] The radar ECU 40 emits a transmission signal from the transmitter 38 into space at predetermined timings (frames). The radar ECU 40 also analyzes the reflected signal received by the receiver 39. As a result, the radar ECU 40 detects object images present in space A frame by frame. These object images include those present around the vehicle, as well as images of vehicles traveling alongside and following on the road, and images (real images) of pedestrians and side walls along the roadside.

[0049] Furthermore, the radar ECU40 calculates information associated with the object image, such as the width of the object image, the position of the representative point of the object image (relative position to the vehicle M), and the behavior of the object image (a vector consisting of the object image's movement speed and direction of movement).

[0050] Furthermore, the radar_ECU40 also calculates the reliability of each detected object image for each frame. In this embodiment, the radar_ECU40 calculates the reliability D of the object image in at least one pattern (for example, three patterns of reliability D1 to D3 in this embodiment) based on a pre-set calculation method. Note that the reliability calculation may be performed using another ECU, or if the communication status with the outside of the vehicle M is good, the calculation may be performed by a server or the like.

[0051] The first confidence level D1 is calculated based, for example, on the relative positional relationship between multiple object images and the radar unit 37. In calculating this first confidence level D1, the first confidence level D1 is basically calculated as "high" for object images detected in locations where detection is expected. On the other hand, the first confidence level D1 is calculated as "low" for object images detected in locations where detection is not expected.

[0052] For example, as shown in Figure 8, if an object image is detected on a wall surface W1 that is inclined relative to the vehicle M and is diagonally in front of the vehicle M, then, in principle, an object image Oi will not be detected at a distance greater than the wall surface W1 relative to the vehicle M. Therefore, the first confidence level D1 for object image Oi is calculated to be "low". However, due to the characteristics of the radar unit 37, each reflection point of the transmitted signal on the wall surface W1 is detected intermittently. Therefore, if it can be determined that an object image Oi was detected through the gaps between each reflection point, the first confidence level D1 for object image Oi is calculated to be "high".

[0053] The second confidence level D2 is a confidence level calculated by taking into account, for example, the characteristics of multipath. That is, if a stationary object that may cause multipath is detected, the predicted position of the moving object image Oi can be calculated from the distance from the radar unit 37 to the stationary object and the velocity of the object image Oi, etc. Therefore, when the object image Oi is within the range of the predicted position, the second confidence level D2 for the object image Oi is calculated to be "low". Conversely, when the object image Oi is outside the range of the predicted position, the second confidence level D2 for the object image Oi is calculated to be "high".

[0054] Furthermore, this second confidence level D2 is judged as "low" less frequently than the other confidence levels. Therefore, if the second confidence level D2 is judged as "low" for an object image, there is a high probability that the object image is a virtual image.

[0055] The third confidence level D3 is a confidence level calculated, for example, based on the time-series behavior of the object image. This third confidence level D3 is a confidence level that changes between "high" and "low" with the highest responsiveness to changes in the detection state of the object image, and corresponds to a specific example of the specific confidence level in this embodiment.

[0056] These calculated first to third confidence levels, D1 to D3, are output to the driving ECU14 along with the object image detection information.

[0057] Thus, in this embodiment, the left front side sensor unit 37lf, the right front side sensor unit 37rf, the left rear side sensor unit 37lr, and the right rear side sensor unit 37rr correspond to specific examples of a driving environment recognition unit that recognizes driving environment information outside the vehicle. Also, in this embodiment, the radar ECU 40 corresponds to a specific example of a reliability calculation unit.

[0058] Furthermore, the coordinates of each object image detected by the left front side sensor unit 37lf, the right front side sensor unit 37rf, the left rear side sensor unit 37lr, and the right rear side sensor unit 37rr are converted, together with the coordinates of each external object included in the driving environment information recognized by the image recognition ECU 13 and the locator unit 36, into coordinates in a three-dimensional coordinate system (see Figure 2) with the center of the vehicle M as the origin.

[0059] When object image detection information and reliability information are input from the left front side sensor unit 37lf, the right front side sensor unit 37rf, the left rear side sensor unit 37lr, and the right rear side sensor unit 37rr, the driving ECU 14 determines whether each object image is a real image or not.

[0060] When determining whether this object image is a real image, the driving ECU14 calculates an evaluation value Ve. This evaluation value Ve includes a first evaluation value Ve1. The first evaluation value Ve1 is an evaluation value whose frequency is cumulatively added when the confidence level of the object image is predetermined to be high.

[0061] The first evaluation value Ve1 is an evaluation value to which a frequency is added when, for example, the first to third confidence levels D1 to D3 for the object image are all "high", and the third confidence level D3 remains "high" for a set number of frames or longer. More specifically in calculating this first evaluation value Ve1, the driving_ECU14 adds a frequency of "1" to the first evaluation value Ve1 when the first to third confidence levels D1 to D3 for the object image are all "high", and the third confidence level D3 in the previous frame and the current frame remains "high" consecutively.

[0062] Furthermore, the evaluation value Ve includes a second evaluation value Ve2. The second evaluation value Ve2 is an evaluation value whose frequency is cumulatively added when the confidence level for the object image is predetermined to be low.

[0063] The second evaluation value Ve2 is an evaluation value to which a frequency is added when, for example, at least one of the first and second confidence levels D1 to D3 for the object image is "low". More specifically in calculating this second evaluation value Ve2, the driving_ECU14 adds a frequency of "1" to the second evaluation value Ve2 when at least one of the first and second confidence levels D1 to D3 for the object image is "low". However, if the second confidence level D2 is "low", the driving_ECU14 adds a frequency of "2" to the second evaluation value Ve2 because there is a high probability that the object image in question is a virtual image.

[0064] The driving ECU14 then determines that an object image is a real image when a set number of frames (for example, 8 frames or more) have elapsed since the object image was detected, and the cumulative frequency of the first evaluation value Ve1 exceeds the cumulative frequency of the second evaluation value Ve2. In other words, the driving ECU14 recognizes that the object corresponding to the object image it has determined to be a real image exists in real space.

[0065] The driving ECU 14 then uses only the object images that are determined to be real images from the object images detected by the radar unit 37 as the control targets for various driving assistance controls.

[0066] Thus, in this embodiment, the driving ECU14 corresponds to a specific example of the first evaluation value calculation unit, the second evaluation value calculation unit, and the real image determination unit, and further corresponds to a specific example of an object recognition device.

[0067] The driving ECU14 has settings for driving modes, including a manual driving mode and two driving control modes: a first driving control mode and a second driving control mode. Each of these driving modes can be selectively switched in the driving ECU14 based on, for example, the operation status of the mode switching switch provided on the HMI31.

[0068] Here, manual driving mode refers to a driving mode that requires the driver to maintain steering. In other words, manual driving mode is a driving mode in which the vehicle M is driven according to driving operations such as steering, accelerating, and braking performed by the driver.

[0069] The first driving control mode is also a driving mode that requires the driver to maintain steering. In other words, the first driving control mode is a semi-autonomous driving mode in which the vehicle M assists the driver while reflecting the driver's driving operations. This first driving control mode is realized, for example, by the driving_ECU14 outputting various control signals to the E / G_ECU22, BK_ECU24, and PS_ECU25. In the first driving control mode, adaptive cruise control (ACC), active lane keep centering (ALKC), active lane departure prevention (ALKB), and lane change control are mainly performed in appropriate combinations. As a result, the vehicle M can drive along the target driving path. Furthermore, in the first driving control mode, lane change control can also be performed when the driver operates the turn signal switch (direction indicator).

[0070] Here, the adaptive cruise control is basically performed based on driving environment information input from the image recognition ECU13, etc.

[0071] To explain in more detail, the Driving ECU14, for example, if a preceding vehicle is recognized in front of the vehicle M by the Image Recognition ECU13, performs follow-up driving control as part of follow-up distance control. In this follow-up driving control, the Driving ECU14 sets a target distance Lt and target speed Vt based on the speed Vl of the preceding vehicle. Then, the Driving ECU14 performs acceleration and deceleration control for the vehicle M based on the target distance Lt and target speed Vt. As a result, the Driving ECU14 basically maintains the distance L at the target distance Lt and the speed V at the target speed Vt, causing the vehicle M to follow the preceding vehicle.

[0072] On the other hand, if, for example, the image recognition ECU14 does not recognize a preceding vehicle in front of the vehicle M, the driving ECU14 performs constant speed driving control as part of the follow distance control. In this constant speed driving control, the driving ECU14 sets the set vehicle speed Vs input by the driver as the target vehicle speed Vt. Then, the driving ECU14 performs acceleration and deceleration control for the vehicle M based on the target vehicle speed Vt. As a result, the driving ECU14 maintains the vehicle speed V of the vehicle M at the set vehicle speed Vs.

[0073] Furthermore, lane centering control and lane departure prevention control are basically performed based on driving environment information input from at least one of the image recognition ECU 13 and the locator unit 36. That is, the driving ECU 14 sets a target path Rm along the left and right lane markings in the center of the vehicle's driving lane, based on lane marking information included in the driving environment information, for example. Then, based on the target path Rm, the driving ECU 14 maintains the vehicle M in the center of the lane by performing feedforward control and feedback control for steering. In addition, when the driving ECU 14 determines that there is a high possibility that the vehicle M will deviate from its driving lane due to the effects of crosswinds or road banking, it suppresses lane departure by forcibly controlling the steering.

[0074] The second driving control mode is a driving mode in which the vehicle M is driven without requiring steering, acceleration, or braking by the driver. In other words, the second driving control mode is an autonomous driving mode in which the vehicle M is driven autonomously without requiring any driving operation by the driver. This second driving control mode is realized, for example, by the driving_ECU14 outputting various control signals to the E / G_ECU22, BK_ECU24, and PS_ECU25. In the second driving control mode, the preceding vehicle following control, lane centering control, and lane departure prevention control are mainly performed in appropriate combinations. As a result, the vehicle M is able to drive according to the target route (route map information).

[0075] Furthermore, the driving ECU14 performs emergency collision avoidance control as appropriate for obstacles such as vehicles that are highly likely to collide with the vehicle M. This emergency collision avoidance control includes, for example, emergency braking (AEB: Autonomous Emergency Braking (collision damage mitigation brake)) control.

[0076] Emergency braking control is basically a control system that uses braking to avoid collisions with obstacles located ahead of the vehicle M on its target path Rm. When emergency braking control is performed, the driving ECU 14 sets a target travel area Am in front of the vehicle M. This target travel area Am has a predetermined width (for example, greater than or equal to the width of the vehicle M) centered on the target travel path Rm. The driving ECU 14 also detects obstacles such as preceding vehicles or stationary vehicles located on the target travel area Am based on driving environment information. Furthermore, the driving ECU 14 calculates the predicted collision time (longitudinal collision time) TTCz for the vehicle M in the longitudinal direction as the predicted collision time with the obstacle. This longitudinal collision time TTCz is calculated based on the relative speed and relative distance between the vehicle M and the obstacle.

[0077] When the longitudinal collision prediction time TTCz becomes smaller than a first threshold value Tth1 set in advance, the traveling ECU 14 executes primary brake control. This primary brake control is, for example, a so-called warning brake for prompting the driver to pay attention by deceleration. When the primary brake control is started, the traveling ECU 14 decelerates the host vehicle M using a first target deceleration a1 (for example, 0.4G) set in advance. When executing this primary brake control, it is also possible to use, for example, displays and audible warnings in combination via the HMI 31.

[0078] Furthermore, when the longitudinal collision prediction time TTCz becomes smaller than a second threshold value Tth2 (where Tth2 < Tth1) set in advance, the traveling ECU 14 executes secondary brake control. When the secondary brake control is started, the traveling ECU 14 decelerates the host vehicle M using a second target deceleration a2 (for example, 1G) set in advance until the relative speed with the obstacle becomes "0".

[0079] [ Such emergency brake control is extended, for example, to pedestrians or the like having a vector that enters the target traveling area Am of the host vehicle M from a sidewalk or the like beside the road. In this case, information on pedestrians or the like is mainly obtained from information on the object image detected by each radar unit 37. However, the object image detected by each radar unit 37 does not become a control target such as emergency brake control until it is recognized as a real image in the traveling ECU 14. This is to prevent incorrect emergency brake control or the like from being executed for virtual images generated by multipath or the like.

[0080] Next, the object image detection performed in the radar ECU 40 will be described according to the flowchart of the object image detection routine shown in FIG. 4. This routine is repeatedly executed at each set time.

[0081] When the routine starts, the radar ECU 40, in step S101, drives and controls the transmitter 38 to irradiate the three-dimensional space around the host vehicle M with millimeter waves, which are transmission signals.

[0082] In the following step S102, the radar ECU 40 acquires the millimeter-wave reflected wave (reflected signal) received by the receiver 39.

[0083] In the subsequent step S103, the radar ECU 40 detects an object image based on the acquired reflected waves and calculates information associated with the object image, such as the width of the object image, the position of the representative point of the object image, and the movement and direction of movement (vector) of the object image.

[0084] In the following step S104, the radar_ECU40 checks whether there are any object images among the multiple object images detected that have not yet been processed for the calculation of the first to third confidence levels D1 to D3, which will be described later. That is, the radar_ECU40 checks whether there are any object images among the multiple object images detected that have not yet had their first to third confidence levels D1 to D3 calculated.

[0085] Then, if it is determined in step S104 that there are unprocessed object images (step S104: YES), the radar_ECU40 proceeds to step S105.

[0086] In step S105, the radar_ECU40 extracts unprocessed object images from the object images detected this time.

[0087] Then, in steps S106-S107, the radar ECU40 sequentially calculates the first reliability D1, the second reliability D2, and the third reliability D3, and then returns to step S104.

[0088] Furthermore, if it is determined in step S104 that there are no unprocessed object images among the known object images (step S104: NO), the radar_ECU40 exits the routine.

[0089] Next, the real-image determination process for object images performed in the driving ECU14 will be explained according to the flowchart of the real-image determination routine shown in Figure 5. This routine is executed repeatedly at set intervals.

[0090] When the real image determination routine starts, the driving ECU 14 checks in step S201 whether there are any object images among the object images input from the radar unit 37 that have not yet been processed for real image determination.

[0091] Then, if it is determined in step S201 that there are unprocessed object images (step S201: YES), the driving_ECU14 proceeds to step S202.

[0092] In step S202, the driving_ECU14 extracts unprocessed object images from the object images input this time.

[0093] In the following step S203, the driving ECU14 checks whether all of the first to third confidence levels D1 to D3 for the extracted object image are "high".

[0094] Then, in step S203, if it is determined that at least one of the first to third reliability levels D1 to D3 is "low" (step S203: NO), the driving ECU14 proceeds to step S205.

[0095] On the other hand, if in step S203 all of the first to third reliability levels D1 to D3 are determined to be "high" (step S203: YES), the driving ECU14 proceeds to step S204.

[0096] When the process moves from step S203 to step S204, the driving ECU14 checks whether the previous third confidence level D3 was "high" for the object image that is identical to the object image extracted this time.

[0097] Then, in step S204, if it is determined that the reliability D3 in the previous step 3 was "low" (step S204: NO), the driving ECU14 proceeds to step S209.

[0098] On the other hand, if in step S204 the previous third reliability D3 was determined to be "high" (step S204: YES), the driving ECU14 proceeds to step S206.

[0099] In step S206, the driving ECU14 adds a frequency of "1" to the first evaluation value Ve1.

[0100] Furthermore, when moving from step S203 to step S205, the driving ECU14 checks whether the second confidence level D2 for the object image is "low".

[0101] Then, in step S205, if the second reliability level D2 is determined to be "high" (step S205: NO), the driving ECU14 proceeds to step S207.

[0102] In step S207, the driving ECU14 adds a frequency of "1" to the second evaluation value Ve2.

[0103] On the other hand, if the second reliability level D2 is determined to be "low" in step S205 (step S205: YES), the driving ECU14 proceeds to step S208.

[0104] In step S208, the driving ECU14 adds a frequency of "2" to the second evaluation value Ve2.

[0105] When the vehicle proceeds from step S204, step S206, step S207, or from step S208 to step S209, the driving ECU 14 checks whether, for example, a set number of millimeter-wave radar frames (e.g., 8 frames) has elapsed since the object image currently being extracted was first detected.

[0106] Then, in step S209, if it is determined that the set number of frames (for example, 8 frames) has not elapsed since the object image was detected (step S209: NO), the driving_ECU14 returns to step S201.

[0107] On the other hand, if in step S209 it is determined that a set number of frames (for example, 8 frames) have elapsed since the object image was detected (step S209: YES), the driving ECU 14 proceeds to step S210.

[0108] In step S210, the driving ECU14 checks whether the first evaluation value Ve1 is greater than the second evaluation value Ve2.

[0109] Then, in step S210, if it is determined that the first evaluation value Ve1 is less than or equal to the second evaluation value Ve2 (step S210: NO), the driving ECU14 returns to step S201.

[0110] On the other hand, if in step S210 it is determined that the first evaluation value Ve1 is greater than the second evaluation value Ve2 (step S210: YES), the driving ECU14 proceeds to step S211.

[0111] In step S211, the driving ECU14 determines that the currently extracted object image of the vehicle M is a real image, and then returns to step S201.

[0112] Then, in step S201, if it is determined that there are no unprocessed object images (step S201: NO), the driving_ECU14 exits the routine.

[0113] Next, an example of the effects of this type of processing will be explained with reference to Figures 8 to 11.

[0114] For example, Figure 8 shows a state in which an object image (virtual image Oi) of the vehicle M is detected at a distance greater than the wall W1 due to multipath interference from the wall W1. This virtual image Oi is detected, for example, when a transmission signal emitted from the transmitter 38 of the left front side sensor unit 37lf is received by the receiver 39 after being reflected by the wall W1, reflected by the vehicle M, and / or re-reflected by the wall W1. In Figure 8, a pedestrian is moving along the wall W2 which is approximately parallel to the direction of travel of the vehicle M, but pedestrians have a weaker reflection intensity to millimeter waves compared to metal, etc. Therefore, at the stage shown in Figure 8, the pedestrian is not detected by the radar unit 37.

[0115] As shown in Figure 9, the virtual image Oi of the vehicle M detected in this way moves toward the wall W2 as the vehicle M moves.

[0116] During this time, for example, as shown up to t+1 in Figure 6, the first to third confidence levels D1 to D3 fluctuate between "high" and "low". In particular, the third confidence level D3 fluctuates frequently between "high" and "low" with high responsiveness. Consequently, as shown in Figure 7, the frequency of the second evaluation value Ve2 increases significantly, but the frequency of the first evaluation value Ve1 does not increase as much as the frequency of the second evaluation value Ve2.

[0117] Figure 10 illustrates, for example, a scenario where a pedestrian happens to be present near a virtual image Oi of the vehicle M as the virtual image Oi approaches the wall W2. In this case, when the radar unit 37 receives a reflected wave from the pedestrian, which was located along the wall W2 approximately parallel to the direction of travel of the vehicle M, the pedestrian's physical image may be confused with the virtual image Oi due to their close proximity (see Oi' in Figure 10).

[0118] In this way, the pedestrian object image Oi', which is detected by being confused with the virtual image Oi, inherits the behavior of the virtual image Oi (movement speed and direction, etc.) and the frequencies of the first and second evaluation values ​​Ve1 and Ve2.

[0119] In the illustrated example, since the pedestrian is real, the first to third confidence levels D1 to D3 for the object image Oi' are maintained at a relatively stable "high" state (see t+n onwards in Figure 6).

[0120] However, the frequency of the first evaluation value Ve1 remains sufficiently low compared to the frequency of the second evaluation value Ve2 and is carried over to the object image Oi'. Therefore, the recognition of the object image Oi' as a real image at a relatively early stage after detection is suppressed.

[0121] Then, until the frequency of the first evaluation value Ve1 becomes equal to the frequency of the second evaluation value Ve2 (until t+n+m in Figure 6), the influence of the behavior inherited from the virtual image Oi gradually decreases, and the behavior of the object image Oi' approaches the behavior of an actual pedestrian.

[0122] Subsequently, when the frequency of the first evaluation value Ve1 exceeds the frequency of the second evaluation value Ve2, the object image Oi' is recognized as a real image Ri (see Figure 11).

[0123] Therefore, the execution of incorrect emergency braking control, etc., in response to the pedestrian object image (real image Ri) detected by the radar unit 37 is accurately prevented.

[0124] Figures 12 and 13 show the changes in the first and second evaluation values ​​Ve1 and Ve2, calculated using other methods, for the first to third confidence levels D1 to D3 shown in Figure 6.

[0125] The first evaluation value Ve1 shown in Figure 12 is the evaluation value obtained when all of the first to third confidence levels D1 to D3 are "high", and a frequency of "1" is added regardless of the confidence level state of the previous frame.

[0126] Furthermore, the second evaluation value Ve2 shown in Figure 12 is the evaluation value obtained when a frequency of "1" is added regardless of the type of confidence level when at least one of the first to third confidence levels D1 to D3 is "low".

[0127] The first evaluation value Ve1 shown in Figure 13 is the same as the first evaluation value Ve1 shown in Figure 12.

[0128] Furthermore, the second evaluation value Ve2 shown in Figure 13 is the same as the second evaluation value Ve2 shown in Figure 7.

[0129] If these evaluation values ​​determine that the object image Oi' is a real image Ri, then the determination that the object image Oi' is a real image Ri occurs before the influence of the virtual image Oi's behavior (vector) decreases, which may result in unnecessary emergency braking control. In other words, it is determined that the real image Ri is flying out in the direction of the vehicle M's movement.

[0130] According to this embodiment, when the reliability of all patterns for the object image (first to third reliability D1 to D3) is high, and the third reliability D3 remains high for two frames or more consecutively, the driving ECU 14 adds a frequency of "1" to the first evaluation value Ve1, which indicates a high probability that the object image is a real image. When the first evaluation value Ve1 exceeds the comparison value, the ECU 14 determines that the object image is a real image.

[0131] This makes it possible to suppress the misrecognition of the behavior of an object even when its image is mistakenly detected as a virtual image.

[0132] In other words, by calculating the first evaluation value Ve1 as described above, even when the object image of a real object is detected in confusion with a virtual image, or when the behavior of the virtual image is inherited as the behavior of the object image, it is possible to accurately prevent the object image from being judged as a real image until the influence of the virtual image is sufficiently reduced.

[0133] In this case, by calculating a second evaluation value Ve2 as a comparison value, which is added when at least one of the first to third confidence levels D1 to D3 for the object image is low, an appropriate comparison value can be set according to the fluctuation state of the first to third confidence levels D1 to D3.

[0134] Furthermore, by setting the third confidence level D3, which changes with the highest responsiveness in response to changes in the detection state of the object image, as a specific confidence level among the first to third confidence levels D1 to D3, it is possible to accurately prevent the frequency of the first evaluation value Ve1 from rising unnecessarily quickly.

[0135] In the above-described embodiment, an example of a configuration in which object image detection and calculation of first to third reliability levels D1 to D3 are performed in the radar unit 37 and object image information is output to the driving ECU 14 has been explained, but the present invention is not limited to this. For example, as shown in Figure 14, it is also possible to output only object image detection information from the radar unit 37 to the driving ECU 14 and to perform the calculation of first to third reliability levels D1 to D3 in the driving ECU 14. Also, for example, as shown in Figure 15, it is also possible to output only the received signal from the radar unit 37 to the driving ECU 14 and to perform object image detection and the calculation of first to third reliability levels D1 to D3 in the driving ECU 14. In these cases, the driving ECU 14 corresponds to a specific example of a reliability calculation unit. Furthermore, any recognition device that may recognize a false image is acceptable, not limited to radar.

[0136] Furthermore, in the above-described embodiment, the image recognition_ECU13, driving_ECU14, CP_ECU21, E / G_ECU22, T / M_ECU23, BK_ECU24, PS_ECU25, and radar_ECU40, etc., are composed of a well-known microcomputer equipped with a CPU, RAM, ROM, non-volatile memory, etc., and its peripheral devices, with fixed data such as programs executed by the CPU and data tables pre-stored in the ROM. Note that all or part of the processor's functions may be composed of logic circuits or analog circuits, and the processing of various programs may be realized by electronic circuits such as FPGAs.

[0137] The inventions described in the above embodiments are not limited to those forms, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, each of the above embodiments includes inventions at various stages, and various inventions can be extracted by appropriate combinations of the multiple constituent elements disclosed.

[0138] For example, if the problem described can be solved and the effects described can be obtained even if some of the constituent elements shown in each form are removed, then the configuration with the removed constituent elements can be extracted as an invention.

[0139] For example, instead of using the second evaluation value Ve2 as a comparison value, it is also possible to use a pre-set frequency threshold.

[0140] Furthermore, the confidence level only needs to be 1 or higher, including a confidence level equivalent to the third confidence level D3, and of course, other confidence levels are not limited to the first and second confidence levels D1 and D2 mentioned above. [Explanation of Symbols]

[0141] 1. Driving assistance system 10 ... Camera unit 11… Stereo camera 11a ... Main camera 11b ... Sub-camera 13 … Image Recognition_ECU 14 … Driving_ECU 37 … Radar Unit 37lf ... Left front side sensor unit 37lr … Left rear side sensor unit 37rf ... Right front side sensor unit 37rr ... Right rear side sensor unit 38… Transmitter 39… Receiver 40 … Radar ECU

Claims

1. An object recognition device that detects an object image frame by frame based on the reflected signal of a transmitted signal, calculates the reliability of multiple patterns of the detected object image based on a preset calculation method, and performs object recognition based on the calculated reliability, A first evaluation value calculation unit adds up the frequency of a first evaluation value indicating that the object image is likely to be a real image when the reliability of all patterns for the object image is high, and a predetermined specific reliability among the reliability of multiple patterns remains high for a set frame or longer. The system includes a real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds the comparison value, The object recognition device is characterized in that the specific confidence level is the confidence level calculated from among a plurality of confidence levels based on the time-series behavior of the object image.

2. A reliability calculation unit calculates the reliability of multiple patterns of object images detected frame by frame based on the reflected signal of the transmitted signal, using a pre-set calculation method. A first evaluation value calculation unit adds up the frequency of a first evaluation value indicating that the object image is likely to be a real image when the reliability of all patterns for the object image is high, and a predetermined specific reliability among the reliability of multiple patterns remains high for a set frame or longer. The system includes a real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds the comparison value, The object recognition device is characterized in that the specific confidence level is the confidence level calculated from among a plurality of confidence levels based on the time-series behavior of the object image.

3. Furthermore, the system includes a second evaluation value calculation unit that adds a second evaluation value frequency indicating a high probability that the object image is a virtual image when at least one of the multiple confidence levels for the object image is low. The object recognition device according to claim 1 or 2, characterized in that the real image determination unit determines that the object image is a real image after a set number of frames have elapsed since detecting the object image, and when the frequency of the first evaluation value exceeds the frequency of the second evaluation value.

4. An object recognition device that detects an object image frame by frame based on a reflected signal of a transmitted signal, calculates the reliability of at least one pattern of the detected object image based on a preset calculation method, and performs object recognition based on the calculated reliability, A first evaluation value calculation unit adds up the frequency of a first evaluation value indicating that the object image is likely to be a real image when the reliability of all patterns for the object image is high, and a predetermined specific reliability among the reliability of at least one pattern remains high for a set frame or longer. A real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds the comparison value, The system includes a second evaluation value calculation unit that adds a second evaluation value frequency indicating a high probability that the object image is a virtual image when at least one of the reliability values ​​of the at least one pattern for the object image is low, The object recognition device is characterized in that the real image determination unit determines that the object image is a real image after a set number of frames have elapsed since the detection of the object image, and when the frequency of the first evaluation value exceeds the frequency of the second evaluation value.

5. A reliability calculation unit that calculates the reliability of at least one pattern of object images detected for each frame based on the reflected signal of the transmitted signal, using a preset calculation method, A first evaluation value calculation unit adds up the frequency of a first evaluation value indicating that the object image is likely to be a real image when the reliability of all patterns for the object image is high, and a predetermined specific reliability among the reliability of at least one pattern remains high for a set frame or longer. A real image determination unit that determines that the object image is a real image when the cumulative frequency of the first evaluation value exceeds the comparison value, The system includes a second evaluation value calculation unit that adds a frequency of a second evaluation value indicating a high probability that the object image is a virtual image when at least one of the reliability values ​​of the at least one pattern for the object image is low. The object recognition device is characterized in that the real image determination unit determines that the object image is a real image after a set number of frames have elapsed since the detection of the object image, and when the frequency of the first evaluation value exceeds the frequency of the second evaluation value.

Images