Vehicle detection device

Abstract

To facilitate association of vehicles detected by a camera and a detector.SOLUTION: A detection device 50 for vehicle for detecting a vehicle around an own vehicle comprises: a camera 1a which images the periphery of the own vehicle by an image sensor to acquire image information; a detector 1b which receives a reflection signal from a vehicle in an imaging region of the camera 1a to acquire first distance information to the vehicle; a distance estimation unit 14a which estimates second distance information to the vehicle on the basis of the size of the image of the vehicle included in the image information; and a correction unit 14b which corrects the second distance information so as to be closer to the first distance information acquired by the detector 1b.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a vehicle detection device that detects vehicles existing around a vehicle based on information from a sensor that detects the external environment. [Background technology]

[0002] One known technology of this type uses sensor fusion using a camera and radar to recognize a preceding vehicle traveling ahead of the vehicle, and performs driving control of the vehicle, such as speed control, based on the recognition results (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3619628 Summary of the Invention [Problem to be solved by the invention]

[0004] However, for example, when a preceding vehicle detected by a sensor is located far away from the vehicle, the vehicle detected based on the image information from the camera and the vehicle detected by the radar may not be considered to be the same vehicle. [Means for solving the problem]

[0005] A vehicle detection device according to one aspect of the present invention is a vehicle detection device that detects vehicles around the vehicle, and includes a camera that captures images of the surroundings of the vehicle using an image sensor to obtain image information, a detector that receives reflected signals from vehicles within the camera's imaging area to obtain first distance information to the vehicle, and a distance estimation unit that estimates second distance information to the vehicle based on the size of the image of the vehicle included in the image information. If the difference between the second distance information and the first distance information is smaller than a predetermined value, the second distance information is determined to be correct. If the difference between the second distance information and the first distance information is larger than the predetermined value, the second distance information is determined to be incorrect. If the difference between the second distance information and the first distance information is determined to be incorrect, a correction unit that corrects the second distance information so that the second distance information approaches the first distance information acquired by the detector; a linking unit that links the targets detected by the camera and the detector so that the targets for which the first distance information and the second distance information have been detected or estimated are treated as the same target when the correction unit makes a positive determination, and links the targets detected by the camera and the detector so that the targets for which the first distance information and the second distance information corrected by the correction unit have been detected or estimated are treated as the same target when the correction unit makes a negative determination; Equipped with. [Effects of the Invention]

[0006] According to the present invention, it is possible to easily link vehicles detected by a camera and a detector. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram illustrating an overall configuration of a vehicle control system including a detection device for a vehicle according to an embodiment; [Figure 2] 1 is a block diagram showing a configuration of a main part of a detection device for a vehicle according to an embodiment; [Figure 3] 10 is a flowchart illustrating the flow of processing executed by a calculation unit. [Figure 4] FIG. 10 is a diagram illustrating an example of a curve of a theoretical value as conversion information. [Figure 5] FIG. 10 is a diagram illustrating an example of the relationship between the difference calculated by equation (4) and the distance from the host vehicle to the preceding vehicle. [Figure 6] A plot of sample data along with the theoretical curve. [Figure 7] FIG. 10 is a diagram for explaining symbols wi and Wi in equation (8). [Figure 8] FIG. 10 is a diagram for explaining symbols wi and Wi in equation (8). DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to FIGS. A vehicle detection device according to an embodiment of the present invention can be applied to both vehicles with an automatic driving function, i.e., an automatic driving vehicle, and a manually driven vehicle without an automatic driving function. A manually driven vehicle includes a vehicle equipped with a driving assistance function. An example of applying the vehicle detection device to an automatically driven vehicle will be described below. The vehicle to which the vehicle detection device according to the embodiment is applied may be referred to as the host vehicle to distinguish it from other vehicles.

[0009] The host vehicle may be an engine vehicle that has an internal combustion engine (engine) as a driving source, an electric vehicle that has a traction motor as a driving source, or a hybrid vehicle that has an engine and a traction motor as driving sources. The host vehicle (autonomous vehicle) can run not only in an autonomous driving mode that does not require driving operation by the driver, but also in a manual driving mode (in which driving assistance functions can be used) in which the driver operates the vehicle.

[0010] <Outline of Autonomous Driving Configuration> First, a schematic configuration related to autonomous driving will be described. Fig. 1 is a block diagram showing a schematic overall configuration of a vehicle control system 100 for an autonomous driving vehicle having a vehicle detection device according to an embodiment. As shown in Fig. 1, the vehicle control system 100 mainly includes a controller 10, a group of external sensors 1 and a group of internal sensors 2, each of which is communicatively connected to the controller 10 via a CAN communication line or the like, an input / output device 3, a positioning unit 4, a map database 5, a navigation device 6, a communication unit 7, and a driving actuator AC.

[0011] The external sensor group 1 is a collective term for a plurality of sensors (referred to as external sensors) that detect the external situation, which is information about the surroundings of the host vehicle. For example, the external sensor group 1 includes a LiDAR (Laser Imaging Detection and Ranging) that detects the position (distance and direction from the host vehicle) of an object around the host vehicle by emitting laser light and detecting reflected light, a Radar that detects the position of an object around the host vehicle by emitting electromagnetic waves and detecting reflected waves, and a camera that has an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor and captures an image of the surroundings of the host vehicle. The imaging element is also called an image sensor. LiDAR and radar can detect objects within the imaging area of ​​the camera. LiDAR and radar may also be called detectors.

[0012] The internal sensor group 2 is a collective term for a plurality of sensors (internal sensors) that detect the driving state of the host vehicle. The internal sensor group 2 includes, for example, a vehicle speed sensor that detects the driving speed (vehicle speed) of the host vehicle, an acceleration sensor that detects the acceleration in the forward / backward and left / right directions of the host vehicle, a gyro sensor that detects the rotation and change in direction of the host vehicle as angular velocity, and a rotation speed sensor that detects the rotation speed of the driving source. The internal sensor group 2 also includes sensors that detect the driving operations of the driver in manual driving mode, such as operation of the accelerator pedal, operation of the brake pedal, operation of the steering wheel, etc.

[0013] The input / output device 3 is a general term for devices that input commands from the driver and output information to the driver. For example, the input / output device 3 includes various switches through which the driver inputs various commands by operating operating members, a microphone through which the driver inputs commands by voice, a display that provides information to the driver via displayed images, a speaker that provides information to the driver by voice, etc.

[0014] The positioning unit (GNSS (Global Navigation Satellite System) unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. The positioning sensor can also be included in the internal sensor group 2. The positioning satellite is an artificial satellite such as a GPS satellite or a quasi-zenith satellite. The positioning unit 4 measures the current position (latitude, longitude, altitude) of the vehicle using the positioning information received by the positioning sensor.

[0015] The map database 5 is a device that stores general map information used in the navigation device 6, and is configured with, for example, a hard disk or semiconductor elements. The map information includes road position information, road shape information (curvature, etc.), and position information of intersections and branching points. The map information stored in the map database 5 is different from the highly accurate map information stored in the storage unit 12 of the controller 10.

[0016] The navigation device 6 searches for a target route on roads to a destination input by the driver and provides guidance along the target route. The input of the destination and guidance along the target route are performed via the input / output device 3. The target route is calculated based on the current position of the vehicle measured by the positioning unit 4 and map information stored in the map database 5. The current position of the vehicle can also be measured using detection values ​​from the external sensor group 1, and the target route can be calculated based on this current position and high-precision map information stored in the memory unit 12.

[0017] The communication unit 7 communicates with various servers (not shown) via networks including wireless communication networks such as the Internet and mobile phone networks, and acquires map information, driving history information, traffic information, and the like from the servers periodically or at any timing. Networks include not only public wireless communication networks but also closed communication networks established for each predetermined management area, such as wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like. The acquired map information is output to the map database 5 and the storage unit 12, where the map information is updated.

[0018] Actuators AC are driving actuators for controlling the driving of the host vehicle. When the driving source is an engine, actuators AC include a throttle actuator that adjusts the opening of the engine's throttle valve (throttle opening). When the driving source is a driving motor, actuators AC include the driving motor. Actuators AC also include a brake actuator that operates the host vehicle's braking device and a steering actuator that drives the steering device.

[0019] The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 includes a computer having an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM (read only memory) and a RAM (random access memory), and other peripheral circuits (not shown) such as an I / O interface. Note that although multiple ECUs with different functions, such as an engine control ECU, a traction motor control ECU, and a braking device ECU, can be provided separately, the controller 10 is shown in FIG. 1 as a collection of these ECUs for convenience.

[0020] High-precision road map information is stored in the memory unit 12. This road map information includes road position information, road shape (curvature, etc.), road gradient information, intersection and branch point position information, number of lanes, lane width and lane position information (information on lane center positions and lane boundary lines), landmarks (traffic lights, signs, buildings, etc.) as markers on the map, and road surface profile information such as road surface irregularities. Landmark information includes landmark shape (outline), characteristics, position, etc. The storage unit 12 further stores conversion information, which will be described later.

[0021] The calculation unit 11 has, as functional components, a vehicle position recognition unit 13, an external environment recognition unit 14, a behavior plan generation unit 15, a driving control unit 16, and a sensor group management unit 17.

[0022] The vehicle position recognition unit 13 recognizes the position of the vehicle on the map (own vehicle position) based on the vehicle position information obtained by the positioning unit 4 and the map information in the map database 5. The vehicle position may be recognized using the map information stored in the storage unit 12 and information about the surroundings of the vehicle detected by the external sensor group 1, thereby enabling the vehicle position to be recognized with high accuracy. Note that when the vehicle position can be measured by an external sensor installed on or beside the road, the vehicle position can also be recognized by communicating with the sensor via the communication unit 7. The vehicle position recognition unit 13 can also calculate the azimuth angle of the direction in which the camera constituting the external sensor group 1 captures an image based on the output of the gyro sensor constituting the internal sensor group 2.

[0023] The external environment recognition unit 14 recognizes the external situation around the vehicle based on signals from the external sensor group 1, such as detectors and cameras. For example, it recognizes the positions, speeds, and accelerations of surrounding vehicles (vehicles ahead and vehicles behind) traveling around the vehicle, the positions of surrounding vehicles stopped or parked around the vehicle, and the positions and states of other objects. Examples of other objects include signs, traffic lights, markings such as road dividing lines and stop lines, buildings, guardrails, utility poles, signs, pedestrians, bicycles, tunnel entrances, etc. Examples of the states of other objects include the color of traffic lights (red, green, yellow), the moving speed and direction of pedestrians and bicycles, etc.

[0024] The object detected by the external sensor group 1 is called a target. Targets include both people and objects, and both moving and stationary objects. The external environment recognition unit 14 performs integrated processing (fusion processing) of the detection data of different types of sensors (e.g., cameras and detectors) that make up the external sensor group 1, determines whether to treat targets detected by each sensor as the same target, and derives target position data. Generally, detectors have superior measurement accuracy compared to cameras, and cameras have superior resolution at close range compared to detectors. Fusion processing makes it possible to appropriately combine the characteristics of each sensor. For example, distance information of a preceding vehicle, etc., used for vehicle control such as automatic braking (e.g., collision mitigation braking) and adaptive cruise control (ACC) can be obtained by using distance information based on camera detection data or distance information based on detector detection data.

[0025] When targets detected based on different types of sensors can be treated as the same target, the external environment recognition unit 14 derives position data of the target by performing fusion processing such as coordinate conversion of the detected data, data interpolation, averaging, etc. In other words, it becomes possible to link (or associate) vehicles detected based on each sensor and recognize the vehicle position with high accuracy. Furthermore, if targets detected by different types of sensors cannot be treated as the same target, the external environment recognition unit 14 corrects the distance information based on the camera detection data. This correction makes it easier to link vehicles detected by each sensor. The process by which the external environment recognition unit 14 determines whether or not to treat targets detected based on the sensors as the same target will be described in detail later.

[0026] The behavior plan generation unit 15 generates a traveling trajectory (target trajectory) of the host vehicle from the current time to a predetermined time ahead based on, for example, a target route calculated by the navigation device 6, map information stored in the memory unit 12, the host vehicle position recognized by the host vehicle position recognition unit 13, and external conditions (such as the position of targets) recognized by the external environment recognition unit 14. When there are multiple trajectories that are candidates for the target trajectory on the target route, the behavior plan generation unit 15 selects an optimal trajectory from among them that satisfies criteria such as compliance with laws and regulations and efficient and safe traveling, and sets the selected trajectory as the target trajectory. The behavior plan generation unit 15 then generates a behavior plan according to the generated target trajectory. The behavior plan generation unit 15 generates various behavior plans corresponding to overtaking driving to overtake a preceding vehicle, lane-changing driving to change the traveling lane, following driving to follow a preceding vehicle, lane-keeping driving to maintain the traveling lane without deviating from the traveling lane, constant speed driving, decelerating driving, accelerating driving, etc. When generating a target trajectory, the behavior plan generating unit 15 first determines a traveling mode, and generates the target trajectory based on the traveling mode.

[0027] In the autonomous driving mode, the driving control unit 16 controls the actuators AC so that the host vehicle travels along the target trajectory generated by the behavior plan generation unit 15. More specifically, the driving control unit 16 calculates a required driving force for achieving the target acceleration per unit time calculated by the behavior plan generation unit 15, taking into account the driving resistance determined by the road gradient, etc., in the autonomous driving mode. Then, for example, the driving control unit 16 feedback-controls the actuators AC so that the actual acceleration detected by the internal sensor group 2 becomes the target acceleration. In other words, the driving control unit 16 controls the actuators AC so that the host vehicle travels at the target vehicle speed and target acceleration. In the manual driving mode, the driving control unit 16 controls each actuator AC in response to a driving command (such as a steering operation) from the driver acquired by the internal sensor group 2.

[0028] The sensor group management unit 17 manages the status of different types of sensors (cameras and detectors) that make up the external sensor group 1. For example, it determines whether the imaging environment for the camera has deteriorated due to backlighting or weather, or whether the measurement environment for the radar has deteriorated due to reflective objects such as walls.

[0029] <Information used for vehicle control> The controller 10 constituting the above-described vehicle control system 100 controls each actuator AC using information about targets that are linked as the same target among targets detected based on detection data from different types of sensors. For example, among information about targets detected by both of different types of sensors, information based on detection data from one sensor is used for vehicle control such as automatic braking and ACC. This configuration increases redundancy and ensures safety by allowing information obtained based on the other sensor to be used for vehicle control even if the condition of one sensor deteriorates. Furthermore, when the calculation unit 11 is unable to link targets detected based on different types of sensors, it performs the linking after correcting the information detected based on one of the sensors. This configuration makes it easier to link targets detected based on the detection data from each sensor. However, the corrected information is not used for vehicle control such as automatic braking and ACC, and the information detected based on the other sensor is used for vehicle control. Using information that does not require correction for vehicle control can further improve safety. The details of the processing performed by the controller 10 will be further explained with reference to FIGS.

[0030] <Configuration of main components of vehicle detection device> Fig. 2 is a block diagram showing the configuration of a main part of a vehicle detection device 50 according to an embodiment. The vehicle detection device 50 constitutes a part of the vehicle control system 100 shown in Fig. 1. As shown in Fig. 2, the vehicle detection device 50 includes a camera 1a, a detector 1b, a controller 10, and an actuator AC.

[0031] The camera 1a is a monocular camera having an imaging element such as a CCD or CMOS sensor, and constitutes part of the external sensor group 1 in FIG. 1. The camera 1a is attached, for example, at a predetermined position in front of the vehicle and continuously captures images of the space ahead of the vehicle to acquire images of targets (camera images). Targets include preceding vehicles traveling ahead of the vehicle, people, structures, etc. The position and type of the target can be recognized based on the camera images. That is, if the horizontal direction of the two-dimensional camera image is the Y direction and the vertical direction is the X direction, the position of the target in the vehicle width direction can be determined from the position in the Y direction on the camera image, and the position of the target in the height direction and direction of travel can be determined from the position in the X direction. In other words, it is possible to acquire position data (position information) of the target based on the image information from the camera 1a.

[0032] Detector 1b is a detector that detects the distance from the vehicle to the target based on the reflected wave from the detection object (target), and includes either radar or lidar, or both. Detector 1b can acquire position data (position information) of the target relative to the vehicle. The position data includes data on the position and speed of the target. The detection range of detector 1b is included in the imaging area of ​​camera 1a. Therefore, if the target imaged by camera 1a and the target detected by detector 1b are the same, the position and speed of the target can be derived by performing fusion processing.

[0033] The controller 10 in FIG. 2 has a distance estimation unit 14a, a correction unit 14b, a linking unit 14c, and a determination unit 14d as functional components performed by the calculation unit 11 (FIG. 1). The distance estimation unit 14a is provided to estimate distance information based on image information captured by the camera 1a. The correction unit 14b is provided to correct the distance information estimated by the distance estimation unit 14a. The linking unit 14c is provided to link vehicles detected by the camera 1a and the detector 1b. The determination unit 14d is provided to determine the distance information and the like used for vehicle control.

[0034] <Explanation of the flowchart> FIG. 3 is a flowchart illustrating the flow of processing executed by the calculation unit 11 when information acquired by different types of sensors (camera 1a and detector 1b) is used for vehicle control such as automatic braking and ACC. The camera 1a and the detector 1b each repeatedly capture images or perform measurements at a predetermined frame rate and output detection data. The calculation unit 11 executes the process illustrated in FIG. 3 every time detection data is output from the camera 1a and the detector 1b.

[0035] In step S1, the distance estimation unit 14a of the calculation unit 11 estimates distance information based on image information captured by the camera 1a. In this embodiment, a monocular camera 1a is used as one of the sensors. Because the camera 1a is a monocular camera, distance information cannot be directly obtained from captured image information, as is the case with a stereo camera. Therefore, the distance estimation unit 14a estimates the distance information from the host vehicle to the preceding vehicle based on the size of the captured image of the preceding vehicle (distance estimation).

[0036] - Distance estimation - The distance estimation unit 14a performs the distance estimation in step S1 in the following procedure. Step 1. Classify the leading vehicle. Step 2. Select the conversion information that corresponds to the class. Step 3: Based on the converted information, estimate the distance from the vehicle to the leading vehicle.

[0037] Each step of distance estimation will be described below. 1. Class placement The distance estimation unit 14a classifies the preceding vehicle into classes corresponding to the vehicle type by performing pattern matching processing based on the shape of the image of the preceding vehicle captured by the camera 1a (for example, the outline of the image of the preceding vehicle). There are four classes: trucks (large vehicles including trucks, trailers, buses, etc.), four-wheeled vehicles (medium and small vehicles smaller than large vehicles), light vehicles (four-wheeled vehicles smaller than small vehicles), and two-wheeled vehicles. The number of classes to be divided is not limited to four and may be changed as appropriate.

[0038] 2.Conversion information In the embodiment, the vehicle width (referred to as the reference vehicle width) of a standard vehicle (referred to as the reference vehicle) included in each of the four classes is defined as, for example, 2.5 m for trucks, 1.8 m for four-wheeled vehicles, 1.5 m for light vehicles, and 1.0 m for two-wheeled vehicles. The memory unit 12 stores, for each class (in other words, for each vehicle type), conversion information indicating the relationship between the theoretical size dr of the image of the preceding vehicle (for example, the theoretical horizontal width of the image of the preceding vehicle on the image sensor) obtained when the preceding vehicle whose width Wc is equal to the reference vehicle width is imaged by the camera 1a, and the distance Zr from the host vehicle to the preceding vehicle. The conversion information may indicate the relationship between the size dr of the image of the reference vehicle obtained when the reference vehicle having the reference width is imaged by the camera 1a and the distance Zr from the host vehicle to the reference vehicle.

[0039] The conversion information is expressed by the following formula (1). dr=Wc×f / Zr (1) The symbol Wc corresponds to the width of the preceding vehicle (the above-mentioned reference vehicle width). The symbol f indicates the focal length of the optical system of the camera 1a. The symbol Zr indicates the distance from the host vehicle to the reference vehicle. If the size per pixel (which may also be called the pixel pitch) of the image sensor of camera 1a is λ, the theoretical number of pixels drp in the vehicle width direction (horizontal direction) that make up the image of the preceding vehicle captured by camera 1a is expressed by the following equation (2): drp=dr / λ=Wc×f / (Zr×λ) ·····(2)

[0040] - Theoretical value curve - 4 is a diagram illustrating a theoretical curve as conversion information showing the relationship between the number of pixels drp calculated by the above formula (2) and the distance Zr. The vertical axis represents the number of pixels drp in the vehicle width direction of the image of the preceding vehicle (referred to as the image width), and the horizontal axis represents the distance Zr from the host vehicle to the preceding vehicle. A curve 41 indicates the theoretical value when the preceding vehicle is a truck or the like, and a curve 42 indicates the theoretical value when the preceding vehicle is a light vehicle or the like. For example, curve 41 indicates that when a truck with a width Wc=2.5 m is located 32 m ahead of the vehicle, the theoretical number of horizontal pixels drp constituting the image of the truck captured by camera 1a is 120. Although not shown in the figure, in addition to curves 41 and 42, the conversion information also includes information indicating a theoretical value curve when the preceding vehicle is a four-wheeled vehicle, etc., and a theoretical value curve when the preceding vehicle is a two-wheeled vehicle, etc. In step 2, the distance estimation unit 14a selects conversion information corresponding to the class obtained by classification in step 1 from the four types of conversion information corresponding to the four classes described above.

[0041] The conversion information may be stored as table values ​​in the storage unit 12. Storing the conversion information as table values ​​makes it possible to read out the corresponding number of pixels or distance from the storage unit 12 using the distance or number of pixels as an argument.

[0042] 3. Distance Estimation In step 3, the distance estimation unit 14a detects the size dc of the image of the preceding vehicle actually captured by the camera 1a (for example, the horizontal width of the image of the preceding vehicle on the image sensor), and estimates distance information corresponding to the image size dc based on the conversion information selected corresponding to the classified class. When the pixel pitch of the camera 1a is represented by the symbol λ, the number of pixels dcp in the horizontal direction that constitute the image of the leading vehicle captured by the camera 1a is calculated by the following equation (3). dcp=dc / λ (3)

[0043] The distance estimation unit 14a refers to the conversion information and estimates the distance corresponding to the number of pixels dcp in the horizontal direction calculated by the above formula (3). The estimated distance corresponds to the distance in the vertical direction (X direction) on the camera image, so it may also be called the vertical distance x. After estimating the distance information as described above, the calculation unit 11 proceeds to step S2 in FIG.

[0044] In step S2, the calculation unit 11 (external environment recognition unit 14) acquires distance information measured by the detector 1b, and the process proceeds to step S3.

[0045] In step S3, the calculation unit 11 (correction unit 14b) determines the accuracy of the distance information estimated based on the image information captured by the camera 1a. If the correction unit 14b determines that the distance information is correct, it makes an affirmative decision in step S3 and proceeds to step S5. If the correction unit 14b determines that the distance information is incorrect, it makes a negative decision in step S3 and proceeds to step S4.

[0046] The corrector 14b determines whether the distance information is correct based on whether the class obtained in step S1 is appropriate. Whether the class determined in step S1 is valid or not is determined by comparing the theoretical size dr (which may be the number of pixels drp) of the image of the preceding vehicle corresponding to the distance information detected by detector 1b with the size dc (which may be the number of pixels dcp) of the image of the preceding vehicle actually captured by camera 1a. First, the correction unit 14b obtains the theoretical number of pixels drp based on the conversion information selected corresponding to the current class.

[0047] Next, the correction unit 14b calculates the difference R between the theoretical number of pixels drp and the number of pixels dcp of the image of the leading vehicle actually captured by the camera 1a, using the following equation (4). R=│drp-dcp│ (4)

[0048] FIG. 5 is a diagram illustrating the relationship between the difference R calculated by the above formula (4) and the distance Zr from the host vehicle to the preceding vehicle. While the host vehicle is actually traveling, the distance Zr from the host vehicle to the preceding vehicle is repeatedly detected at predetermined time intervals by detector 1b, and the preceding vehicle is repeatedly photographed by camera 1a to collect multiple sample data. The diagram plots the difference R calculated based on each data by distance. Curve (or line) 51 is a function G(x) that indicates the boundary line dividing the collected sample data into two groups. The data group below curve (or line) 51 indicates cases where the classes classified in step S1 are valid (OK), and the data group above curve (or line) 51 indicates cases where the classes classified in step S1 are invalid (NG).

[0049] The function G(x) that divides multiple sample data into the two groups described above may be generated empirically by verifying a large amount of sample data, or may be calculated using a machine learning method such as SVM (Support Vector Machine). According to Figure 5, when the distance from the host vehicle to the leading vehicle is shorter than d, the vehicle is more likely to be included in the "OK" group even if the difference R is large compared to when the distance is longer than d. One reason for this is that the shorter the distance, the larger the number of horizontal pixels dcp that make up the image captured by camera 1a, and therefore the larger the tolerance for the difference R becomes.

[0050] If the difference R calculated by the above formula (4) using the image information used to estimate the distance information in step S1 and the data of the distance information (the distance from the host vehicle to the preceding vehicle) acquired in step S2 are included in the "OK" region of FIG. 5, the correction unit 14b determines that the class assigned in step S1 is valid. If the class assigned is valid, the reliability of the distance information estimated based on the camera 1a is high (in other words, the difference between the distance information detected or estimated based on the detector 1b and the camera 1a is smaller than a predetermined value), and therefore a positive determination is made in step S3. On the other hand, if the data of the difference R and the distance are included in the "NG" region of FIG. 5, the correction unit 14b determines that the class assigned in step S1 is invalid. If the class assigned is invalid, the reliability of the distance information estimated based on the camera 1a is low (in other words, the difference between the distance information detected or estimated based on the detector 1b and the camera 1a is larger than a predetermined value), and therefore a negative determination is made in step S3.

[0051] In step S4, to which the process proceeds after making a negative decision in step S3, the calculation unit 11 (correction unit 14b) corrects the distance information (longitudinal distance x) estimated based on the camera 1a.

[0052] As an example of correcting the distance information, the corrector 14b changes the class selected in the distance estimation procedure 1. More specifically, N (e.g., 5) pieces of actual sample data (N pieces of data indicating the distance Zr from the host vehicle to the preceding vehicle detected by the detector 1b and the number of pixels dcp of the image of the preceding vehicle actually captured by the camera 1a) are plotted on coordinates similar to those in Fig. 4, and the class is changed to that corresponding to the curve of the converted information (the curve of the theoretical value) that is most similar to the regression curve of the N pieces of data.

[0053] Figure 6 is a diagram in which N pieces of sample data are plotted together with two theoretical curves 41 and 42 similar to those in Figure 4. The horizontal and vertical axes of Figure 6 respectively represent the distance Zr from the host vehicle to the preceding vehicle detected by detector 1b and the number of pixels (image width) dcp of the image of the preceding vehicle actually captured by camera 1a. As described above, the distance Zr corresponds to the vertical distance x in the vertical direction (X direction) on the camera image captured by the camera 1a, so the above equation (2) is rewritten as a function F(x) of the vertical distance x as shown in the following equation (5). F(x)=Wc×f / (x×λ) ·····(5) The symbol Wc corresponds to the vehicle width of the preceding vehicle (the reference vehicle width). The symbol f indicates the focal length of the optical system of the camera 1a. The symbol x indicates the vertical distance from the host vehicle to the preceding vehicle, and the symbol λ indicates the pixel pitch of the camera 1a.

[0054] The correction unit 14b calculates a regression curve based on N pieces of data using the following equation (6). f(x)=ax 3 +bx 2 +cx+d ·····(6) The coefficients a, b, c, and d are calculated by the least squares method using the following equation (7).

number

[0055] Correction unit 14b calculates the similarity D between the theoretical value curve F(x) and the regression curve f(x) using the following equation (8), determines the class corresponding to the curve F(x) with the smallest similarity D among the four classes of curves F(x), and changes the currently selected class to this class. Since the theoretical value curve as conversion information changes with the change in class, the distance information corresponding to the size dc of the image of the leading vehicle actually captured by camera 1a is substantially corrected compared to before the class change.

number

[0056] Figures 7 and 8 are diagrams explaining the symbols wi and Wi in equation (8), respectively. Both figures are enlarged views of parts of Figure 6, with the horizontal axis representing the longitudinal distance x from the host vehicle to the preceding vehicle, and the vertical axis representing the number of pixels (image width) dcp of the image of the preceding vehicle actually captured by camera 1a. The equally spaced sampling width l in Figures 7 and 8 is, for example, 1 m.

[0057] As described above, once the calculation unit 11 has corrected the distance information obtained based on the camera 1a, the process proceeds to step S5 in FIG.

[0058] In step S5, the calculation unit 11 (linking unit 14c) performs a linking process to treat vehicles whose distance information has been detected or estimated based on the detector 1b and the camera 1a as the same vehicle. The linking unit 14c links the targets detected by each sensor so that the targets detected by each sensor are treated as the same target when, for example, a vector corresponding to distance information (indicating, for example, the distance from the host vehicle to the preceding vehicle) of the target detected or estimated by the detector 1b and the camera 1a constituting the external sensor group 1 and a vector corresponding to speed information (the above-mentioned distance information per unit time) are within an empirically determined predetermined threshold called the Mahalanobis distance. When the preceding vehicle as a target is linked, the position data of the preceding vehicle can be derived by performing coordinate conversion, data interpolation, averaging, etc. on the detection data of the different sensors, thereby making it possible to accurately recognize the position information of the preceding vehicle. After performing this linking process, the calculation unit 11 proceeds to step S6.

[0059] In step S6, the calculation unit 11 (decision unit 14d) decides information to be used for vehicle control such as automatic braking and ACC. The determination unit 14d determines control data necessary for vehicle control based on the processing result of step S3. More specifically, the determination unit 14d selects the distance information detected by the camera 1a or the distance information detected by the detector 1b as the information to be used for vehicle control.

[0060] When the determination unit 14d judges step S3 to be positive (in other words, the reliability of the distance information obtained based on the camera 1a is high), it selects and determines the distance information obtained based on the camera 1a from the distance information of the target linked by the linking unit 14c. When the calculation unit 11 judges step S3 to be negative (in other words, the reliability of the distance information obtained based on the camera 1a is low), it selects and determines the distance information obtained based on the detector 1b from the distance information of the target linked by the external environment recognition unit 14. After determining the distance information to be used for vehicle control in this way, the calculation unit 11 ends the processing shown in FIG.

[0061] According to the embodiment described above, the following advantageous effects are achieved. (1) A vehicle detection device 50 for detecting vehicles around the host vehicle includes a camera 1a that captures images of the surroundings of the host vehicle using an image sensor to acquire image information, a detector 1b that receives reflected signals from a preceding vehicle within the imaging area of ​​the camera 1a to acquire a distance Zr as first distance information to the preceding vehicle, a distance estimation unit 14a that estimates a longitudinal distance x as second distance information to the preceding vehicle based on the size dc of the image of the preceding vehicle included in the image information, and a correction unit 14b that corrects the longitudinal distance x to approximate the distance Zr acquired by the detector 1b. This configuration makes it possible to easily link targets detected based on detection data from the camera 1a and the detector 1b, which are different types of sensors, based on the corrected longitudinal distance x and distance Zr.

[0062] (2) In the vehicle detection device 50 described in (1) above, the distance estimation unit 14a estimates the distance corresponding to the size of the image of the preceding vehicle contained in the image information as the longitudinal distance x based on conversion information indicating the relationship between the size of the image of the reference vehicle and the distance to the reference vehicle. With this configuration, it is possible to appropriately estimate the longitudinal distance x from the host vehicle to the leading vehicle based on the size of the image of the leading vehicle captured by the camera 1a.

[0063] (3) In the vehicle detection device 50 described in (1) above, the correction unit 14b acquires the size dr of the image of the reference vehicle corresponding to the distance Zr acquired by the detector 1b based on the conversion information, and corrects the vertical distance x based on the difference between the size dr of the image of the reference vehicle and the size dc of the image of the preceding vehicle included in the image information captured by the camera 1a. With this configuration, it is possible to appropriately correct the vertical distance x estimated based on the image information from the camera 1a.

[0064] (4) In the vehicle detection device 50 described in (2) and (3) above, the conversion information includes four pieces of conversion information that respectively indicate the relationship between the image sizes dr of multiple reference vehicles corresponding to multiple vehicle types and the distances Zr to the multiple reference vehicles, and the distance estimation unit 14a selects the conversion information to be used to estimate the longitudinal distance x from among the multiple pieces of conversion information based on the shape of the image of the preceding vehicle included in the image information. With this configuration, it becomes possible to appropriately select conversion information corresponding to a class classified based on the shape of the image of the preceding vehicle from a plurality of pieces of conversion information corresponding to a plurality of vehicle types.

[0065] (5) In the vehicle detection device 50 described in (4) above, the correction unit 14b corrects the longitudinal distance x by changing the conversion information selected by the distance estimation unit 14a. With this configuration, for example, if an incorrect longitudinal distance x is estimated as a result of conversion information for a vehicle type different from that of the preceding vehicle being selected due to the small size of the image of a distant preceding vehicle, it is possible to change to the conversion information for the correct vehicle type and correct the longitudinal distance x to the correct value.

[0066] (6) In the vehicle detection device 50 described in (1) to (5) above, the correction unit 14b corrects the longitudinal distance x when the difference between the longitudinal distance x and the distance Zr is greater than a predetermined value. The vehicle detection device 50 further includes a linking unit 14c as an associating unit that associates the preceding vehicle included in the image information acquired by the camera 1a with the preceding vehicle whose distance Zr was acquired by the detector 1b when the difference between the longitudinal distance x and the distance Zr before or after the correction is equal to or less than a predetermined value. With this configuration, targets detected based on detection data from different types of sensors, camera 1a and detector 1b, can be appropriately linked based on the vertical distance x estimated based on image information and the distance Zr obtained by detector 1b.

[0067] The above embodiment can be modified in various ways, and modifications will be described below. (Variation 1) In step S4 of FIG. 3 described above, the similarity D between the theoretical value curve F(x) and the regression curve f(x) may be calculated using the following equation (9).

number

[0068] (Variation 2) The number N of sample data (data indicating the distance Zr from the vehicle to the preceding vehicle detected by detector 1b and the number of pixels dcp of the image of the preceding vehicle actually captured by camera 1a) illustrated in Figures 6 to 8 is not limited to the five points illustrated in the figures and may be changed as appropriate.

[0069] The above description is merely an example, and the present invention is not limited to the above-described embodiment and modifications as long as the features of the present invention are not impaired. One or more of the above-described embodiment and modifications can be arbitrarily combined, and modifications can also be combined with each other. [Explanation of symbols]

[0070] 1a camera, 1b detector, 10 controller, 12 memory unit, 14 external environment recognition unit, 14a distance estimation unit, 14b correction unit, 14c linking unit, 14d determination unit, 16 driving control unit, 17 sensor group management unit, 50 vehicle detection device, AC actuator

Claims

1. A vehicle detection device that detects vehicles around a vehicle, a camera that captures an image of the surroundings of the vehicle using an image sensor to acquire image information; a detector that receives a reflected signal from a vehicle within an imaging area of ​​the camera and acquires first distance information to the vehicle; a distance estimation unit that estimates second distance information to the vehicle based on a size of an image of the vehicle included in the image information; a correction unit that makes a positive determination that the second distance information is correct when a difference between the second distance information and the first distance information is smaller than a predetermined value, and makes a negative determination that the second distance information is incorrect when a difference between the second distance information and the first distance information is larger than a predetermined value, and corrects the second distance information so that it approaches the first distance information acquired by the detector when the negative determination is made; a linking unit that links the targets detected by the camera and the detector so that, when a positive determination is made by the correction unit, the targets detected or estimated by the first distance information and the second distance information are treated as the same target, and that, when a negative determination is made by the correction unit, links the targets detected or estimated by the camera and the detector so that the targets detected or estimated by the first distance information and the second distance information corrected by the correction unit are treated as the same target; A detection device for a vehicle, comprising:

2. 2. The vehicle detection device according to claim 1, The distance estimation unit estimates the distance corresponding to the size of the image of the vehicle included in the image information as the second distance information based on conversion information indicating the relationship between the size of the image of the reference vehicle and the distance to the reference vehicle.

3. 3. The vehicle detection device according to claim 2, The vehicle detection device is characterized in that the correction unit obtains the size of the image of the reference vehicle corresponding to the first distance information obtained by the detector based on the conversion information, and corrects the second distance information based on the difference between the size of the image of the reference vehicle and the size of the image of the vehicle included in the image information.

4. 4. The vehicle detection device according to claim 2, the conversion information includes a plurality of pieces of conversion information indicating the relationship between the sizes of the images of a plurality of reference vehicles corresponding to a plurality of vehicle types and the distances to the plurality of reference vehicles, A vehicle detection device characterized in that the distance estimation unit selects conversion information to be used for estimating the second distance information from the plurality of conversion information based on the shape of the image of the vehicle included in the image information.

5. 5. The vehicle detection device according to claim 4, The detection device for a vehicle, wherein the correction unit corrects the second distance information by changing the conversion information selected by the distance estimation unit.

6. 2. The vehicle detection device according to claim 1, the correction unit corrects the second distance information when a difference between the second distance information and the first distance information is greater than a predetermined value; A vehicle detection device characterized by further comprising an association unit that associates a vehicle included in the image information acquired by the camera with a vehicle for which the first distance information was acquired by the detector when the difference between the second distance information before or after the correction and the first distance information is less than a predetermined value.