Object detection device

Abstract

An object detection device (100A) comprises: a radar signal processor (3) that uses a radar (1) to detect a radar-detected position (31) and a velocity (32) of a first object; a camera image processor (4A) that detects a camera-detected position (41), pixel coordinates (42), and an object type (43) of a second object on the basis of image data from the camera (2); and a fusion processor (5A) that transmits the radar-detected position, the velocity, and the object type to a vehicle control device (30) when the first object and the second object are identical and transmits the camera-detected position and the object type to the vehicle control device when the first object and the second object are not identical. The camera image processor updates a position conversion table indicating a correspondence relationship between the pixel coordinates and the radar-detected position on the basis of the camera update data in which the pixel coordinates and the radar-detected position are associated with each other when the identity is determined by the identity determination and detects the camera-detected position corresponding to the pixel coordinates on the basis of the position conversion table.

Description

[Technical Field]

[0001] The present disclosure relates to an object detection device that is mounted on a vehicle and detects an object. [Background technology]

[0002] An on-board object detection device is a device that can quickly detect objects (people, structures, etc.) that exist in the vehicle's environment and use the information for vehicle control, warning notifications, etc., thereby enabling comfortable vehicle driving. These object detection devices use a variety of sensors, including radar, cameras, Lidar (Light Detection and Ranging, Laser Imaging Detection and Ranging), and ultrasonic sensors, but in recent years, with the widespread use of various sensors, fusion-type object detection devices that combine multiple sensors to improve performance have come into widespread use.

[0003] When such a fusion-type object detection device is configured using radar and a camera, radar has excellent accuracy in detecting position and speed, and a camera has excellent accuracy in detecting objects, so it is common for radar to be used for detecting position and speed, and a camera to be used for object recognition.

[0004] The external environment recognition device for a vehicle described in Patent Document 1 determines whether an object is a pedestrian or not based on radar information, which is information on the position and speed of the object detected by radar, and image data of the object detected by a camera. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2010 / 119860 Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the technology of Patent Document 1, when an object to be detected approaches a structure with high radar reflection intensity (e.g., a utility pole, a wall, a guardrail, a shelf pillar, a delineator, a traffic light, etc.), the radar reflected wave signal from the object is buried in the radar reflected wave signal from the structure, resulting in the loss of radar detection data and making it difficult to detect the object. In this case, object detection can be maintained by replacing the radar detection data with camera detection data, but a low-performance monocular camera (hereinafter simply referred to as a camera) has low detection accuracy for the relative position (hereinafter also referred to as position) between the object and the vehicle, making it difficult to accurately detect the object. For this reason, with the technology of Patent Document 1, when a camera with low object position detection accuracy is used, the position of the object cannot be accurately detected when the object approaches a structure with high radar reflection intensity.

[0007] The present disclosure has been made in consideration of the above, and aims to provide an object detection device that can accurately detect the position of an object even when a camera with low object position detection accuracy is used, even if the object approaches a structure with high radar reflection intensity. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems and achieve the object, the object detection device of the present disclosure includes a radar signal processor that detects the radar detection position (position of a first object) and the speed of the first object based on a reflected signal of an electromagnetic wave emitted by a radar toward a first object to be detected. The object detection device of the present disclosure also includes a camera image processor that detects the camera detection position (position of a second object), the pixel coordinates of the second object, and the object type (type of the second object) based on image data obtained by a camera capturing an image of the second object. The object detection device of the present disclosure also includes a fusion processor that performs an identity determination based on the radar detection position and the camera detection position to determine whether the first object detected using the radar and the second object detected using the camera are the same, and if it determines that they are the same, sends the radar detection position, speed, and object type to a vehicle control device that controls the vehicle, and if it determines that they are not the same, sends the camera detection position and object type to the vehicle control device. If the fusion processor determines that the images are identical, it sends camera update data associating pixel coordinates with radar detection positions to the camera image processor. The camera image processor has a conversion table update unit that updates a position conversion table indicating the correspondence between pixel coordinates and radar detection positions based on the camera update data, and an object position detection unit that detects camera detection positions corresponding to the pixel coordinates based on the position conversion table. [Effects of the Invention]

[0009] The object detection device according to the present disclosure has the advantage that, when a camera with low object position detection accuracy is used, the position of an object can be accurately detected even if the object approaches a structure with high radar reflection intensity. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a configuration of an object detection device according to a first embodiment; [Figure 2] 1 is a flowchart showing a processing procedure executed by an object detection device according to a first embodiment; [Figure 3]FIG. 10 is a diagram illustrating a configuration of an object detection device according to a second embodiment. [Figure 4] 10 is a flowchart showing a processing procedure executed by an object detection device according to a second embodiment; [Figure 5] FIG. 1 is a diagram illustrating a configuration example of a processing circuit provided in an object detection device according to first and second embodiments, when the processing circuit is realized by a processor and a memory. [Figure 6] FIG. 1 is a diagram illustrating an example of a processing circuit when the processing circuit included in the object detection device according to the first and second embodiments is configured with dedicated hardware. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an object detection device according to an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0012] Embodiment 1 FIG. 1 is a diagram illustrating the configuration of an object detection device according to a first embodiment. Object detection device 100A is an object detection device for vehicle installation that is installed in a vehicle and detects objects such as moving objects (e.g., people, other vehicles) and fixed structures (e.g., walls). Hereinafter, a detection target object such as a moving object or a fixed object detected by object detection device 100A may be simply referred to as an object. Furthermore, a vehicle on which object detection device 100A is installed may be simply referred to as a vehicle, and a vehicle detected by object detection device 100A may be simply referred to as another vehicle.

[0013] The object to be detected in the first embodiment is an object to be detected in order to perform vehicle control or issue an alarm notification. The object detection device 100A is connected to a vehicle control device 30 that controls the operation of the vehicle. The object detection device 100A transmits data used for controlling the operation of the vehicle to the vehicle control device 30. The data used for controlling the operation of the vehicle includes, for example, the position, speed, and type of the object.

[0014] The object detection device 100A includes a radar 1, a camera 2, a radar signal processor 3, a camera image processor 4A, and a fusion processor 5A. At least one of the radar 1 and the camera 2 may be configured separately from the object detection device 100A. The object detected by the radar 1 is a first object, and the object detected by the camera 2 is a second object. There are cases where the first object and the second object are determined to be the same object, and cases where they are determined to be different objects.

[0015] The radar 1 emits electromagnetic waves toward an object to be detected and receives a reflected signal (radar reflection signal) from the object. The radar 1 for vehicle installation is a radar that operates, for example, using the FMCW (Frequency Modulated Continuous Wave) method or the FCM (Fast Chirp Modulation) method. The radar 1 is configured using high-frequency semiconductor components, power semiconductor components, substrates, crystal devices, chip components, antennas, etc. The radar 1 sends the received reflection signal to a radar signal processor 3.

[0016] The radar signal processor 3 detects the position and speed of an object by processing the reflected signal received by the radar 1. The radar signal processor 3 uses a processor or the like, which will be described later.

[0017] The radar signal processor 3 has a distance detection unit 6, a speed detection unit 7, a horizontal angle detection unit 8, and a radar detection data storage unit 9. The distance detection unit 6, the speed detection unit 7, and the horizontal angle detection unit 8 each perform fast Fourier transformation (FFT) in the distance direction, the speed direction, and the horizontal angle direction to detect the distance, speed, and horizontal angle of an object.

[0018] That is, the distance detection unit 6 detects the distance from the vehicle to the object (hereinafter referred to as distance data) based on the reflected signal received by the radar 1. The distance detection unit 6 sends the distance data to the speed detection unit .

[0019] The speed detection unit 7 detects the relative speed of the object with respect to the vehicle based on the reflected signal received by the radar 1. Hereinafter, the data of the relative speed of the object with respect to the vehicle detected by the speed detection unit 7 will be referred to as speed 32. The speed detection unit 7 sends the speed 32 and distance data to the horizontal angle detection unit 8.

[0020] Based on the reflected signal received by the radar 1, the horizontal angle detection unit 8 detects the horizontal angle (the angle between the vehicle's traveling direction and the direction connecting the object and the vehicle) relative to the vehicle's traveling direction. Hereinafter, the horizontal angle of the object detected by the horizontal angle detection unit 8 is referred to as horizontal angle data. The horizontal angle detection unit 8 calculates the relative position of the object with respect to the vehicle based on the horizontal angle data and distance data. Hereinafter, the data of the object's relative position with respect to the vehicle calculated by the horizontal angle detection unit 8 is referred to as the radar detection position 31. The radar detection position 31 is data indicating the position of the object detected using the radar 1. The horizontal angle detection unit 8 stores the speed 32 and the radar detection position 31 in the radar detection data storage unit 9.

[0021] The speed 32 may be stored in the radar detection data storage unit 9 by the speed detection unit 7. The distance detection unit 6 may send the distance data directly to the horizontal angle detection unit 8 without going through the speed detection unit 7.

[0022] The radar detection data storage unit 9 is a storage device such as a memory that stores the speed 32 and the radar detection position 31. The radar detection data storage unit 9 is connected to the fusion processor 5A. The speed 32 and the radar detection position 31 stored in the radar detection data storage unit 9 are read out by the subsequent fusion processor 5A. The radar signal processor 3 may transmit the speed 32 and the radar detection position 31 to the fusion processor 5A.

[0023] Camera 2 captures an image of an object to obtain image data of the object. Camera 2 is composed of components such as a lens, a holder, a CMOS (Complementary Metal Oxide Semiconductor) sensor, power semiconductor components, and a quartz device. Camera 2 sends the image data of the object to camera image processor 4A.

[0024] The camera image processor 4A detects the position of the object, pixel coordinates 42 which are the coordinates of the pixels of the object, and the object type which is the type of the object, by performing signal processing on the image data acquired by the camera 2. A processor or the like, which will be described later, is used as the camera image processor 4A.

[0025] The camera image processor 4A has an object recognition unit 10, a detection frame assignment unit 11, a pixel coordinate assignment unit 12, an object position detection unit 13, a camera detection data memory unit 14A, a parameter memory unit 17, a position conversion table memory unit 18, and a conversion table update unit 19.

[0026] The object recognition unit 10 detects the object type based on the image data acquired by the camera 2. The object recognition unit 10 recognizes objects and detects the object type by using feature data obtained by machine learning, deep learning, etc. as a database (not shown). The object recognition unit 10 sends the object type 43, which is data on the object type, and the image data to the detection frame assignment unit 11. The object recognition unit 10 also stores the object type 43 in the camera detection data storage unit 14A.

[0027] The detection frame assigning unit 11 assigns a frame to the area of ​​the object to be recognized on the image based on the image data and the object recognition result (object type 43) obtained by the object recognition unit 10. The detection frame assigning unit 11 sends the image to which the frame indicating the area of ​​the object has been assigned to the pixel coordinate assigning unit 12 as frame-assigned image data.

[0028] The pixel coordinate assigning unit 12 assigns pixel coordinates 42 to pixels of an object on the image based on the frame-assigned image data. For example, the pixel coordinate assigning unit 12 assigns pixel coordinates 42 to the center point of the bottom side of the frame of the object. In this way, the pixel coordinate assigning unit 12 assigns pixel coordinates 42 to the object on the image data captured by the camera 2. The pixel coordinate assigning unit 12 sends the pixel coordinates 42, which are data of the coordinates assigned to the pixel, to the object position detection unit 13. The pixel coordinate assigning unit 12 also stores the pixel coordinates 42 in the camera detection data storage unit 14A.

[0029] The object position detection unit 13 calculates the relative position of the object with respect to the vehicle based on the pixel coordinates 42. Hereinafter, the data of the relative position of the object with respect to the vehicle calculated by the object position detection unit 13 will be referred to as the camera detected position 41. The camera detected position 41 is data indicating the position of the object detected using the camera 2.

[0030] The object position detection unit 13 calculates the camera detected position 41 by converting the pixel coordinates 42 into position data indicating the position of the object based on a position conversion table (described later) stored in the position conversion table storage unit 18.

[0031] Only in the initial state immediately after object detection device 100A is attached to the vehicle, object position detection unit 13 calculates camera detected position 41 by referring to camera internal parameters 51 and camera external parameters 52 stored in parameter storage unit 17. Object position detection unit 13 stores camera detected position 41 in camera detected data storage unit 14A.

[0032] The parameter storage unit 17 is a storage device such as a memory that stores camera internal parameters 51 and camera external parameters 52. The camera internal parameters 51 are parameters determined by components such as the lens and CMOS sensor included in the camera 2. The camera internal parameters 51 include focal length, image data size, pixel center, lens distortion coefficient, etc. The camera internal parameters 51 are adjusted for each individual camera 2 during product shipping inspection of the camera 2 and are saved in the parameter storage unit 17.

[0033] The camera extrinsic parameters 52 are parameters determined by how the camera 2 is attached to the vehicle. The camera extrinsic parameters 52 include the attachment height, attachment attitude angle, and the like after the camera 2 is attached to the vehicle. The camera extrinsic parameters 52 are adjusted for each individual object detection device 100A during product shipping inspection of the object detection device 100A, and are stored in the parameter storage unit 17. Note that the camera extrinsic parameters 52 may also be adjusted for each individual camera 2 when the camera 2 is attached to the vehicle, and stored in the parameter storage unit 17.

[0034] The camera internal parameters 51 vary with temperature and aging. The camera external parameters 52 vary with aging changes in the attitude of the camera 2, image blurring caused by vibrations that occur while the vehicle is traveling, and the like. Therefore, the camera internal parameters 51 and the camera external parameters 52 are referenced only in the initial state immediately after the camera 2 is attached to the vehicle, and are not referenced thereafter. That is, in the initial state, the object position detection unit 13 references the position conversion table, the camera internal parameters 51, and the camera external parameters 52, and thereafter, the object position detection unit 13 references the position conversion table updated in real time by the conversion table update unit 19.

[0035] The position conversion table storage unit 18 is a storage device such as a memory that stores a position conversion table that is updated in accordance with changes in the radar detection position 31. The position conversion table is a data table that indicates the correspondence between the pixel coordinates of an object and the position of the object. In other words, the position conversion table is a data table for converting the pixel coordinates of an object into the position of the object.

[0036] The object position detection unit 13 of embodiment 1 calculates the camera detection position 41 corresponding to the pixel coordinates 42 using a position conversion table that is updated in accordance with changes in the radar detection position 31, so that the object position can be calculated accurately.

[0037] The conversion table update unit 19 updates the position conversion table stored in the position conversion table storage unit 18. The conversion table update unit 19 updates the position conversion table based on camera update data (first camera update data) described below that is stored in the fusion processor 5A. Specifically, the conversion table update unit 19 updates the position conversion table based on the radar detection position 31 and pixel coordinates 42 included in the camera update data.

[0038] The camera detection data storage unit 14A stores the object type 43 sent from the object recognition unit 10, the pixel coordinates 42 sent from the pixel coordinate assignment unit 12, and the camera detection position 41 sent from the object position detection unit 13. The camera detection data storage unit 14A is connected to the fusion processor 5A.

[0039] The object type 43, pixel coordinates 42, and camera detection position 41 stored in the camera detection data storage unit 14A are read out by the subsequent fusion processor 5A. Alternatively, the camera image processor 4A may transmit the object type 43, pixel coordinates 42, and camera detection position 41 to the fusion processor 5A.

[0040] The fusion processor 5A has an object identity determination unit 20, a detection data selection unit 21, a fusion detection data storage unit 22, and a camera update data storage unit 24A. The object identity determination unit 20 reads out the speed 32 and the radar detection position 31 from the radar signal processor 3. The object identity determination unit 20 also reads out the object type 43, pixel coordinates 42, and the camera detection position 41 from the camera image processor 4A.

[0041] The object identity determination unit 20 executes an object identity determination process based on the radar detection position 31 detected using the radar 1 and the camera detection position 41 detected using the camera 2. That is, the object identity determination unit 20 determines whether the object detected using the radar 1 and the object detected using the camera 2 are the same or not, based on the radar detection position 31 and the camera detection position 41.

[0042] When the object identification determination unit 20 determines through the identification determination process that the object detected using the radar 1 and the object detected using the camera 2 are the same, it associates the radar detection position 31 detected using the radar 1 with pixel coordinates 42 in the object recognition data detected using the camera 2. In this case, the object identification determination unit 20 stores camera update data, which is data associating the radar detection position 31 with the pixel coordinates 42, in the camera update data storage unit 24A.

[0043] Furthermore, if the object identity determination unit 20 determines that the object detected using the radar 1 is the same as the object detected using the camera 2, it sends correspondence data (hereinafter sometimes referred to as first correspondence data) that associates the radar detection position 31, speed 32, and object type 43 to the detection data selection unit 21.

[0044] On the other hand, if the object identification determination unit 20 determines that the object detected using the radar 1 is not the same as the object detected using the camera 2, it considers that the detection data by the radar 1 has been lost and sends the data detected using the camera 2 to the detection data selection unit 21. In other words, if the object detected using the radar 1 is not the same as the object detected using the camera 2, the object identification determination unit 20 sends to the detection data selection unit 21 correspondence data (hereinafter, sometimes referred to as second correspondence data) that associates the camera detection position 41 with the object type 43.

[0045] As a result, if the detection data selection unit 21 succeeds in making an identity determination (if it is determined to be identical), it stores in the fusion detection data storage unit 22 first correspondence data that associates the radar detection position 31 detected using the radar 1, the speed 32, and the object type 43 detected using the camera 2.

[0046] On the other hand, if the detection data selection unit 21 fails to make an identity determination (if it is determined that the objects are not identical), it stores second correspondence data in the fusion detection data storage unit 22, which associates the camera detection position 41 detected using the camera 2 with the object type 43.

[0047] The first data or second data stored in the fusion detection data storage unit 22 is data for vehicle control. Note that FIG. 1 illustrates a case where the data stored in the fusion detection data storage unit 22 are a detected position 61, a speed 32, and an object type 43. If the identity determination is successful, the detected position 61 is the radar detected position 31, and if the identity determination is unsuccessful, the detected position 61 is the camera detected position 41. Note that if the identity determination is unsuccessful, the speed 32 is not stored in the fusion detection data storage unit 22.

[0048] The camera update data storage unit 24A is a storage device such as a memory that stores camera update data in which the radar detection position 31 is associated with the pixel coordinates 42. The camera update data stored in the camera update data storage unit 24A is read out by the conversion table update unit 19 of the camera image processor 4A. The fusion processor 5A may also transmit the camera update data to the conversion table update unit 19.

[0049] Conversion table update unit 19 updates the position conversion table when it acquires camera update data. As a result, conversion table update unit 19 updates the position conversion table in real time. For example, conversion table update unit 19 updates the position conversion table for each frame of image data captured by camera 2. In this way, object detection device 100A updates the position conversion table in real time, thereby improving the accuracy of position detection by camera 2.

[0050] The fusion detection data storage unit 22 is a storage device such as a memory that stores data in which the detection position 61, the speed 32, and the object type 43 are associated with each other. If the identity determination is successful, the fusion detection data storage unit 22 stores first association data in which the radar detection position 31, the speed 32, and the object type 43 are associated with each other. If the identity determination is unsuccessful, the fusion detection data storage unit 22 stores second association data in which the camera detection position 41 and the object type 43 are associated with each other.

[0051] The vehicle control device 30 reads data from the fusion detection data storage unit 22 and controls the operation of the vehicle using the read data. Specifically, if the identity determination is successful, the vehicle control device 30 controls the operation of the vehicle based on the radar detection position 31, speed 32, and object type 43. If the identity determination is successful, the object detection device 100A may transmit the radar detection position 31, speed 32, and object type 43 to the vehicle control device 30.

[0052] Furthermore, if the identity determination fails, the vehicle control device 30 controls the operation of the vehicle based on the camera-detected position 41 and the object type 43. If the identity determination fails, the object detection device 100A may transmit the camera-detected position 41 and the object type 43 to the vehicle control device 30.

[0053] Next, a processing procedure of the processing executed by the object detection device 100A will be described. FIG. 2 is a flowchart showing the processing procedure of the processing executed by the object detection device according to the first embodiment. The object detection device 100A starts frame detection (step S1). That is, the radar 1 emits electromagnetic waves to an object to be detected and acquires a reflected signal (received data) from the object. Furthermore, the camera 2 acquires image data of the object by capturing an image of the object.

[0054] When the radar 1 acquires the received data, it outputs the acquired received data to the radar signal processor 3 (step S2). The radar signal processor 3 detects the position and speed of the object based on the received data. That is, the distance detector 6, speed detector 7, and horizontal angle detector 8 of the radar signal processor 3 detect a radar detection position 31 indicating the position of the object and a speed 32 indicating the speed of the object based on the received data (step S3). The radar signal processor 3 stores the radar detection position 31 and the speed 32 in the radar detection data storage unit 9 (step S4).

[0055] Furthermore, when the camera 2 acquires the image data, it outputs the acquired image data to the camera image processor 4A (step S5). The object recognition unit 10 of the camera image processor 4A recognizes the object based on the image data (step S6). That is, the object recognition unit 10 detects the object type 43, which is the type of the object, based on the image data. The object recognition unit 10 sends the object type 43 and the image data to the detection frame assignment unit 11. Furthermore, the object recognition unit 10 sends the object type 43 to the camera detection data storage unit 14A.

[0056] The detection frame assigning unit 11 assigns a frame to the area of ​​the object on the image based on the image data and the object type 43. The detection frame assigning unit 11 sends the image with the frame indicating the area of ​​the object assigned to the pixel coordinate assigning unit 12 as frame-assigned image data.

[0057] The pixel coordinate assigning unit 12 assigns pixel coordinates 42 to pixels of the object on the image based on the frame-assigned image data. That is, the pixel coordinate assigning unit 12 assigns pixel coordinates 42 to the recognized object (step S7). The pixel coordinate assigning unit 12 sends the pixel coordinates 42 to the object position detecting unit 13. The pixel coordinate assigning unit 12 also sends the pixel coordinates 42 to the camera detection data storage unit 14A.

[0058] The object position detection unit 13 detects the camera detected position 41, which is the relative position of the object with respect to the vehicle, by referring to the position conversion table for the pixel coordinates 42 (step S8). In this case, since the position conversion table for the previous frame is updated in real time, the camera image processor 4A can detect the position of the object with high accuracy using the camera 2. The object position detection unit 13 sends the camera detected position 41 to the camera detected data storage unit 14A.

[0059] In the initial state immediately after object detection device 100A is attached to a vehicle, object position detection unit 13 calculates camera detected position 41 by referring to camera internal parameters 51, camera external parameters 52, and the position conversion table.

[0060] The camera image processor 4A stores the camera detection position 41, pixel coordinates 42, and object type 43 in the camera detection data storage unit 14A (step S9).

[0061] The processing of steps S2 to S4 and the processing of steps S5 to S9 are executed in parallel. The camera-detection data storage unit 14A may store the object type 43 before the processing of steps S7 and S8, as long as the processing of step S6 has been executed. Furthermore, the camera-detection data storage unit 14A may store the pixel coordinates 42 before the processing of step S8, as long as the processing of step S7 has been executed.

[0062] The object identity determination unit 20 of the fusion processor 5A reads out the radar detection position 31 and the speed 32 from the radar detection data storage unit 9. The object identity determination unit 20 also reads out the camera detection position 41, pixel coordinates 42, and object type 43 from the camera detection data storage unit 14A.

[0063] The object identity determination unit 20 of the fusion processor 5A executes an object identity determination process (step S10) based on the radar detection position 31 and the camera detection position 41. Through this identity determination process, the object identity determination unit 20 associates the radar detection position 31 and speed 32 detected using the radar 1 with the object recognition data detected using the camera 2.

[0064] The object identity determination unit 20 determines whether the identity determination has been successful (step S11). If the object identity determination unit 20 determines that the identity determination has been unsuccessful, it executes the process of step S12. On the other hand, if the object identity determination unit 20 determines that the identity determination has been successful, it executes the processes of steps S13 and S17.

[0065] That is, if the object identity determination unit 20 fails to make the identity determination (step S11, No), it sends second correspondence data that associates the camera detection position 41 with the object type 43 to the detection data selection unit 21. As a result, the detection data selection unit 21 outputs the camera detection position 41 and the object type 43 detected using the camera 2 to the fusion detection data storage unit 22 (step S12).

[0066] On the other hand, if the object identity determination is successful (step S11, Yes), the object identity determination unit 20 sends first correspondence data associating the radar detection position 31, the speed 32, and the object type 43 to the detection data selection unit 21. As a result, the detection data selection unit 21 outputs the radar detection position 31 and the speed 32 detected using the radar 1 and the object type 43 detected using the camera 2 to the fusion detection data storage unit 22 (step S13).

[0067] The fusion processor 5A stores the data obtained in step S12 or the data obtained in step S13 in the fusion detection data storage unit 22 (step S14). That is, if the detection data selection unit 21 fails to make the identity determination, it stores second correspondence data that associates the camera detection position 41 with the object type 43 in the fusion detection data storage unit 22. On the other hand, if the detection data selection unit 21 succeeds in making the identity determination, it stores second correspondence data that associates the radar detection position 31, speed 32, and object type 43 in the fusion detection data storage unit 22.

[0068] The fusion processor 5A outputs the data stored in the fusion detection data storage unit 22 to the vehicle control device 30. That is, the fusion processor 5A outputs the data of the object detection position 61, speed 32, and object type 43, which are the data stored in the fusion detection data storage unit 22 in the processing of step S14, to the vehicle control device 30 (step S15). Specifically, if the identity determination fails, the fusion processor 5A outputs second correspondence data that associates the camera detection position 41 with the object type 43 to the vehicle control device 30. On the other hand, if the identity determination is successful, the fusion processor 5A outputs first correspondence data that associates the radar detection position 31, speed 32, and object type 43 to the vehicle control device 30.

[0069] The vehicle control device 30 controls the operation of the vehicle using the data acquired from the fusion processor 5A (step S16). Specifically, if the identity determination fails, the vehicle control device 30 controls the vehicle based on the camera detection position 41 and the object type 43. If the identity determination is successful, the vehicle control device 30 controls the vehicle based on the radar detection position 31, the speed 32, and the object type 43.

[0070] Furthermore, if the object identity determination unit 20 succeeds in the identity determination in the processing of step S11 (step S11, Yes), it stores the camera update data in which the radar detection position 31 and the pixel coordinates 42 are associated in the camera update data storage unit 24A (step S17).

[0071] The fusion processor 5A transmits the camera update data stored in the camera update data storage unit 24A to the conversion table update unit 19 of the camera image processor 4A. As a result, the conversion table update unit 19 of the camera image processor 4A updates the position conversion table based on the camera update data (step S18).

[0072] When the process of step S12 is executed, the processes of steps S14 to S16 are executed, and the processes of steps S13, S17, and S18 are not executed. On the other hand, when the process of step S13 is executed, the processes of steps S14 to S18 are executed, and the process of step S12 is not executed. Note that when the process of step S13 is executed, the processes of steps S13 to S16 and the processes of steps S17 and S18 are executed in parallel.

[0073] When the processes of steps S16 and S18 are completed, object detection device 100A proceeds to the next frame process of object detection (step S19). Then, object detection device 100A repeats the processes of steps S2 to S19 in FIG.

[0074] Object detection device 100A achieves fusion-type object detection using radar 1 and camera 2. When an object to be detected approaches the vicinity of a structure with high radar reflection intensity, the radar reflected wave signal from the object is buried in the radar reflected wave signal from the structure, causing the detection data of radar 1 to be lost and making it difficult for radar 1 to detect the object. Object detection device 100A maintains the detection accuracy of fusion-type object detection by replacing the detection data of radar 1 with the detection data of camera 2. That is, object detection device 100A calculates camera detection position 41 based on radar detection position 31, and generates vehicle control data for controlling the vehicle using the calculated camera detection position 41 instead of radar detection position 31.

[0075] In the first embodiment, even if the accuracy of object position detection by camera 2 is low, object detection device 100A can update the position conversion table in real time based on radar detection position 31 by the processing of steps S17 and S18. As a result, even if the detection data of radar 1 is lost, object detection device 100A can detect the object position using the position conversion table that has been updated in accordance with changes in radar detection position 31. This makes it possible to prevent a decrease in object position detection accuracy and maintain high position detection accuracy.

[0076] For example, the camera 2's characteristics change due to temperature and aging. Furthermore, the position detection accuracy of the camera 2 decreases due to aging changes in the attitude of the camera 2 after it is attached to the vehicle, blurring of the image due to vibrations that occur while the vehicle is traveling, and the like. Even if the detection data of the radar 1 is lost in such a case, the object detection device 100A can detect the object's position using the position conversion table that is updated in accordance with changes in the radar detection position 31. This prevents a decrease in the object's position detection accuracy and maintains high position detection accuracy.

[0077] As described above, in the first embodiment, object detection device 100A performs object identity determination based on data in which radar 1 and camera 2 simultaneously detected an object (e.g., a person) in a past frame, and adds radar detection position 31, which is position detection information of radar 1, to the image data of camera 2. That is, for a frame in which object detection device 100A determines that the object detected by radar 1 and the object detected by camera 2 are the same based on radar detection position 31 and camera detection position 41, object detection device 100A generates camera update data that associates pixel coordinates 42 with radar detection position 31. Then, object detection device 100A updates a position conversion table indicating the correspondence between the pixel coordinates of the object and the object's position based on the camera update data, and calculates camera detection position 41 corresponding to pixel coordinates 42 based on the position conversion table. If radar 1 detection data is lost in a subsequent frame, object detection device 100A provides vehicle control device 30 with camera detection position 41 calculated based on the position conversion table that has been updated in accordance with changes in radar detection position 31. This enables the object detection device 100A to improve and maintain high position detection accuracy of the camera 2. Therefore, even if the detection data of the radar 1 is lost, the object detection device 100A can provide the vehicle control device 30 with a highly accurate detected position 61, and the vehicle control device 30 can control the vehicle based on the highly accurate detected position 61.

[0078] As described above, according to the first embodiment, when object detection device 100A determines that the object detected by radar 1 and the object detected by camera 2 are the same, it generates camera update data in which pixel coordinates 42 correspond to radar detection positions 31. Then, based on the camera update data, object detection device 100A updates the position conversion table indicating the correspondence between pixel coordinates 42 and radar detection positions 31, and detects camera detection positions 41 corresponding to pixel coordinates 42 based on the position conversion table. Therefore, when camera 2 with low object position detection accuracy is used, object detection device 100A can accurately detect the position of an object even if the object approaches a structure with high radar reflection intensity.

[0079] Embodiment 2 Next, a second embodiment will be described with reference to Figures 3 and 4. In the second embodiment, the object detection device updates a distance conversion table indicating the correspondence relationship between the pixel size of an object and the distance from the vehicle to the object, based on the pixel size of the object and the distance to the object detected using radar 1. Then, the object detection device detects the distance corresponding to the pixel size by referring to the distance conversion table, and detects the camera detection position 41 based on the detected distance.

[0080] Fig. 3 is a diagram showing the configuration of an object detection device according to embodiment 2. Among the components in Fig. 3, components that achieve the same functions as those in object detection device 100A according to embodiment 1 shown in Fig. 1 are assigned the same reference numerals, and duplicated explanations will be omitted.

[0081] Object detection device 100B of embodiment 2 is an object detection device mounted on a vehicle that detects objects, similar to object detection device 100A of embodiment 1. In embodiment 1, object detection device 100A updates the position conversion table based on camera update data (first camera update data) in which radar detection positions 31 and pixel coordinates 42 are associated with each other, and detects camera detection positions 41 corresponding to the pixel coordinates by referring to the position conversion table.

[0082] In the second embodiment, object detection device 100B updates a distance conversion table indicating the correspondence relationship between the pixel size of an object and the distance from the vehicle (camera 2) to the object, based on camera update data (second camera update data) in which the pixel size in an image of the object is associated with the distance from radar 1 to the object. Then, object detection device 100B detects the distance corresponding to the pixel size by referring to the distance conversion table, and detects camera detection position 41 based on the detected distance. Note that the distance to the object in the second embodiment may be the distance from camera 2 to the object, or the distance from the vehicle to the object.

[0083] The pixel size in the second embodiment does not correspond to the size of one pixel, but corresponds to the number of pixels in a specific region of an object. Therefore, even for the same object, the closer the object is to camera 2, the larger the pixel size will be, and the farther the object is from camera 2, the smaller the pixel size will be. In other words, the pixel size in the second embodiment changes depending on the distance between camera 2 and the object. In other words, if the pixel size is known, the distance between camera 2 and the object can also be calculated.

[0084] Compared to object detection device 100A, object detection device 100B includes a camera image processor 4B and a fusion processor 5B instead of camera image processor 4A and fusion processor 5A. That is, object detection device 100B includes radar 1, camera 2, radar signal processor 3, camera image processor 4B, and fusion processor 5B.

[0085] Compared to camera image processor 4A, camera image processor 4B has a camera detection data storage unit 14B instead of camera detection data storage unit 14A. Also, compared to camera image processor 4A, camera image processor 4B has a distance conversion table storage unit 16 instead of position conversion table storage unit 18. Also, camera image processor 4B has a detection frame pixel size assigning unit 15. That is, camera image processor 4B has an object recognition unit 10, a detection frame assigning unit 11, a pixel coordinate assigning unit 12, an object position detection unit 13, a detection frame pixel size assigning unit 15, camera detection data storage unit 14B, a parameter storage unit 17, a distance conversion table storage unit 16, and a conversion table update unit 19.

[0086] In the second embodiment, the detection frame assigning unit 11 sends the framed image data to the pixel coordinate assigning unit 12 and the detection frame pixel size assigning unit 15. The detection frame pixel size assigning unit 15 assigns a pixel size (number of pixels) to the detection frame assigned to the framed image data. Hereinafter, the pixel size data assigned to the detection frame by the detection frame pixel size assigning unit 15 will be referred to as the pixel size 44.

[0087] The detection frame pixel size assigning unit 15 sends the pixel size 44 to the object position detecting unit 13. The detection frame pixel size assigning unit 15 also stores the pixel size 44 in the camera detection data storage unit 14B.

[0088] Pixel coordinate assignment unit 12 of the second embodiment sends pixel coordinates 42 to object position detection unit 13, but does not have to store them in camera detection data storage unit 14B. Object position detection unit 13 of the second embodiment detects the orientation of the object with respect to camera 2 based on pixel coordinates 42 sent from pixel coordinate assignment unit 12.

[0089] Furthermore, object position detection unit 13 detects the distance from camera 2 to the object based on pixel size 44 sent from detection frame pixel size assignment unit 15. At this time, object position detection unit 13 calculates the distance from camera 2 to the object by converting pixel size 44 into the distance to the object based on a distance conversion table stored in distance conversion table storage unit 16.

[0090] The distance conversion table storage unit 16 is a storage device such as a memory that stores a distance conversion table that is updated in accordance with changes in the radar detection position 31. The distance conversion table is a data table that indicates the correspondence between the pixel size 44 of an object and the distance from the radar 1 to the object. In other words, the distance conversion table is a data table for converting the pixel size 44 of an object into the distance from the radar 1 to the object.

[0091] The object position detection unit 13 of the second embodiment calculates the distance to the object corresponding to the pixel size 44 using a distance conversion table that is updated in response to changes in the radar detection position 31, so that the distance to the object can be calculated accurately. The object position detection unit 13 detects the camera detection position 41, which is the relative position of the object with respect to the vehicle, based on the distance to the object and the direction in which the object is located. The object position detection unit 13 stores the camera detection position 41 in the camera detection data storage unit 14B.

[0092] The conversion table update unit 19 updates the distance conversion table stored in the distance conversion table storage unit 16. The conversion table update unit 19 updates the distance conversion table based on camera update data (second camera update data) described below that is stored in the fusion processor 5B. Specifically, the conversion table update unit 19 updates the distance conversion table based on the pixel size 44 and the distance 33 included in the camera update data. The distance 33 is the distance from the radar 1 to the object, calculated based on the radar detection position 31.

[0093] The camera detection data storage unit 14B stores the object type 43 sent from the object recognition unit 10, the pixel size 44 sent from the detection frame pixel size assignment unit 15, and the camera detection position 41 sent from the object position detection unit 13. The camera detection data storage unit 14B is connected to the fusion processor 5B.

[0094] The object type 43, pixel size 44, and camera detection position 41 stored in the camera detection data storage unit 14B are read out by the subsequent fusion processor 5B. Alternatively, the camera image processor 4B may transmit the object type 43, pixel size 44, and camera detection position 41 to the fusion processor 5B.

[0095] Compared to the fusion processor 5A, the fusion processor 5B has a camera update data storage unit 24B instead of the camera update data storage unit 24A. That is, the fusion processor 5B has an object identity determination unit 20, a detection data selection unit 21, a fusion detection data storage unit 22, and a camera update data storage unit 24B.

[0096] If the object identification determination unit 20 of the second embodiment is successful in the identification determination, it calculates the distance 33 from the vehicle to the object and the direction of the object relative to the vehicle based on the radar detection position 31. That is, the object identification determination unit 20 calculates the distance 33 to the object using the radar 1 by breaking down the radar detection position 31 into the distance 33 to the object and the direction to the object. If the object identification determination is successful, the object identification determination unit 20 associates the distance 33 with the pixel size 44 in the object recognition data detected using the camera 2. In this case, the object identification determination unit 20 stores camera update data, which is data that associates the distance 33 with the pixel size 44, in the camera update data storage unit 24B.

[0097] Furthermore, if the object identity determination unit 20 succeeds in the identity determination, it sends the first correspondence data in which the radar detection position 31, the speed 32, and the object type 43 are associated with each other to the detection data selection unit 21.

[0098] As a result, if the detection data selection unit 21 succeeds in making an identity determination, it stores in the fusion detection data storage unit 22 first correspondence data that associates the radar detection position 31 detected using the radar 1, the speed 32, and the object type 43 detected using the camera 2.

[0099] On the other hand, if the object identity determination unit 20 fails to make the identity determination, it considers that the detection data by the radar 1 has been lost, and sends the data detected using the camera 2 to the detection data selection unit 21. In other words, if the object detected using the radar 1 and the object detected using the camera 2 are not the same, the object identity determination unit 20 sends second correspondence data that associates the camera detection position 41 with the object type 43 to the detection data selection unit 21.

[0100] As a result, if the detection data selection unit 21 fails to make an identity determination, it stores second correspondence data in the fusion detection data storage unit 22, which associates the camera detection position 41 detected using the camera 2 with the object type 43.

[0101] 3 illustrates a case where the data stored in the fusion detection data storage unit 22 is a detected position 61, a speed 32, and an object type 43. If the identity determination is successful, the detected position 61 is a radar detected position 31, and if the identity determination is unsuccessful, the detected position 61 is a camera detected position 41. If the identity determination is unsuccessful, the speed 32 is not stored in the fusion detection data storage unit 22.

[0102] The camera update data storage unit 24B is a storage device such as a memory that stores camera update data in which the distance 33 is associated with the pixel size 44. The camera update data stored in the camera update data storage unit 24B is read out by the conversion table update unit 19 of the camera image processor 4B. The fusion processor 5B may also transmit the camera update data to the conversion table update unit 19.

[0103] Conversion table update unit 19 updates the distance conversion table when it acquires camera update data. As a result, conversion table update unit 19 updates the distance conversion table in real time. Conversion table update unit 19 updates the position conversion table for each frame of image data captured by camera 2, for example. In this way, object detection device 100B updates the position conversion table in real time, thereby improving the accuracy of position detection by camera 2.

[0104] Next, the processing procedure executed by object detection device 100B will be described. Fig. 4 is a flowchart showing the processing procedure executed by the object detection device according to the second embodiment. Among the processing shown in Fig. 4, the same steps as those executed by object detection device 100A according to the first embodiment shown in Fig. 2 are assigned the same step numbers, and duplicated explanations will be omitted.

[0105] Compared to object detection device 100A of embodiment 1, object detection device 100B of embodiment 2 executes the process of step S7A in addition to the process of step S7, and executes the processes of steps S8A and S9A instead of the processes of steps S8 and S9. Also, compared to object detection device 100A, object detection device 100B executes the processes of steps S17A and S18A instead of the processes of steps S17 and S18.

[0106] That is, when the object recognition unit 10 of the camera image processor 4B recognizes an object based on the image data (step S6), it sends the frame-added image data to the pixel coordinate assignment unit 12 and the detection frame pixel size assignment unit 15. As a result, the pixel coordinate assignment unit 12 assigns pixel coordinates 42 to the recognized object (step S7). Then, the pixel coordinate assignment unit 12 sends the pixel coordinates 42 to the object position detection unit 13.

[0107] Furthermore, detection frame pixel size assigning unit 15 assigns pixel size 44 to the detection frame assigned to the frame-assigned image data. That is, detection frame pixel size assigning unit 15 assigns pixel size 44 of the detection frame to the recognized object (step S7A). Then, detection frame pixel size assigning unit 15 sends pixel size 44 to object position detection unit 13 and camera-detected data storage unit 14B.

[0108] The object position detection unit 13 detects the orientation of the object relative to the camera 2 based on the pixel coordinates 42 sent from the pixel coordinate assignment unit 12. The object position detection unit 13 also detects the distance to the object (the distance from the camera 2 to the object) corresponding to the pixel size 44 of the object by referring to a distance conversion table for the pixel size 44 (step S8A). In this case, the distance conversion table for the previous frame is updated in real time, so the camera image processor 4B can detect the distance to the object with high accuracy using the camera 2.

[0109] Based on the distance to the object and the orientation in which the object is located, object position detection unit 13 detects camera detected position 41, which is the relative position of the object with respect to camera 2. Object position detection unit 13 sends camera detected position 41 to camera detected data storage unit 14B.

[0110] In the initial state immediately after object detection device 100B is attached to a vehicle, object position detection unit 13 calculates camera detected position 41 by referring to camera internal parameters 51, camera external parameters 52, and the distance conversion table.

[0111] The camera image processor 4B stores the camera detection position 41, pixel size 44, and object type 43 in the camera detection data storage unit 14B (step S9A).

[0112] The processing of steps S2 to S4 and the processing of steps S5 to S9A are executed in parallel. The camera-detection data storage unit 14B may store the object type 43 before the processing of steps S7, S7A, and S8A, as long as the processing of step S6 has been executed.

[0113] Furthermore, the object identity determination unit 20 of the camera image processor 4B determines whether the identity determination has been successful (step S11), and if it determines that the identity determination has been unsuccessful, it executes the process of step S12. On the other hand, if it determines that the identity determination has been successful, it executes the processes of steps S13 and S17A.

[0114] If the object identity determination unit 20 succeeds in determining identity in the processing of step S11 (step S11, Yes), it stores camera update data that corresponds to the distance 33 and pixel size 44 extracted from the radar detection position 31 in the camera update data memory unit 24B (step S17A).

[0115] The fusion processor 5B transmits the camera update data stored in the camera update data storage unit 24B to the conversion table update unit 19 of the camera image processor 4B. As a result, the conversion table update unit 19 of the camera image processor 4B updates the distance conversion table based on the camera update data (step S18A).

[0116] When the process of step S12 is executed, the processes of steps S14 to S16 are executed, and the processes of steps S13, S17A, and S18A are not executed. On the other hand, when the process of step S13 is executed, the processes of steps S14 to S16, S17A, and S18A are executed, and the process of step S12 is not executed. Note that when the process of step S13 is executed, the processes of steps S13 to S16 and the processes of steps S17A and S18A are executed in parallel.

[0117] When the processes of step S16 and step S18A are completed, object detection device 100B proceeds to the next frame process of object detection (step S19). Then, object detection device 100B repeats the processes of steps S2 to S19 in FIG.

[0118] In this way, like object detection device 100A, object detection device 100B achieves fusion-type object detection using radar 1 and camera 2. Then, like object detection device 100A, object detection device 100B maintains the detection accuracy of fusion-type object detection by replacing the detection data of radar 1 with the detection data of camera 2. That is, object detection device 100B calculates camera detection position 41 based on radar detection position 31, and generates vehicle control data for controlling the vehicle using the calculated camera detection position 41 instead of radar detection position 31.

[0119] In the second embodiment, even if the accuracy of object position detection by camera 2 is low, object detection device 100B can update the distance conversion table in real time according to changes in radar detection position 31 by the processing of steps S17A and S18A. As a result, even if the detection data of radar 1 is lost, object detection device 100B can detect the object position using the distance conversion table updated based on radar detection position 31, and therefore it is possible to prevent a decrease in object position detection accuracy and maintain high position detection accuracy.

[0120] For example, the camera characteristics of the camera 2 vary with temperature and aging. Furthermore, the position detection accuracy of the camera 2 decreases due to aging changes in the attitude of the camera 2 after it is attached to the vehicle, blurring of the image due to vibrations that occur while the vehicle is traveling, and the like. Even if the detection data of the radar 1 is lost in such a case, the object detection device 100A can detect the object position using the distance conversion table that has been updated based on the radar detection position 31, so that it is possible to prevent a decrease in the object position detection accuracy and maintain high position detection accuracy.

[0121] Thus, according to the second embodiment, as in the first embodiment, when a camera 2 with low object position detection accuracy is used, the object position can be accurately detected even if the object approaches a structure with high radar reflection intensity.

[0122] Here, a hardware configuration of the object detection devices 100A and 100B according to the first and second embodiments will be described. The object detection devices 100A and 100B are realized by a processing circuit. The processing circuit may be a processor and memory that executes a program stored in a memory, or may be dedicated hardware.

[0123] 5 is a diagram illustrating a configuration example of a processing circuit when the processing circuit included in the object detection device according to the first and second embodiments is realized by a processor and a memory. A processing circuit 90 illustrated in FIG. 5 includes a processor 91 and a memory 92.

[0124] When the processing circuit 90 is configured with a processor 91 and a memory 92, each function of the processing circuit 90 is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a data processing program and stored in the memory 92. In the processing circuit 90, each function is realized by the processor 91 reading and executing the data processing program stored in the memory 92. That is, the processing circuit 90 includes the memory 92 for storing a data processing program that results in the processing of the object detection devices 100A, 100B. This data processing program can also be said to be a program that causes the object detection devices 100A, 100B to execute each function realized by the processing circuit 90. This data processing program may be provided by a storage medium on which the data processing program is stored, or by other means such as a communication medium.

[0125] Here, the processor 91 is, for example, a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor), etc. Furthermore, the memory 92 is, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (registered trademark) (Electrically EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc).

[0126] FIG. 6 is a diagram illustrating an example of a processing circuit included in the object detection device according to the first and second embodiments, configured with dedicated hardware. The processing circuit 93 illustrated in FIG. 6 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The processing circuit 93 may be partially implemented with dedicated hardware and partially implemented with software or firmware. In this way, the processing circuit 93 can achieve the above-described functions with dedicated hardware, software, firmware, or a combination thereof.

[0127] In the object detection device 100A, the radar signal processor 3, the camera image processor 4A, and the fusion processor 5A may each be implemented by separate processing circuits. In this case, one or more of the radar signal processor 3, the camera image processor 4A, and the fusion processor 5A are implemented by the processing circuit 90 or the processing circuit 93 described above. In this case, the object detection device 100A has multiple processing circuits.

[0128] Furthermore, in the object detection device 100B, the radar signal processor 3, the camera image processor 4B, and the fusion processor 5B may each be implemented by separate processing circuits. In this case, one or more of the radar signal processor 3, the camera image processor 4B, and the fusion processor 5B are implemented by the processing circuit 90 or the processing circuit 93 described above. In this case, the object detection device 100B has multiple processing circuits.

[0129] Furthermore, the camera image processors 4A and 4B may be realized using an MCU (Micro Control Unit) or the like. Furthermore, the camera image processors 4A and 4B may be realized using a GPU (Graphics Processing Unit) or the like.

[0130] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, or different embodiments may be combined with each other. It is also possible to omit or modify parts of the configurations as long as they do not deviate from the gist of the invention. [Explanation of symbols]

[0131] 1 radar, 2 camera, 3 radar signal processor, 4A, 4B camera image processor, 5A, 5B fusion processor, 6 distance detection unit, 7 speed detection unit, 8 horizontal angle detection unit, 9 radar detection data storage unit, 10 object recognition unit, 11 detection frame assignment unit, 12 pixel coordinate assignment unit, 13 object position detection unit, 14A, 14B camera detection data storage unit, 15 detection frame pixel size assignment unit, 16 distance conversion table storage unit, 17 parameter storage unit, 18 position conversion table storage unit, 19 conversion table update unit, 20 object identity determination unit, 21 detection data selection unit, 22 fusion detection data storage unit, 24A, 24B camera update data storage unit, 30 vehicle control device, 31 radar detection position, 32 speed, 33 distance, 41 camera detection position, 42 pixel coordinates, 43 object type, 44 Pixel size, 51 camera internal parameters, 52 camera external parameters, 61 detection position, 90, 93 processing circuit, 91 processor, 92 memory, 100A, 100B object detection device.

Claims

1. a radar signal processor that detects a radar detection position, which is the position of a first object, and a speed of the first object based on a reflected signal of an electromagnetic wave emitted by the radar toward the first object; a camera image processor that detects a camera detection position, which is the position of a second object, pixel coordinates of the second object, and an object type, which is the type of the second object, based on image data obtained by capturing an image of the second object with a camera; a fusion processor that determines whether the first object detected using the radar and the second object detected using the camera are the same or not based on the radar detection position and the camera detection position, and if it determines that they are the same, sends the radar detection position, the speed, and the object type to a vehicle control device that controls the vehicle, and if it determines that they are not the same, sends the camera detection position and the object type to the vehicle control device; Equipped with The fusion processor comprises: If it is determined that the pixels are identical, camera update data that associates the pixel coordinates with the radar detection position is sent to the camera image processor; The camera image processor includes: a conversion table update unit that updates a position conversion table indicating a correspondence relationship between the pixel coordinates and the radar detection positions based on the camera update data; an object position detection unit that detects the camera detection position corresponding to the pixel coordinates based on the position conversion table; having An object detection device characterized by:

2. The camera image processor includes: a parameter storage unit that stores camera internal parameters that are parameters determined by components included in the camera and camera external parameters that are parameters determined by how the camera is attached to the vehicle; the object position detection unit, in an initial state immediately after the camera is attached to the vehicle, detects the camera detection position corresponding to the pixel coordinates based on the camera internal parameters, the camera external parameters, and the position conversion table; 2. The object detection device according to claim 1.

3. a radar signal processor that detects a radar detection position, which is the position of a first object, and a speed of the first object based on a reflected signal of an electromagnetic wave emitted by the radar toward the first object; a camera image processor that detects, based on image data obtained by a camera capturing an image of a second object, a camera detection position that is the position of the second object, a pixel size that corresponds to the number of pixels on the image of the second object, and an object type that is the type of the second object; a fusion processor that determines whether the first object detected using the radar and the second object detected using the camera are the same or not based on the radar detection position and the camera detection position, and if it determines that they are the same, sends the radar detection position, the speed, and the object type to a vehicle control device that controls the vehicle, and if it determines that they are not the same, sends the camera detection position and the object type to the vehicle control device; Equipped with The fusion processor comprises: If it is determined that the two objects are identical by the identity determination, camera update data in which the pixel size and the distance from the radar to the first object calculated from the radar detection position are associated with each other is sent to the camera image processor; The camera image processor includes: a conversion table update unit that updates a distance conversion table indicating a correspondence relationship between the pixel size and the distance based on the camera update data; an object position detection unit that detects the distance corresponding to the pixel size based on the distance conversion table, detects an orientation of the second object relative to the camera based on the image data, and detects the camera detection position corresponding to the pixel size based on the distance and the orientation; having An object detection device characterized by:

4. The camera image processor includes: a parameter storage unit that stores camera internal parameters that are parameters determined by components included in the camera and camera external parameters that are parameters determined by how the camera is attached to the vehicle; the object position detection unit, in an initial state immediately after the camera is attached to the vehicle, detects the camera detection position corresponding to the pixel size based on the camera internal parameters, the camera external parameters, and the distance conversion table; 4. The object detection device according to claim 3.

Images