Object detection device

Abstract

An object detection device (100) comprises: a housing that is attached to a vehicle; a radar that is housed in the housing and that detects, on the basis of a reflection signal of an electromagnetic wave emitted to an object to be detected on the front side of the housing, radar object detection data that is detection data of the object; a camera that is housed in the housing, that acquires, by imaging an object to be detected on the front side of the housing, image data of the object, and that detects, on the basis of the image data, camera object detection data that is detection data of the object; and a fusion processor that is housed in the housing and that performs, on the basis of the radar object detection data and the camera object detection data, identical object determination processing as to whether or not the object detected by the radar and the object detected by the camera are the same. The extension direction of the emission center axis (20a) of the electromagnetic wave emitted from the radar and the extension direction of the optical axis (40a) of the camera are different from each other in terms of the height direction of the housing.

Description

Object detection device

[0001] The present disclosure relates to an object detection device that detects an object from a vehicle.

[0002] Vehicle-mounted object detection devices have the function of quickly detecting objects such as people and structures in the vehicle's environment and controlling the vehicle and issuing warnings, thereby contributing to safe vehicle driving. Object detection devices use various sensors such as radar, cameras, LiDAR (Light Detection and Ranging), and ultrasonic sensors.

[0003] In recent years, cameras have been increasingly being installed in specialized vehicles such as excavators, wheel loaders, bulldozers, and dump trucks used at construction or civil engineering sites. The introduction of cameras in specialized vehicles primarily detects people working on-site, contributing to the safe and secure operation of on-site work.

[0004] The cameras installed on special vehicles use wide-angle lenses, which allow them to detect the presence or absence of objects over a wide angle in the vicinity of the special vehicle, but they have the disadvantage of reduced object detection performance in the distant range, which is relatively far from the special vehicle. Therefore, by adding radar to the special vehicle equipped with a camera, it is possible to complement the object detection performance in the distant range of the special vehicle.

[0005] Patent Document 1 discloses an integrated radar-camera sensor module for vehicle installation, which is a sensor module that detects objects near the vehicle, in which the camera and radar are housed in a single common module housing, achieving compact integration.

[0006] JP 2014-51284 A

[0007] The integrated radar-camera sensor module described in Patent Document 1 has a structure in which the traveling direction of the electromagnetic waves emitted from the radar and the extending direction of the optical axis of the camera coincide with each other in order to achieve compact integration. The detection range in which radar provides good object detection performance is a close range. On the other hand, to complement the object detection performance of special vehicles in the long range, it is necessary to detect objects in a range farther than the radar. However, a structure in which the traveling direction of the electromagnetic waves emitted from the radar and the extending direction of the optical axis of the camera coincide with each other has the problem that it is not possible to achieve and optimize both the object detection performance of the radar in the short range and the object detection performance of the camera in the long range.

[0008] The present disclosure has been made in consideration of the above, and aims to obtain an object detection device that can improve object detection performance by optimizing the detection ranges of the radar and camera housed within the housing.

[0009] In order to solve the above-mentioned problems and achieve the object, the object detection device according to the present disclosure includes a housing attached to a vehicle, a radar housed in the housing and detecting radar object detection data, which is object detection data, based on a reflected signal of electromagnetic waves emitted from an object to be detected located in front of the housing, a camera housed in the housing and capturing an image of the object to be detected located in front of the housing to obtain image data of the object and detecting camera object detection data, which is object detection data, based on the image data, and a fusion processor housed in the housing and performing identity determination processing to determine whether the object detected by the radar and the object detected by the camera are the same, based on the radar object detection data and the camera object detection data. The direction of extension of the emission center axis of the electromagnetic waves emitted from the radar and the direction of extension of the optical axis of the camera are different in the height direction of the housing.

[0010] According to the present disclosure, it is possible to optimize the detection ranges of the radar and camera housed in the housing, thereby improving the object detection performance.

[0011] FIG. 1 is a perspective view of an object detection device according to a first embodiment, as seen from the front side; FIG. 2 is a first view showing the internal configuration of an object detection device according to a first embodiment, as seen from the side; FIG. 3 is a second view showing the internal configuration of an object detection device according to a first embodiment, as seen from the side; FIG. 4 is a diagram showing the configuration of an object detection device according to a first embodiment;

[0012] An object detection device according to an embodiment will be described in detail below with reference to the drawings.

[0013] First Embodiment Fig. 1 is a perspective view of an object detection device according to a first embodiment, as seen from the front side. Fig. 2 is a first diagram showing the internal configuration of the object detection device according to the first embodiment, as seen from the side. Fig. 2 shows a radar and a camera as seen when the inside of the object detection device is viewed from the direction of arrow A in Fig. 1, penetrating the side surface of the object detection device. Fig. 3 is a second diagram showing the internal configuration of the object detection device according to the first embodiment, as seen from the side. Fig. 3 shows a radar and a camera as seen when the inside of the object detection device is viewed from the direction of arrow B in Fig. 1, penetrating the side surface of the object detection device. Fig. 4 is a diagram showing the configuration of the object detection device according to the first embodiment.

[0014] The object detection device 100 is a fusion-type object detection device for mounting on a vehicle, equipped with a radar 20 and a camera 40, and detects objects using the radar 20 and the camera 40. The object detection device 100 is mounted on a vehicle (not shown). The output of the object detection device 100 is input to a vehicle control device 200 mounted on the vehicle and used to control the vehicle. In the following description, an object refers to an object that is desired to be detected when controlling the vehicle, and corresponds to a person, a vehicle, an obstacle, etc. Examples of obstacles include utility poles, walls, guardrails, shelf pillars, delineators, traffic lights, etc.

[0015] 1 to 3, the X direction corresponds to the width direction of object detection device 100. In FIGS. 1 to 3, the Y direction corresponds to the depth direction of object detection device 100. In FIGS. 1 to 3, the Z direction corresponds to the height direction of object detection device 100. In FIGS. 1 to 3, the front side of the paper in the Y direction corresponds to the front side of object detection device 100. In FIGS. 1 to 3, the back side of the paper, which is opposite the front side in the Y direction, corresponds to the back side of object detection device 100.

[0016] The object detection device 100 includes a housing 1 , a radome 2 , a lens cover 3 , a connector 4 , a radar 20 , a camera 40 , and a fusion processor 60 .

[0017] The housing 1 constitutes part of the outer shell of the object detection device 100, and protects the radar 20, the camera 40, and the fusion processor 60 from external environmental factors such as wind, rain, sunlight, dust, sand, and mud. The housing 1 has, for example, a rectangular parallelepiped shape with one face on the front side open, and is made of a material such as metal or resin.

[0018] The radome 2 transmits electromagnetic signals transmitted and received by the radar 20 to the radar 20 with low loss and low reflection. The radome 2 constitutes part of the outer shell of the object detection device 100 to protect the radar 20, the camera 40, and the fusion processor 60 from external environmental factors such as wind, rain, sunlight, dust, sand, and mud. The radome 2 has, for example, a rectangular parallelepiped shape with an opening on one rear surface and is made of resin. The radome 2 is attached to the front side of the housing 1 with the open rear surface facing the open front surface of the housing 1, thereby closing the opening on the front side of the housing 1. The front side of the radome 2 has an opening (not shown) through which a lens 7 and a holder 8 (described later) protrude toward the front side of the radome 2.

[0019] The lens cover 3 transmits the visible light received by the camera 40 to the inside of the radome 2 with low loss and low reflection. The lens cover 3 constitutes part of the outer shell of the object detection device 100 to protect the radar 20, the camera 40, and the fusion processor 60 from the external environment such as wind, rain, sunlight, dust, sand, and mud. The lens cover 3 has, for example, a hemispherical shape and is made of resin. The lens cover 3 is attached to the radome 2 from the front side of the radome 2 with its convex front side covering the opening on the front side of the radome 2, and closes the opening on the front side of the radome 2.

[0020] The housing 1, the radome 2, and the lens cover 3 form the outer shell of the object detection device 100. The housing 1 is the main component of the outer shell of the object detection device 100, and the housing 1, the radome 2, and the lens cover 3 can be broadly considered to be the housing of the object detection device 100.

[0021] The connector 4 is an interface component for connecting the object detection device 100 and the vehicle control device 200 via a cable (not shown). The vehicle control device 200 and the object detection device 100 are connected using the connector 4 as an interface.

[0022] The radar 20 emits electromagnetic waves toward objects such as people, vehicles, and obstacles, and receives reflected signals from the objects. The radar 20 receives reflected signals from both moving and stationary objects. In vehicle-mounted applications, the radar 20 operates, for example, using the FMCW (Frequency Modulated Continuous Wave) method or the FCM (Fast Chirp Modulation) method. The radar 20 is configured using high-frequency semiconductor components, power semiconductor components, substrates, quartz devices, chip components, antennas, and the like.

[0023] The radar 20 includes a radar antenna board 5, a radar control board 6, and a radar depression angle unit 11. The radar antenna board 5 and the radar control board 6 are integrated with an adhesive and fixed to the housing 1 by the radar depression angle unit 11. The radar antenna board 5 may be integrally molded with the radar control board 6.

[0024] The radar antenna board 5 includes a transmitting antenna 21 and a receiving antenna 22, and functions as a radar antenna in which the transmitting antenna 21 emits electromagnetic waves toward an object and the receiving antenna 22 receives a signal reflected from the object. The transmitting antenna 21 and the receiving antenna 22 may both be formed as multi-channel antennas in order to improve the horizontal resolution of the object. For example, the transmitting antenna 21 is configured with three transmitting channels. For example, the receiving antenna 22 is configured with four receiving channels.

[0025] The radar control board 6 functions as a radar control unit that controls the radar 20. The radar control board 6 includes a transmission circuit 23, a modulation control circuit 24, a local oscillation circuit 25, a reception circuit 26, and a radar signal processing circuit 31.

[0026] The transmission circuit 23 is a circuit for supplying power to the transmission antenna 21, and functions as a transmission section for amplifying the power supplied to the transmission antenna 21 to a desired power level.

[0027] The local oscillator circuit 25 is a circuit for generating a source signal of the electromagnetic wave used in the radar 20, and has a function as a local oscillator that multiplies an oscillation signal with high frequency accuracy using a quartz device.

[0028] The modulation control circuit 24 functions as a modulation control unit that modulates the oscillation signal by controlling the frequency of the oscillation signal generated by the local oscillation circuit 25. The modulation control circuit 24 uses an FMCW system or an FCM system.

[0029] The receiving circuit 26 amplifies the power received from the receiving antenna 22 with low noise and functions as a power signal converter that converts analog signals into digital signals.

[0030] The radar signal processing circuit 31 functions as a radar signal processing unit that detects the position and speed of an object by performing signal processing on a reflected signal, which is a received signal received by the radar 20. That is, the radar signal processing circuit 31 detects the position and speed of an object by performing signal processing on a received signal that is received by the receiving antenna 22 and then converted into a digital signal by the receiving circuit 26. The radar signal processing circuit 31 uses an MCU (Micro Control Unit), a CPU (Central Processing Unit), etc.

[0031] The radar signal processing circuit 31 includes a distance detection unit 27 , a speed detection unit 28 , a horizontal angle detection unit 29 , and a radar detection data storage unit 30 .

[0032] The distance detection unit 27, the speed detection unit 28, and the horizontal angle detection unit 29 perform a fast Fourier transformation (FFT) in the distance direction, the speed direction, and the horizontal angle direction, respectively, to detect the distance, the speed, and the horizontal angle of an object.

[0033] That is, the distance detection unit 27 detects the distance from the vehicle to the object based on the reflected signal received by the radar 20. The distance detection unit 27 sends the distance data to the speed detection unit .

[0034] The speed detection unit 28 detects the relative speed of the object with respect to the vehicle based on the reflected signal received by the radar 20. Hereinafter, the relative speed of the object with respect to the vehicle detected by the speed detection unit 28 will be referred to as "speed." The speed detection unit 28 sends the speed data and distance data to the horizontal angle detection unit 29.

[0035] The horizontal angle detection unit 29 detects the horizontal angle with respect to the traveling direction of the vehicle based on the reflected signal received by the radar 20. The horizontal angle with respect to the traveling direction of the vehicle is the angle between the traveling direction of the vehicle and the direction connecting the object and the vehicle. The horizontal angle of the object detected by the horizontal angle detection unit 29 is called horizontal angle data. The horizontal angle detection unit 29 calculates the relative position of the object with respect to the vehicle based on the horizontal angle data and distance data. The relative position of the object with respect to the vehicle calculated by the horizontal angle detection unit 29 can be said to be the position detected by the radar 20. The position detected by the radar 20 is the position of the object detected using the radar 20. The horizontal angle detection unit 29 stores the speed and position data in the radar detection data storage unit 30.

[0036] The radar detection data storage unit 30 is a storage device such as a memory that stores object speed data and object position data detected by the radar 20. The radar detection data storage unit 30 is connected to the subsequent fusion control board 10. The object speed data and object position data stored in the radar detection data storage unit 30 are read out by the subsequent fusion control board 10. The radar control board 6 may also transmit the object speed data and object position data to the fusion control board 10. The object position and speed detected by the radar 20 are called radar detection data.

[0037] The radar depression angle unit 11 is an angle adjustment component that provides the radar 20 with a radar depression angle θradar relative to the height direction of the housing 1 inside the housing 1. The radar depression angle θradar is the angle formed by the height direction of the radar 20 relative to the height direction of the housing 1. The radar depression angle θradar is the smaller of the angles formed by the height direction of the housing 1 and the height direction of the radar 20.

[0038] The direction of travel of the electromagnetic waves emitted from the transmitting antenna 21 at the center of the radar detection coverage area, which is the detection coverage area of ​​the radar 20, forms a predetermined angle, for example, 90 degrees, with the height direction of the radar 20. The height direction of the radar 20 is, for example, the same direction as the height direction of the radar antenna board 5 arranged inside the housing 1. In this case, when the height direction of the housing 1 and the height direction of the radar 20 are the same direction, the radar depression angle θ radar is 0 degrees.

[0039] As shown in Fig. 2 , the radar depression unit 11 adjusts the attitude of the radar 20 housed in the housing 1 to adjust the extension direction of the emission center axis 20a of the electromagnetic waves in the height direction of the housing 1 when the housing 1 is attached to the vehicle. That is, the radar depression unit 11 adjusts the attitude of the radar 20 housed in the housing 1 to adjust the extension direction of the emission center axis 20a of the electromagnetic waves in the up-down direction when the housing 1 is attached to the vehicle. The emission center axis 20a of the electromagnetic waves is the axis of the traveling direction of the electromagnetic waves emitted to the center of the radar detection coverage area, which is the detection coverage area of ​​an object by the radar 20. As shown in Fig. 2 , the radar depression unit 11 fixes the radar 20 to the housing 1 inside the housing 1 while applying a radar depression angle θ radar to the radar 20 in the height direction of the housing 1.

[0040] A radar height axis 20b, which is the axis in the height direction of the radar 20, is inclined by a radar depression angle θradar from a housing height axis 1a, which is the axis in the height direction of the housing 1. Furthermore, the extension direction of the electromagnetic wave emission central axis 20a is inclined by the radar depression angle θradar from the Y direction, i.e., the depth direction of the object detection device 100. In Fig. 2, the radar 20 is fixed in a position in which the extension direction of the electromagnetic wave emission central axis 20a is inclined downward by the radar depression angle θradar from the depth direction of the object detection device 100 in the YZ plane of the housing 1. Note that the radar depression unit 11 can also fix the radar 20 in a position in which the extension direction of the electromagnetic wave emission central axis 20a is inclined upward by the radar depression angle θradar from the depth direction of the object detection device 100 in the YZ plane of the housing 1.

[0041] The radar depression angle adjustment unit 11 can provide the radar 20 with a radar depression angle θ radar of any angle by changing the fixed angle of the radar 20 with respect to the housing 1. In other words, by changing the fixed angle of the radar 20 with respect to the housing 1, the radar depression angle adjustment unit 11 can adjust the extension direction of the emission center axis 20 a of the electromagnetic waves in any direction in the vertical direction when the housing 1 is attached to the vehicle.

[0042] The radar depression angle portion 11 is formed of a material such as metal or resin. The radar depression angle portion 11 may be fixed to the housing 1 with screws or adhesive, or may be integrally molded with the housing 1. The radar antenna board 5 and the radar control board 6 are integrated with adhesive and fixed to the housing 1 by the radar depression angle portion 11.

[0043] The camera 40 captures images of objects around the vehicle to obtain image data of the objects. The camera 40 includes a lens 7, a holder 8, a camera control board 9, and a camera depression unit 12.

[0044] The holder 8 is a lens holding unit to which the lens 7, which has undergone optical focus adjustment, is fixed, and is mounted on the camera control board 9. The holder 8 has, for example, a cylindrical shape, and is fixed to the housing 1 by a camera depression angle part 12. The lens 7 and the holder 8 are fixed with an adhesive or screws. The holder 8 and the camera control board 9 are fixed with an adhesive or screws.

[0045] As shown in FIG. 3, a holder central axis 8a, which is the central axis of the holder 8, is coaxial with an optical axis 40a of the camera, that is, an optical axis 7a of the lens.

[0046] The camera control board 9 functions as a camera control unit that detects the position and type of an object based on image data captured by the image sensor 41. The image data captured by the camera 40 is also called captured image data. The detection data for the position and type of an object detected by the camera 40 is called camera object detection data. The camera control board 9 includes the image sensor 41, an image processing circuit 42, and a camera recognition processing circuit 49.

[0047] 3 , the camera control board 9 is disposed in a position inclined by a camera depression angle θcamera (described later) from a housing height axis 1a, which is the axis in the height direction of the housing 1. That is, the camera control board height axis 9a, which is the axis in the height direction of the camera control board 9, is inclined by the camera depression angle θcamera from the housing height axis 1a. The holder central axis 8a, the camera optical axis 40a, and the lens optical axis 7a are, for example, perpendicular to the camera control board 9. In this case, the holder central axis 8a, the camera optical axis 40a, and the lens optical axis 7a are inclined by the camera depression angle θcamera from the Y direction, i.e., the depth direction of the object detection device 100.

[0048] The imaging element 41 converts the optical signal focused by the lens 7 into electrical digital data to capture an image within the range of the angle of view of the lens 7. The imaging element 41 sends the digital data to an image processing circuit 42. The imaging element 41 may be a sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

[0049] The image processing circuit 42 has a function of generating a color captured image by performing image quality adjustment processing such as white balance and demosaicing processing on the digital data converted by the image sensor 41. Note that the digital data converted by the image sensor 41 may be replaced with a non-color captured image such as monochrome. The image processing circuit 42 sends color captured image data, which is data of the generated color captured image, to the camera recognition processing circuit 49.

[0050] The camera recognition processing circuit 49 functions as a camera recognition processing unit that detects the position and type of an object by performing signal processing on the color captured image data generated by the image processing circuit 42. An MCU, a CPU, a GPU (Graphics Processing Unit), or the like is used for the camera recognition processing circuit 49. The camera recognition processing circuit 49 has an object recognition unit 43, an object position detection unit 44, a camera object detection data storage unit 45, a camera parameter storage unit 46, a position conversion table storage unit 47, and an object recognition feature database 48.

[0051] The object recognition unit 43 acquires the color captured image data generated by the image processing circuit 42, and refers to the object recognition feature database 48 to recognize objects such as people and vehicles from the color captured image data generated by the image processing circuit 42, detects the object type, and creates object type data. The object recognition unit 43 also assigns pixel coordinates of the recognized object to the color captured image data. The object recognition unit 43 sends the color captured image data generated by the image processing circuit 42, the object type data, and the object pixel coordinate data to the object position detection unit 44. The object recognition unit 43 also sends the object type data to the camera object detection data storage unit 45 for storage.

[0052] The object recognition feature database 48 is a database of features obtained by machine learning or deep learning.

[0053] The object position detection unit 44 detects the position of an object by converting the pixel coordinates of the object recognized by the object recognition unit 43 into object position data by referring to a position conversion table (described later) stored in the position conversion table storage unit 47. The pixel coordinates of the object are the coordinates of the pixel of the object on the image. The object position detection unit 44 stores the object position data in the camera object detection data storage unit 45. The position detected by the object position detection unit 44 is the position of the object detected using the camera 40.

[0054] The camera object detection data storage unit 45 stores the object type data sent from the object recognition unit 43 and the position data sent from the object position detection unit 44. The camera object detection data storage unit 45 is connected to the fusion control board 10 at the subsequent stage.

[0055] The object type data and position data stored in the camera object detection data storage unit 45 are read out by the subsequent fusion control board 10. The camera recognition processing circuit 49 may also transmit the object type data and position data to the fusion control board 10.

[0056] The camera parameter storage unit 46 is a storage device such as a memory that stores camera internal parameters and camera external parameters.

[0057] The camera internal parameters are parameters determined by the components of the camera 40, such as the lens 7 and the image sensor 41. The camera internal parameters include the focal length, the image data size, the pixel center, the lens distortion coefficient, etc. The camera internal parameters are adjusted for each individual camera 40 during product shipping inspection of the camera 40, and are stored in the camera parameter storage unit 46.

[0058] The camera extrinsic parameters are parameters determined by how the camera 40 is attached to the vehicle. The camera extrinsic parameters include the attachment height of the camera 40 after the camera 40 is attached to the vehicle, the attachment attitude angle of the camera 40, and the like. The camera extrinsic parameters are adjusted for each individual object detection device 100 during product shipping inspection of the object detection device 100, and are stored in the camera parameter storage unit 46. Note that the camera extrinsic parameters may also be adjusted for each individual camera 40 when the camera 40 is attached to the vehicle, and be stored in the camera parameter storage unit 46.

[0059] The position conversion table storage unit 47 is a storage device such as a memory that stores a position conversion table. The position conversion table is a data table that indicates the correspondence between the pixel coordinates of an object and the position of the object. In other words, the position conversion table is a data table for converting the pixel coordinates of an object into the position of the object.

[0060] Camera depression angle unit 12 is an angle adjustment component that provides camera 40 with a camera depression angle θcamera relative to the height direction of housing 1 inside housing 1. Camera depression angle θcamera is the angle formed by the height direction of camera 40 relative to the height direction of housing 1. Camera depression angle θcamera is the smaller of the angles formed by the height direction of housing 1 and the height direction of camera 40.

[0061] The camera optical axis 40a forms a predetermined angle, for example, 90 degrees, with the height direction of the camera 40. The height direction of the camera 40 is, for example, the same direction as the height direction of the camera control board 9 disposed inside the housing 1. In this case, when the height direction of the housing 1 and the height direction of the camera 40 are the same direction, the camera depression angle θcamera is 0 degrees.

[0062] As shown in FIG. 3 , the camera depression unit 12 adjusts the attitude of the camera 40 stored in the housing 1 to adjust the extension direction of the camera's optical axis 40a in the height direction of the housing 1 when the housing 1 is attached to a vehicle. That is, the camera depression unit 12 adjusts the attitude of the camera 40 stored in the housing 1 to adjust the extension direction of the camera's optical axis 40a in the up-down direction when the housing 1 is attached to a vehicle. The camera's optical axis 40a faces the center of the camera detection coverage area, which is the detection coverage area of ​​the camera 40. As shown in FIG. 3 , the camera depression unit 12 fixes the camera 40 to the housing 1 while applying a camera depression angle θcamera to the camera 40 in the height direction of the housing 1 inside the housing 1.

[0063] Camera height axis 40b, which is the axis in the height direction of camera 40, is inclined by camera depression angle θcamera from housing height axis 1a, which is the axis in the height direction of housing 1. Furthermore, the extension direction of camera optical axis 40a is inclined by camera depression angle θcamera from the Y direction, i.e., the depth direction of object detection device 100. In Fig. 3, camera 40 is fixed in a position in which camera optical axis 40a is inclined downward by camera depression angle θcamera from the depth direction of object detection device 100 in the YZ plane of housing 1.

[0064] Camera depression angle unit 12 can provide camera 40 with any desired camera depression angle θcamera by changing the fixed angle of camera 40 relative to housing 1. In other words, camera depression angle unit 12 can adjust the extension direction of camera optical axis 40a in any desired vertical direction when housing 1 is attached to a vehicle by changing the fixed angle of camera 40 relative to housing 1.

[0065] Camera depression section 12 is formed from a material such as metal or resin. Camera depression section 12 may be fixed to housing 1 with screws or adhesive, or may be integrally molded with housing 1. Camera control board 9 is fixed to housing 1 by camera depression section 12.

[0066] The fusion processor 60 includes a fusion control board 10 and a fusion mounting portion 13 .

[0067] The fusion mounting part 13 is a mounting part for mounting the fusion control board 10 inside the housing 1. The fusion mounting part 13 is formed of a material such as metal or resin. The fusion mounting part 13 may be fixed to the housing 1 with screws or adhesive, or may be molded integrally with the housing 1.

[0068] The fusion control board 10 includes a fusion processing circuit 63. The fusion control board 10 uses an MCU, a CPU, or the like.

[0069] The fusion processing circuit 63 is composed of an object identity determination unit 61 and a fusion object detection data storage unit 62 .

[0070] The object identity determination unit 61 reads speed and position data from the radar signal processing circuit 31 of the radar control board 6. The object identity determination unit 61 also reads object type and position data from the camera recognition processing circuit 49 of the camera control board 9. The object identity determination unit 61 performs object identity determination processing based on the position data of the object detected using the radar 20 and the position data of the object detected using the camera 40. That is, the object identity determination unit 61 determines whether the object detected using the radar 20 and the object detected using the camera 40 are the same or not based on the position data of the object detected using the radar 20 and the position data of the object detected using the camera 40.

[0071] If the object identity determination process is successful, i.e., if the object detected by the radar 20 and the object detected by the camera 40 are determined to be the same by the identity determination process, the object identity determination unit 61 associates the object position data and object speed data detected by the radar 20 with the object type data detected by the camera 40. The correspondence data in which the object position data and object speed data detected by the radar 20 are associated with the object type data detected by the camera 40 is referred to as first correspondence data. The object identity determination unit 61 transmits the object position data and object speed data detected by the radar 20 and the object type data detected by the camera 40, which are associated with each other, to the fusion object detection data storage unit 62 for storage.

[0072] On the other hand, if the object identity determination process fails, that is, if the object detected by the radar 20 and the object detected by the camera 40 are determined not to be the same by the identity determination process, the object identity determination unit 61 transmits and stores second correspondence data in which the object position data and object speed data detected by the radar 20 are associated with each other, or third correspondence data in which the object position data and object type data detected by the camera 40 are associated with each other, to the fusion object detection data storage unit 62. The fusion object detection data storage unit 62 transmits the stored object position data, object speed data, and object type data to the vehicle control device 200 to use for vehicle control.

[0073] The first correspondence data, the second correspondence data, or the third correspondence data stored in the fusion object detection data storage unit 62 is data for vehicle control. Note that Fig. 4 illustrates a case where the data stored in the fusion object detection data storage unit 62 is data on the object position, data on the object speed, and data on the object type.

[0074] The vehicle control device 200 reads data from the fusion object detection data storage unit 62 and controls the operation of the vehicle using the read data. Specifically, if the object identity determination unit 61 succeeds in the identity determination, the vehicle control device 200 controls the operation of the vehicle based on first correspondence data in which object position data and object speed data detected by the radar 20 are associated with object type data detected by the camera 40. Note that if the object identity determination unit 61 succeeds in the identity determination, the object detection device 100 may transmit the first correspondence data to the vehicle control device 200.

[0075] In addition, if the object identity determination unit 61 fails to make an identity determination, the vehicle control device 200 controls the operation of the vehicle based on second correspondence data in which the object position data detected by the radar 20 and the object speed data are correlated with each other, or based on third correspondence data in which the object position data detected by the camera 40 and the object type data are correlated with each other.

[0076] Next, a description will be given of a processing procedure executed by the object detection device 100 according to the first embodiment. Fig. 5 is a flowchart showing the processing procedure executed by the object detection device according to the first embodiment.

[0077] In step S10, object detection device 100 starts a frame of object detection processing, and then proceeds to steps S20 and S40.

[0078] In step S20, the radar 20, to which the radar depression angle θ radar has been given by the radar depression angle adjustment unit 11, detects an object. Specifically, the radar antenna board 5 of the radar 20 emits electromagnetic waves toward the object to be detected and acquires a reflected signal from the object, which is received data. The radar antenna board 5 transmits the received data to the radar control board 6. In the radar control board 6, the receiving circuit 26 receives the received data and transmits the received data to the radar signal processing circuit 31. Then, the process proceeds to step S30.

[0079] In step S30, the radar signal processing circuit 31 detects the position and speed of the object based on the received data. The radar signal processing circuit 31 stores the detected object position and speed data in the radar detection data storage unit 30. Then, the process proceeds to step S60.

[0080] In step S40, camera 40, to which camera depression angle θcamera has been given by camera depression angle setting unit 12, detects an object. Specifically, image sensor 41 of camera 40 captures an image of the object to generate digital data of the image of the object. Image sensor 41 transmits the digital data to image processing circuit 42. Image processing circuit 42 performs various processes on the digital data to generate a color captured image, and transmits the generated color captured image data to camera recognition processing circuit 49. Then, the process proceeds to step S50.

[0081] In step S50, the camera recognition processing circuit 49 recognizes the object by performing signal processing on the color captured image data generated by the image processing circuit 42, and detects the object's position and object type. The camera recognition processing circuit 49 stores the detected object's position and object type data in the camera object detection data storage unit 45. Then, the process proceeds to step S60.

[0082] The processes of steps S20 and S30 and the processes of steps S40 and S50 are executed in parallel.

[0083] In step S60, the object identity determination unit 61 of the fusion processing circuit 63 of the fusion control board 10 in the fusion processor 60 performs object identity determination processing based on the position data detected by the radar 20 and the position data detected by the camera 40. Specifically, the object identity determination unit 61 of the fusion processing circuit 63 reads object position and speed data from the radar detection data storage unit 30. The object identity determination unit 61 also reads object position and object type data from the camera object detection data storage unit 45. The object identity determination unit 61 performs identity determination processing based on the read data. Then, the process proceeds to step S70.

[0084] In step S70, the object identity determination unit 61 determines whether the identity determination has been successful. If it is determined that the identity determination has been successful, the answer is Yes in step S70, and the process proceeds to step S80. If it is determined that the identity determination has not been successful, the answer is No in step S70, and the process proceeds to step S90.

[0085] In step S80, the object identity determination unit 61 outputs first correspondence data that associates the object position data and object speed data detected by the radar 20 with the object type data detected by the camera 40 to the fusion object detection data storage unit 62. Then, the process proceeds to step S100.

[0086] In step S90, the object identity determination unit 61 outputs second correspondence data that associates object position data and object speed data detected by the radar 20, or third correspondence data that associates object position data and object type data detected by the camera 40, to the fusion object detection data storage unit 62. Then, the process proceeds to step S100.

[0087] In step S100, fusion object detection data storage unit 62 stores the data output from object identity determination unit 61 in step S80 or step S90. That is, fusion object detection data storage unit 62 stores the first correspondence data transmitted from object identity determination unit 61 in step S80. Alternatively, fusion object detection data storage unit 62 stores the second correspondence data or third correspondence data output from object identity determination unit 61 in step S90. Then, the process proceeds to step S110.

[0088] In step S110, the fusion processor 60 outputs the object position, speed, and object type data stored in the fusion object detection data storage unit 62 to the vehicle control device 200. That is, the fusion processing circuit 63 of the fusion control board 10 of the fusion processor 60 outputs the object position, speed, and object type data stored in the fusion object detection data storage unit 62 in the processing of step S100 to the vehicle control device 200.

[0089] Specifically, if the identity determination fails, the fusion processing circuit 63 outputs second correspondence data that associates object position data and object speed data detected by the radar 20, or third correspondence data that associates object position data and object type data detected by the camera 40, to the vehicle control device 200. On the other hand, if the identity determination is successful, the fusion processing circuit 63 outputs first correspondence data that associates object position data and object speed data detected by the radar 20 with object type data detected by the camera 40, to the vehicle control device 200. Then, the process proceeds to step S120.

[0090] In step S120, the vehicle control device 200 controls the operation of the vehicle using the data acquired from the fusion processor 60. Specifically, if the identity determination has failed, the vehicle control device 200 controls the vehicle based on second correspondence data that associates object position data and object speed data detected by the radar 20, or third correspondence data that associates object position data and object type data detected by the camera 40. On the other hand, if the identity determination has succeeded, the vehicle control device 200 controls the vehicle based on first correspondence data that associates object position data and object speed data detected by the radar 20 with object type data detected by the camera 40. Then, the process proceeds to step S130.

[0091] In step S130, object detection device 100 proceeds to processing the next frame of the object detection process, and then repeats the processes from step S20 to step S130 in FIG.

[0092] Next, features of the object detection device 100 according to the first embodiment will be described. Fig. 6 is a schematic diagram showing an example of a usage form of the object detection device according to the first embodiment. In Fig. 6, the detection coverage area of ​​an object by the camera 40 is referred to as a camera detection coverage area 110, the detection coverage area of ​​an object by the radar 20 of the object detection device 100 is referred to as a radar detection coverage area 120, a special vehicle equipped with the object detection device 100 is referred to as a special vehicle 300, a person in a nearby range that is relatively close to the special vehicle 300 is referred to as a nearby person 310, and a person in a distant range that is relatively far from the special vehicle 300 is referred to as a distant person 320.

[0093] Object detection device 100 is attached to special vehicle 300 with housing 1 at a predetermined height and in a predetermined posture, i.e., with the angle between the height direction of object detection device 100 and the vertical direction set to a predetermined angle. Object detection device 100 includes radar depression unit 11 and camera depression unit 12 as described above.

[0094] As a result, the radar 20 is fixed to the housing 1 with a radar depression angle θ radar given to it in the height direction of the housing 1 inside the housing 1. The radar depression unit 11 adjusts the extension direction of the central emission axis 20 a of the electromagnetic waves emitted from the radar 20 in the up-down direction when the housing 1 is attached to the vehicle. Specifically, the radar 20 is adjusted by the radar depression unit 11 so that the extension direction of the central emission axis 20 a of the electromagnetic waves emitted from the radar 20 in the up-down direction faces a direction suitable for detecting detection data of an object in a radar detection coverage area 120, which is an object detection coverage area of ​​the radar 20 and is a region relatively far from the vehicle.

[0095] Furthermore, camera 40 is fixed to housing 1 with a camera depression angle θcamera given to it in the height direction of housing 1 inside housing 1. The extending direction of camera optical axis 40a of camera 40 when housing 1 is attached to a vehicle is adjusted in the up-down direction by camera depression unit 12. Specifically, camera 40 is adjusted by camera depression unit 12 so that the extending direction of camera optical axis 40a in the up-down direction faces a direction suitable for detecting object detection data in camera detection coverage area 110, which is an object detection coverage area of ​​camera 40 and is a region relatively close to the vehicle.

[0096] That is, the attitude of the camera 40 is adjusted by the camera depression unit 12 so that the camera 40 faces in a direction suitable for detecting a nearby person 310 in the camera detection coverage area 110. The attitude of the radar 20 is adjusted by the radar depression unit 11 so that the camera 40 faces in a direction suitable for detecting a distant person 320 in the radar detection coverage area 120.

[0097] According to the object detection device 100 of the first embodiment described above, an object detection device is realized that includes a housing attached to a vehicle, a radar housed in the housing and that detects radar object detection data, which is object detection data, based on a reflected signal of electromagnetic waves emitted at an object to be detected located on the front side of the housing, a camera housed in the housing and that captures an image of the object to be detected located on the front side of the housing to obtain image data of the object, and detects camera object detection data, which is object detection data, based on the image data, and a fusion processor housed in the housing and that performs identity determination processing to determine whether the object detected by the radar and the object detected by the camera are the same, based on the radar object detection data and the camera object detection data, and in which the extension direction of the central axis of the electromagnetic waves emitted from the radar and the extension direction of the optical axis of the camera differ in the height direction of the housing.

[0098] In the object detection device 100 configured as described above, the camera 40 uses a wide-angle lens to enable it to detect objects over a wide angle in the camera detection coverage area 110, which is a nearby area relatively close to the special vehicle 300, and can reliably detect nearby people 310 within the nearby area.

[0099] Furthermore, in the object detection device 100, the radar 20 is capable of detecting objects in the radar detection coverage area 120, which is a distant range area that is relatively far from the special vehicle 300, and can reliably detect distant people 320 in the distant range.

[0100] Therefore, by providing radar depression section 11 and camera depression section 12 within housing 1, object detection device 100 makes it possible for the direction in which emission center axis 20a of electromagnetic waves emitted from radar 20 and the direction in which optical axis 40a of the camera extend to differ in the up-down direction when housing 1 is attached to special vehicle 300. In other words, by providing radar depression section 11 and camera depression section 12 within housing 1, object detection device 100 makes it possible for the direction in which emission center axis 20a of electromagnetic waves emitted from radar 20 and the direction in which optical axis 40a of the camera extend to not coincide in the up-down direction when housing 1 is attached to special vehicle 300.

[0101] Therefore, the object detection device 100 can detect objects in a nearby range that is relatively close to the special vehicle 300 using the camera 40, and can detect objects in a distant range that is relatively far from the special vehicle 300 using the radar 20. This allows the object detection device 100 to reliably detect objects in both the nearby range and the distant range of the special vehicle 300, and an object detection device with high performance for detecting objects around the special vehicle 300 can be provided.

[0102] Note that object detection device 100 may include at least one of radar depression unit 11 and camera depression unit 12. For example, assume that camera optical axis 40a is disposed perpendicular to housing height axis 1a, which is the axis in the height direction of housing 1, and the front surfaces of housing 1 and radome 2.

[0103] When the object detection device 100 includes only the radar depression angle unit 11, the housing 1 of the object detection device 100 is attached to the special vehicle 300 in an attitude suitable for the camera 40 to detect an object in the camera detection coverage area 110. Then, inside the housing 1, the radar depression angle θ radar is applied to the radar 20 by the radar depression angle unit 11 so that the attitude of the radar antenna board 5 of the radar 20 becomes an attitude suitable for the radar 20 to detect an object in the radar detection coverage area 120.

[0104] That is, inside the housing 1 attached to the special vehicle 300, the radar depression angle θ radar is applied to the radar 20 by the radar depression angle unit 11 so that the attitude of the radar antenna board 5 of the radar 20 is suitable for the radar 20 to detect objects in the radar detection coverage area 120.

[0105] In this case, the front of the housing 1 attached to the special vehicle 300 faces the camera detection coverage area 110, which is an area closer to the special vehicle 300 than the radar detection coverage area 120. Therefore, in the YZ plane of the housing 1 attached to the special vehicle 300, the radar antenna board 5 of the radar 20 is fixed by the radar depression angle part 11 in a position tilted upward by the radar depression angle θ radar from the housing height axis 1 a, which is the axis in the height direction of the housing 1.

[0106] This provides the same effect as when the object detection device 100 is equipped with the radar depression angle unit 11 and the camera depression angle unit 12.

[0107] Furthermore, for example, when object detection device 100 includes only camera depression angle portion 12, housing 1 of object detection device 100 is attached to special vehicle 300 in a posture suitable for detection of objects in radar detection coverage area 120 by radar 20. In this case, the shape of camera depression angle portion 12 is designed so that, inside housing 1, the extension direction of camera optical axis 40a faces a direction suitable for detection of objects in camera detection coverage area 110 by camera 40.

[0108] In this case, the front of the housing 1 attached to the special vehicle 300 faces the radar detection coverage area 120, which is an area farther from the special vehicle 300 than the camera detection coverage area 110. Therefore, in the YZ plane of the housing 1 attached to the special vehicle 300, the camera optical axis 40a is fixed by the camera depression angle unit 12 in an attitude in which it is tilted downward by the camera depression angle θcamera from the housing height axis 1a, which is the axis in the height direction of the housing 1.

[0109] This provides the same effect as when the object detection device 100 is equipped with the radar depression angle unit 11 and the camera depression angle unit 12.

[0110] As described above, the object detection device 100 according to the first embodiment has the effect of optimizing the detection ranges of the radar 20 and the camera 40 housed in the housing 1, thereby improving the object detection performance.

[0111] Next, a hardware configuration of the object detection device 100 according to the first embodiment will be described. Various functions of the object detection device 100 are realized by a processing circuit. The processing circuit may be a processor and memory that executes a program stored in a memory, or may be dedicated hardware.

[0112] 7 is a diagram illustrating a configuration example of a processing circuit included in the object detection device according to the first embodiment, when the processing circuit is realized by a processor and a memory. The processing circuit 90 illustrated in FIG. 7 corresponds to each of the circuits included in the radar control board 6 included in the radar 20, the circuits included in the camera control board 9 included in the camera 40, and the circuits included in the fusion control board 10 included in the fusion processor 60. Each function of the processing circuit 90 according to the first embodiment is realized by the processing circuit 90. The processing circuit 90 illustrated in FIG. 7 includes a processor 91 and a memory 92.

[0113] When the processing circuit 90 is configured with a processor 91 and a memory 92, each function of the processing circuit 90 is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a data processing program and stored in the memory 92. In the processing circuit 90, each function is realized by the processor 91 reading and executing the data processing program stored in the memory 92. That is, the processing circuit 90 includes the memory 92 for storing a data processing program that results in the processing of the object detection device 100 being executed. This data processing program can also be said to be a program that causes the object detection device 100 to execute each function realized by the processing circuit 90. This data processing program may be provided by a storage medium in which the data processing program is stored, or by other means such as a communication medium.

[0114] Here, the processor 91 is, for example, a CPU, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor), etc. The memory 92 is, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc).

[0115] FIG. 8 is a diagram illustrating an example of a processing circuit included in the object detection device according to the first embodiment, configured with dedicated hardware. The processing circuit 93 illustrated in FIG. 8 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The processing circuit 93 may be partially implemented with dedicated hardware and partially implemented with software or firmware. In this way, the processing circuit 93 can realize each of the above-described functions by dedicated hardware, software, firmware, or a combination thereof.

[0116] In the object detection device 100, the circuit of the radar control board 6 included in the radar 20, the circuit of the camera control board 9 included in the camera 40, and the circuit of the fusion control board 10 included in the fusion processor 60 may each be realized by separate processing circuits. In this case, one or more of the circuit of the radar control board 6 included in the radar 20, the circuit of the camera control board 9 included in the camera 40, and the circuit of the fusion control board 10 included in the fusion processor 60 are realized by the above-mentioned processing circuit 90 or processing circuit 93. In this case, the object detection device 100 has multiple processing circuits.

[0117] The configurations shown in the above embodiments are merely examples, and may be combined with other known technologies, and parts of the configurations may be omitted or modified without departing from the spirit of the invention.

[0118] REFERENCE SIGNS LIST 1 Housing, 1a Housing height axis, 2 Radome, 3 Lens cover, 4 Connector, 5 Radar antenna board, 6 Radar control board, 7 Lens, 7a Lens optical axis, 8 Holder, 8a Holder central axis, 9 Camera control board, 9a Camera control board height axis, 10 Fusion control board, 11 Radar depression angle portion, 12 Camera depression angle portion, 13 Fusion mounting portion, 20 Radar, 20a Electromagnetic wave emission central axis, 20b Radar height axis, 21 Transmitting antenna, 22 Receiving antenna, 23 Transmitting circuit, 24 Modulation control circuit, 25 Local oscillator circuit, 26 Receiving circuit, 27 Distance detection unit, 28 Speed ​​detection unit, 29 Horizontal angle detection unit, 30 Radar detection data storage unit, 31 Radar signal processing circuit, 40 Camera, 40a Camera optical axis, 40b Camera height axis, 41 Image pickup element, 42 image processing circuit, 43 object recognition unit, 44 object position detection unit, 45 camera object detection data storage unit, 46 camera parameter storage unit, 47 position conversion table storage unit, 48 object recognition feature database, 49 camera recognition processing circuit, 60 fusion processor, 61 object identity determination unit, 62 fusion object detection data storage unit, 63 fusion processing circuit, 90, 93 processing circuit, 91 processor, 92 memory, 100 object detection device, 110 camera detection coverage, 120 radar detection coverage, 200 vehicle control device, 300 special vehicle, 310 nearby person, 320 distant person, θcamera camera depression angle, θradar radar depression angle.

Claims

1. An object detection device comprising: a housing to be attached to a vehicle; a radar housed in the housing and configured to detect radar object detection data, which is detection data of an object to be detected located in front of the housing, based on a reflected signal of electromagnetic waves emitted at the object; a camera housed in the housing and configured to capture an image of the object to be detected located in front of the housing, obtain image data of the object, and detect camera object detection data, which is detection data of the object, based on the image data; and a fusion processor housed in the housing and configured to perform identity determination processing to determine whether the object detected by the radar and the object detected by the camera are the same, based on the radar object detection data and the camera object detection data; wherein the extension direction of the emission center axis of the electromagnetic waves emitted from the radar and the extension direction of the optical axis of the camera are different in the height direction of the housing.

2. The object detection device according to claim 1, wherein the radar detects detection data of the object in a radar detection coverage area, which is an area within the radar's detection coverage area that is relatively far from the vehicle, and the camera detects detection data of the object in a camera detection coverage area, which is an area within the camera's detection coverage area that is relatively close to the vehicle.

3. An object detection device as described in claim 1 or 2, characterized in that it is provided with a radar depression unit that adjusts the attitude of the radar housed in the housing to adjust the extension direction of the emission center axis of the electromagnetic waves in the height direction of the housing.

4. An object detection device as described in any one of claims 1 to 3, characterized in that it is provided with a camera depression unit that adjusts the attitude of the camera housed in the housing to adjust the extension direction of the optical axis of the camera in the height direction of the housing.

Images