Radar image object detection device and radar image object detection program

Abstract

To integrate radar objects and image objects to output highly accurate distances and azimuths.SOLUTION: A radar image object detection device 3 is provided, comprising: a radar object detection unit 31 configured to acquire a radar signal, detect a radar object based on the radar signal, and measure the distance to and azimuth of the radar object; an image object detection unit 32 configured to acquire a camera image, detect an image object based on the camera image, and measure the distance to and azimuth of the image object; a radar image object integration unit 33 configured to integrate the radar object and the image object when a difference between the distance to the radar object and the distance to the image object is equal to or less than a given difference and a difference between the azimuth of the image object and the azimuth of the radar object is equal to or less than a given difference; and a collision risk determination unit 36 configured to determine the risk of collision with each of multiple objects obtained by integrating radar objects and image objects on the basis of velocity vectors.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technology for detecting a target based on a radar signal and a camera image. [Background technology]

[0002] A technology for detecting a target based on a radar signal and a camera image is disclosed in Patent Document 1 etc. In Patent Document 1, a radar signal is plotted on a camera image, an edge analysis is performed in the vicinity of the radar signal, and a quadrangular pyramid frustum that borders the measurement target (e.g., a vehicle, etc.) is detected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-211402 A [Patent Document 2] JP 2019-211403 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, a target detected based on a radar signal (hereinafter referred to as a radar target) and a target detected based on a camera image (hereinafter referred to as an image target) are not integrated. Therefore, in Patent Document 1, as the distance of the integrated radar target and image target, it is not possible to output a highly accurate distance of the radar target, instead of a low-accuracy distance of the image target. And, in Patent Document 1, as the orientation of the integrated radar target and image target, it is not possible to output a highly accurate orientation of the image target, instead of a low-accuracy orientation of the radar target.

[0005] Therefore, in order to solve the above problem, an object of the present disclosure is to output highly accurate distance and direction as the distance and direction of a combination of a radar target and an image target. [Means for solving the problem]

[0006] In order to solve the above problem, when the difference between the distance of the radar target and the distance of the image target is equal to or smaller than a predetermined difference and the difference between the orientation of the image target and the orientation of the radar target is equal to or smaller than a predetermined difference, the radar target and the image target are integrated. Therefore, as the distance of the integrated result of the radar target and the image target, a highly accurate distance of the radar target can be output instead of a low accuracy distance of the image target. And, as the orientation of the integrated result of the radar target and the image target, a highly accurate orientation of the image target can be output instead of a low accuracy orientation of the radar target.

[0007] Then, the radar target and the image target are tracked to calculate a velocity vector. Furthermore, for each of the radar target and the image target, a collision risk is determined based on the velocity vector. Thus, for each of the radar target and the image target, a collision risk can be determined based on the degree of approach within a predetermined period.

[0008] Specifically, the present disclosure provides a radar image target detection device comprising: a radar object detection unit that acquires a radar signal, detects a radar object based on the radar signal, and measures a distance and an orientation of the radar object; an image object detection unit that acquires a camera image, detects an image object based on the camera image, and measures an orientation and distance of the image object; a radar image object integration unit that integrates the radar object and the image object when a difference between the distance of the radar object and the distance of the image object is equal to or less than a predetermined difference and a difference between the orientation of the image object and the orientation of the radar object is equal to or less than a predetermined difference; a radar image object tracking unit that tracks the integrated radar object and the image object and calculates a velocity vector; and a collision risk determination unit that determines a collision risk for a plurality of integrated radar objects and image objects based on the velocity vector.

[0009] The present disclosure also relates to a radar image target detection program for causing a computer to execute a radar target detection step of acquiring a radar signal, detecting a radar target based on the radar signal, and measuring a distance and a direction of the radar target; an image target detection step of acquiring a camera image, detecting an image target based on the camera image, and measuring a direction and a distance of the image target; a radar image target integration step of integrating the radar target and the image target when a difference between the distance of the radar target and the distance of the image target is equal to or less than a predetermined difference and a difference between the direction of the image target and the direction of the radar target is equal to or less than a predetermined difference; a radar image target tracking step of tracking the integrated radar target and the image target and calculating a velocity vector; and a collision risk determination step of determining a collision risk for a plurality of integrated radar targets and image targets based on the velocity vector.

[0010] According to these configurations, it is possible to integrate the radar target and the image target, and to determine the collision risk for a plurality of integrated radar targets and image targets based on the degree of approach within a predetermined period of time.

[0011] The present disclosure also relates to a radar image target detection device, wherein the collision risk determination unit calculates a minimum value of absolute differences in position vectors within a predetermined period for a plurality of integrated objects, each of which is the radar target and the image target, based on the velocity vector, and determines that the risk of collision is high if the minimum value is smaller than a threshold value, and determines that the risk of collision is low if the minimum value is equal to or greater than the threshold value.

[0012] According to this configuration, the risk of collision can be determined for a plurality of integrated radar targets and image targets based on the degree of approach within a predetermined period of time.

[0013] The present disclosure also relates to a radar image target detection device, characterized in that the collision risk determination unit determines the collision risk for multiple objects each obtained by integrating the radar target and the image target, based on the type and / or size.

[0014] According to this configuration, the risk of collision can be determined with high accuracy for a plurality of integrated radar targets and image targets, based on factors other than the degree of approach within a predetermined period of time.

[0015] The present disclosure also relates to a radar image target detection device, characterized in that the collision risk determination unit determines the risk of collision with a structure for an integrated result of the radar target and the image target, based on the distance and direction of the structure detected by a radar transmitting / receiving device and a camera imaging device, or based on three-dimensional map information of the structure that the collision risk determination unit has in advance.

[0016] According to this configuration, the risk of collision with a structure can be determined for an integrated radar target and image target after detecting the structure with the radar transmitting / receiving device and the camera imaging device, or without detecting the structure with the radar transmitting / receiving device and the camera imaging device.

[0017] The present disclosure also relates to a radar image target detection device, characterized in that, when the radar image target integration unit does not detect the radar target that can be integrated with the image target, it detects a cluster of radar signal points having an orientation and distance closest to the orientation and distance of the image target, and integrates the cluster of radar signal points with the image target.

[0018] According to this configuration, even when a radar target that can be combined with the image target is not detected, the cluster of radar signal points and the image target can be combined.

[0019] The present disclosure also relates to a radar image target detection device, characterized in that, when the image target that can be integrated with the radar target is not detected, the radar image target integration unit redetects a redetected image target having a distance and direction close to the distance and direction of the radar target, and integrates the redetected image target with the radar target.

[0020] According to this configuration, even when an image object that can be combined with a radar object is not detected, the redetected image object can be combined with the radar object.

[0021] The present disclosure also relates to a radar image target detection device, wherein the radar image target integration unit outputs the distance of the radar target and the direction of the image target as the distance and direction of the integrated result of the radar target and the image target.

[0022] According to this configuration, it is possible to output a highly accurate distance to the radar target and a highly accurate direction of the image target as the distance and direction of the integrated radar target and image target.

[0023] The present disclosure also relates to a radar image target detection device, wherein the radar image target integration unit outputs a height of the integrated radar target and image target that is based on a distance to the radar target and a vertical pixel count of the image target.

[0024] According to this configuration, the height of the radar target and the image target combined can be output based on the highly accurate distance of the radar target and the highly accurate number of vertical pixels of the image target.

[0025] The present disclosure also relates to a radar image target detection device, further comprising an image target contour extraction unit that extracts a contour of the image target from an integrated result of the radar target and the image target, and outputs a three-dimensional position of the contour of the image target.

[0026] According to this configuration, for an integrated radar target and image target, the radar image target integration unit determines the size based on the radar signal point distribution or image rectangular shape information, and the image target contour extraction unit can extract a three-dimensional contour based on the determined size. Effect of the Invention

[0027] In this manner, the present disclosure can output highly accurate distance and direction as the distance and direction of the integrated radar target and image target. [Brief description of the drawings]

[0028] [Figure 1] 1 is a diagram showing a configuration of a radar image target detection system according to the present disclosure; [Diagram 2] FIG. 13 is a diagram showing a procedure of radar target center integration processing according to the present disclosure. [Diagram 3] FIG. 13 is a diagram showing a procedure of image target center integration processing according to the present disclosure. [Figure 4] 1A to 1C are diagrams illustrating a specific example of radar image target integration processing according to the present disclosure. [Diagram 5] 11A and 11B are diagrams illustrating a specific example of a re-detection process of an image target according to the present disclosure. [Figure 6] FIG. 2 is a diagram showing a specific example of a process for detecting clusters of radar signal points according to the present disclosure. [Figure 7] FIG. 4 is a diagram showing a procedure of output processing of the radar target center according to the present disclosure. [Figure 8] FIG. 11 is a diagram showing a procedure of an output process of an image target center according to the present disclosure. [Figure 9] 10A to 10C are diagrams showing a specific example of output processing of distance and azimuth of a radar image target according to the present disclosure. [Figure 10] FIG. 11 is a diagram showing a specific example of a radar image target height output process according to the present disclosure. [Figure 11] 10 is a diagram showing a procedure of a collision risk determination process according to the present disclosure. [Figure 12] 1A to 1C are diagrams illustrating a specific example of a contour extraction process for an image target according to the present disclosure. [Figure 13]1A to 1C are diagrams illustrating a specific example of a radar image target tracking process according to the present disclosure. [Figure 14] 10A to 10C are diagrams illustrating a specific example of a process for determining a collision risk between targets according to the present disclosure. [Figure 15] 10A to 10C are diagrams illustrating a specific example of a process for determining a collision risk between targets according to the present disclosure. [Figure 16] 11A to 11C are diagrams showing a specific example of a process for determining the risk of collision with a structure according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present disclosure will be described with reference to the accompanying drawings. The embodiments described below are examples of the present disclosure, and the present disclosure is not limited to the following embodiments.

[0030] (Radar image target detection system of the present disclosure) The configuration of the radar image target detection system of the present disclosure is shown in Fig. 1. The radar image target detection system T includes a radar transmitting / receiving device 1, a camera imaging device 2, a radar image target detection device 3, a collision risk display device 4, and a collision risk notification device 5. The radar image target detection device 3 includes a radar target detection unit 31, an image target detection unit 32, a radar image target integration unit 33, an image target contour extraction unit 34, a radar image target tracking unit 35, and a collision risk determination unit 36. The radar image target detection device 3 can be realized by installing the radar image target detection programs shown in Figs. 2, 3, 7, 8, and 11 on a computer.

[0031] (Radar image target integration processing of the present disclosure) The procedure of the radar target center integration process of the present disclosure is shown in Fig. 2. The procedure of the image target center integration process of the present disclosure is shown in Fig. 3. A specific example of the radar image target integration process of the present disclosure is shown in Fig. 4.

[0032] The radar target detection unit 31 acquires a radar signal from the radar transceiver 1 (step S1), detects radar targets R1, R2, and R3 based on the radar signal (step S2), and measures the distances and directions of the radar targets R1, R2, and R3 (step S3).

[0033] The image target detection unit 32 acquires a camera image from the camera imaging device 2 (step S11), detects image targets P1, P4, and P5 based on the camera image (step S12), and measures the direction and distance of the image targets P1, P4, and P5 (step S13).

[0034] Here, the distance of the radar target is highly accurate, but the direction of the radar target is low accuracy. Furthermore, the radar target is susceptible to the effects of multipath and false images, and the shape and color cannot be determined. On the other hand, the direction of the image target is highly accurate, but the distance of the image target is low accuracy. Furthermore, the image target is susceptible to the effects of dark places and bad weather, and overlapping images cannot be separated. Therefore, in the present disclosure, the advantages of the radar target and the image target are utilized to compensate for the disadvantages of the image target and the radar target.

[0035] First, the radar target R1 and the image target P1 will be described. The difference between the distance of the radar target R1 and the distance of the image target P1 is equal to or less than a predetermined difference, and the difference between the orientation of the image target P1 and the orientation of the radar target R1 is equal to or less than a predetermined difference (steps S4, S14 and steps S5, S15, YES). Here, the predetermined difference for the distance difference between the radar target and the image target may be set to the distance accuracy difference between the radar target and the image target, and the predetermined difference for the orientation difference between the image target and the radar target may be set to the orientation accuracy difference between the image target and the radar target. Then, the radar image target integration unit 33 integrates the radar target R1 and the image target P1 into the radar image integration target C1 (steps S6, S16).

[0036] Next, the radar target R2 will be described. The difference between the distance of the radar target R2 and the distance of the image targets P1, P4, and P5 is larger than a predetermined difference, and the difference between the orientation of the image targets P1, P4, and P5 and the orientation of the radar target R2 is also larger than a predetermined difference (steps S4 and S5, NO). That is, the radar image target integration unit 33 recognizes that an image target that can be integrated with the radar target R2 is not detected. Therefore, the radar image target integration unit 33 redetects the image target P2 having a distance and orientation close to the distance and orientation of the radar target R2 using a method different from that of the image target detection unit 32 (steps S7 and S8, YES), and integrates the image target P2 and the radar target R2 into the radar image integrated target C2 (step S9).

[0037] A specific example of the re-detection process of the image object of the present disclosure is shown in FIG. 5. In the left column of FIG. 5, the radar object detection unit 31 detects the radar object R1 and also detects the radar object R2. In the right column of FIG. 5, the image object detection unit 32 detects the image object P1 but does not detect other image objects. Since the human is superimposed on the vehicle, the white of the human's upper body is distinguished from the black of the vehicle, but the black of the human's lower body is not distinguished from the black of the vehicle. Therefore, the radar image object integration unit 33 re-detects the image object P2 in the azimuth range in which the radar object R2 is located, and recognizes that the image object P2 and the radar object R2 are close to each other.

[0038] Next, the radar target R3 will be described. The difference between the distance of the radar target R3 and the distance of the image targets P1, P4, P5 is larger than a predetermined difference, and the difference between the orientation of the image targets P1, P4, P5 and the orientation of the radar target R3 is also larger than a predetermined difference (steps S4 and S5, NO). That is, the radar image target integration unit 33 recognizes that an image target that can be integrated with the radar target R3 is not detected. However, the radar image target integration unit 33 does not redetect an image target having a distance and orientation close to the distance and orientation of the radar target R3 (steps S7 and S8, NO), and does not integrate the radar target R3 with the image targets P1, P4, P5, and outputs it as a radar non-integrated target C3 (step S10).

[0039] Next, the image target P4 will be described. The difference between the direction of the image target P4 and the direction of the radar targets R1, R2, and R3 is greater than a predetermined difference, and the difference between the distance of the image target P4 and the distance of the radar targets R1, R2, and R3 is also greater than a predetermined difference (steps S14 and S15, NO). That is, the radar image target integration unit 33 recognizes that a radar target that can be integrated with the image target P4 is not detected. Therefore, the radar image target integration unit 33 detects a cluster R4 of radar signal points having a direction and distance closest to the direction and distance of the image target P4 (steps S17 and S18, YES), and integrates the cluster R4 of radar signal points and the image target P4 into a radar image integrated target C4 (step S19).

[0040] A specific example of the detection process of radar signal point clusters according to the present disclosure is shown in FIG. 6. In the left column of FIG. 6, the image target detection unit 32 detects an image target P4. In the same left column, the radar target detection unit 31 does not detect a radar target, but detects radar signal point clusters R4, R6, and R7. The radar signal point clusters R4, R6, and R7 are not large enough to constitute radar targets, but are not small enough to be recognized as clutter. In the right column of FIG. 6, the radar image target integration unit 33 detects the radar signal point cluster R4 in the azimuth range in which the image target P4 is located, and recognizes that the radar signal point cluster R4 and the image target P4 are close to each other.

[0041] Next, the image target P5 will be described. The difference between the direction of the image target P5 and the direction of the radar targets R1, R2, and R3 is greater than a predetermined difference, and the difference between the distance of the image target P5 and the distance of the radar targets R1, R2, and R3 is also greater than a predetermined difference (steps S14 and S15, NO). That is, the radar image target integration unit 33 recognizes that a radar target that can be integrated with the image target P5 is not detected. However, the radar image target integration unit 33 does not detect a cluster of radar signal points having a direction and distance closest to the direction and distance of the image target P5 (steps S17 and S18, NO), and does not integrate the image target P5 with the radar targets R1, R2, and R3, and outputs it as an image non-integrated target C5 (step S20).

[0042] In this way, the radar target R1 and the image target P1 can be integrated into the radar image integrated target C1. Then, even when an image target that can be integrated with the radar target R2 is not detected, the redetected image target P2 and the radar target R2 can be integrated into the radar image integrated target C2. Furthermore, even when a radar target that can be integrated with the image target P4 is not detected, the cluster R4 of radar signal points and the image target P4 can be integrated into the radar image integrated target C4. Then, as will be described shortly, the distance and direction of the radar image integrated targets C1, C2, and C4 can be output with high accuracy.

[0043] (Radar image target output processing of the present disclosure) The procedure for output processing of the radar target center of the present disclosure is shown in Fig. 7. The procedure for output processing of the image target center of the present disclosure is shown in Fig. 8. A specific example of output processing of the distance and azimuth of the radar image target of the present disclosure is shown in Fig. 9. A specific example of output processing of the height of the radar image target of the present disclosure is shown in Fig. 10.

[0044] First, the radar image integrated object C1 will be described. In the left column of FIG. 9, the radar image target integration unit 33 calculates the low-precision distance d P Instead, the highly accurate distance d R In the same left column, the radar image target integration unit 33 outputs the low-precision direction a of the radar target R1 as the direction of the radar image integrated target C1. R Instead, the highly accurate azimuth a of the image target P1 P (steps S21 and S31).

[0045] In the left column of FIG. 10, the radar image target integration unit 33 calculates the height of the radar image integrated target C1 by calculating the highly accurate distance d R and the number of highly accurate vertical pixels p of the image target P1 H Based on height h P (Steps S22 and S32). Here, the height h P hP =p H *d R *Can be calculated as sensor size of camera imaging device 2 / focal length of camera imaging device 2 / number of image pixels of camera imaging device 2.

[0046] Next, the radar image integrated object C2 will be described. In the middle column of FIG. 9, the radar image target integration unit 33 calculates the low-precision distance d P Instead, the highly accurate distance d R In the middle column, the radar image target integration unit 33 outputs the low-precision direction a of the radar target R2 as the direction of the radar image integration target C2. R Instead, the highly accurate azimuth a of the re-detected image target P2 is P (Step S23). The image target P2 is redetected, but the highly accurate azimuth a P has.

[0047] In the middle column of FIG. 10, the radar image target integration unit 33 calculates the height of the radar image integrated target C2 by calculating the highly accurate distance d R and the number of highly accurate vertical pixels p of the redetected image target P2 H Based on height h P (Step S24). The image target P2 is redetected, but the number of vertical pixels p H In the re-detected image target P2, the upper half of the vehicle is detected and the lower half of the vehicle is not detected, but the number of vertical pixels p H The distance d of the radar target R2 is R Then, the position of the lower side of the image target P2 is calculated based on the above, and the number of pixels in the vertical direction from the position of the upper side to the position of the lower side of the image target P2 is used.

[0048] Next, the radar non-integrated target C3 will be described. Although not shown in FIG. 9, the radar image target integration unit 33 calculates the distance of the radar non-integrated target C3 by calculating a highly accurate distance d RAlthough not shown in FIG. 9, the radar image target integration unit 33 outputs the low-precision direction a of the radar target R3 as the direction of the radar non-integrated target C3. R (Step S25). Since the radar image target integration unit 33 cannot calculate the height of the radar target R3, it cannot output the height of the radar non-integrated target C3.

[0049] Next, the radar image integrated object C4 will be described. In the right column of FIG. 9, the radar image target integration unit 33 uses the low-precision azimuth a of the cluster R4 of radar signal points as the azimuth of the radar image integrated object C4. R Instead, the highly accurate azimuth a of the image target P4 P In the same right column, the radar image target integration unit 33 outputs a low-precision distance d P Instead, we use the highly accurate distance d R (Step S33). The cluster R4 of radar signal points is not the entire radar target but a part of the radar target, but is a highly accurate distance d R has.

[0050] In the right column of FIG. 10, the radar image target integration unit 33 calculates the height of the radar image integrated target C4 by calculating the highly accurate distance d R and the number of highly accurate vertical pixels p of the image target P4 H Based on height h P The cluster R4 of radar signal points is not the entire radar target but a part of the radar target, but is a highly accurate distance d R has.

[0051] Next, the image non-integrated target C5 will be described. Although not shown in FIG. 9, the radar image target integration unit 33 uses a highly accurate azimuth a of the image target P5 as the azimuth of the image non-integrated target C5. P Although not shown in FIG. 9, the radar image target integration unit 33 outputs the low-precision distance d PAlthough not shown in FIG. 10, the radar image target integration unit 33 outputs the low-precision distance d P (The radar target distance cannot be output with high accuracy.) And the number of vertical pixels p H Based on height h P is output (step S36).

[0052] In this way, the distance and direction of the radar image integrated target C1, etc. are calculated by using a highly accurate distance d R and high-precision orientation a of the image target P1, etc. P Then, the highly accurate distance d of the radar target R1 can be output as the height of the radar image integrated target C1, etc. R and the number of highly accurate vertical pixels p of the image target P1, etc. H The output can be based on the following:

[0053] (Collision Risk Determination Process of the Present Disclosure) The procedure of the collision risk determination process of the present disclosure is shown in Fig. 11. A specific example of the image target contour extraction process of the present disclosure is shown in Fig. 12. A specific example of the radar image target tracking process of the present disclosure is shown in Fig. 13. A specific example of the collision risk determination process between targets of the present disclosure is shown in Figs. 14 and 15. A specific example of the collision risk determination process with a structure of the present disclosure is shown in Fig. 16.

[0054] In FIG. 12, the image target contour extraction unit 34 extracts the contour E of the image target P for the radar image-integrated target C (including non-image-integrated targets) (the radar target R has no height), and outputs the three-dimensional position of the contour E of the image target P to coordinate G (step S41).

[0055] Here, the image target contour extraction unit 34 can output the three-dimensional position of the contour E of the image target P to the coordinate G based on the distance d and the direction a output in Fig. 7 to Fig. 10. In this way, for the radar image-integrated target C (including the image non-integrated target), the radar image target integration unit 33 can determine the size based on the radar signal point distribution or the image rectangular shape information, and the image target contour extraction unit 34 can extract the three-dimensional contour based on the determined size.

[0056] In FIG. 13, the radar image target tracking unit 35 performs tracking of the radar image-integrated target C (including non-radar integrated targets and non-image integrated targets) and calculates the velocity vectors v(t1→t2) and v(t2→t3) (step S42).

[0057] Here, the radar image target tracking unit 35 can calculate the velocity vectors v(t1→t2), v(t2→t3) based on the distances d(t1), d(t2), d(t3) and the directions a(t1), a(t2), a(t3) output in Figures 7 to 10 using an αβγ (position, velocity, acceleration) filter, a Kalman filter, a particle filter, or the like.

[0058] In FIG. 14, the collision risk determination unit 36 ​​calculates a velocity vector v for a plurality of radar image-integrated targets Ci, Cj (including radar non-integrated targets and image non-integrated targets), or for each of these targets and the radar transmitting / receiving device 1 and the camera imaging device 2. i , v j Based on this (assumed to be constant over time), the risk of collision is determined (step S43).

[0059] Here, the collision risk determination unit 36 ​​calculates the velocity vector v i , v j Based on the position vector r i (t), r j (t) is calculated as the Closest Point of Approach within a given period T, and the CPA i、j =min(|r i (t)-r j(t)| 2 ) and calculate the CPA i、j If is smaller than the threshold, it is determined that the risk of collision is high and CPA i、j is equal to or greater than the threshold, the risk of collision is determined to be low. In this way, the risk of collision can be determined for multiple radar image-integrated targets Ci, Cj (including non-radar integrated targets and non-image integrated targets), or for each of these targets and the radar transmitting / receiving device 1 and the camera imaging device 2, based on the degree of approach within a predetermined period T.

[0060] In the left column of Figure 15, the collision risk determination unit 36 ​​determines the collision risk for multiple radar image-integrated targets Ci, Cj (including non-radar integrated targets and non-image integrated targets), or for each of these targets and the radar transceiver device 1 and camera imaging device 2, based on their type (e.g., human or vehicle, etc. and also human age, etc.) (step S44).

[0061] Here, the collision risk determination unit 36 ​​determines the collision risk to be low if the radar image integrated objects Ci, Cj are a human and a human, and determines the collision risk to be high if the radar image integrated objects Ci, Cj are a human and a vehicle, based on the types of the radar image integrated objects Ci, Cj determined by the radar image target integration unit 33. The radar image target integration unit 33 makes a final determination of the types of the radar image integrated objects Ci, Cj based on the type determination result of the radar target detection unit 31 or the type determination result of the image target detection unit 32.

[0062] In the right column of Figure 15, the collision risk determination unit 36 ​​determines the collision risk for multiple radar image-integrated targets Ci, Cj (including non-radar integrated targets and non-image integrated targets), or for each of these targets and the radar transceiver device 1 and camera imaging device 2, based on their size (e.g., humans are small and vehicles are large) (step S44).

[0063] Here, the collision risk determination unit 36 ​​determines the collision risk to be low if the radar image integrated targets Ci, Cj are both small targets based on the sizes of the radar image integrated targets Ci, Cj determined by the radar image target integrating unit 33, and determines the collision risk to be high if the radar image integrated targets Ci, Cj are both large targets. The radar image target integrating unit 33 determines the size of the radar image integrated targets Ci, Cj based on the radar signal point distribution of the radar target detecting unit 31 or the image rectangular shape information of the image target detecting unit 32.

[0064] In this way, the risk of collision can be determined with high accuracy for multiple radar image-integrated targets Ci, Cj (including non-radar integrated targets and non-image integrated targets), or for each of these targets and the radar transceiver device 1 and camera imaging device 2, based on parameters (type and / or size) other than the degree of approach within a specified period T.

[0065] In FIG. 16, the collision risk determination unit 36 ​​determines the risk of collision with the structure S for the radar image-integrated target C (including non-radar-integrated targets and non-image-integrated targets) based on the distance and direction of the structure S detected by the radar transceiver 1 and the camera imaging device 2, or based on three-dimensional map information of the structure S that the collision risk determination unit 36 ​​has in advance (including the position and shape of the structure S) (step S45).

[0066] Here, the collision risk determination unit 36 ​​superimposes the distance and direction of the detected structure S or the three-dimensional position and three-dimensional shape of the structure S based on the previously held three-dimensional map information of the structure S on the three-dimensional position and three-dimensional shape of the contour E output in Fig. 12. Then, if the velocity vector v calculated in Fig. 13 is approaching the structure S, it determines that the risk of collision with the structure S is high, and if the velocity vector v calculated in Fig. 13 is moving away from the structure S, it determines that the risk of collision with the structure S is low. In this way, for the radar image-integrated target C (including radar non-integrated targets and image non-integrated targets), it is possible to determine the risk of collision with the structure S after the radar transmitting and receiving device 1 and the camera imaging device 2 detect the structure S, or even if the radar transmitting and receiving device 1 and the camera imaging device 2 do not detect the structure S.

[0067] The collision risk determination unit 36 ​​outputs the distance d, direction a, height h, contour E, three-dimensional position r, velocity vector v and collision risk for the radar-image-integrated target C (including non-radar-integrated targets and non-image-integrated targets). The collision risk display device 4 displays these parameters on a screen. The collision risk notification device 5 notifies these parameters by voice or the like. The collision risk display device 4 and the collision risk notification device 5 may alert the user to the risk of collision for both targets of a plurality of targets with a high collision risk, or may alert the user to the risk of collision for only one of them. [Industrial Applicability]

[0068] The radar image target detection device and radar image target detection program disclosed herein are applicable to environmental recognition in autonomous driving, collision prevention in safety monitoring, and high-precision detection in intrusion detection, etc. [Explanation of symbols]

[0069] T: Radar image target detection system R1, R2, R3: Radar target R4, R6, R7: Clusters of radar signal points R: Radar target P1, P2, P4, P5: Image targets P: Image target C1, C2, C4: Radar image integrated target C3: Radar non-integrated target C5: Image non-integrated target C, Ci, Cj: Radar image integrated target E:Contour G: Coordinates S: Structure 1: Radar transmitter and receiver 2: Camera imaging device 3: Radar image target detection device 4: Collision risk indicator 5. Collision risk notification device 31: Radar target detection unit 32: Image target detection unit 33: Radar image target integration unit 34: Image target contour extraction unit 35: Radar image target tracking unit 36: Collision risk assessment unit

Claims

1. a radar target detection unit that acquires a radar signal, detects a target (hereinafter referred to as a radar target) based on the radar signal, and measures a distance and a direction of the radar target; an image target detection unit that acquires a camera image, detects a target (hereinafter referred to as an image target) based on the camera image, and measures a direction and a distance of the image target; a radar image target integration unit that integrates the radar target and the image target when a difference between a distance of the radar target and a distance of the image target is equal to or smaller than a predetermined difference and a difference between an orientation of the image target and an orientation of the radar target is equal to or smaller than a predetermined difference; a radar image target tracking unit that performs tracking on a combination of the radar target and the image target and calculates a velocity vector; a collision risk determination unit that determines a collision risk for each of the plurality of integrated radar targets and image targets based on the velocity vector; A radar image target detection device comprising:

2. The collision risk determination unit calculates a minimum value of absolute differences between position vectors within a predetermined period for a plurality of objects obtained by integrating the radar target and the image target, based on the velocity vector, and determines that the risk of collision is high if the minimum value is smaller than a threshold value, and determines that the risk of collision is low if the minimum value is equal to or larger than the threshold value.

2. The radar image target detection device according to claim 1, wherein:

3. The collision risk determination unit determines a collision risk for each of the radar targets and the image targets based on the type and / or size of the target.

3. The radar image target detection device according to claim 1 or 2.

4. The collision risk determination unit determines a risk of collision with the structure based on a distance and a direction of the structure detected by the radar transmitting / receiving device and the camera imaging device, or based on three-dimensional map information of the structure that the collision risk determination unit has in advance, for the integrated radar target and the image target.

4. The radar image target detection device according to claim 1, wherein the radar image target detection device further comprises a first detecting means.

5. The radar image target integration unit detects a cluster of radar signal points having a direction and distance closest to the direction and distance of the image target when the radar target that can be integrated with the image target is not detected, and integrates the cluster of radar signal points with the image target.

5. The radar image target detection device according to claim 1, wherein the radar image target detection device further comprises a first detecting means.

6. The radar image target integration unit, when the image target that can be integrated with the radar target is not detected, redetects a redetected image target having a distance and a direction close to the distance and direction of the radar target, and integrates the redetected image target with the radar target.

6. The radar image target detection device according to claim 1, wherein the radar image target detection device further comprises a first detecting means.

7. The radar image target integration unit outputs a distance of the radar target and a direction of the image target as a distance and a direction of an integration of the radar target and the image target.

7. The radar image target detection device according to claim 1, wherein the radar image target detection device further comprises a first detecting means.

8. The radar image target integration unit outputs a height obtained by integrating the radar target and the image target, the height being based on a distance to the radar target and a vertical pixel count of the image target.

8. The radar image target detection device according to claim 1, wherein the radar image target detection device is a radar image detection device.

9. The radar system further includes an image target contour extraction unit that extracts a contour of the image target from an integrated result of the radar target and the image target, and outputs a three-dimensional position of the contour of the image target.

9. The radar image target detection device according to claim 1, wherein the radar image target detection device is a radar image detection device.

10. a radar target detection step of acquiring a radar signal, detecting a target (hereinafter referred to as a radar target) based on the radar signal, and measuring a distance and a direction of the radar target; an image target detection step of acquiring a camera image, detecting a target (hereinafter referred to as an image target) based on the camera image, and measuring a direction and a distance of the image target; a radar image target integration step of integrating the radar target and the image target when a difference between a distance of the radar target and a distance of the image target is equal to or smaller than a predetermined difference and a difference between an orientation of the image target and an orientation of the radar target is equal to or smaller than a predetermined difference; a radar image target tracking step of tracking the integrated radar target and the image target and calculating a velocity vector; a collision risk determination step of determining a collision risk for each of the plurality of integrated radar targets and image targets based on the velocity vector; A radar image target detection program for causing a computer to execute the above.

Images