Radar image target detection device and radar image target detection program

Abstract

To integrate radar objects and image objects to output highly accurate distances and azimuths.SOLUTION: A radar image object detection device 3 is provided, comprising: a radar object detection unit 31 configured to acquire a radar signal, detect a radar object based on the radar signal, and measure the distance to and azimuth of the radar object; an image object detection unit 32 configured to acquire a camera image, detect an image object based on the camera image, and measure the distance to and azimuth of the image object; and a radar image object integration unit 33 configured to integrate the radar object and the image object when a difference between the distance to the radar object and the distance to the image object is equal to or less than a given difference and a difference between the azimuth of the image object and the azimuth of the radar object is equal to or less than a given difference. The radar image object integration unit 33 outputs a height based on the distance to the radar object and the number of pixels of the image object in a longitudinal direction as a height of an object obtained by integrating the radar object and the image object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a technology for detecting a target based on a radar signal and a camera image. [Background technology]

[0002] A technology for detecting a target based on a radar signal and a camera image is disclosed in Patent Document 1, etc. In Patent Document 1, a radar signal is plotted on a camera image, an edge analysis is performed near the radar signal, and a truncated pyramid that borders the measurement target (for example, a vehicle) is detected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-211402 [Patent Document 2] Japanese Patent Application Publication No. 2019-211403 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, a target detected based on a radar signal (hereinafter referred to as a radar target) and a target detected based on a camera image (hereinafter referred to as an image target) are not integrated. Therefore, in Patent Document 1, it is not possible to output a highly accurate distance of the radar target as the distance of the integrated radar target and image target, rather than a low-accuracy distance of the image target. Furthermore, in Patent Document 1, it is not possible to output a highly accurate bearing of the image target as the bearing of the integrated radar target and image target, rather than a low-accuracy bearing of the radar target.

[0005] Therefore, in order to solve the above problem, an object of the present disclosure is to output highly accurate distance and bearing as the distance and bearing of a combination of a radar target and an image target. [Means for solving the problem]

[0006] To solve the above problem, when the difference between the distance of the radar target and the distance of the image target is equal to or less than a predetermined difference and the difference between the bearing of the image target and the bearing of the radar target is equal to or less than a predetermined difference, the radar target and the image target are integrated. Thus, as the distance of the integrated result of the radar target and the image target, a highly accurate distance of the radar target can be output instead of a low accuracy distance of the image target. And, as the bearing of the integrated result of the radar target and the image target, a highly accurate bearing of the image target can be output instead of a low accuracy bearing of the radar target.

[0007] Then, the height of the radar target and the image target combined together is output based on the distance of the radar target and the number of vertical pixels of the image target. Therefore, the height of the radar target and the image target combined together can be output based on the highly accurate distance of the radar target and the highly accurate number of vertical pixels of the image target.

[0008] Specifically, the present disclosure provides a radar image target detection device including: a radar target detection unit that acquires a radar signal, detects a radar target based on the radar signal, and measures the distance and orientation of the radar target; an image target detection unit that acquires a camera image, detects an image target based on the camera image, and measures the orientation and distance of the image target; and a radar image target integration unit that integrates the radar target and the image target when a difference between the distance of the radar target and the distance of the image target is equal to or less than a predetermined difference and a difference between the orientation of the image target and the orientation of the radar target is equal to or less than a predetermined difference, wherein the radar image target integration unit outputs a height of the integrated radar target and the image target that is based on the distance of the radar target and the number of vertical pixels of the image target.

[0009] The present disclosure also provides a radar image target detection program that causes a computer to execute the following steps: a radar target detection step of acquiring a radar signal, detecting a radar target based on the radar signal, and measuring the distance and orientation of the radar target; an image target detection step of acquiring a camera image, detecting an image target based on the camera image, and measuring the orientation and distance of the image target; and a radar image target integration step of integrating the radar target and the image target when a difference between the distance of the radar target and the distance of the image target is equal to or less than a predetermined difference and a difference between the orientation of the image target and the orientation of the radar target is equal to or less than a predetermined difference, wherein the radar image target integration step outputs a height of the integrated radar target and the image target that is based on the distance of the radar target and the number of vertical pixels of the image target.

[0010] These configurations enable the radar target and the image target to be integrated, and the height of the integrated radar target and image target to be output based on the highly accurate distance of the radar target and the highly accurate number of vertical pixels of the image target.

[0011] The present disclosure also provides a radar image target detection device, characterized in that, when the radar image target integrating unit does not detect a radar target that can be integrated with the image target, it detects a cluster of radar signal points having a direction and distance closest to the direction and distance of the image target, and integrates the cluster of radar signal points with the image target.

[0012] According to this configuration, even when a radar target that can be combined with an image target is not detected, the cluster of radar signal points can be combined with the image target.

[0013] The present disclosure also provides a radar image target detection device, characterized in that, when the image target that can be integrated with the radar target is not detected, the radar image target integration unit redetects a redetected image target having a distance and direction close to the distance and direction of the radar target, and integrates the redetected image target with the radar target.

[0014] According to this configuration, even when an image target that can be combined with a radar target is not detected, the redetected image target and the radar target can be combined.

[0015] The present disclosure also provides a radar image target detection device, characterized in that the radar image target integration unit outputs the distance of the radar target and the direction of the image target as the distance and direction of the integrated result of the radar target and the image target.

[0016] According to this configuration, it is possible to output a highly accurate distance to the radar target and a highly accurate direction of the image target as the distance and direction of the integrated radar target and image target.

[0017] The present disclosure also provides a radar image target detection device, further comprising an image target contour extraction unit that extracts the contour of the image target from the integrated radar target and image target, and outputs the three-dimensional position of the contour of the image target.

[0018] According to this configuration, the radar image target integration unit determines the size of the integrated radar target and image target based on the radar signal point distribution or image rectangular shape information, and the image target contour extraction unit can extract the three-dimensional contour based on the determined size.

[0019] The present disclosure also provides a radar image target detection device, further comprising a radar image target tracking unit that performs tracking on the integrated radar target and the image target and calculates a velocity vector.

[0020] The present disclosure also provides a radar image target detection device further comprising a collision risk determination unit that determines a collision risk for a plurality of integrated radar targets and image targets, or for each of these targets, a radar transmitting / receiving device, and a camera imaging device, based on the velocity vector.

[0021] With this configuration, the risk of collision can be determined for multiple integrated radar targets and image targets, or for each of these targets and the radar transmitting / receiving device and camera imaging device, based on the degree of proximity within a specified period of time.

[0022] The present disclosure also provides a radar image target detection device, characterized in that the collision risk determination unit determines the collision risk for a plurality of integrated radar targets and image targets, or for each of these targets, the radar transmitting / receiving device, and the camera imaging device, based on the type and / or size.

[0023] With this configuration, the risk of collision can be determined with high accuracy for multiple integrated radar targets and image targets, or for each of these targets and the radar transmitting / receiving device and camera imaging device, based on factors other than the degree of proximity within a specified period of time.

[0024] The present disclosure also provides a radar image target detection device, characterized in that the collision risk determination unit determines the risk of collision with a structure for an integrated result of the radar target and the image target, based on the distance and direction of the structure detected by the radar transmitting / receiving device and the camera imaging device, or based on three-dimensional map information of the structure that the collision risk determination unit has in advance.

[0025] According to this configuration, the risk of collision with a structure can be determined for an integrated radar target and image target after detecting the structure using the radar transmitting / receiving device and the camera imaging device, or without detecting the structure using the radar transmitting / receiving device and the camera imaging device. [Effects of the Invention]

[0026] In this way, the present disclosure can output highly accurate distance and bearing as the distance and bearing of the integrated radar target and image target. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a diagram illustrating a configuration of a radar image target detection system according to the present disclosure. [Figure 2] FIG. 10 is a diagram showing a procedure for radar target center integration processing according to the present disclosure. [Figure 3] FIG. 10 is a diagram showing a procedure for image target center integration processing according to the present disclosure. [Figure 4] 10A and 10B are diagrams illustrating a specific example of radar image target integration processing according to the present disclosure. [Figure 5] 10A and 10B are diagrams illustrating a specific example of the re-detection process of an image target according to the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating a specific example of a process for detecting clusters of radar signal points according to the present disclosure. [Figure 7] FIG. 10 is a diagram showing a procedure for output processing of the radar target center according to the present disclosure. [Figure 8] FIG. 10 is a diagram showing a procedure for outputting an image target center according to the present disclosure. [Figure 9] 10A and 10B are diagrams illustrating a specific example of a process for outputting the distance and azimuth of a radar image target according to the present disclosure. [Figure 10] 10A and 10B are diagrams illustrating a specific example of a radar image target height output process according to the present disclosure. [Figure 11] FIG. 4 is a diagram showing a procedure for determining a collision risk level according to the present disclosure. [Figure 12] 10A and 10B are diagrams illustrating a specific example of a contour extraction process for an image target according to the present disclosure. [Figure 13] 1A and 1B are diagrams illustrating a specific example of a radar image target tracking process according to the present disclosure. [Figure 14] 10A and 10B are diagrams illustrating a specific example of a process for determining a collision risk between targets according to the present disclosure. [Figure 15] 10A and 10B are diagrams illustrating a specific example of a process for determining a collision risk between targets according to the present disclosure. [Figure 16] 10A and 10B are diagrams showing a specific example of a process for determining the risk of collision with a structure according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0028]

[0023] The following embodiments of the present disclosure will be described with reference to the accompanying drawings. The embodiments described below are examples of implementation of the present disclosure, and the present disclosure is not limited to the following embodiments.

[0029] (Radar image target detection system of the present disclosure) The configuration of a radar image target detection system according to the present disclosure is shown in Fig. 1. The radar image target detection system T includes a radar transmitting / receiving device 1, a camera imaging device 2, a radar image target detection device 3, a collision risk display device 4, and a collision risk notification device 5. The radar image target detection device 3 includes a radar target detection unit 31, an image target detection unit 32, a radar image target integration unit 33, an image target contour extraction unit 34, a radar image target tracking unit 35, and a collision risk determination unit 36. The radar image target detection device 3 can be implemented by installing the radar image target detection programs shown in Figs. 2, 3, 7, 8, and 11 on a computer.

[0030] (Radar image target integration processing of the present disclosure) The procedure for radar target center integration processing of the present disclosure is shown in Fig. 2. The procedure for image target center integration processing of the present disclosure is shown in Fig. 3. A specific example of radar image target integration processing of the present disclosure is shown in Fig. 4.

[0031] The radar target detection unit 31 acquires a radar signal from the radar transceiver 1 (step S1), detects radar targets R1, R2, and R3 based on the radar signal (step S2), and measures the distance and direction of the radar targets R1, R2, and R3 (step S3).

[0032] The image target detection unit 32 acquires a camera image from the camera imaging device 2 (step S11), detects image targets P1, P4, and P5 based on the camera image (step S12), and measures the direction and distance of the image targets P1, P4, and P5 (step S13).

[0033] Here, the distance of a radar target is highly accurate, but the azimuth of the radar target is low. Furthermore, radar targets are susceptible to multipath and false images, making it impossible to determine their shape and color. On the other hand, the azimuth of an image target is highly accurate, but the distance of the image target is low. Furthermore, image targets are susceptible to the effects of darkness and bad weather, making it impossible to separate overlapping images. Therefore, the present disclosure utilizes the advantages of both radar targets and image targets to compensate for the disadvantages of both image targets and radar targets.

[0034] First, the radar target R1 and the image target P1 will be described. The difference between the distance of the radar target R1 and the distance of the image target P1 is equal to or less than a predetermined difference, and the difference between the bearing of the image target P1 and the bearing of the radar target R1 is also equal to or less than a predetermined difference (steps S4, S14 and steps S5, S15, YES). Here, the predetermined difference for the distance difference between the radar target and the image target may be set to the distance accuracy difference between the radar target and the image target, and the predetermined difference for the bearing difference between the image target and the radar target may be set to the bearing accuracy difference between the image target and the radar target. Therefore, the radar image target integration unit 33 integrates the radar target R1 and the image target P1 into a radar image integrated target C1 (steps S6, S16).

[0035] Next, the radar target R2 will be described. The difference between the distance of the radar target R2 and the distance of the image targets P1, P4, and P5 is greater than a predetermined difference, and the difference between the azimuths of the image targets P1, P4, and P5 and the azimuth of the radar target R2 is also greater than a predetermined difference (steps S4 and S5, NO). In other words, the radar image target integration unit 33 recognizes that no image target that can be integrated with the radar target R2 has been detected. Therefore, the radar image target integration unit 33 redetects the image target P2 having a distance and azimuth close to those of the radar target R2 using a method different from that of the image target detection unit 32 (steps S7 and S8, YES), and integrates the image target P2 and the radar target R2 into a radar image integrated target C2 (step S9).

[0036] A specific example of the image target redetection process of the present disclosure is shown in FIG. 5. In the left column of FIG. 5, the radar target detection unit 31 detects radar target R1 and radar target R2. In the right column of FIG. 5, the image target detection unit 32 detects image target P1 but does not detect other image targets. Because the human is superimposed on the vehicle, the white of the human's upper body is distinguishable from the black of the vehicle, but the black of the human's lower body is not distinguishable from the black of the vehicle. Therefore, the radar image target integration unit 33 redetects the image target P2 in the azimuth range in which the radar target R2 is located and recognizes that the image target P2 and the radar target R2 are close to each other.

[0037] Next, the radar target R3 will be described. The difference between the distance of the radar target R3 and the distances of the image targets P1, P4, and P5 is greater than a predetermined difference, and the difference between the orientations of the image targets P1, P4, and P5 and the orientation of the radar target R3 is also greater than a predetermined difference (steps S4 and S5, NO). That is, the radar image target integrating unit 33 recognizes that no image targets that can be integrated with the radar target R3 have been detected. However, the radar image target integrating unit 33 does not redetect an image target having a distance and orientation close to the distance and orientation of the radar target R3 (steps S7 and S8, NO), and outputs the radar target R3 as a radar non-integrated target C3 without integrating it with the image targets P1, P4, and P5 (step S10).

[0038] Next, the image target P4 will be described. The difference between the azimuth of the image target P4 and the azimuths of the radar targets R1, R2, and R3 is greater than a predetermined difference, and the difference between the distance of the image target P4 and the distance of the radar targets R1, R2, and R3 is also greater than the predetermined difference (steps S14 and S15, NO). That is, the radar image target integrating unit 33 recognizes that no radar target that can be integrated with the image target P4 has been detected. Therefore, the radar image target integrating unit 33 detects a cluster R4 of radar signal points having an azimuth and distance closest to the azimuth and distance of the image target P4 (steps S17 and S18, YES), and integrates the cluster R4 of radar signal points and the image target P4 into a radar image integrated target C4 (step S19).

[0039] A specific example of the radar signal point cluster detection process of the present disclosure is shown in FIG. 6. In the left column of FIG. 6, the image target detection unit 32 detects image target P4. In the same left column, the radar target detection unit 31 does not detect a radar target, but detects radar signal point clusters R4, R6, and R7. The radar signal point clusters R4, R6, and R7 are not large enough to constitute radar targets, but are not small enough to be recognized as clutter. In the right column of FIG. 6, the radar image target integration unit 33 detects radar signal point cluster R4 in the azimuth range in which image target P4 is located, and recognizes that radar signal point cluster R4 and image target P4 are close to each other.

[0040] Next, the image target P5 will be described. The difference between the azimuth of the image target P5 and the azimuths of the radar targets R1, R2, and R3 is greater than a predetermined difference, and the difference between the distance of the image target P5 and the distance of the radar targets R1, R2, and R3 is also greater than a predetermined difference (steps S14 and S15, NO). That is, the radar image target integrating unit 33 recognizes that no radar target that can be integrated with the image target P5 has been detected. However, the radar image target integrating unit 33 does not detect a cluster of radar signal points having an azimuth and distance closest to the azimuth and distance of the image target P5 (steps S17 and S18, NO), and outputs the image target P5 as an image non-integrated target C5 without integrating it with the radar targets R1, R2, and R3 (step S20).

[0041] In this way, the radar target R1 and the image target P1 can be integrated into the radar-image integrated target C1. Even when an image target that can be integrated with the radar target R2 is not detected, the redetected image target P2 and the radar target R2 can be integrated into the radar-image integrated target C2. Furthermore, even when a radar target that can be integrated with the image target P4 is not detected, the radar signal point cluster R4 and the image target P4 can be integrated into the radar-image integrated target C4. This allows highly accurate distances and azimuths to be output as the distances and azimuths of the radar-image integrated targets C1, C2, and C4, as will be described shortly.

[0042] (Radar image target output processing of the present disclosure) The procedure for outputting the radar target center according to the present disclosure is shown in Fig. 7. The procedure for outputting the image target center according to the present disclosure is shown in Fig. 8. A specific example of outputting the distance and azimuth of the radar image target according to the present disclosure is shown in Fig. 9. A specific example of outputting the height of the radar image target according to the present disclosure is shown in Fig. 10.

[0043] First, the radar image integrated target C1 will be described. In the left column of Fig. 9, the radar image target integration unit 33 calculates the low-precision distance d P Instead, the highly accurate distance d of the radar target R1 R Similarly, in the left column, the radar image target integration unit 33 outputs the low-precision direction a of the radar target R1 as the direction of the radar image integrated target C1. R Instead, the highly accurate direction a of the image target P1 P is output (steps S21 and S31).

[0044] In the left column of FIG. 10, the radar image target integration unit 33 calculates the height of the radar image integrated target C1 by calculating the highly accurate distance d R and the number of highly accurate vertical pixels p of the image target P1 H Based on the height h P (Steps S22 and S32). Here, the height h of the radar image integrated target C1 is P is, h P =p H *d R *Can be calculated as sensor size of camera imaging device 2 / focal length of camera imaging device 2 / number of image pixels of camera imaging device 2.

[0045] Next, the radar image integrated target C2 will be described. In the middle column of Fig. 9, the radar image target integration unit 33 calculates the low-accuracy distance d of the re-detected image target P2 as the distance of the radar image integrated target C2. P Instead, the highly accurate distance d of the radar target R2 R Similarly, in the middle column, the radar image target integration unit 33 outputs the low-precision direction a of the radar target R2 as the direction of the radar image integrated target C2. RInstead, the highly accurate direction a of the re-detected image target P2 is P (Step S23). The image target P2 is redetected, but the highly accurate azimuth a P It has.

[0046] In the middle column of FIG. 10, the radar image target integration unit 33 calculates the height of the radar image integrated target C2 by calculating the highly accurate distance d R and the number of highly accurate vertical pixels p of the re-detected image target P2 H Based on the height h P (Step S24). The image target P2 is redetected, but the number of vertical pixels p H In the re-detected image target P2, the upper half of the vehicle is detected and the lower half of the vehicle is not detected, but the number of vertical pixels p H The distance d of the radar target R2 is R The position of the bottom side of the image target P2 is calculated based on the above equation, and the number of pixels in the vertical direction from the top side position to the bottom side position of the image target P2 is used.

[0047] Next, the radar non-integrated target C3 will be described. Although not shown in FIG. 9, the radar image target integrating unit 33 calculates the distance of the radar non-integrated target C3 by calculating the highly accurate distance d R Although not shown in FIG. 9, the radar image target integration unit 33 outputs the low-accuracy direction a of the radar target R3 as the direction of the radar non-integrated target C3. R (Step S25). Since the radar image target integration unit 33 cannot calculate the height of the radar target R3, it cannot output the height of the radar non-integrated target C3.

[0048] Next, the radar image integrated target C4 will be described. In the right column of Fig. 9, the radar image target integration unit 33 uses the low-precision direction a of the cluster R4 of radar signal points as the direction of the radar image integrated target C4. R Instead, the highly accurate direction a of the image target P4 PSimilarly, in the right column, the radar image target integration unit 33 outputs the low-precision distance d of the image target P4 as the distance of the radar image integrated target C4. P Instead, the high-precision distance d R The cluster R4 of radar signal points is not the entire radar target but a part of the radar target, but it is possible to obtain a highly accurate distance d R It has.

[0049] In the right column of FIG. 10, the radar image target integration unit 33 calculates the height of the radar image integrated target C4 by calculating the highly accurate distance d R and the number of highly accurate vertical pixels p of the image target P4 H Based on the height h P (Step S34). The cluster R4 of radar signal points is not the entire radar target but a part of the radar target, but it is possible to obtain a highly accurate distance d R It has.

[0050] Next, the image non-integrated target C5 will be described. Although not shown in FIG. 9, the radar image target integration unit 33 uses the highly accurate direction a of the image target P5 as the direction of the image non-integrated target C5. P Although not shown in FIG. 9, the radar image target integration unit 33 outputs the low-accuracy distance d of the image target P5 as the distance of the image non-integrated target C5. P Although not shown in FIG. 10, the radar image target integration unit 33 outputs the low-accuracy distance d P (High-precision distance of radar target cannot be output.) And high-precision vertical pixel count p of image target P5 H Based on the height h P is output (step S36).

[0051] In this way, the distance and direction of the radar image integrated target C1 etc. are calculated by the highly accurate distance d R and highly accurate direction a of the image target P1, etc. P Then, the highly accurate distance d of the radar target R1 etc. can be output as the height of the radar image integrated target C1 etc.R and the number of highly accurate vertical pixels p of the image target P1, etc. H can be output based on

[0052] (Collision Risk Determination Process of the Present Disclosure) The procedure for the collision risk determination process of the present disclosure is shown in Fig. 11. A specific example of the image target contour extraction process of the present disclosure is shown in Fig. 12. A specific example of the radar image target tracking process of the present disclosure is shown in Fig. 13. A specific example of the target-to-target collision risk determination process of the present disclosure is shown in Figs. 14 and 15. A specific example of the collision risk determination process with a structure of the present disclosure is shown in Fig. 16.

[0053] In Figure 12, the image target contour extraction unit 34 extracts the contour E of the image target P for the radar image-integrated target C (including non-image-integrated targets) (the radar target R has no height), and outputs the three-dimensional position of the contour E of the image target P to coordinate G (step S41).

[0054] Here, the image target contour extraction unit 34 can output the three-dimensional position of the contour E of the image target P to the coordinate G based on the distance d and the direction a output in Figures 7 to 10. In this way, for the radar image-integrated target C (including non-image-integrated targets), the radar image target integration unit 33 can determine the size based on the radar signal point distribution or image rectangular shape information, and the image target contour extraction unit 34 can extract the three-dimensional contour based on the determined size.

[0055] In FIG. 13, the radar image target tracking unit 35 performs tracking of the radar image-integrated target C (including non-radar integrated targets and non-image integrated targets) and calculates the velocity vectors v(t1→t2) and v(t2→t3) (step S42).

[0056] Here, the radar image target tracking unit 35 can calculate the velocity vectors v(t1→t2), v(t2→t3) using an αβγ (position, velocity, acceleration) filter, a Kalman filter, a particle filter, or the like, based on the distances d(t1), d(t2), d(t3) and the directions a(t1), a(t2), a(t3) output in Figures 7 to 10.

[0057] In FIG. 14, the collision risk determination unit 36 ​​calculates a velocity vector v for a plurality of radar image-integrated targets Ci, Cj (including radar non-integrated targets and image non-integrated targets), or for each of these targets and the radar transmitting / receiving device 1 and the camera imaging device 2. i , v j Based on this (assumed to be constant over time), the risk of collision is determined (step S43).

[0058] Here, the collision risk determination unit 36 ​​calculates the velocity vector v i , v j Based on the position vector r i (t), r j (t) is calculated and the CPA is calculated as the closest point of approach within the specified period T. i、j =min(|r i (t)-r j (t)| 2 ) and calculate the CPA i、j If is smaller than the threshold, it is determined that the risk of collision is high, and CPA i、j is equal to or greater than the threshold, the risk of collision is determined to be low. In this way, the risk of collision can be determined for multiple radar-image-integrated targets Ci, Cj (including non-radar-integrated targets and non-image-integrated targets), or for each of these targets and the radar transmitting / receiving device 1 and camera imaging device 2, based on the degree of proximity within the predetermined period T.

[0059] In the left column of Figure 15, the collision risk determination unit 36 ​​determines the collision risk for multiple radar image-integrated targets Ci, Cj (including non-radar integrated targets and non-image integrated targets), or for each of these targets and the radar transceiver device 1 and camera imaging device 2, based on their type (e.g., human or vehicle, etc., and also the age of the human, etc.) (step S44).

[0060] Here, the collision risk determination unit 36 ​​determines the collision risk to be low if the radar-image-integrated objects Ci, Cj are a human and a human, and determines the collision risk to be high if the radar-image-integrated objects Ci, Cj are a human and a vehicle, based on the types of the radar-image-integrated objects Ci, Cj determined by the radar-image-target integration unit 33. The radar-image-target integration unit 33 makes a final determination of the types of the radar-image-integrated objects Ci, Cj based on the type determination result of the radar target detection unit 31 or the type determination result of the image target detection unit 32.

[0061] In the right column of Figure 15, the collision risk determination unit 36 ​​determines the collision risk for multiple radar image-integrated targets Ci, Cj (including non-radar integrated targets and non-image integrated targets), or for each of these targets and the radar transceiver device 1 and camera imaging device 2, based on their size (for example, humans are small and vehicles are large) (step S44).

[0062] Here, the collision risk determination unit 36 ​​determines the collision risk to be low if the radar-image-integrated targets Ci and Cj are both small targets, and determines the collision risk to be high if the radar-image-integrated targets Ci and Cj are both large targets, based on the sizes of the radar-image-integrated targets Ci and Cj determined by the radar-image-target integration unit 33. The radar-image-target integration unit 33 determines the sizes of the radar-image-integrated targets Ci and Cj based on the radar signal point distribution of the radar target detection unit 31 or the image rectangular shape information of the image target detection unit 32.

[0063] In this way, the risk of collision can be determined with high accuracy for multiple radar image-integrated targets Ci, Cj (including non-radar integrated targets and non-image integrated targets), or for each of these targets and the radar transmitting / receiving device 1 and camera imaging device 2, based on parameters (type and / or size) other than the degree of proximity within a predetermined period T.

[0064] In Figure 16, the collision risk determination unit 36 ​​determines the risk of collision with the structure S for the radar image-integrated target C (including non-radar-integrated targets and non-image-integrated targets) based on the distance and direction of the structure S detected by the radar transceiver 1 and the camera imaging device 2 or the three-dimensional map information of the structure S that the collision risk determination unit 36 ​​has in advance (including the position and shape of the structure S) (step S45).

[0065] Here, the collision risk determination unit 36 ​​superimposes the distance and direction of the detected structure S or the three-dimensional position and three-dimensional shape of the structure S based on pre-stored three-dimensional map information of the structure S on the three-dimensional position and three-dimensional shape of the contour E output in Fig. 12. Then, if the velocity vector v calculated in Fig. 13 is approaching the structure S, it determines that the risk of collision with the structure S is high, and if the velocity vector v calculated in Fig. 13 is moving away from the structure S, it determines that the risk of collision with the structure S is low. In this way, for radar-image-integrated targets C (including non-radar-integrated targets and non-image-integrated targets), the risk of collision with the structure S can be determined after the structure S is detected by the radar transceiver 1 and the camera imaging device 2, or even if the structure S is not detected by the radar transceiver 1 and the camera imaging device 2.

[0066] The collision risk determination unit 36 ​​outputs the distance d, direction a, height h, contour E, three-dimensional position r, velocity vector v, and collision risk for the radar-image-integrated target C (including non-radar-integrated targets and non-image-integrated targets). The collision risk display device 4 displays these parameters on a screen. The collision risk notification device 5 notifies these parameters by voice or the like. The collision risk display device 4 and the collision risk notification device 5 may alert the user to the risk of collision for both or one of multiple targets with a high risk of collision. [Industrial Applicability]

[0067] The radar image target detection device and radar image target detection program disclosed herein are applicable to environmental recognition in autonomous driving, collision prevention in safety monitoring, and high-precision detection in intrusion detection. [Explanation of symbols]

[0068] T: Radar image target detection system R1, R2, R3: Radar targets R4, R6, R7: Clusters of radar signal points R: Radar target P1, P2, P4, P5: Image targets P: Image target C1, C2, C4: Radar image integrated target C3: Radar non-integrated target C5: Image non-integrated target C, Ci, Cj: Radar image integrated target E: Contour G: Coordinates S: Structure 1: Radar transmitter and receiver 2: Camera imaging device 3: Radar image target detection device 4: Collision risk indicator 5: Collision risk notification device 31: Radar target detection unit 32: Image target detection unit 33: Radar image target integration unit 34: Image target contour extraction unit 35: Radar image target tracking unit 36: Collision risk assessment unit

Claims

1. a radar target detection unit that acquires a radar signal, detects a target (hereinafter referred to as a radar target) based on the radar signal, and measures the distance and direction of the radar target; an image target detection unit that acquires a camera image, detects a target (hereinafter referred to as an image target) based on the camera image, and measures the direction and distance of the image target; a radar image target integrating unit that integrates the radar target and the image target when a difference between the distance of the radar target and the distance of the image target is equal to or smaller than a predetermined difference and a difference between the orientation of the image target and the orientation of the radar target is equal to or smaller than a predetermined difference, When the radar image target integration unit outputs a height based on the distance to the radar target and the number of vertical pixels of the image target as the integrated height of the radar target and the image target, if the upper side of the image target is detected but the lower side of the image target is not detected, the radar image target integration unit calculates the position of the lower side of the image target based on the distance to the radar target and uses the number of vertical pixels from the position of the upper side of the image target to the position of the lower side of the image target. A radar image target detection device characterized by:

2. The radar image target integrating unit detects a cluster of radar signal points having a direction and distance closest to the direction and distance of the image target when the radar target that can be integrated with the image target is not detected, and integrates the cluster of radar signal points with the image target.

2. The radar image target detection device according to claim 1, wherein:

3. The radar image target integrating unit, when the image target integrable with the radar target is not detected, redetects a redetected image target having a distance and a direction close to the distance and direction of the radar target, and integrates the redetected image target with the radar target.

3. The radar image target detection device according to claim 1 or 2, wherein:

4. The radar image target integration unit outputs the distance of the radar target and the direction of the image target as the distance and direction of the integrated result of the radar target and the image target.

4. The radar image target detection device according to claim 1, wherein:

5. The apparatus further includes an image target contour extraction unit that extracts a contour of the image target from the combination of the radar target and the image target and outputs a three-dimensional position of the contour of the image target.

5. The radar image target detection device according to claim 1, wherein the radar image target detection device is a radar image detection device.

6. The radar image target tracking unit further includes a radar image target tracking unit that tracks the integrated radar target and image target and calculates a velocity vector.

6. The radar image target detection device according to claim 1, wherein:

7. The system further includes a collision risk determination unit that determines a collision risk for each of the radar target and the image target, or for each of the radar target and the image target, the radar transmitting / receiving device, and the camera imaging device, based on the velocity vector.

7. The radar image target detection device according to claim 6, wherein:

8. The collision risk determination unit determines the collision risk for a plurality of the radar targets and the image targets, or for each of these targets, the radar transmitting / receiving device, and the camera imaging device, based on the type and / or size.

8. The radar image target detection device according to claim 7, wherein:

9. The collision risk determination unit determines the risk of collision with the structure based on the integrated radar target and the image target, based on the distance and direction of the structure detected by the radar transmitting / receiving device and the camera imaging device, or based on three-dimensional map information of the structure that the collision risk determination unit has in advance.

9. The radar image target detection device according to claim 7 or 8, wherein:

10. a radar target detection step of acquiring a radar signal, detecting a target (hereinafter referred to as a radar target) based on the radar signal, and measuring the distance and direction of the radar target; an image target detection step of acquiring a camera image, detecting a target (hereinafter referred to as an image target) based on the camera image, and measuring the azimuth and distance of the image target; a radar image target integration step of integrating the radar target and the image target when a difference between the distance of the radar target and the distance of the image target is equal to or smaller than a predetermined difference and a difference between the orientation of the image target and the orientation of the radar target is equal to or smaller than a predetermined difference; on the computer, In the radar image target integration step, when an upper side of the image target is detected but a lower side of the image target is not detected, the position of a lower side of the image target is calculated based on the distance of the radar target, and the number of vertical pixels from the position of the upper side of the image target to the position of the lower side of the image target is used as the height of the integrated result of the radar target and the image target. A radar image target detection program comprising:

Images