Detection device

Abstract

To accurately detect a position and speed of a pedestrian.SOLUTION: A position and speed of a pedestrian existing in a detection area are detected based on three-dimensional point group information of the detection area acquired by a rider and the position and speed of an object existing in the detection area acquired by a millimeter wave radar.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a detection device. [Background technology]

[0002] There is a technology that detects pedestrians crossing a crosswalk and warns the driver of a vehicle to be careful (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Utility Model Application Publication No. 59-160400 Summary of the Invention [Problem to be solved by the invention]

[0004] Various technologies have been proposed for detecting pedestrians. For example, Patent Document 1 discloses photoelectric sensors, ultrasonic sensors, microwave Doppler sensors, and the like as technologies for detecting pedestrians crossing a crosswalk. However, the technologies proposed so far each have their advantages and disadvantages.

[0005] Therefore, an object of the present invention is to accurately detect the position and speed of a pedestrian. [Means for solving the problem]

[0006] In order to solve the above problem, a detection device according to one embodiment of the present invention has a lidar that acquires three-dimensional point cloud information of a detection area, a millimeter-wave radar that acquires the position and speed of an object present in the detection area, and a control unit that detects the position and speed of a pedestrian present in the detection area based on the three-dimensional point cloud information of the detection area acquired by the lidar and the position and speed of an object present in the detection area acquired by the millimeter-wave radar.

[0007] The control unit may detect a pedestrian present in the detection area based on three-dimensional point cloud information of the detection area acquired by the lidar.

[0008] The control unit may detect a position of the detected pedestrian based on three-dimensional point cloud information of the detection area acquired by the lidar.

[0009] The control unit may detect the speed of each of the detected pedestrians based on the position and speed of the object acquired by the millimeter wave radar.

[0010] A detection method according to one embodiment of the present invention is a detection method executed by a computer, which detects the position and speed of a pedestrian in a detection area based on three-dimensional point cloud information of the detection area acquired by a lidar and the position and speed of an object in the detection area acquired by a millimeter-wave radar.

[0011] A detection program according to an embodiment of the present invention causes a computer to execute the detection method.

[0012] A computer-readable storage medium according to an embodiment of the present invention stores the detection program. [Effects of the Invention]

[0013] The present invention makes it possible to accurately detect the position and speed of a pedestrian. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram showing a detection device 100 according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing an example of a processing operation in a control unit 130. [Figure 3] FIG. 2 is a diagram illustrating an example of the installation position of a millimeter wave radar 120. [Figure 4]FIG. 2 is a diagram illustrating an example of the installation position of a millimeter wave radar 120. [Figure 5] FIG. 2 is a diagram illustrating an example of the installation position of a millimeter wave radar 120. [Figure 6] FIG. 2 is a diagram illustrating an example of the installation position of a millimeter wave radar 120. DETAILED DESCRIPTION OF THE INVENTION

[0015] <Detection device 100> 1 is a diagram showing a detection device 100 according to one embodiment of the present invention. The detection device 100 includes a LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) 110, a millimeter-wave radar 120, and a control unit 130.

[0016] The lidar 110 is a device that acquires a three-dimensional point cloud of an object by irradiating the object with laser light and receiving the light reflected from the object. In this embodiment, the lidar 110 acquires information (three-dimensional point cloud information) related to the three-dimensional point cloud of an object existing in a first area A1. Here, the first area A1 is, for example, a three-dimensional area within the field of view of the lidar 110.

[0017] The millimeter-wave radar 120 is a device that acquires the position and speed of an object by irradiating the object with millimeter waves and receiving the reflected waves from the object. In this embodiment, the millimeter-wave radar 120 detects an object present in a second area and acquires the position and speed of the object (object detected by the millimeter-wave radar). Here, the second area A2 is, for example, a three-dimensional area within the field of view of the millimeter-wave radar 120.

[0018] In the case of an object whose extent is greater than the resolution of the millimeter-wave radar 120, the millimeter-wave radar 120 detects the object as a cluster of multiple points. Therefore, the millimeter-wave radar 120, for example, clusters the detected points of the object to identify the object, and sets a representative point among the clustered points (e.g., the center point of the clustered points) as the position of the identified object. In this case, if the millimeter-wave radar 120 is a three-dimensional millimeter-wave radar, it acquires a three-dimensional position as the position of the object, and if it is a two-dimensional millimeter-wave radar, it acquires a two-dimensional position (e.g., distance, azimuth angle) as the position of the object. The millimeter-wave radar 120 detects the speed of the object using the Doppler effect in the waves reflected by the object. Therefore, the millimeter-wave radar 120 detects the speed of the object in the direction of the millimeter-wave radar 120 as the speed of the object.

[0019] In this embodiment, the first area A1 and the second area A2 are set to overlap each other. Therefore, in this embodiment, for an object existing in the area where the first area A1 and the second area A2 overlap, a three-dimensional point cloud is acquired by the lidar 110, and the position and velocity of the object are acquired by the millimeter-wave radar 120.

[0020] The control unit 130 is configured by a computer, and detects the position and speed of a pedestrian present in the detection area DA based on the three-dimensional point cloud information acquired by the LIDAR 110 and the position and speed of the object acquired by the millimeter-wave radar 120. The detection area DA is an area included in the overlapping portion of the first area A1 and the second area A2, and is, for example, a three-dimensional area. Therefore, in this embodiment, for an object present in the detection area DA, the three-dimensional point cloud is acquired by the LIDAR 110, and the position and speed are acquired by the millimeter-wave radar 120.

[0021] At this time, the control unit 130 first detects pedestrians present in the detection area DA based on the three-dimensional point cloud information acquired by the LIDAR 110. The control unit 130 identifies objects present in the detection area DA, for example, by clustering the three-dimensional point cloud acquired by the LIDAR 110, and detects pedestrians present in the detection area DA by recognizing the shapes of the identified objects.

[0022] The control unit 130 then detects the position of each detected pedestrian based on the three-dimensional point cloud information acquired by the LIDAR 110. The control unit 130 may, for example, set the position of a representative point of the point cloud of the detected pedestrian as the position of the pedestrian. Here, the representative point of the pedestrian point cloud may be set as appropriate. For example, the center point of the pedestrian point cloud may be set as the representative point of the pedestrian point cloud.

[0023] The control unit 130 detects the speed of each detected pedestrian based on the position and speed of the object acquired by the millimeter-wave radar 120. For example, for each detected pedestrian, the control unit 130 identifies an object corresponding to the pedestrian from among the objects detected by the millimeter-wave radar, and sets the speed of the identified object (i.e., the speed of the object detected by the millimeter-wave radar 120) as the speed of the pedestrian. At this time, for each detected pedestrian, the control unit 130 identifies, for example, an object detected by the millimeter-wave radar within a predetermined distance from the position of the pedestrian as the object corresponding to the pedestrian. When calculating the distance between the position of the pedestrian and the position of the object detected by the millimeter-wave radar, it is preferable to appropriately perform coordinate conversion between the position of the pedestrian and the position of the object detected by the millimeter-wave radar so that the position of the pedestrian and the position of the object detected by the millimeter-wave radar are expressed in the same coordinate system.

[0024] As described above, in this embodiment, a pedestrian and his / her position are detected using the three-dimensional point cloud information acquired by the LIDAR 110, and the speed of the object acquired by the millimeter-wave radar 120 is used as the speed of the detected pedestrian. Using the data acquired by the LIDAR 110 makes it possible to more accurately detect the position of an object present in the detection area, and to more accurately detect whether the object is a pedestrian. Furthermore, using the data acquired by the millimeter-wave radar 120 makes it possible to more accurately detect the speed of an object present in the detection area, than using the data acquired by the LIDAR 110. Therefore, in this embodiment, it is possible to accurately detect a pedestrian in the detection area, and to accurately detect the position and speed of the pedestrian.

[0025] The control unit 130 may calculate the time it takes for the pedestrian to reach the target position TP based on the position and speed of the pedestrian. In this way, it becomes possible to calculate the time it takes for the pedestrian to complete crossing the crosswalk, the time it takes for the pedestrian to reach the crosswalk, the time it takes for the pedestrian to complete crossing the vehicle entrance / exit, the time it takes for the pedestrian to reach the vehicle entrance / exit, etc., as will be described in detail below.

[0026] The detection device 100 may further include a notification unit 140 that notifies information (for example, a display device that displays information, an audio output device that outputs audio related to the information, or a communication device that transmits information to another device). The control unit 130 may then use the notification unit 140 to notify the driver of the time it takes for the pedestrian to complete crossing the crosswalk, the time it takes for the pedestrian to reach the crosswalk, the time it takes for the pedestrian to complete crossing the vehicle entrance / exit, the time it takes for the pedestrian to reach the vehicle entrance / exit, etc. This makes it possible to notify drivers of vehicles traveling toward the crosswalk and drivers traveling toward the entrance / exit of the timing when they need to pay attention to pedestrians.

[0027] 2 is a diagram showing an example of a processing operation in the control unit 130. The control unit 130 detects pedestrians present in the detection area DA based on the three-dimensional point cloud information acquired by the LIDAR 110 (S201). The control unit 130 detects the position of each detected pedestrian based on the three-dimensional point cloud information acquired by the LIDAR 110 (S202). The control unit 130 detects the speed of each detected pedestrian based on the position and speed of the object acquired by the millimeter-wave radar 120 (S203).

[0028] The control unit 130 may track pedestrians by repeatedly performing the processing operations in FIG.

[0029] The data acquired by the LIDAR 110 and the data acquired by the millimeter-wave radar 120 used in the processing operation of Fig. 2 should be acquired at the same time. If they cannot be acquired at the same time, the difference in acquisition timing between the data acquired by the LIDAR 110 and the data acquired by the millimeter-wave radar 120 used in the processing operation of Fig. 2 should be set to a predetermined time or less. In this case, the predetermined distance used when identifying an object corresponding to a pedestrian from among the objects detected by the millimeter-wave radar should be determined based on this predetermined time.

[0030] <Detecting pedestrians at crosswalks> The detection area DA may include, for example, the crosswalk. The detection area DA may include, for example, the crosswalk and the area above it. This makes it possible to detect pedestrians walking on the crosswalk and to detect the position and speed of the pedestrians. In this case, the leading edge of the crosswalk is set as the target position TP, and the control unit 130 calculates the time it takes for the pedestrian to complete crossing the crosswalk, and the notification unit 140 notifies the user of the time it takes for the pedestrian to complete crossing the crosswalk.

[0031] When the detection area DA includes a pedestrian crossing, the lidar 110 may be installed in any position as long as the pedestrian crossing is included in the first area A1. In this case, the millimeter-wave radar 120 should be installed in a position where the pedestrian crossing is included in the second area A2 (i.e., the pedestrian crossing is included in the two-dimensional area of ​​the second area A2 that intersects with the ground) and where the speed of pedestrians crossing the pedestrian crossing can be detected using the Doppler effect. Therefore, if the detection area DA includes a pedestrian crossing, the millimeter-wave radar 120 should be installed so that the direction RD from the millimeter-wave radar 120 to each point on the pedestrian crossing PC is not perpendicular to the walking direction of pedestrians walking on the pedestrian crossing PC (e.g., the crossing direction D1 of the pedestrian crossing), as shown in FIG. 3 . In other words, the millimeter-wave radar 120 should be installed in an area ahead of and / or behind the crossing direction D1 of the pedestrian crossing.

[0032] The detection area DA may include a predetermined area PA1 on the sidewalk in front of the crosswalk. For example, the detection area DA may be the predetermined area PA1 and the area above it. This makes it possible to detect pedestrians approaching the crosswalk and to determine their position and speed. The edge of the crosswalk on the sidewalk where the pedestrian is walking is set as the target position TP, and the control unit 130 calculates the time at which the pedestrian will reach the crosswalk and notifies the notification unit 140 of the time at which the pedestrian will reach the crosswalk.

[0033] When the detection area DA includes a predetermined area PA1 of the sidewalk in front of the crosswalk, the lidar 110 may be installed in any position as long as the predetermined area PA1 of the sidewalk in front of the crosswalk is included in the first area A1. Meanwhile, in this case, the millimeter-wave radar 120 should be installed in a position where the second area A2 includes the predetermined area PA1 of the sidewalk in front of the crosswalk (that is, the predetermined area PA1 is included in the two-dimensional area part of the second area A2 that intersects with the ground), and where the speed of pedestrians walking toward the crosswalk can be detected by the Doppler effect.

[0034] As shown in FIG. 4, in the case of a crosswalk PC installed at an intersection, there are two walking directions in which pedestrians walk toward the crosswalk PC (directions D2 and D3 in FIG. 4). Therefore, in order to detect the speed of a pedestrian walking toward the crosswalk PC at an intersection, at least two millimeter-wave radars 120 are required, for example, as shown in FIG. 4. Therefore, if the detection area DA is an area that includes a predetermined area PA1 on the sidewalk SW in front of the crosswalk PC installed at the intersection, it is advisable to install the two millimeter-wave radars 120 so that the field-of-view center directions CD1 and CD2 of the two millimeter-wave radars 120 are not parallel to each other, as shown in FIG. 4. In the example shown in FIG. 4, the field-of-view center direction CD1 of one of the two millimeter-wave radars 120 is parallel to one of the two walking directions, walking direction D2, and the field-of-view center direction CD2 of the other of the two millimeter-wave radars 120 is parallel to the other of the two walking directions, walking direction D3.

[0035] As shown in Fig. 5, in the case of a crosswalk PC installed at a position other than an intersection, there is only one walking direction toward the crosswalk PC (direction D4 in Fig. 5). Therefore, if the detection area DA is an area that includes a predetermined area PA1 of the sidewalk SW in front of the crosswalk PC installed at a position other than an intersection, for example, as shown in Fig. 5, it is advisable to install the millimeter-wave radar 120 so that the direction RD from the millimeter-wave radar 120 to each point in the predetermined area PA1 is not perpendicular to the walking direction of pedestrians walking in the predetermined area PA1 (for example, the walking direction of the sidewalk SW in front of the crosswalk PC (i.e., the direction in which the sidewalk SW extends) D4). In other words, it is advisable to install the millimeter-wave radar 120 in an area ahead of and / or behind the walking direction D4 of the predetermined area PA1.

[0036] <Detecting pedestrians crossing the road in front of a vehicle entrance / exit> The detection area DA may include, for example, a predetermined area PA2 of a road (e.g., a sidewalk) that crosses in front of a vehicle entrance / exit to a roadway. The detection area DA may be, for example, the predetermined area PA2 and the area above it. This makes it possible to detect pedestrians walking in front of a vehicle entrance / exit or pedestrians walking toward the vehicle entrance / exit, and to detect the position and speed of the pedestrians. When a pedestrian is walking in front of the entrance / exit, the leading edge of the pedestrian crossing the entrance / exit may be set as the target position TP, and the control unit 130 may calculate the time it takes for the pedestrian to complete crossing in front of the entrance / exit, and the notification unit 140 may notify the pedestrian of the time it takes for the pedestrian to complete crossing in front of the entrance / exit. Alternatively, when a pedestrian is walking toward the entrance / exit, the near end of the entrance / exit may be set as the target position TP, and the control unit 130 may calculate the time it takes for the pedestrian to reach the entrance / exit, and the notification unit 140 may notify the pedestrian of the time it takes for the pedestrian to complete crossing in front of the entrance / exit.

[0037] In this case, the lidar 110 may be installed at any position as long as it is included in the predetermined area PA2 in the first area A1. Meanwhile, the millimeter-wave radar 120 is installed at a position that is included in the predetermined area PA2 in the second area A2 (i.e., the predetermined area PA2 is included in the two-dimensional area of ​​the second area A2 that intersects with the ground), and that allows the speed of a pedestrian walking in the predetermined area PA to be detected by the Doppler effect. Therefore, if the detection area DA is an area including a predetermined area PA2 of a road that crosses in front of a vehicle entrance / exit, for example, as shown in FIG. 6, it is advisable to install the millimeter-wave radar 120 so that the direction from the millimeter-wave radar 120 to each point in the predetermined area PA2 is not perpendicular to the walking direction D5 of a pedestrian walking in the predetermined area PA2 (for example, the walking direction D5 of the sidewalk SW in front of the entrance / exit EN to the roadway DW (i.e., the crossing direction of the entrance / exit EN)). In other words, it is advisable to install the millimeter-wave radar 120 in an area ahead of and / or behind the walking direction D5 of the predetermined area PA2.

[0038] The present invention has been described above in terms of preferred embodiments thereof. While the present invention has been described herein with reference to specific examples, various modifications and variations can be made to these examples without departing from the spirit and scope of the present invention as set forth in the claims. [Explanation of symbols]

[0039] 100 Detection device 110 Rider 120 mm wave radar 130 Control Unit 140 Notification Department

Claims

1. a lidar that acquires three-dimensional point cloud information of a detection area; a millimeter wave radar for acquiring the position and velocity of an object present in the detection area; a control unit that detects the position and speed of a pedestrian present in the detection area based on three-dimensional point cloud information of the detection area acquired by the lidar and the position and speed of an object present in the detection area acquired by the millimeter-wave radar, the detection area includes a predetermined area of ​​the sidewalk in front of the crosswalk, The control unit calculates a time at which the pedestrian will reach the crosswalk based on the position and speed of the detected pedestrian.

2. a lidar that acquires three-dimensional point cloud information of a detection area; a millimeter wave radar for acquiring the position and velocity of an object present in the detection area; a control unit that detects the position and speed of a pedestrian present in the detection area based on three-dimensional point cloud information of the detection area acquired by the lidar and the position and speed of an object present in the detection area acquired by the millimeter-wave radar, the detection area includes a predetermined area of ​​a road crossing in front of a vehicle entrance / exit; The control unit calculates a time for the pedestrian to cross in front of the entrance / exit or a time for the pedestrian to reach the entrance / exit based on the position and speed of the detected pedestrian.

3. The detection device according to claim 1 , wherein the control unit detects a pedestrian present in the detection area based on three-dimensional point cloud information of the detection area acquired by the lidar.

4. The detection device according to claim 3 , wherein the control unit detects a position of the detected pedestrian based on three-dimensional point cloud information of the detection area acquired by the lidar.

5. The detection device according to claim 3 , wherein the control unit detects the speed of each of the detected pedestrians based on the position and speed of the object acquired by the millimeter-wave radar.

Images