Object detection device and object detection method

Abstract

To provide an object detection device and an object detection method which can accurately perform clustering regardless of the distance from a sensor head (LiDAR).SOLUTION: An object detection device 10 includes: an input part 11a in which point group information is input from a LiDAR 2 installed on the ground; and a processing part 11b for executing clustering processing on the basis of a threshold set for a point-to-point distance between respective points which are included in the point group information input in the input part 11a. The processing part 11b executes the clustering processing by each of the plurality of thresholds set for the point-to-point distance so as to execute duplicated cluster organization processing for deleting duplicated clusters among clusters obtained in the clustering processing.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an object detection device and an object detection method for detecting an object on, for example, a road. [Background technology]

[0002] As an object detection device for detecting objects on a road, a method is known in which an object is detected using clusters generated by clustering a point cloud detected by LiDAR (Light Detection And Ranging).

[0003] For example, Patent Document 1 describes clustering into clusters that are subsets of a group of ranging points based on the inter-point distances between the ranging points. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-006716 A Summary of the Invention [Problem to be solved by the invention]

[0005] The clustering described above is performed based on a predetermined threshold value for the distance between points. Because the LiDAR laser has a radial spread, the number of reflection points per unit area decreases and the distance between points increases the farther away from the sensor head. If the threshold value for the distance between points is not set large enough at distant points, there is a risk that reflection points from the same object will be treated as separate clusters.

[0006] On the other hand, if the threshold for the distance between points is too large, there is a risk that reflection points from multiple objects that are close to the sensor head may be treated as a single cluster.

[0007] Therefore, an object of the present invention is to provide an object detection device and an object detection method that are capable of performing clustering with high accuracy regardless of the distance from a sensor head (LiDAR). [Means for solving the problem]

[0008] The invention described in claim 1, which has been made to solve the above problem, is an object detection device comprising: an input unit to which point cloud information is input from a sensor that is installed on the ground, intermittently irradiates light to scan a predetermined space, receives reflected light of the light, and acquires point cloud information based on the reflected light; and a processing unit that performs clustering processing based on a threshold value set for the inter-point distance of each point including the point cloud information input to the input unit, wherein the processing unit performs clustering processing based on each of a plurality of threshold values ​​set for the inter-point distance, and performs processing to delete overlapping clusters from the clusters obtained by the clustering processing. Effect of the Invention

[0009] According to the present invention, clustering processing is performed using multiple inter-point distance thresholds, so that what would be a single cluster containing multiple objects using a single threshold can be separated into multiple clusters. Also, if there are overlapping clusters generated using multiple thresholds, one of the clusters can be deleted to align the clusters with the real objects. [Brief description of the drawings]

[0010] [Figure 1] 1 is an example of installation of an object detection device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of the object detection device shown in FIG. [Diagram 3] 3 is a flowchart of a process for creating a list of distances between points in the object detection device shown in FIG. 2. [Figure 4] FIG. 11 is an explanatory diagram of the maximum depth distance dmax and the maximum point-to-point distance wmax to be detected. [Diagram 5]3 is a flowchart of a multi-clustering process in the object detection device shown in FIG. 2. [Figure 6] FIG. 11 is an explanatory diagram of a multi-clustering process. [Figure 7] FIG. 13 is a diagram showing an example in which a cluster of interest is included only once in another cluster. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, one embodiment of the present invention will be described with reference to Figures 1 to 7. Figure 1 shows an example of installation of an object detection device according to one embodiment of the present invention.

[0012] The object detection device 10 shown in Fig. 1 is installed on a pillar P fixed near an intersection, for example. The object detection device 10 detects moving objects such as pedestrians and vehicles passing through the intersection. That is, the object detection device 10 is installed on the ground. Installed on the ground here means that it is fixed to the ground directly or indirectly, that is, it means that it is installed on an object that is fixedly installed on the ground, such as a building, rather than on a moving object such as a vehicle.

[0013] The object detection device 10 includes a main body 1 and a LiDAR 2. The main body 1 is provided as a control box or the like at the bottom of a support P. As shown in the block diagram of FIG. 2, the main body 1 includes a control unit 11 and a communication unit 12.

[0014] The control unit 11 is configured with, for example, a computer, etc. The control unit 11 is responsible for the overall control of the object detection device 10. In Fig. 1, the control unit 11 functionally includes an input unit 11a, a processing unit 11b, and a detection unit 11c.

[0015] The input unit 11a receives point cloud information acquired by the LiDAR 2. The processing unit 11b performs clustering processing based on a threshold value set for the inter-point distances of each point included in the point cloud information input to the input unit 11a. The detection unit 11c detects an object based on the point cloud clustered by the processing unit 11b. The detection unit 11c determines the type of moving object, such as a pedestrian or a vehicle, from the shape, size, etc. of the cluster, for example.

[0016] The communication unit 12 outputs information about the object detected by the control unit 11 to the outside. The communication unit 12 may be configured as, for example, a wireless communication device (or a wired communication device). Information about the object may include the type, moving direction, and moving speed of the object. Specific examples of output destinations include moving bodies such as vehicles passing near the pillar P, and traffic control centers.

[0017] The LiDAR2 is a sensor that intermittently irradiates a laser beam to scan a predetermined space, receives reflected light of the laser beam, and acquires point cloud information (three-dimensional point cloud information) based on the reflected light. The LiDAR2 discretely measures the distance to an object present in the space of the irradiation range by the reflected light of the irradiated laser beam, and recognizes the position and shape of the object as a three-dimensional point cloud. In this embodiment, the LiDAR2 is installed on the top of the support P as shown in FIG. 1, and scans the road surface R (the road surface R and the space above it) as the predetermined space.

[0018] Next, the clustering operation of the object detection operation in the object detection device 10 having the above configuration will be described with reference to Fig. 3 to Fig. 7. Fig. 3 is a flowchart of the point-to-point distance list creation process. The flowchart in Fig. 3 is executed before executing the flowchart in Fig. 5 described later.

[0019] The point-to-point distance list is a list of thresholds for the point-to-point distance used in the multi-clustering process described later. In other words, multiple thresholds are generated and listed by executing the flowchart in FIG.

[0020] First, the processing unit 11b calculates the maximum inter-point distance wmax at the maximum depth distance dmax to be detected by the LiDAR 2 (step S11). Here, the maximum depth distance dmax and the maximum inter-point distance wmax will be described with reference to FIG. 4. FIG. 4 is a diagram showing a schematic diagram of a laser beam intermittently emitted from the LiDAR 2. In FIG. 4, the laser beam emitted from the LiDAR 2 reaches a point at a depth distance d. The depth is the direction in which the laser beam is emitted as seen from the LiDAR 2, as shown in FIG. 4.

[0021] If the horizontal resolution of the LiDAR2 is θ, then the inter-point distances w1, w2, ..., wn at the depth distance d are as shown in the formula. The formula for calculating wn among the formulas shown in FIG. 4 is shown below as formula (1). wn=d*tan(nθ)-d*tan((n-1)θ)···(1)

[0022] That is, the maximum depth distance dmax to be detected is the maximum depth distance d to be detected by the LiDAR2. The maximum point-to-point distance wmax is the point-to-point distance wn between the outermost illuminated point at the maximum depth position dmax and the illuminated point inside it. Due to the structure of the LiDAR2, the maximum point-to-point distance at the maximum depth position dmax is this wn. Therefore, if the above-mentioned maximum depth position dmax, the resolution θ, and the number of horizontal shots n on one side of the LiDAR2 are known, the maximum point-to-point distance wmax can be obtained.

[0023] Returning to the explanation of Fig. 3, the processing unit 11b then calculates a difference value wdiff between the inter-point distance list elements (step S12). The difference value wdiff is calculated by the following formula (2). wdiff = (wmax - wmin) / j (2)

[0024] In equation (2), wmin is a numerical value that is sufficiently smaller than the minimum distance between nearby objects that is desired to be separated (hereinafter, referred to as the minimum inter-point distance wmin). For example, if it is desired to separate pedestrians walking side by side when they are 500 mm or more apart, but if they are 499 mm apart, separation is acceptable even if they cannot be separated, then the minimum inter-point distance wmin is wmin=499 mm. In equation (2), j is the number by which the inter-point distance list is to be divided. This number j to be divided is set to a value of 1 or more. The upper limit of the number j to be divided depends on the processing capacity of the processing unit 11b, but it is desirable to set it to approximately 10 or less.

[0025] The number j to be divided and the difference value wdiff are explained below. For example, suppose the point distance list has two elements (1000 mm and 500 mm). In this case, if clustering processing is performed using a point distance of 1000 mm as the standard (threshold), it is possible to group the points even at a depth point where the laser irradiation interval spreads to 999 mm. On the other hand, two objects moving side by side with an interval of 999 mm or less will be merged. If clustering processing is performed using a point distance of 500 mm as the standard, two objects moving side by side with an interval of 501 mm or more can be detected separately. However, clustering processing based on a point distance of 500 mm as the standard does not work effectively at a depth point where the laser irradiation interval spreads to 501 mm or more.

[0026] Here, when detecting two objects moving side by side at an interval of 800 mm at a depth point where the laser irradiation interval widens to 749 mm, clustering processing based on a point-to-point distance of 1000 mm will detect them as one object. On the other hand, clustering processing based on a point-to-point distance of 500 mm will not create groups of points and all points will become isolated points. Therefore, if clustering processing with a point-to-point distance of 750 mm is performed between clustering processing with a point-to-point distance of 1000 mm and clustering processing with a point-to-point distance of 500 mm, it will be possible to detect them as two objects. In other words, the number of divisions j is a value that determines how many divisions to make between the maximum point-to-point distance wmax and the minimum point-to-point distance wmin.

[0027] For example, if a request is received saying, "I want to detect objects with a depth distance of 100 m or less. Also, I want to detect objects moving at intervals of 501 mm or more separately. However, if they move closer together by 500 mm or less, it is acceptable even if they cannot be separated," then the maximum reference point distance (wmax) will be the laser irradiation interval at a depth distance of 100 m as described above. For example, if the irradiation interval widens to 999 mm, a point distance (threshold) of 1000 mm is required. On the other hand, the minimum point distance (wmin) will be 500 mm based on the separation request. The number j to be divided as described above is, in other words, the number of extra clustering processes that can be added, and the difference value wdiff is equal to the section from the maximum point distance wmax to the minimum point distance wmin according to this j.

[0028] Next, the processing unit 11b calculates the i-th point-to-point distance list L (step S13). The i-th point-to-point distance Li can be calculated by the following formula (3). Here, i is an integer, and assuming that i starts from 0 in this step S13, i is incremented by 1 each time step S13 is executed, and is repeated as long as the relationship i≦j is satisfied. In other words, step S13 is repeated j+1 times. Li = wmax-wdiff*i (3)

[0029] In this way, the inter-point distance list L is created. For example, if the maximum inter-point distance wmax = 1000 mm, the minimum inter-point distance wmin = 500 mm, and the number of divisions j = 2, the difference value wdiff will be 250, and the inter-point distance list L will have three elements: (1000 mm, 750 mm, 500 mm).

[0030] Next, the multi-clustering process will be described with reference to the flowchart in Fig. 5. The multi-clustering process is a process of performing clustering multiple times using the point-to-point distances included in the point-to-point distance list L generated by executing the flowchart in Fig. 3 as threshold values.

[0031] First, the following steps S21 and S22 are repeated for the length of the point-to-point distance list L. For example, if the point-to-point distance list L is L0=1000, L1=750, and L2=500, the list is repeated three times from L0 to L2. The point-to-point distance list L contains a point-to-point distance of 2 or more because the number j to be divided is 1 or more and because of equation (2). Therefore, the loop (clustering loop) of steps S21 and S22 is executed multiple times.

[0032] In step S21, the processing unit 11b executes the clustering process. The clustering process is a well-known process in which point cloud information acquired by the LiDAR 2 is input to the input unit 11a, and clusters are generated for the input point cloud information based on the inter-point distance (threshold value) selected from the inter-point distance list L, as described above.

[0033] In the next step S22, the processing unit 11b excludes points that are not included in each cluster. This step S22 is performed to speed up processing because, when repeating the clustering process, points that were not included in a cluster in clustering with a larger inter-point distance are unlikely to be included in a cluster in clustering with a smaller inter-point distance. By performing the process of this step S22, all points that are the subject of clustering when the next loop is executed are points that are included in the clusters.

[0034] The following steps S23 to S25 are repeated for the number of clusters generated in step S21. These steps S23 to S25 are also called an overlapping cluster sorting loop. In step S23, the processing unit 11b determines whether or not multiple clusters are included in a cluster that is being focused on as a processing target in each loop process (hereinafter, a "focused cluster"). The "focused cluster" includes multiple clusters when cluster α2 includes clusters β1 and β2, as shown in FIG. 6. In FIG. 6, when there is a laser beam irradiated from the LiDAR2 and objects V (vehicle), P1 (pedestrian), and P2 (pedestrian) within the range of the laser beam, clusters α1 and α2 are generated at a point-to-point distance a, and clusters β1 and β2 are generated at a point-to-point distance b that is shorter than the point-to-point distance a.

[0035] Returning to the explanation of Fig. 5, if the result of the determination in step S23 indicates that multiple clusters are included (step S23: yes), the processing unit 11b deletes the cluster of interest (step S24). This step S24 corresponds to, for example, the case in Fig. 6 where the cluster of interest is cluster α2. In this case, cluster α2 includes cluster β1 and cluster β2. This is because, for example, cluster α2 was generated based on inter-point distance a, but since it actually includes objects P1 and P2, cluster α2 is unnecessary, and it is appropriate to leave only cluster β1 and cluster β2 generated based on inter-point distance b.

[0036] On the other hand, if the result of the determination in step S23 is that multiple clusters are not included (step S23: no), the processing unit 11b determines whether the target cluster is included only once in another cluster (step S25). Step S25 will be described with reference to Fig. 7. Fig. 7 shows a side view of a track T. It is assumed that a cluster γ1 and a cluster γ2 are generated for this track T, and that the cluster γ2 is included only once in the cluster γ1. It is also assumed that the black circles in the figure represent points of the point cloud.

[0037] This means that the state in which the cluster of interest is contained only once in another cluster means that in Fig. 7, the density of points varies on the horizontal part of an object, such as the bed of truck T. Therefore, it is appropriate to delete the small cluster γ2 and leave only the large cluster γ1.

[0038] As a result of the determination in step S24, when the cluster of interest is included only once in the other clusters (step S24: yes), the processing unit 11b proceeds to step S25, that is, deletes the cluster of interest as described above.

[0039] On the other hand, if the cluster of interest is not included in only one other cluster (step S24: no), the processing unit 11b ends the loop of steps S23 to S25 and organizes the overlapping clusters of the next cluster. When the organization of the overlapping clusters of all clusters has been completed, the flow chart ends.

[0040] Here, an example will be used to explain the overlapping cluster sorting loop in Fig. 5. For example, if the inter-point distance list L has three elements (1000mm, 750mm, 500mm), the clustering loop (steps S21, S22) is executed three times.

[0041] In this case, if there are two objects moving side by side with an interval of 800 mm at a depth point where the laser irradiation interval widens to 749 mm, clustering processing with a point-to-point distance of 1000 mm will output one large cluster surrounding the two objects. Clustering processing with a point-to-point distance of 750 mm will output two clusters surrounding each of the side-by-side objects. Clustering processing with a point-to-point distance of 500 mm will not output any clusters. As a result, if all of the output clusters are adopted, the result will be unrealistic, with three outputs even though there are only two objects, so it is necessary to organize the overlapping clusters.

[0042] Therefore, in the overlapping cluster sorting loop, the clusters output as a result of the clustering loop are looked at in order and processed. When the cluster output by the clustering process with a point-to-point distance of 1000 mm is looked at, the spatial spread of that cluster contains two clusters output by the clustering process with a point-to-point distance of 750 mm. Therefore, the cluster output by the clustering process with a point-to-point distance of 1000 mm is deleted. As a result, what actually exists are two objects running side by side, and there are two clusters remaining, so they match.

[0043] According to the above-mentioned flowchart, step S21 functions as an input step, and the clustering loop functions as a processing step.

[0044] According to this embodiment, the object detection device 10 includes an input unit 11a to which point cloud information is input from a LiDAR 2 installed on the ground, and a processing unit 11b that performs clustering processing based on a threshold value set for the inter-point distance of each point included in the point cloud information input to the input unit 11a. The processing unit 11b performs clustering processing using each of a plurality of threshold values ​​set for the inter-point distance, and performs overlapping cluster sorting processing to delete overlapping clusters from among the clusters obtained by the clustering processing.

[0045] By configuring object detection device 10 as described above, clustering processing is performed using multiple point-to-point distance thresholds, so that what would be one cluster containing multiple objects using only one threshold can be separated into multiple clusters. Also, if there are overlaps in the clusters generated using multiple point-to-point distances, they can be organized and deleted to match the real objects with the clusters.

[0046] Moreover, the processing unit 11b calculates a plurality of thresholds in advance based on the depth distance d from the LiDAR 2. In this way, it is possible to set thresholds such that the inter-point distance increases as the distance from the LiDAR 2 increases, for example.

[0047] Moreover, the processing unit 11b calculates a plurality of thresholds based on the maximum depth distance dmax and the minimum inter-point distance wmin that can be detected by the LiDAR 2. In this way, it is possible to obtain the maximum inter-point distance wmax at the maximum depth distance dmax, and it becomes possible to include at least the maximum inter-point distance wmax and the minimum inter-point distance wmin in the thresholds.

[0048] Moreover, the processing unit 11b calculates a plurality of thresholds by equally dividing the distance between the maximum inter-point distance wmax at the maximum depth distance dmax that the LiDAR 2 can detect and the minimum inter-point distance wmin. In this way, in addition to the maximum inter-point distance wmax and the minimum inter-point distance wmin, the inter-point distance corresponding to the intermediate distance between them can also be set as a threshold. Therefore, the clustering process can be performed with higher accuracy.

[0049] Furthermore, the processing unit 11b deletes a cluster of interest when the cluster of interest contains multiple clusters, thereby making it possible to delete overlapping clusters.

[0050] Furthermore, the processing unit 11b deletes the cluster of interest when the cluster of interest is the only one included in another cluster. In this way, when the cluster of interest is the only one included in another cluster, such as in a state where the density of points varies in a horizontal part of the object, the cluster of interest can be deleted to match the real object with the cluster.

[0051] The present invention is not limited to the above-described embodiment. In other words, a person skilled in the art can implement the present invention by modifying it in various ways in accordance with conventionally known knowledge without departing from the gist of the present invention. As long as the object detection device and object detection method of the present invention are still included in the scope of the present invention even after such modification, the object detection device and object detection method of the present invention are of course included in the scope of the present invention. [Explanation of symbols]

[0052] 1 Main unit 2 LiDAR (sensor) 10 Object detection device 11 Control section 11a Input section 11b Processing section 11c Detection unit

Claims

1. an input unit that receives the point cloud information from a sensor that is installed on the ground, intermittently irradiates light to scan a predetermined space, receives reflected light of the light, and acquires point cloud information based on the reflected light; a processing unit that performs a clustering process based on a threshold value set for a distance between each point including the point cloud information input to the input unit; Equipped with the processing unit performs a clustering process using each of a plurality of thresholds set for the inter-point distance, and performs a process of deleting overlapping clusters from among the clusters obtained by the clustering process.

1. An object detection device comprising:

2. The object detection device according to claim 1 , wherein the processing unit calculates the plurality of thresholds based on a maximum depth distance that the sensor can detect and a minimum inter-point distance based on a minimum desired separation distance between adjacent objects.

3. 3. The object detection device according to claim 2, wherein the processing unit calculates the plurality of thresholds by equally dividing a range between a maximum inter-point distance at the maximum depth distance and the minimum inter-point distance into a plurality of ranges.

4. The object detection device according to claim 1 , wherein the processing unit deletes a cluster when a plurality of clusters are included in the cluster.

5. The object detection device according to claim 1 , wherein the processing unit deletes a cluster when the cluster is included only once in another cluster.

6. The object detection device according to claim 1 , further comprising a detection unit that detects an object based on the point cloud clustered by the processing unit.

7. An object detection method implemented by an object detection device having a sensor that is installed on the ground, intermittently irradiates light to scan a predetermined space, receives reflected light of the light, and acquires point cloud information based on the reflected light, an input step of inputting the point cloud information from the sensor; a processing step of performing a clustering process based on a threshold value set for the inter-point distances of each point included in the point cloud information input in the input step; Including, The processing step performs a clustering process using each of a plurality of thresholds set for the inter-point distance, and deletes overlapping clusters from among the clusters obtained by the clustering process.

13. An object detection method comprising:

Images