Object detection system, object detection device, object detection method, and program

Abstract

To detect an object when detecting an object based on a three-dimensional point group even if the three-dimensional point group corresponding to the object is not obtained.SOLUTION: The object detection system comprises point group defect area extraction means for extracting a point group defect area, which is an area in a three-dimensional point group where point group data is missing, and object estimation means for estimating the presence of an object based on the point group defect area.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an object detection system, an object detection device, an object detection method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology for detecting utility poles, traffic lights, and guardrails based on a three-dimensional point cloud. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-3527 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, a LiDAR (light detection and ranging) device is known as a means for generating a three-dimensional point cloud. The LiDAR device generates a three-dimensional point cloud of a distance measurement target by using a laser light. Here, the LiDAR device is a distance measurement device that assumes that the laser light emitted toward the distance measurement target is reflected on the surface of the distance measurement target and input again to the LiDAR device.

[0005] Therefore, if a metal object that specularly reflects the laser light emitted from the LiDAR device is included in the distance measurement target, a 3D point cloud corresponding to the metal object is not generated, so the metal object cannot be detected based on the 3D point cloud. The same problem can be said about a black body that does not easily reflect the laser light emitted from the LiDAR device, such as charcoal.

[0006] An object of the present disclosure is to provide a technique for detecting an object based on a three-dimensional point cloud even if a three-dimensional point cloud corresponding to the object cannot be obtained. [Means for solving the problem]

[0007] According to the present disclosure, there is provided an object detection system including a point cloud missing area extraction means that extracts a point cloud missing area, which is an area where a point cloud is missing in a three-dimensional point cloud, and an object estimation means that estimates the presence of an object based on the point cloud missing area.

[0008] a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; Including, An object sensing device is provided.

[0009] A point cloud missing region is extracted, which is a region where a point cloud is missing in the three-dimensional point cloud; Estimating the presence of an object based on the point cloud missing region; A method for object detection is provided.

[0010] Computer, a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; A program is provided to function as a Effect of the Invention

[0011] According to the present disclosure, when detecting an object based on a three-dimensional point cloud, the object can be detected even if a three-dimensional point cloud corresponding to the object is not obtained. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram of an object detection system. [Diagram 2] FIG. 2 is a block diagram of an object detection device. [Diagram 3] 3 is a control flow of the object detection device. [Figure 4] 13 is a flow chart showing a flow of extracting a point cloud missing region. [Diagram 5] FIG. 13 is a distribution diagram of point density index values. [Figure 6] 1 is a flow of object estimation. [Figure 7] 1 is a flow of object estimation. [Figure 8] 13 is a flow chart showing a flow of extracting a point cloud missing region. [Figure 9] 13 is a flow chart showing a flow of extracting a point cloud missing region. [Figure 10] FIG. 13 is a distribution diagram of point density index values. [Figure 11] 13 is a diagram showing a specific example of an appearance mode of a point cloud missing region. [Figure 12] 13 is a diagram showing a specific example of an appearance mode of a point cloud missing region. [Figure 13] 1 shows a flow of extracting a point cloud missing region and a flow of estimating an object. [Figure 14] FIG. 13 is a diagram showing two point cloud missing regions that partially overlap each other. [Figure 15] FIG. 1 is a block diagram of a train equipped with an object detection device. [Figure 16] FIG. 1 is a diagram showing a case where a processing circuit included in an object detection device is configured with a processor and a memory. [Figure 17] FIG. 13 is a diagram showing a case where a processing circuit included in an object detection device is configured with dedicated hardware. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] (Summary of the Disclosure) First, an overview of the present disclosure will be described. FIG 1 shows a block diagram of an object detection system.

[0014] As shown in FIG. 1, the object detection system 100 includes a point cloud missing region extraction unit 101 and an object estimation unit 102.

[0015] The point cloud missing region extraction means 101 extracts a point cloud missing region, which is a region where a point cloud is missing from a three-dimensional point cloud.

[0016] The object estimation means 102 estimates the presence of an object based on the point cloud missing region.

[0017] According to the above configuration, when detecting an object based on a three-dimensional point cloud, the object can be detected even if a three-dimensional point cloud corresponding to the object is not obtained.

[0018] (First embodiment) Next, a first embodiment of the present disclosure will be described. Fig. 2 is a block diagram of an object detection device 1. The object detection device 1 shown in Fig. 2 is a device that detects an object based on a three-dimensional point cloud.

[0019] The object detection device 1 includes a point cloud acquisition unit 10, a point cloud missing region extraction unit 11, an object estimation unit 12, and an output unit 13.

[0020] The point cloud acquisition unit 10 acquires a three-dimensional point cloud. The three-dimensional point cloud can typically be generated using a LiDAR device (Light Detection And Ranging), a RADAR device (Radio Detection And Ranging), a stereo camera, or a combination thereof. Alternatively, the three-dimensional point cloud may be generated from a plurality of two-dimensional images by SfM (Structure from Motion).

[0021] The point cloud acquisition unit 10 may acquire a three-dimensional point cloud from a LiDAR device (not shown) included in the object detection device 1. The point cloud acquisition unit 10 may acquire a three-dimensional point cloud from a LiDAR device provided outside the object detection device 1. The point cloud acquisition unit 10 may also acquire a three-dimensional point cloud from an external device via a communication network.

[0022] The three-dimensional point cloud acquired by the point cloud acquisition unit 10 includes data such as the reflected luminance value of each point, the distance measurement position of the LiDAR device during distance measurement, etc., in addition to the three-dimensional coordinate value of each point. The three-dimensional coordinate value of each point is expressed in a coordinate system fixed to the LiDAR device. The distance measurement position of the LiDAR device is typically expressed in a world coordinate system.

[0023] The point cloud missing region extraction unit 11 extracts a point cloud missing region, which is a region where a point cloud is locally missing from a three-dimensional point cloud.

[0024] The object estimation unit 12 estimates the presence of an object based on the point cloud missing region extracted by the point cloud missing region extraction unit 11.

[0025] The output unit 13 outputs the estimation result by the object estimation unit 12. The output unit 13 is, for example, a display or a speaker. When the object estimation unit 12 estimates that an object exists, the output unit 13 displays a warning screen on the display. Alternatively, when the object estimation unit 12 estimates that an object exists, the output unit 13 may output a warning sound from the speaker.

[0026] Fig. 3 shows an operation flow of the object detection device 1. As shown in Fig. 3, first, the point cloud acquisition unit 10 acquires a three-dimensional point cloud (S100). Next, the point cloud missing region extraction unit 11 extracts a point cloud missing region, which is a region where a point cloud is locally missing in the three-dimensional point cloud (S110). Next, the object estimation unit 12 estimates the presence of an object based on the point cloud missing region (S120). Next, the output unit 13 outputs the estimation result by the object estimation unit 12 (S130).

[0027] Next, the point cloud missing region extraction unit 11 and the object estimation unit 12 will be described in detail in this order.

[0028] (Point cloud missing area extraction unit 11) As described above, the point cloud missing region extraction unit 11 extracts a point cloud missing region, which is a region where a point cloud is locally missing in a three-dimensional point cloud. FIG. 4 shows an operation flow of the point cloud missing region extraction unit 11.

[0029] S200: First, the point cloud missing region extraction unit 11 detects a surface based on the three-dimensional point cloud. The surface is typically a ground surface composed of a plane or a curved surface. The point cloud missing region extraction unit 11 detects the surface by, for example, clustering the three-dimensional point cloud. In this case, when multiple point clouds belonging to one cluster have almost the same vertical coordinates, the point cloud missing region extraction unit 11 can detect one surface based on the multiple point clouds.

[0030] S210: Next, the point cloud missing region extraction unit 11 divides the surface detected in step S200 into a plurality of cells in a lattice shape. FIG. 5 shows the surface 14 detected in step S200. FIG. 5 also shows a plurality of cells 15 arranged in a lattice shape. In this embodiment, the plurality of cells 15 have the same area. However, instead of this, the plurality of cells 15 may have areas different from each other.

[0031] S220: Next, the point cloud missing region extraction unit 11 calculates a point density index value of each cell 15. The point density index value is an index value that directly or indirectly indicates the density of the point cloud in each cell 15. In this embodiment, the point density index value indirectly indicates the density of the point cloud in each cell 15. That is, in this embodiment, since the areas of the multiple cells 15 have the same area, the point density index value can be defined as the number of point clouds belonging to the corresponding cell 15. For example, when the point density index value is 65, it means that 65 points exist in the corresponding cell 15. In FIG. 5, the point density index value is directly written in each cell 15. The point density index value may be defined as the number of point clouds belonging to the corresponding cell 15 divided by the area of ​​the cell 15. In this case, the point density index value directly indicates the density of the point cloud in each cell 15.

[0032] S230: Next, the point cloud missing region extraction unit 11 extracts cells whose point density index value satisfies a predetermined condition. In this embodiment, when the point density index value is equal to or less than a predetermined value, it is determined that the point density index value satisfies the predetermined condition. In this embodiment, as shown in FIG. 5, the point density index values ​​of the multiple cells 15 are mostly distributed around 5 and around 60. Therefore, in this embodiment, the point cloud missing region extraction unit 11 extracts cells whose point density index value is 30 or less.

[0033] S240: Next, the point cloud missing region extraction unit 11 determines the set of cells 15 extracted in step S230 as a point cloud missing region 16. In Fig. 5, the point cloud missing region 16 is surrounded by a thick solid line.

[0034] (Object estimation unit 12) As described above, the object estimation unit 12 estimates the presence of an object based on the point cloud missing region 16. The estimation by the object estimation unit 12 may be, for example, the following first estimation method, second estimation method, and third estimation method, but is not limited to these.

[0035] <First estimation method> FIG. 6 shows a specific example of an estimation flow of the object estimation unit 12. In FIG.

[0036] As shown in FIG. 6, the object estimation unit 12 determines whether the point cloud missing region 16 is elongated (S300). Specifically, the object estimation unit 12 calculates a rectangle circumscribing the point cloud missing region 16, and can determine whether the point cloud missing region 16 is elongated based on the aspect ratio of the rectangle. When the point cloud missing region 16 is determined to be elongated (S300: YES), the object estimation unit 12 estimates that the object 18 is present in the point cloud missing region 16 (S310). Specifically, the object estimation unit 12 may estimate that the object 18 is present in any one of the ends 16a and 16b in the longitudinal direction of the point cloud missing region 16. Furthermore, the object estimation unit 12 may estimate that the object 18 is present in the end 16a, which is closer to the LiDAR device 17, among the ends 16a and 16b in the longitudinal direction of the point cloud missing region 16.

[0037] On the other hand, if it is determined that the point cloud missing region 16 is not elongated (S300: NO), the object estimation unit 12 estimates that no object exists in the point cloud missing region 16 (S320) and ends the process. This is because, if the measurement target is, for example, 600 meters away from the LiDAR device in horizontal distance, the point cloud missing region 16 should be elongated if it is caused by an object 18 placed on the ground.

[0038] <Second estimation method> FIG. 7 shows another specific example of the estimation flow of the object estimation unit 12.

[0039] As shown in FIG. 7, the object estimation unit 12 determines whether the point cloud missing region 16 is elongated (S400). When it is determined that the point cloud missing region 16 is elongated (S400: YES), the object estimation unit 12 determines whether the LiDAR device 17 is on the extension line of the point cloud missing region 16 in the longitudinal direction (S410). When it is determined that the LiDAR device 17 is on the extension line of the point cloud missing region 16 in the longitudinal direction (S410: YES), the object estimation unit 12 estimates that an object 18 exists in the point cloud missing region 16 (S420). Specifically, the object estimation unit 12 may estimate that an object exists in any one of the ends 16a and 16b in the longitudinal direction of the point cloud missing region 16. Furthermore, the object estimation unit 12 may estimate that an object exists in the end 16a, which is closer to the LiDAR device 17, among the ends 16a and 16b in the longitudinal direction of the point cloud missing region 16.

[0040] On the other hand, when it is determined that the point cloud missing region 16 is not elongated (S400: NO), the object estimation unit 12 estimates that no object exists in the point cloud missing region 16 (S430) and ends the process. This is because, if the measurement target is, for example, 600 meters away from the LiDAR device in horizontal distance, the point cloud missing region 16 should be elongated if it is caused by an object 18 placed on the ground.

[0041] Furthermore, when it is determined that the LiDAR device 17 is not present on the longitudinal extension of the point cloud missing region 16 (S410: NO), the object estimation unit 12 estimates that no object exists in the point cloud missing region 16 (S430) and ends the process. This is because, when the measurement target is, for example, 600 meters away from the LiDAR device in horizontal distance, if the point cloud missing region 16 is caused by an object 18 placed on the ground, the point cloud missing region 16 should extend in a direction away from the LiDAR device 17.

[0042] <Third estimation method> When the point cloud missing region extraction unit 11 extracts the point cloud missing region 16, the object estimation unit 12 may estimate that an object 18 exists in the point cloud missing region 16, regardless of the shape or extension direction of the point cloud missing region 16. For example, when the LiDAR device 17 is placed at a position higher in the vertical direction than the object 18, the point cloud missing region 16 caused by the object 18 placed on the ground does not necessarily extend in an elongated manner.

[0043] The first embodiment of the present disclosure has been described above, and the above embodiment has the following features.

[0044] The object detection device 1 includes a point cloud missing region extraction unit 11 that extracts a point cloud missing region 16, which is a region where a point cloud is missing in a three-dimensional point cloud, and an object estimation unit 12 that estimates the presence of an object 18 based on the point cloud missing region 16. According to the above configuration, when detecting an object based on a three-dimensional point cloud, the object can be detected even if a three-dimensional point cloud corresponding to the object is not obtained.

[0045] As shown in FIG. 5, point cloud missing region 16 is a region within surface 14 detected based on the three-dimensional point cloud.

[0046] As shown in Fig. 5, the point cloud missing region extraction unit 11 divides a surface 14 into a plurality of cells 15 in a lattice pattern. The point cloud missing region extraction unit 11 calculates a point density index value for each cell 15. The point cloud missing region extraction unit 11 extracts a set of cells 15 whose point density index values ​​satisfy a predetermined condition as a point cloud missing region 16. With the above configuration, the calculation cost for extracting a point cloud missing region can be reduced.

[0047] 5, the object estimation unit 12 estimates that an object 18 exists at ends 16a and 16b in the longitudinal direction of the point cloud missing region 16. With the above configuration, the position of the object 18 can be estimated with high accuracy.

[0048] Second embodiment Next, a second embodiment of the present disclosure will be described. Below, the differences between the second embodiment and the first embodiment will be mainly described, and overlapping descriptions will be omitted. The operation of the point cloud missing region extraction unit 11 in this embodiment is different from that in the first embodiment.

[0049] Fig. 8 shows an operation flow of the point cloud missing region extraction unit 11. In the operation flow shown in Fig. 8, steps S200, S210, and S240 are the same as those in the first embodiment.

[0050] S220: The point cloud missing region extraction unit 11 calculates a reflection luminance index value of each cell 15. The reflection luminance index value is an index value that directly or indirectly indicates the reflection luminance of the point cloud in each cell 15. In this embodiment, the reflection luminance index value is a representative value of the reflection luminance of the point cloud in each cell 15. The representative value of the reflection luminance of the point cloud is, for example, but is not limited to, the average value (Mean) of the reflection luminance of the point cloud, the median value (Median) of the reflection luminance of the point cloud, or the mode (Mode) of the reflection luminance of the point cloud.

[0051] S230: Next, the point cloud missing region extraction unit 11 extracts a cell whose reflection luminance index value satisfies a predetermined condition for each cell 15. In this embodiment, when the reflection luminance index value is equal to or less than a predetermined value, it is determined that the reflection luminance index value satisfies the predetermined condition.

[0052] Thus, in this embodiment, the point cloud missing region extraction unit 11 extracts the point cloud missing region 16 based on the reflection luminance of the point cloud. For example, if the object 18 detected by the object detection device 1 is a light-transmitting material that has both reflective and transmissive properties, such as glass, the reflection luminance of the point cloud in the point cloud missing region 16 corresponding to the object 18 will be distributed around 30, for example. Therefore, by setting the predetermined value to 10 in step S230, the object 18 can be detected without any problems even if the object 18 is a light-transmitting material.

[0053] Third embodiment Next, a third embodiment of the present disclosure will be described. Below, the differences between the third embodiment and the first embodiment will be mainly described, and overlapping descriptions will be omitted. The operation of the point cloud missing region extraction unit 11 in this embodiment is different from that in the first embodiment.

[0054] Fig. 9 shows the operation flow of the point cloud missing region extraction unit 11. In the operation flow shown in Fig. 8, steps S200 to S220 are the same as those in the first embodiment. In this embodiment, the point cloud missing region extraction unit 11 detects the contour of the point cloud missing region 16 based on the difference in point density index value between two adjacent cells 15, and extracts the point cloud missing region 16 based on the detection result. Specifically, it is as follows.

[0055] S250: First, the point cloud missing region extraction unit 11 calculates the difference in point density index value between two adjacent cells 15. In FIG. 10, the two adjacent cells 15 are surrounded by a solid line 19. The difference in point density index value between the two cells 15 surrounded by the solid line 19 is 53. If this difference exceeds a predetermined value, the point cloud missing region extraction unit 11 sets a provisional contour line 20 between the two cells 15. Similarly, the point cloud missing region extraction unit 11 executes the above difference calculation for all cells 15 and sets a plurality of provisional contour lines 20.

[0056] S260: Next, the point cloud missing region extraction unit 11 determines whether the multiple temporary contours 20 set in step S250 form a closed loop. If it is determined that the multiple temporary contours 20 form a closed loop (S260: YES), the point cloud missing region extraction unit 11 extracts the multiple cells 15 surrounded by the multiple temporary contours 20. On the other hand, if it is determined that the multiple temporary contours 20 do not form a closed loop (S260: NO), the point cloud missing region extraction unit 11 concludes that there is no point cloud missing region and ends the process.

[0057] S270: Next, the point cloud missing region extraction unit 11 determines the set of cells 15 extracted in step S260 as a point cloud missing region 16.

[0058] In this manner, in this embodiment, the point cloud missing region extraction unit 11 extracts the point cloud missing region 16 by detecting the contour of the point cloud missing region 16. As is well known, the point density of the three-dimensional point cloud increases or decreases depending on the distance from the LiDAR device. For example, the point density is relatively low in an area relatively far from the LiDAR device, and the point density is relatively high in an area relatively close to the LiDAR device. Thus, in the point cloud missing region 16, the point density is relatively low in an area relatively far from the LiDAR device, and the point density is relatively high in an area relatively close to the LiDAR device. Even if there is a gradient in the point density depending on the distance from the LiDAR device, the extraction method according to this embodiment can reliably extract the point cloud missing region 16.

[0059] It should be noted that the point cloud loss region extraction unit 11 may calculate a reflection luminance index value of each cell 15 instead of calculating a point density index value of each cell 15 .

[0060] Also, instead of calculating the point density index value or the reflection brightness index value of each cell 15, the point cloud missing region extraction unit 11 may first divide the space into a voxel grid shape, calculate the difference in point density between adjacent voxel grids, and extract the point cloud missing region 16 based on the difference. Secondly, the point cloud missing region extraction unit 11 may calculate the point density index value or the reflection brightness index value for each point constituting the three-dimensional point cloud. For example, the number of points in a virtual sphere having a predetermined radius centered on a certain point constituting the three-dimensional point cloud may be set as the point density index value of the point. In short, it is sufficient to be able to calculate the distribution state of the three-dimensional point cloud acquired by the point cloud acquisition unit 10 to an extent that the point cloud missing region 16 can be extracted. Therefore, instead of calculating the point density index value or the reflection brightness index value of each cell 15, the point cloud missing region extraction unit 11 may calculate representative values ​​of polarization, phase, and frequency obtained from the characteristics of light of the received light pulse corresponding to the point cloud of each cell 15.

[0061] (Fourth embodiment) Next, a fourth embodiment of the present disclosure will be described. Figures 11 and 12 show a specific example of the appearance of the point cloud missing region 16. As shown in Figures 11 and 12, the point cloud missing region 16 has a property of appearing in an elongated manner along a direction away from the LiDAR device 17. That is, as shown in Figures 11 and 12, the point cloud missing region 16 appears in an elongated manner along an extension of a line connecting the LiDAR device 17 and the object 18.

[0062] If the point cloud missing region 16 in Fig. 11 and Fig. 12 does not appear due to the object 18, but appears due to the characteristics of the ground, that is, for example, a mirror placed on the ground or charcoal powder scattered on the ground, the point cloud missing region 16 extracted by the point cloud missing region extractor 11 will always have the same shape and be extracted at the same position, regardless of the ranging position of the LiDAR device 17. That is, the point cloud missing region 16 shown in Fig. 11 and the point cloud missing region 16 shown in Fig. 12 will completely overlap. In short, the point cloud missing region 16 shown in Fig. 11 and the point cloud missing region 16 shown in Fig. 12 will have the same contour.

[0063] Therefore, by comparing the two point cloud missing areas 16 extracted from the two three-dimensional point clouds generated before and after the ranging position of the LiDAR device 17 is changed, it will be possible to determine whether these point cloud missing areas 16 are caused by an object 18 placed on the ground or by the characteristics of the ground itself.

[0064] FIG. 13 shows a processing flow of the point cloud missing region extraction unit 11 and the object estimation unit 12.

[0065] S500: First, the point cloud missing region extraction unit 11 aligns the first and second three-dimensional point clouds, which have different ranging positions (i.e., the location of the LiDAR device 17 when ranging is performed) of the LiDAR device 17, with each other, for example, by using ICP (Iterative Closest Point). However, if the coordinate accuracy of both the first and second three-dimensional point clouds is good, the alignment can be omitted.

[0066] S510: Next, the point cloud missing region extraction unit 11 extracts a first point cloud missing region and a second point cloud missing region in the first three-dimensional point cloud and the second three-dimensional point cloud, respectively. In Fig. 14, the first point cloud missing region 21 and the second point cloud missing region 22 are indicated by a thick solid line and a dashed line, respectively.

[0067] S520: Next, the object estimation unit 12 determines whether the first point cloud missing region 21 and the second point cloud missing region 22 overlap each other. If the first point cloud missing region 21 and the second point cloud missing region 22 are separated from each other without overlapping each other (S520: NO), the object estimation unit 12 estimates that no object exists (S550) and ends the process. On the other hand, if the first point cloud missing region 21 and the second point cloud missing region 22 partially overlap each other (S520: YES), the object estimation unit 12 advances the process to step S530. Here, "overlapping each other" means "partially overlapping each other" or "at least partially overlapping each other".

[0068] S530: Next, the object estimation unit 12 determines whether the contours of the first point cloud missing region 21 and the second point cloud missing region 22 are not the same. If the contours of the first point cloud missing region 21 and the second point cloud missing region 22 are not the same (S530: YES), the object estimation unit 12 estimates that an object 18 exists in the overlapping region 23 of the first point cloud missing region 21 and the second point cloud missing region 22 (S540). On the other hand, if the contours of the first point cloud missing region 21 and the second point cloud missing region 22 are the same (S530: NO), the object estimation unit 12 estimates that no object exists in the overlapping region 23 of the first point cloud missing region 21 and the second point cloud missing region 22 (S550) and ends the process.

[0069] In this way, by comparing the point cloud missing areas extracted from two 3D point clouds obtained at different ranging positions, it is possible to prevent erroneous detection of objects.

[0070] The fourth embodiment has been described above. The fourth embodiment has the following features.

[0071] That is, the point cloud missing region extraction unit 11 extracts a first point cloud missing region 21 and a second point cloud missing region 22 in each of the first and second three-dimensional point clouds in which the ranging positions of the LiDAR device 17 (ranging device) for generating the three-dimensional point cloud are different from each other. When the first point cloud missing region 21 and the second point cloud missing region 22 overlap and the contour of the first point cloud missing region 21 and the contour of the second point cloud missing region 22 do not match each other, the object estimation unit 12 estimates that the object 18 exists in the overlapping region 23 of the first point cloud missing region 21 and the second point cloud missing region 22. With the above configuration, it is possible to prevent erroneous detection of an object.

[0072] Fifth embodiment Next, a case will be described where the point cloud generating device 30 that supplies a three-dimensional point cloud to the object detection device 1 is mounted on a train 31. The train 31 shown in Fig. 15 is a specific example of a moving body equipped with the point cloud generating device 30. The moving body is not limited to the train 31, and may be an automobile, a moving robot, an aircraft, or the like.

[0073] As shown in FIG. 15, the point cloud generating device 30 includes a LiDAR device 32, a position correction unit 33, an integrating unit 34, and a point cloud output unit 35.

[0074] The LiDAR device 32 performs distance measurement while the train 31 is moving. If the LiDAR device 32 is a so-called scanning LiDAR device such as a raster scan type or a conical scan type, the distance measurement position and distance measurement direction of the LiDAR device 32 change from moment to moment. Even if the LiDAR device 32 is a so-called flash type LiDAR device that performs distance measurement all at once at a certain time, the distance measurement position and distance measurement direction of the LiDAR device 32 change every time distance measurement is performed.

[0075] The position correction unit 33 corrects the coordinates of each point of the three-dimensional point cloud generated by the LiDAR device 32 based on the ranging position and ranging direction of the LiDAR device 32. For example, the ranging position of the LiDAR device 32 means the acquisition location of the LiDAR device 32 when the LiDAR device 32 performs ranging. The ranging position of the LiDAR device 32 can be acquired by various self-location estimation techniques. A typical self-location estimation technique is a combination of a Global Navigation Satellite System (GNSS) and an Inertial Measurement Unit (IMU). As another self-location estimation technique, Simultaneous Localization and Mapping (SLAM) may be adopted. The ranging direction means the attitude of the LiDAR device 32 at the time of ranging. The ranging direction can be acquired based on the output value of an acceleration sensor or a geomagnetic sensor provided in the LiDAR device 32.

[0076] The accumulating unit 34 accumulates the 3D point cloud obtained in the first time period as a first 3D point cloud and holds it. The accumulating unit 34 also accumulates the 3D point cloud obtained in a second time period that is after the first time period and holds it. The lengths of the first time period and the second time period on the time axis may be appropriately adjusted so as to ensure the point density required to detect the detected object. The lengths of the first time period and the second time period on the time axis may also be appropriately adjusted depending on the running speed of the train 31 and the size of the detected object.

[0077] The point cloud output unit 35 outputs the first three-dimensional point cloud and the second three-dimensional point cloud integrated by the integration unit 34 to the object detection device 1.

[0078] The fifth embodiment has the following features.

[0079] When the LiDAR device 17 is mounted on a train 31 and distance measurement is performed by the LiDAR device 17 while the train 31 is moving, the first three-dimensional point cloud is a point cloud obtained by distance measurement in a first time period, and the second three-dimensional point cloud is a point cloud obtained in a second time period that is after the first time period. According to the above configuration, an object is detected based on multiple sets of point clouds obtained from various angles, so that erroneous detection of an object can be effectively prevented.

[0080] The first to fifth embodiments have been described above. Each of the above embodiments can be modified, for example, as follows.

[0081] For example, the object estimation unit 12 may estimate the shape of the object 18 based on the shape of the point cloud missing region 16 shown in Fig. 5. The object estimation unit 12 may also estimate the shape of the object 18 based on the first point cloud missing region 21 and the second point cloud missing region 22 shown in Fig. 14.

[0082] In addition, the point cloud missing region extraction unit 11 may extract the point cloud missing region 16 based on the difference between two three-dimensional point clouds generated at different times. In this case, it is preferable that the object 18 does not exist at the time when one of the three-dimensional point clouds is generated.

[0083] In the fourth embodiment, it has been described that by comparing two point cloud missing regions 16 extracted from two three-dimensional point clouds generated before and after the change in the measuring position of the LiDAR device 17, it is possible to distinguish whether these point cloud missing regions 16 are caused by an object 18 placed on the ground or are caused by the characteristics of the ground itself. Here, the change in the measuring position of the LiDAR device 17 is not limited to the movement of the measuring position of the LiDAR device 17 in a horizontal plane. The measuring position of the LiDAR device 17 may move in the vertical direction. Even in this case, at least the longitudinal length of the point cloud missing region 16 increases or decreases, so it will be possible to distinguish whether these point cloud missing regions 16 are caused by an object 18 placed on the ground or are caused by the characteristics of the ground itself.

[0084] In the fourth embodiment, the point cloud missing region extracting unit 11 may calculate the difference between two three-dimensional point clouds, and extract as the point cloud missing region 16 a region where the difference is equal to or greater than a threshold value.

[0085] Next, a description will be given of the hardware configuration of the object detection device 1. In the object detection device 1, the point cloud missing region extraction unit 11 and the object estimation unit 12 are realized by a processing circuit. The processing circuit may be a processor and memory that executes a program stored in a memory, or may be dedicated hardware.

[0086] FIG. 16 is a diagram showing an example of a processing circuit included in the object detection device 1, which is configured with a processor and a memory. When the processing circuit is configured with a processor 1000 and a memory 1001, each function of the processing circuit of the object detection device 1 is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 1001. In the processing circuit, each function is realized by the processor 1000 reading and executing the program stored in the memory 1001. That is, the processing circuit includes the memory 1001 for storing a program that will result in the processing of the object detection device 1 being executed. It can also be said that these programs cause a computer to execute the procedures and methods of the object detection device 1.

[0087] Here, the processor 1000 may be a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor), etc. Also, the memory 1001 may be, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), an EEPROM (Electrically EPROM), or other non-volatile or volatile semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc), etc.

[0088] Fig. 17 is a diagram showing an example in which the processing circuit of the object detection device 1 is configured with dedicated hardware. When the processing circuit is configured with dedicated hardware, the processing circuit 1002 shown in Fig. 17 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of the object detection device 1 may be realized by the processing circuit 1002 on a function-by-function basis, or each function may be realized collectively by the processing circuit 1002.

[0089] It should be noted that the functions of the object detection device 1 may be partially realized by dedicated hardware and partially realized by software or firmware. In this manner, the processing circuit can realize each of the above-described functions by dedicated hardware, software, firmware, or a combination of these.

[0090] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be appropriately combined with other embodiments.

[0091] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing may be combined with features or steps shown in one or more other drawings to create an embodiment not expressly shown or described, for example. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0092] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; Including, Object detection system. (Appendix 2) The point cloud missing region is a region within a plane detected based on the three-dimensional point cloud. 2. An object detection system as described in claim 1. (Appendix 3) The point cloud missing region extraction means Dividing the surface into a plurality of cells in a lattice pattern; Calculating a point density index value or a reflected brightness index value for each cell; A set of cells in which the point density index value or the reflected brightness index value satisfies a predetermined condition is extracted as the point cloud missing region. 3. An object detection system as described in claim 2. (Appendix 4) The point cloud missing region extraction means Dividing the surface into a plurality of cells in a lattice pattern; Calculating a point density index value or a reflected brightness index value for each cell; detecting a contour of the point cloud missing area based on a difference in the point density index value or the reflected brightness index value between two adjacent cells; 3. An object detection system as described in claim 2. (Appendix 5) the object estimation means estimates that the object is present at an end portion in a longitudinal direction of the point cloud missing region; 2. An object detection system as described in claim 1. (Appendix 6) The point cloud missing region extraction means extracting a first point cloud missing region and a second point cloud missing region from a first three-dimensional point cloud and a second three-dimensional point cloud, respectively, the first three-dimensional point cloud and the second three-dimensional point cloud being generated at different ranging positions of a ranging device; The object estimation means includes: when the first point cloud missing region and the second point cloud missing region overlap and the contour of the first point cloud missing region and the contour of the second point cloud missing region do not match each other, it is estimated that the object is present in an overlapping region of the first point cloud missing region and the second point cloud missing region. 2. An object detection system as described in claim 1. (Appendix 7) In a case where the distance measuring device is mounted on a moving body and distance measuring is performed by the distance measuring device while the moving body is moving, the first three-dimensional point cloud is a point cloud obtained by distance measurement in a first time period; The second three-dimensional point cloud is a point cloud obtained in a second time period that is after the first time period. 7. An object detection system as described in claim 6. (Appendix 8) a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; Including, Object detection device. (Appendix 9) A point cloud missing region is extracted, which is a region where a point cloud is missing in the three-dimensional point cloud; Estimating the presence of an object based on the point cloud missing region; Object detection methods. (Appendix 10) Computer, a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; A program that functions as a

[0093] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 7 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Notes 8, 9, and 10 in the same dependent relationship as Supplementary Notes 2 to 7. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0094] 1. Object detection device 10 Point cloud acquisition section 11 Point cloud defect area extraction section 12 Object estimation section 13 Output section 14 sides 15 Cells 16 Point cloud missing area 16a End 16b End 17 LiDAR device 18 Object 19 Solid line 20 Temporary contour line 21 First point cloud missing area 22 Second point cloud missing area 23 Overlapping area 30 point cloud generator 31 Train 32 LiDAR device 33 Position correction section 34 Integration section 35 Point cloud output section

Claims

1. a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; Including, Object detection system.

2. The point cloud missing region is a region within a plane detected based on the three-dimensional point cloud. The object detection system of claim 1 .

3. The point cloud missing region extraction means Dividing the surface into a plurality of cells in a lattice pattern; Calculating a point density index value or a reflected brightness index value for each cell; A set of cells in which the point density index value or the reflected brightness index value satisfies a predetermined condition is extracted as the point cloud missing region. The object detection system of claim 2 .

4. The point cloud missing region extraction means Dividing the surface into a plurality of cells in a lattice pattern; Calculating a point density index value or a reflected brightness index value for each cell; detecting a contour of the point cloud missing region based on a difference in the point density index value or the reflected brightness index value between two adjacent cells; The object detection system of claim 2 .

5. the object estimation means estimates that the object is present at an end portion in a longitudinal direction of the point cloud missing region; The object detection system of claim 1 .

6. The point cloud missing region extraction means extracting a first point cloud missing region and a second point cloud missing region from a first three-dimensional point cloud and a second three-dimensional point cloud, respectively, the first three-dimensional point cloud and the second three-dimensional point cloud being generated at different ranging positions of a ranging device; The object estimation means includes: when the first point cloud missing region and the second point cloud missing region overlap and the contour of the first point cloud missing region and the contour of the second point cloud missing region do not match each other, it is estimated that the object is present in an overlapping region of the first point cloud missing region and the second point cloud missing region. The object detection system of claim 1 .

7. In a case where the distance measuring device is mounted on a moving body and distance measuring is performed by the distance measuring device while the moving body is moving, the first three-dimensional point cloud is a point cloud obtained by distance measurement in a first time period, The second three-dimensional point cloud is a point cloud obtained in a second time period that is after the first time period. The object detection system of claim 6.

8. a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; Including, Object detection device.

9. A point cloud missing region is extracted, which is a region where a point cloud is missing in the three-dimensional point cloud; Estimating the presence of an object based on the point cloud missing region; Object detection methods.

10. Computer, a point cloud missing region extraction means for extracting a point cloud missing region, which is a region where a point cloud is missing in a three-dimensional point cloud; an object estimation means for estimating the presence of an object based on the point cloud missing region; A program that functions as a

Images