Object detection system, object detection device, and object detection method

Abstract

The present invention pertains to object detection based on point cloud data acquired by scanning a monitoring range, and provides a mechanism whereby both a stationary object and a moving object can be suitably detected. An object detection system according to one embodiment of the present invention comprises: a LiDAR 10 that acquires point cloud data by scanning a monitoring range; and an object detection server 20 that executes an object detection process on the basis of the point cloud data acquired by the LiDAR 10. In accordance with the setting of a number of accumulated frames for each area obtained by dividing the monitoring range, the object detection server 20 accumulates point cloud data of each area in the monitoring range for each among the number of accumulated frames set for said area and executes the object detection process on the basis of the accumulated point cloud data.

Description

Object detection system, object detection device, and object detection method

[0001] The present invention relates to an object detection system based on point cloud data acquired by scanning a surveillance area.

[0002] Conventionally, object detection systems have been developed that use LiDAR (Light Detection and Ranging) to detect objects present within a surveillance area. Figure 1 shows an example of the configuration of an object detection system. Figure 2 shows an image of the operation of the object detection system.

[0003] The object detection system in Fig. 1 includes a LiDAR 10 and an object detection server 20. As shown in Fig. 2, the LiDAR 10 scans and irradiates a monitoring area with laser light and measures the arrival time of the reflected light to acquire point cloud data representing the three-dimensional position and shape of an object 30 present in the monitoring area. The point cloud data acquired by the LiDAR 10 is transmitted to the object detection server 20 via a network NW. The object detection server 20 executes an object detection process to detect an object 30 present in the monitoring area based on the point cloud data received from the LiDAR 10.

[0004] Ken Nishida and three others, "Verification of LIDAR specifications for autonomous driving," Proceedings of the 32nd Fuzzy Systems Society Fuzzy Systems Symposium FSS (Saga University, 2016), pp. 147-150

[0005] The resolution of the laser light emitted from the LiDAR is determined by the LiDAR's irradiation pattern and the individual LiDAR. The higher the resolution of the laser light, the higher the density of the point cloud data that can be acquired, making it possible to detect smaller or more distant objects. On the other hand, if the resolution of the laser light is low, the density of the point cloud data that can be acquired will be low, making it difficult to detect small or distant objects. This is because conventional object detection systems detect objects using one frame of point cloud data acquired by the LiDAR at a time.

[0006] Figure 3 shows an example of one frame of point cloud data obtained by LiDAR for an object. In the example of Figure 3, the density of the point cloud data is low, making it impossible to accurately identify the object. To address this issue, it has been considered to accumulate and overlay point cloud data for multiple frames before processing, rather than processing the point cloud data acquired by LiDAR on a frame-by-frame basis (see, for example, Non-Patent Document 1).

[0007] Object detection using point cloud data accumulation (overlay) will be described with reference to FIGS. 4 and 5. FIG. 4 shows an example of point cloud data from frames 1 to 4 obtained for an object using LiDAR. Even when the object is stationary, the coordinates of the point cloud data acquired using LiDAR are slightly shifted between successive frames, as shown in FIG. 4. Therefore, by accumulating and overlaying point cloud data from multiple frames, the density of the point cloud data can be artificially increased, as shown in FIG. 5, enabling more accurate object detection and discrimination.

[0008] While the accumulation (overlay) of point cloud data described above is effective for stationary objects (hereinafter referred to as "stationary objects"), it is less effective for moving objects, especially objects with large movements (hereinafter referred to as "moving objects"). This is because, when observing a moving object with LiDAR, only point cloud data with a scattered distribution area can be obtained in each frame. Therefore, even if multiple frames are accumulated and overlaid, only point cloud data dispersed over a wide area can be obtained. This problem can be addressed by using high-performance (high-resolution) LiDAR or by increasing the number of installed LiDARs, but this is hardware-dependent and expensive.

[0009] The present invention has been made in consideration of the above-mentioned conventional circumstances, and aims to provide a mechanism for object detection based on point cloud data acquired by scanning a monitoring range, which is capable of effectively detecting both stationary and moving objects.

[0010] In order to achieve the above object, an object detection system according to one aspect of the present invention is configured as follows: That is, the object detection system includes a point cloud data acquisition device that scans a monitoring range to acquire point cloud data, and an object detection device that executes object detection processing based on the point cloud data acquired by the point cloud data acquisition device, wherein the object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulation frames set for that area, in accordance with a setting of the number of accumulation frames for each area into which the monitoring range is divided, and executes the object detection processing based on the accumulated point cloud data.

[0011] In the above-described object detection system, the number of accumulated frames for each area can be set to a value according to the estimated speed of an object to be detected in each area.

[0012] In the above-described object detection system, the number of accumulated frames for each area may be set to a different value between a time period in which the presence of an object to be detected in each area is expected and other time periods.

[0013] Furthermore, in the above-mentioned object detection system, the object detection device may be configured to re-execute the object detection process for an area in which an object has been detected by the object detection process, based on point cloud data that has accumulated a number of frames greater than the number set for that area.

[0014] An object detection device according to another aspect of the present invention is configured as follows: In the object detection device, which performs object detection processing based on point cloud data acquired by scanning a monitoring range, the device accumulates point cloud data for each area within the monitoring range by the number of accumulation frames set for that area in accordance with a setting of the number of accumulation frames for each area into which the monitoring range is divided, and performs the object detection processing based on the accumulated point cloud data.

[0015] An object detection method according to yet another aspect of the present invention is configured as follows: That is, the object detection method is based on point cloud data acquired by scanning a monitoring range, and is characterized in that an object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulation frames set for that area, in accordance with a setting of the number of accumulation frames for each area into which the monitoring range is divided, and performs object detection processing based on the accumulated point cloud data.

[0016] According to the present invention, it is possible to provide a mechanism for object detection based on point cloud data acquired by scanning a monitoring range, which is capable of suitably detecting both stationary and moving objects.

[0017] FIG. 1 is a diagram illustrating an example of the configuration of an object detection system. FIG. 2 is a diagram illustrating an operational image of the object detection system shown in FIG. 1. FIG. 3 is a diagram illustrating an example of point cloud data for one frame obtained regarding an object. FIG. 4 is a diagram illustrating an example of point cloud data for the first to fourth frames obtained regarding an object. FIG. 5 is a diagram illustrating an example of point cloud data obtained by overlaying four frames. FIG. 6 is a diagram illustrating an example of division of a monitoring range in the proposed method. FIG. 7 is a diagram illustrating an example of accumulated frame count data by area in the proposed method. FIG. 8 is a diagram illustrating an example of a processing flow in the proposed method.

[0018] An embodiment of the present invention will be described with reference to the drawings. The schematic configuration of an object detection system according to the embodiment of the present invention is the same as the object detection system shown in Fig. 1. That is, the object detection system of this example includes a LiDAR 10, which is an example of a point cloud data acquisition device according to the present invention, and an object detection server 20, which is an example of an object detection device according to the present invention.

[0019] The LiDAR 10 scans and irradiates a monitoring area with laser light and measures the arrival time of the reflected light to acquire point cloud data representing the three-dimensional position and shape of an object 30 present in the monitoring area. The point cloud data acquired by the LiDAR 10 is transmitted to the object detection server 20 via the network NW. The object detection server 20 executes an object detection process to detect an object 30 present in the monitoring area based on the point cloud data received from the LiDAR 10. In the object detection process, for example, the point cloud data is processed using background subtraction to determine the size, shape, etc. of the detected object, and the detected object is identified based on the results. The detected object may also be identified by matching with a pre-registered object shape or by using an AI model that has learned the object shape.

[0020] Note that instead of the LiDAR 10, other point cloud data acquisition devices, such as a 3D sensor capable of acquiring point cloud data representing the three-dimensional position and shape of objects present in the monitoring area, may be used. Although FIG. 1 shows only one LiDAR 10, multiple LiDARs 10 may be provided, and the object detection server 20 may perform object detection processing based on point cloud data acquired by each LiDAR 10. The object detection server 20 may be implemented, for example, by a computer equipped with hardware resources such as a processor and memory, and configured to read programs related to each function of the present invention from the memory and execute them using the processor. Furthermore, the object detection server 20 may be implemented by a single computer or by multiple computers connected to each other so that they can communicate with each other.

[0021] In this proposed method, the object detection server 20 divides the monitoring range into multiple areas and executes object detection processing with a different number of accumulated frames for each area. As shown in Figure 6, for example, the monitoring range is divided into an air area whose main purpose is to detect stationary objects (flying objects caught on utility poles or power lines) and a ground area whose main purpose is to detect moving objects (passing people and vehicles).

[0022] 7 shows an example of area-specific accumulated frame count data used for the above control. In the example of FIG. 7, the number of accumulated frames for the "sky area" identified by area ID = "1" is set to "4," and the number of accumulated frames for the "ground area" identified by area ID = "2" is set to "1." Such area-specific accumulated frame count data is stored in the internal memory of the object detection server 20 or in an external device accessible to the object detection server 20. The area-specific accumulated frame count data may be manually set by a user of the object detection system, or may be automatically set by a device such as the object detection server 20 according to the distribution of stationary and moving objects analyzed from past point cloud data.

[0023] The object detection server 20 accumulates point cloud data for each area within the monitoring range by the number of accumulated frames set for that area in accordance with the area-specific accumulated frame number data described above, and performs object detection processing based on the accumulated point cloud data. When the area-specific accumulated frame number data of Figure 7 is used, the object detection server 20 performs object detection processing for the sky area based on point cloud data accumulated and superimposed over four frames, and for the ground area based on point cloud data for one frame.

[0024] An example of the processing flow for the proposed method is shown in Figure 8. The object detection server 20 first initializes (sets to 0) the frame counter for each area and initializes (clears) the accumulated point cloud data for each area (step S11). After that, when the object detection server 20 receives one frame of point cloud data from the LiDAR 10 (step S12), it performs the following processing for each area (steps S13 to S17).

[0025] First, the frame counter for area n (where 1≦n≦number of areas) is incremented (added by 1) (step S13), and the corresponding portion of the received point cloud data is added to the accumulated point cloud data for area n (step S14). Next, it is determined whether the frame counter for area n has reached the number of accumulated frames (step S15). If the frame counter for area n has reached the number of accumulated frames (step S15; Yes), object detection processing is performed based on the accumulated point cloud data for area n (step S16), and the frame counter and accumulated point cloud data for area n are reset (step S17), after which processing proceeds to the next area. On the other hand, if the frame counter for area n has not reached the number of accumulated frames (step S15; Yes), processing proceeds to the next area without performing steps S16 to S17. Furthermore, if processing for all areas is completed, the system waits until the next frame of point cloud data is received (step S12).

[0026] Here, in the processing flow of Figure 8, object detection processing in the sky area is performed every four frames, but by storing point cloud data for the number of accumulated frames as history, it is also possible to perform object detection processing by overlaying point cloud data from the past four frames for each frame (each time point cloud data is received from LiDAR).

[0027] As described above, the object detection system of this example comprises a LiDAR 10 that scans the monitoring range to acquire point cloud data, and an object detection server 20 that performs object detection processing based on the point cloud data acquired by the LiDAR 10.The object detection server 20 accumulates point cloud data for each area within the monitoring range by the number of accumulation frames set for that area, in accordance with the setting of the number of accumulation frames for each area into which the monitoring range is divided, and performs object detection processing based on the accumulated point cloud data.

[0028] This allows, for example, object detection processing for the sky area to be performed based on point cloud data accumulated and overlaid over four frames, while object detection processing for the ground area can be performed based on point cloud data for one frame. As a result, in the sky area, flying objects caught on utility poles or power lines can be detected with high accuracy, and in the ground area, passing people and vehicles can be detected with high accuracy. In this way, it is possible to effectively detect both stationary and moving objects. Moreover, because this can be achieved through software innovation rather than hardware, more advanced detection is possible while reducing the cost of the entire system.

[0029] In the above description, the number of accumulated frames in the sky area, whose main purpose is to detect stationary objects, is set to 4, and the number of accumulated frames in the ground area, whose main purpose is to detect moving objects, is set to 1, but this is merely an example. For example, the number of accumulated frames for each area may be set to a value according to the expected speed of the object that is the detection target in each area. As an example, the number of accumulated frames in an area, whose main purpose is to detect slow-moving moving objects, may be set to 2, and the number of accumulated frames in an area, whose main purpose is to detect fast-moving moving objects, may be set to 1.

[0030] For example, the number of accumulated frames for each area may be set to a different value depending on the time period when the presence of an object to be detected in each area is expected and other time periods. As an example, in each area, the number of accumulated frames may be set to a smaller value during time periods when pedestrians are expected to pass by, and to a larger value during time periods when pedestrians are not expected to pass by.

[0031] For example, the number of accumulated frames for each area may be set to a value according to the sampling period of the LiDAR. As an example, the number of accumulated frames may be increased when the sampling period of the LiDAR is short, and decreased when the sampling period is long.

[0032] For example, the number of accumulated frames for each area may be set to a value according to the urgency of detection in each area (degree of necessity for immediate detection). As an example, the number of accumulated frames may be increased in areas where the urgency of detection is low, and decreased in areas where the urgency of detection is high.

[0033] In the above explanation, the monitoring range is divided into two areas, but it may be divided into three or more areas. For example, when monitoring the inside of a station, it may be divided into an overhead area, an area where passengers walk (platform area), and an area where trains run (railway area), and a different number of accumulated frames may be set for each area.

[0034] As a variation of the object detection system described above, the object detection server 20 may re-execute the object detection process for an area where an object was detected by the object detection process, based on point cloud data accumulated for a number of frames greater than the number set for that area. For example, if an object is detected by performing the object detection process for a ground area based on one frame of point cloud data, the object detection process may be re-executed for the ground area based on four frames of point cloud data. This further improves the accuracy of object detection.

[0035] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications thereof are included in the scope and spirit of the invention described in this specification, etc., and are included in the invention described in the claims and their equivalents.

[0036] Furthermore, the present invention can be provided not only as devices such as those described above or as systems composed of these devices, but also as methods executed by these devices, programs for realizing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable manner.

[0037] The present invention can be used in an object detection system based on point cloud data acquired by scanning a monitoring area.

[0038] 10: LiDAR, 20: Object detection server, 30: Object

Claims

1. An object detection system comprising a point cloud data acquisition device which scans a monitoring range to acquire point cloud data, and an object detection device which executes object detection processing based on the point cloud data acquired by the point cloud data acquisition device, wherein the object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulation frames set for that area, in accordance with a setting of the number of accumulation frames for each area into which the monitoring range is divided, and executes the object detection processing based on the accumulated point cloud data.

2. An object detection system according to claim 1, wherein the number of accumulated frames for each area is set to a value according to the expected speed of the object to be detected in each area.

3. An object detection system as described in claim 1, characterized in that the number of accumulated frames for each area is set to a different value between time periods when the presence of an object to be detected in each area is expected and other time periods.

4. An object detection system as described in claim 1, characterized in that the object detection device re-executes the object detection process for an area in which an object is detected by the object detection process based on point cloud data that has accumulated a number of accumulated frames that is greater than the number set for that area.

5. An object detection device that performs object detection processing based on point cloud data acquired by scanning a monitoring range, the object detection device storing point cloud data for each area within the monitoring range by the number of accumulated frames set for that area in accordance with a setting of the number of accumulated frames for each area into which the monitoring range is divided, and performing the object detection processing based on the accumulated point cloud data.

6. An object detection method based on point cloud data acquired by scanning a monitoring range, characterized in that an object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulated frames set for that area, in accordance with a setting of the number of accumulated frames for each area into which the monitoring range is divided, and performs object detection processing based on the accumulated point cloud data.