Detection device, detection method, and detection program

Abstract

Point cloud data is used to enable accurate detection of objects. [Solution] The voxelization unit 22 associates each point in the point cloud data with the voxel containing the location of that point among a plurality of voxels, thereby generating voxel data. The difference identification unit 23 identifies difference cells for each of the plurality of voxels where there is a difference between the voxel data and the reference data. The accumulation processing unit 25 clusters the difference cells identified from each point cloud data acquired during the detection period, and identifies voxels in overlapping regions where a certain percentage or more of clusters overlap as target cells.

Description

Technical Field

[0001] The present disclosure relates to a technique for detecting an object using point cloud data.

Background Art

[0002] There is a technique for detecting an object such as a displaced object using a visible light camera. When detecting an object using a visible light camera, it is not possible to appropriately detect the object in an environment where the amount of light is insufficient, such as at night, or in an environment where direct sunlight shines into the visible light camera and flare occurs.

[0003] There is an optical sensor such as LiDAR that collects point cloud data by irradiating irradiation light and receiving the reflected light reflected at the reflection points. LiDAR is an abbreviation for Light Detection And Ranging. In an optical sensor, it is possible to obtain point cloud data even in an environment where the amount of light is insufficient, such as at night, or in an environment where direct sunlight shines into the visible light camera and flare occurs. Therefore, detecting an object based on the point cloud data obtained using an optical sensor has been studied.

[0004] Patent Document 1 describes comparing background point cloud data held in a memory with the current point cloud data and identifying that some object has appeared when a difference occurs.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] With optical sensors, it is difficult to always illuminate the same spot with light, resulting in slight deviations. Consequently, the generated point cloud data differs slightly each time. Therefore, as described in Patent Document 1, comparing background point cloud data with current point cloud data may lead to the identification of objects in areas where no objects actually exist. This disclosure aims to enable the appropriate detection of objects using point cloud data. [Means for solving the problem]

[0007] The detection device related to this disclosure is A point cloud acquisition unit repeatedly acquires point cloud data obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection point, A voxelization unit generates voxel data corresponding to the target point cloud data by taking each of the point cloud data repeatedly acquired by the point cloud acquisition unit as the target point, and associating each point of the target point cloud data with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, For each of the plurality of voxels, a difference identification unit identifies difference cells that have a difference between the voxel data generated by the voxelization unit and the reference data for the target point cloud data, based on the presence or absence of associated points. An accumulation processing unit identifies voxels as target cells among the clusters obtained by clustering the difference cells identified by the difference identification unit from each point cloud data acquired during the detection period, in which clusters exceeding a certain percentage overlap. It is equipped with. [Effects of the Invention]

[0008] In this disclosure, the difference is repeatedly identified for each voxel into which the target region has been divided, and the overlapping region of the cluster of difference cells is identified. This makes it possible to appropriately detect the target object even if there is a slight shift in the irradiation position of the irradiation light. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing the configuration of the detection device 10 according to Embodiment 1. [Figure 2] An explanatory diagram of the voxel definition 31 according to Embodiment 1. [Figure 3] A flowchart showing the processing flow of the detection device 10 according to Embodiment 1. [Figure 4] An explanatory diagram of the voxelization process according to Embodiment 1. [Figure 5] Diagram illustrating the difference identification process according to Embodiment 1. [Figure 6] An explanatory diagram of the first filtering process according to Embodiment 1. [Figure 7] An explanatory diagram of the second filtering process according to Embodiment 1. [Figure 8] A diagram showing the configuration of the detection device 10 according to modified example 1. [Figure 9] A flowchart showing the processing flow of the detection device 10 according to modified example 1. [Figure 10] Configuration diagram of the detection device 10 according to Embodiment 2. [Figure 11] A flowchart showing the processing flow of the detection device 10 according to Embodiment 2. [Figure 12] Configuration diagram of the detection device 10 according to Embodiment 3. [Figure 13] A flowchart showing the processing flow of the detection device 10 according to Embodiment 3. [Figure 14] Configuration diagram of the detection device 10 according to Embodiment 4. [Figure 15] An explanatory diagram of the mask definition 34 according to Embodiment 4. [Figure 16] A flowchart showing the processing flow of the detection device 10 according to Embodiment 4. [Figure 17] A flowchart showing the processing flow of the detection device 10 according to Embodiment 5. [Figure 18] An explanatory diagram of the IoU determination process according to Embodiment 5. [Figure 19] An explanatory diagram of the IoU determination process according to Embodiment 5. [Figure 20] Explanatory diagram of the IoU determination process according to Embodiment 5. [Figure 21] Configuration diagram of the detection device 10 according to Embodiment 6. [Figure 22] Flowchart showing the processing flow of the detection device 10 according to Embodiment 6. [Figure 23] Explanatory diagram of the difference identification process according to Embodiment 7. [Figure 24] Configuration diagram of the detection device 10 according to Embodiment 8. [Figure 25] Flowchart showing the processing flow of the detection device 10 according to Embodiment 8. [[ID=1P9]] [Figure 26] Explanatory diagram of the voxel setting process according to Embodiment 8. [Figure 27] Explanatory diagram of the voxel setting process according to Embodiment 8.

Modes for Carrying Out the Invention

[0010] Embodiment 1. ***Explanation of Configuration*** Referring to FIG. 1, the configuration of the detection device 10 according to Embodiment 1 will be described. The detection device 10 is a computer. The detection device 10 includes hardware such as a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware via signal lines and controls these other hardware. [[ID=SO]]

[0011] The processor 11 is an IC that performs processing. IC is an abbreviation for Integrated Circuit. The processor 11 is, as a specific example, a CPU, a DSP, or a GPU. CPU is an abbreviation for Central Processing Unit. DSP is an abbreviation for Digital Signal Processor. GPU is an abbreviation for Graphics Processing Unit.

[0012] Memory 12 is a storage device that temporarily stores data. Specific examples of memory 12 include SRAM and DRAM. SRAM stands for Static Random Access Memory. DRAM stands for Dynamic Random Access Memory.

[0013] Storage 13 is a storage device for storing data. A concrete example of storage 13 is an HDD. HDD stands for Hard Disk Drive. Alternatively, storage 13 may be a portable recording medium such as an SD® memory card, CompactFlash®, NAND flash, flexible disk, optical disk, compact disk, Blu-ray® disc, or DVD. SD stands for Secure Digital. DVD stands for Digital Versatile Disk.

[0014] Communication interface 14 is an interface for communicating with external devices. Specific examples of communication interface 14 include Ethernet®, USB, and HDMI® ports. USB stands for Universal Serial Bulb. It is an abbreviation for 's'. HDMI is an abbreviation for High-Definition Multimedia Interface.

[0015] The detection device 10 is connected to the optical sensor 40 via a communication interface 14. The optical sensor 40 is a device that acquires point cloud data 41 by irradiating a target area with light and receiving the reflected light reflected at the reflection point. The position of each point in the point cloud data 41 is determined from the irradiation angle of the irradiated light and the time from when the irradiated light is irradiated until the reflected light is received.

[0016] The detection device 10 comprises a point cloud acquisition unit 21, a voxelization unit 22, a difference identification unit 23, and a detection unit 24 as functional components. The functions of each functional component of the detection device 10 are implemented by software. Storage 13 stores programs that implement the functions of each functional component of the detection device 10. These programs are loaded into memory 12 by the processor 11 and executed by the processor 11. This enables the functions of each functional component of the detection device 10 to be implemented.

[0017] Additionally, the voxel definition 31 is stored in storage 13.

[0018] In Figure 1, only one processor 11 was shown. However, there may be multiple processors 11, and multiple processors 11 may work together to execute programs that implement each function.

[0019] ***Explanation of operation*** The operation of the detection device 10 according to Embodiment 1 will be explained with reference to Figures 2 to 7. The operation procedure of the detection device 10 according to Embodiment 1 corresponds to the detection method according to Embodiment 1. Furthermore, the program that implements the operation of the detection device 10 according to Embodiment 1 corresponds to the detection program according to Embodiment 1.

[0020] Referring to Figure 2, the voxel definition 31 according to Embodiment 1 will be explained. Voxel definition 31 is data that defines voxels 32. Voxels 32 are three-dimensional regions obtained by dividing the target region 33. The size of voxels 32 can be arbitrarily set according to their position in the target region 33. In Embodiment 1, larger voxels 32 are defined as the distance from the light sensor 40 increases.

[0021] Referring to Figure 3, the processing flow of the detection device 10 according to Embodiment 1 will be explained. In Embodiment 1, the detection device 10 detects objects left behind in the target area 33 as target objects.

[0022] (Step S11: Point cloud acquisition process) The point cloud acquisition unit 21 acquires point cloud data 41 collected by the optical sensor 40.

[0023] (Step S12: Voxelization process) The voxelization unit 22 generates voxel data 42 from the point cloud data 41 acquired in step S11. This will be explained in detail with reference to Figure 4. The voxelization unit 22 sets each point of the point cloud data 41 acquired in step S11 as a target point. The voxelization unit 22 associates the target point with the voxel containing the location of the target point among the multiple voxels obtained by dividing the target region 33. This generates voxel data 42 corresponding to the point cloud data 41.

[0024] (Step S13: Difference Identification Process) The difference identification unit 23 identifies difference cells 44 that have a difference between the voxel data 42 and the reference data 43. In Embodiment 1, the reference data 43 is the voxel data 42 generated from the background point cloud data. In other words, the reference data 43 is the voxel data 42 generated by performing the process in step S12 on the background point cloud data. The background point cloud data is the point cloud data 41 acquired by the optical sensor 40 when no objects or the like exist in the target area 33. Specifically, the difference identification unit 23 sets each of the multiple voxels 32 as the target voxel 32. The difference identification unit 23 determines whether there is a difference between the voxel data 42 and the reference data 43 based on whether or not there is a corresponding point for the target voxel 32, that is, based on the presence or absence of a point in the voxel 32. If there is a difference, the difference identification unit 23 identifies that voxel 32 as the difference cell 44. Here, as shown in Figure 5, the difference identification unit 23 identifies a target voxel 32 as a difference cell 44 if no points are associated with it in the reference data 43, but points are associated with it in the voxel data 42. In other words, the difference identification unit 23 does not identify a voxel 32 that has points associated with it in the reference data 43 as a difference cell 44, even if the number of associated points is different.

[0025] (Step S14: First filtering process) The difference identification unit 23 excludes from the difference cells 44 identified in step S13 any voxels 32 in which the number of points associated with the voxel data 42 is less than or equal to the reference number of points. For example, let's set the baseline score to 2. In this case, as shown in Figure 6, voxels 32 with 2 or fewer corresponding points in the voxel data 42 are excluded from the difference cell 44. In Figure 6, the middle voxel 32 is excluded from the difference cell 44 because it has 2 corresponding points. On the other hand, the bottom voxel 32 remains in the difference cell 44 because it has 3 corresponding points.

[0026] (Step S15: Second filtering process) The difference identification unit 23 clusters the difference cells 44 that were not excluded in step S14 to form a cluster 45. The clustering method can be any existing clustering technique, such as grouping adjacent difference cells 44 into the same cluster. The difference identification unit 23 excludes from the difference cells 44 any voxels 32 that constitute a cluster 45 whose number of voxels 32 included in the obtained cluster 45 is less than or equal to the first number of voxels. For example, let's assume the number of first voxels is 5. In this case, as shown in Figure 7, the voxels 32 that make up cluster 45B, which has 4 voxels 32, are excluded from the difference cells 44. On the other hand, the voxels 32 that make up cluster A, which has 10 voxels 32, remain as difference cells 44.

[0027] (Step S16: Detection process) The detection unit 24 detects the target object based on the difference cells 44 identified by the difference identification unit 23. In other words, the detection unit 24 detects the target object based on the difference cells 44 that were not excluded in step S15. For example, the detection unit 24 detects a cluster 45 composed of difference cells 44 that were not excluded in step S15 as the target object. In the processing flow explained using Figure 3, the number of first voxels and reference points may be set to decrease as the distance from the light sensor 40 increases, similar to the size of the voxel 32. In this case, the setting of the number of first voxels and reference points may or may not be linked to the setting of the size of the voxel 32. Setting only one of the number of first voxels or reference points is also acceptable. Furthermore, the setting of the number of first voxels and reference points may be set to increase as the distance from the light sensor 40 increases. In other words, the size of the voxel 32, the number of first voxels, and the number of reference points can be set to values ​​corresponding to the distance from the optical sensor 40.

[0028] ***Effects of Embodiment 1*** As described above, the detection device 10 according to Embodiment 1 associates each point of the point cloud data 41 with a plurality of voxels 32 that divide the target area 33, identifies the difference based on the presence or absence of the points associated with each voxel 32, and detects the target object. The light sensor 40 has difficulty illuminating the same spot every time, resulting in slight deviations. Consequently, the generated point cloud data 41 differs slightly each time. Therefore, simply comparing the point cloud data 41 with the reference data 43 results in differences in various places, making it difficult to accurately detect the target object. The detection device 10 according to Embodiment 1 performs a comparison for each voxel 32. This makes it less likely for slight deviations in the irradiation position of the irradiated light to appear as differences. As a result, the target object can be detected appropriately.

[0029] The further away the light sensor 40 is, the greater the shift in the position of the reflection point, even if only a slight change in the angle at which the light sensor 40 emits light. Therefore, the further away the light sensor 40 is, the more likely it is that a shift in the emitted light will occur. In Embodiment 1, the size of the voxel 32 increases as the distance from the light sensor 40 increases. Therefore, even in areas far from the light sensor 40, the deviation in the irradiation position of the irradiated light is less likely to appear as a difference, making it possible to appropriately detect the target object.

[0030] When acquiring background point cloud data, an object may cross in front of the light sensor 40. In this case, no point cloud is present in the voxel data 42, while the reference data 43 generated from the background point cloud data contains many voxels 32 that do contain point clouds. In the detection device 10 according to Embodiment 1, voxels 32 that are associated with points in reference data 43 are not identified as difference cells 44. The detection device 10 according to Embodiment 1 identifies voxels 32 that are not associated with points in reference data 43 but are associated with points in voxel data 42 as difference cells 44. Therefore, it is possible to prevent objects that cross in front of the optical sensor 40 from being detected.

[0031] Wind or vibrations may cause the detected location of vegetation or structures to shift. Such shifts may not be fully compensated for by voxelization alone. The detection device 10 according to Embodiment 1 excludes voxels 32 with a number of points less than or equal to the reference number of points from the difference cells 44. Furthermore, the detection device 10 according to Embodiment 1 excludes voxels 32 that constitute a cluster 45 with a number of voxels less than or equal to the first number of voxels from the difference cells 44. As a result, point clouds that appear due to wind or vibration are less likely to be identified as difference cells 44. Consequently, objects can be detected appropriately.

[0032] ***Other configurations*** <Example 1> In Embodiment 1, the detection unit 24, a functional component of the detection device 10, performed the final detection of the target object. However, the final detection of the target object may be configured to be performed by a specific type of object detection device, which is an external device to the detection device 10. In other words, the detection device 10 may perform the process up to generating the difference cell 44, which is information for the final detection of the target object, and the final detection of the target object may be configured to be performed by an external device.

[0033] Referring to Figure 8, the configuration of the detection device 10 according to the modified example 1 will be explained. Unlike the detection device 10 shown in Figure 1, the detection device 10 does not have a detection unit 24 as a functional component. Furthermore, object detection is performed by a specific type of object detection device. The specific type is a person, a vehicle, etc. In Embodiment 3, assuming the specific type is a person, This section describes an example of outputting a difference cell 44 to a detection device.

[0034] Referring to Figure 9, the processing flow of the detection device 10 according to the modified example 1 will be explained. The processes from step S21 to step S25 are the same as the processes from step S11 to step S15 in Figure 3.

[0035] (Step S26: Detection result output processing) The difference identification unit 23 hands over the information of the difference cell 44 to the person detection device. The person detection device uses the information from the difference cell 44 as input information to determine whether or not it is a person, and if it is determined to be a person, it determines the person's position. As described above, in the detection device 10 according to Modification 1, the detection of the target object and the determination of its type are performed by a specific type of object detection device. Therefore, the effect of reducing the processing load on the specific type of object detection device, such as a human detection device, can be obtained. Furthermore, the effect of suppressing false detections on the human detection device can also be expected.

[0036] Embodiment 2. Embodiment 2 differs from Embodiment 1 in that it detects the target object after repeatedly identifying the difference cell 44 over a certain period of time. Embodiment 2 explains this difference, and omits the explanation of the same points.

[0037] ***Explanation of the structure*** Referring to Figure 10, the configuration of the detection device 10 according to Embodiment 2 will be described. The detection device 10 differs from the detection device 10 shown in Figure 1 in that it includes a data storage processing unit 25 as a functional component. The function of the data storage processing unit 25, like other functions, is implemented by software.

[0038] ***Explanation of operation*** Referring to Figure 11, the processing flow of the detection device 10 according to Embodiment 2 will be explained. The process shown in Figure 11 is repeatedly executed at each reference interval. The processes from step S31 to step S35 are the same as the processes from step S11 to step S15 in Figure 3.

[0039] (Step S36: Storage process) The storage processing unit 25 records the voxels 32 that were not excluded in step S35 and remain as difference cells 44 into the memory 12.

[0040] (Step S37: High-Percentage Cell Identification Process) The storage processing unit 25 identifies voxels 32 whose recorded percentage as a difference cell 44 is equal to or greater than the reference percentage as target cells 46. Specifically, let T be the number of times the process shown in Figure 11 was executed during the most recent reference period. The storage processing unit 25 sets each voxel 32 as the target voxel 32. The storage processing unit 25 identifies the number of times C that the target voxel 32 was recorded as a difference cell 44 during the most recent reference period. The storage processing unit 25 divides the number C by the number T to calculate the percentage R that the target voxel 32 was recorded as a difference cell 44 during the most recent reference period. If the percentage R is greater than or equal to the reference percentage α, the storage processing unit 25 identifies the target voxel 32 as the target cell 46.

[0041] The reference ratio α may be set to a value corresponding to the position in the target area 33. In this case, the reference ratio α is set to a lower value the further away the target area 333 is from the optical sensor 40.

[0042] (Step S38: Clustering process) The detection unit 24 clusters the target cells 46 identified in step S37 to form a cluster 47. The clustering method can be any existing clustering technique, such as grouping adjacent difference cells 44 into the same cluster.

[0043] (Step S39: Detection process) The detection unit 24 detects the clusters 47 that satisfy the conditions among the clusters 47 formed in step S38 as target objects. Specifically, the detection unit 24 detects clusters 47 as objects that satisfy both of the following conditions 1 and 2. Condition 1 is that the number of voxels 32 included in the cluster 47 is greater than the number of second voxels. Condition 2 is that the size of the cluster 47 is greater than the standard size. The size of the cluster 47 is either in the vertical direction, horizontal direction, or depth direction, or a combination of at least two of the vertical, horizontal, and depth directions. Here, the number of second voxels may be the same as the number of first voxels, or it may be a different value. Also, similar to the number of first voxels, the number of second voxels may be set to be smaller as the distance from the light sensor 40 increases. In this case, the setting of the number of second voxels may or may not be linked to the setting of the size of voxel 32. Also, the number of second voxels may be set to be larger as the distance from the light sensor 40 increases. In other words, the number of second voxels can be set to a value corresponding to the distance from the light sensor 40.

[0044] ***Effects of Embodiment 2*** As described above, the detection device 10 according to Embodiment 2 detects the target object after repeatedly identifying the difference cell 44 over a certain period of time. This makes it possible to appropriately detect the object that has been left behind.

[0045] The detection device 10 according to Embodiment 2 detects an object using a voxel 32 in which the ratio R recorded as a difference cell 44 is equal to or greater than the reference ratio α. By using the ratio recorded as a difference cell 44, detection is possible even if the object is temporarily obscured, such as being hidden behind an object.

[0046] In Embodiment 2, the reference ratio α is set to a lower value the further away the object is from the optical sensor 40 in the target area 333. This means that objects can be detected even with a lower ratio R at a greater distance. Point clouds tend to be less accurately acquired at greater distances due to the increased likelihood of occlusion. Therefore, by allowing objects to be detected even with a lower ratio R at greater distances, it becomes easier to prevent missed detections of objects at distant locations. On the other hand, at closer locations, a higher reference ratio α makes it easier to prevent false detections of objects.

[0047] Embodiment 3. Embodiment 3 differs from Embodiment 1 in that it identifies the movement trajectory of the object. Embodiment 3 will explain this difference, while the same points will not be explained.

[0048] ***Explanation of the structure*** Referring to Figure 12, the configuration of the detection device 10 according to Embodiment 3 will be described. The detection device 10 differs from the detection device 10 shown in Figure 1 in that, instead of the detection unit 24, it includes a trajectory identification unit 28 and a notification unit 29 as functional components. The functions of the trajectory identification unit 28 and the notification unit 29 are implemented by software, as with the other functions.

[0049] ***Explanation of operation*** Referring to Figure 13, the processing flow of the detection device 10 according to Embodiment 3 will be explained. In Embodiment 3, the detection device 10 detects a moving object as an object when it moves within the target area 33.

[0050] The processes from step S41 to step S45 are repeatedly executed at each reference interval. The processes from step S41 to step S45 are the same as the processes from step S11 to step S15 in Figure 3. However, in step S45, the identified cluster 45 is recorded in memory 12 for a certain period of time.

[0051] (Step S46: Trajectory identification process) The trajectory identification unit 28 identifies the movement trajectory by associating the cluster 45 identified in the most recent step S45 with the corresponding cluster 45 identified in step S45 executed one cycle earlier. Specifically, the trajectory identification unit 28 compares the position of the cluster 45 identified in the most recent step S45 with the cluster 45 identified in the previously executed step S45. The trajectory identification unit 28 determines that both clusters 45 are the same object if the positional relationship is as expected from the direction and speed of movement of the object. In this case, the trajectory identification unit 28 may also consider the condition that the difference in size between the two clusters 45 is less than an upper limit when determining whether the two clusters 45 are the same object.

[0052] (Step S47: Notification Processing) The notification unit 29 notifies the movement trajectory identified in step S46. Specifically, the notification unit 29 identifies movement trajectories that satisfy the conditions for having the characteristics of a moving object, such as having the same direction of movement and a constant speed of movement. It then notifies the identified movement trajectories. For example, the notification unit 29 notifies by displaying the identified movement trajectories.

[0053] ***Effects of Embodiment 3*** As described above, the detection device 10 according to Embodiment 3 identifies and notifies the movement trajectory of the target object. This makes it possible to detect a moving object while simultaneously identifying its movement trajectory.

[0054] The detection device 10 according to Embodiment 3 notifies only of movement trajectories that satisfy the condition of having the characteristics of a moving object. As a result, the notification unit 29 can notify objects that continue to move in an irregular direction within the same range, such as swaying plants.

[0055] ***Other configurations*** <Modification 2> In Embodiment 1, voxel data 42 generated from background point cloud data was used as the reference data 43. In Embodiment 3, voxel data 42 generated in step S42 one cycle earlier may be used as the reference data 43. This makes it possible to omit processes such as acquiring background point cloud data in advance and periodically updating the background point cloud data.

[0056] Embodiment 4. Embodiment 4 differs from Embodiments 1 to 4 in that it sets an area within the target area 33 where no object is detected. Embodiment 4 explains this difference, while omitting explanations of the same points. Embodiment 4 describes a case where functionality is added to Embodiment 1. However, it is also possible to add functionality to Embodiments 2 and 3.

[0057] ***Explanation of the structure*** Referring to Figure 14, the configuration of the detection device 10 according to Embodiment 4 will be described. The detection device 10 differs from the detection device 10 shown in Figure 1 in that it includes a mask processing unit 30 as a functional component. The function of the mask processing unit 30, like other functions, is implemented by software. Additionally, the mask definition 34 is stored in storage 13.

[0058] ***Explanation of operation*** Referring to Figure 15, the mask definition 34 according to Embodiment 4 will be described. The mask definition 34 indicates the area within the target region 33 that will be excluded from detection. For example, if the target object is an object placed on the ground, the area in the air that is a certain distance away from the ground will be set as the area to be excluded from detection. The mask definition 34 may also accept and set the designation of areas to be excluded from detection. Alternatively, the mask definition 34 may accept the designation of areas to be designated as target areas 33 from among a given area, and the areas not designated may be set as areas to be excluded from detection.

[0059] Referring to Figure 16, the processing flow of the detection device 10 according to Embodiment 4 will be explained. The process in step S51 is the same as the process in step S11 in Figure 3. The processes in steps S55 to S57 are the same as the processes in steps S14 to S16 in Figure 3.

[0060] (Step S52: Masking) The mask processing unit 30 sets the area indicated by the mask definition 34 within the target area 33 as the mask area.

[0061] (Step S53: Voxelization process) The voxelization unit 22 generates voxel data 42 from the point cloud data 41, similar to step S12 in Figure 3. In this process, the voxelization unit 22 associates points only with the voxels 32 of the target region 33, excluding the mask region set in step S52.

[0062] (Step S54: Difference Identification Process) The difference identification unit 23 identifies difference cells 44 that have a difference between the voxel data 42 and the reference data 43, similar to step S13 in Figure 3. In this case, the difference identification unit 23 compares only the voxels 32 that constitute the target area 33 excluding the mask area set in step S52, and identifies the difference cells 44.

[0063] ***Effects of Embodiment 4*** As described above, the detection device 10 according to Embodiment 4 sets a mask region and detects objects only in the target region 33 excluding the mask region. This makes it possible to reduce the computational load and to suppress unnecessary false detections.

[0064] Embodiment 5. Embodiment 5 differs from Embodiment 2 in the method of identifying the target cell 46. Embodiment 5 explains this difference, while omitting explanations of the similarities.

[0065] ***Explanation of operation*** Referring to Figure 17, the processing flow of the detection device 10 according to Embodiment 5 will be explained. The processing from steps S61 to S66 is the same as the processing from steps S31 to S36 in Figure 11. The processing from steps S68 to S69 is the same as the processing from steps S38 to S39 in Figure 11.

[0066] (Step S67: IoU determination process) The storage processing unit 25 identifies the target cell 46 from the recorded difference cell 44 using IoU. IoU stands for Intersection over Union. Specifically, first, the storage processing unit 25 identifies each point cloud data 41 as the target point cloud data 41, and sets the difference cells 44 identified from the target point cloud data 41 as the target difference cells 44. The storage processing unit 25 clusters the target difference cells 44 to form a cluster 48. The clustering method can be any existing clustering technique, such as making adjacent difference cells 44 the same cluster. Then, the storage processing unit 25 sets the smallest rectangular area surrounding the cluster 48 as the area of ​​the cluster 48. Next, the storage processing unit 25 identifies voxels in overlapping regions of clusters 48 formed from the difference cells 44 of each point cloud data 41, where the overlapping regions of clusters 48 exceed a certain threshold, as target cells 46. Let T be the number of times the process shown in Figure 17 has been executed in the most recent reference period. The storage processing unit 25 identifies the number C of clusters 48 regions that include the target voxel 32. The storage processing unit 25 calculates the overlap ratio R, which is the percentage of clusters 48 regions that overlap with the target voxel 32, by dividing the number C by the number T. If the overlap ratio R is greater than or equal to the threshold α, the storage processing unit 25 identifies the target voxel 32 as a target cell 46.

[0067] Please refer to Figure 18 for further explanation. In Figure 18, T=3. Figure 18 shows the region of cluster 48A obtained from point cloud data 41A, the region of cluster 48B obtained from point cloud data 41B, and the region of cluster 48C obtained from point cloud data 41C. Let's assume the reference ratio α = 1. In this case, the area where the regions of the clusters 48 of the three point cloud data 41 all overlap becomes the target cell 46. In other words, the voxel 32 in the hatched area in Figure 18 becomes the target cell 46.

[0068] The storage processing unit 25 may also identify voxels 32 in the regions included in cluster 48, including the overlapping region, as target cells 46. As in the example in Figure 18, the reference ratio α = 1. In this case, voxels 32 included in the regions of cluster 48A, cluster 48B, and cluster 48C, which are hatched in Figure 19, are identified as target cells 46. The storage processing unit 25 may also identify the voxel 32 in the region surrounding the cluster 48 that includes the overlapping region as the target cell 46. As in the example in Figure 18, the reference ratio α = 1. In this case, the voxel 32 included in the region surrounding the hatched areas of cluster 48A, cluster 48B, and cluster 48C in Figure 20 is identified as the target cell 46.

[0069] ***Effects of Embodiment 5*** As described above, the detection device 10 according to Embodiment 5 identifies the target cell 46 using IoU. When identifying the target cell 46 using the method described in Embodiment 2, the computational load increases as the number of difference cells 44 increases. In contrast, when identifying the target cell 46 using IoU, the number of difference cells 44 does not significantly affect the computational load, and the computational load increases as the number of clusters 48 increases. The increase in the number of clusters 48 is more gradual than the increase in the number of difference cells 44. Therefore, the method of identifying the target cell 46 using IoU has the effect of reducing the computational load compared to the method described in Embodiment 2.

[0070] Embodiment 6. Embodiment 6 differs from Embodiments 1 to 5 in that it excludes detected areas from detection for a specified period of time. Embodiment 6 explains this difference, while omitting explanations of the same points. Embodiment 6 describes a case where functionality is added to Embodiment 1. However, it is also possible to add functionality to Embodiments 2 to 5.

[0071] ***Explanation of the structure*** Referring to Figure 21, the configuration of the detection device 10 according to Embodiment 6 will be described. The detection device 10 differs from the detection device 10 shown in Figure 1 in that it includes a completed area setting unit 211 as a functional component. The function of the completed area setting unit 211, like other functions, is implemented by software.

[0072] ***Explanation of operation*** Referring to Figure 22, the processing flow of the detection device 10 according to Embodiment 6 will be explained. The process from step S71 to step S72 is the same as the process from step S11 to step S12 in Figure 3. The process from step S75 to step S77 is the same as the process from step S14 to step S16 in Figure 3.

[0073] (Step S73: Process to set completed area) The completed area setting unit 211 sets the portion of the target area 33 that was identified as an object by the detection unit 24 in the most recently executed step S77 as a detected area until the reference time has elapsed. In other words, the completed area setting unit 211 sets the portion of the voxel 32 that was identified as an object as a detected area. The reference time is a time set in advance, and it is the interval time that prevents repeated detection of the same object.

[0074] (Step S74: Difference Identification Process) The difference identification unit 23 identifies difference cells 44 that have a difference between the voxel data 42 and the reference data 43 for the voxels that make up the area of ​​the target area 33 excluding the detected area set in step S73.

[0075] ***Effects of Embodiment 6*** As described above, the detection device 10 according to Embodiment 6 sets the portion region detected as an object as a detected region and removes it from the specified target of the difference cell 44 for a reference time. Removing it from the specified target of the difference cell 44 means removing it from the detection region of the object. In other words, the region that was detected as an object will no longer be detected as an object for the reference time. This prevents the system from continuously detecting the same object, thus avoiding unnecessary notifications. Furthermore, reducing the number of specific targets in the 44 differential cells reduces the computational load.

[0076] Embodiment 7. Embodiment 7 differs from Embodiments 1 to 6 in that it identifies the difference cell 44 using the inter-frame difference, which is the difference with the point cloud data from a specific time earlier. Embodiment 7 explains this difference, and omits the explanation of the points that are the same. Embodiment 7 describes a case where functionality is added to Embodiment 1. However, it is also possible to add functionality to Embodiments 2 to 6.

[0077] ***Explanation of operation*** Referring to Figure 3, the processing flow of the detection device 10 according to Embodiment 7 will be explained. Except for the processing in step S13, the procedure is the same as in Embodiment 1.

[0078] (Step S13: Difference Identification Process) The difference identification unit 23 identifies voxels 32 that have a difference between the voxel data 42 and the reference data 43, and that do not have a difference between the voxel data 42 and the previous frame data, as difference cells 44. The previous frame data is the voxel data 42 generated from the point cloud data 41 from a specific time before. Here, the point cloud data 41 from a specific time before is the point cloud data 41 acquired one frame earlier, that is, the point cloud data from the previous frame. Specifically, the difference identification unit 23 sets each of the multiple voxels 32 as the target voxel 32. The difference identification unit 23 determines whether there is a difference between the voxel data 42 and the reference data 43 based on whether or not there is a corresponding point for the target voxel 32, that is, based on the presence or absence of a point in the voxel 32. The difference identification unit 23 also determines whether there is a difference between the voxel data 42 and the previous frame data based on whether or not there is a corresponding point for the target voxel 32, that is, based on the presence or absence of a point in the voxel 32. Then, if there is a difference between the voxel data 42 and the reference data 43, and there is no difference between the voxel data 42 and the previous frame data, the difference identification unit 23 identifies that voxel 32 as the difference cell 44. The difference identification unit 23 determines whether there is a difference between the voxel data 42 and the previous frame data if, for the target voxel 32, a point is associated with the voxel data 42 in one case, but not with the other. The difference identification unit 23 may also determine whether there is a difference between the voxel data 42 and the previous frame data if the number of points associated with the voxel data 42 differs from the previous frame data by a certain number or more.

[0079] In other words, the difference identification unit 23 compares the voxel data 42 not only with the reference data 43 but also with the previous frame data. Then, as shown in Figure 23, the difference identification unit 23 identifies voxels 32 that have a difference between the voxel data 42 and the reference data 43, and that do not have a difference between the voxel data 42 and the previous frame data, as difference cells 44.

[0080] ***Effects of Embodiment 7*** As described above, the detection device 10 according to Embodiment 7 identifies voxels 32 that have no difference between the voxel data 42 and the previous frame data as difference cells 44. The parts that have a difference between the voxel data 42 and the previous frame data are parts that have time-series movement. Parts that have time-series movement are unlikely to be abandoned objects. Therefore, when an abandoned object is the target object, appropriate detection becomes possible by targeting only the parts that have no difference between the voxel data 42 and the previous frame data.

[0081] Embodiment 8. Embodiment 8 differs from Embodiments 1 to 7 in that the size of the voxel 32 is dynamically changed. Embodiment 8 will explain this difference, and the same points will not be explained. Embodiment 8 describes a case where functionality is added to Embodiment 1. However, it is also possible to add functionality to Embodiments 2 to 7.

[0082] ***Explanation of the structure*** Referring to Figure 24, the configuration of the detection device 10 according to Embodiment 8 will be described. The detection device 10 differs from the detection device 10 shown in Figure 1 in that it includes a voxel setting unit 212 as a functional component. The function of the voxel setting unit 212, like other functions, is implemented by software.

[0083] ***Explanation of operation*** Referring to Figure 25, the processing flow of the detection device 10 according to Embodiment 8 will be explained. The processes from step S81 to step S86 are the same as the processes from step S11 to step S16 in Figure 3. As a prerequisite for starting the process, the voxel setting unit 212 divides the target area 33 according to the voxel definition 31 and sets up multiple voxels 32. Here, the size of each voxel 32 defined in the voxel definition 31 is called the default size.

[0084] (Step S87: Voxel setting process) The voxel setting unit 212 divides the voxel 32 identified as the difference cell 44 into multiple parts to reduce its size. The voxel 32 identified as the difference cell 44 is the voxel 32 that remained as the difference cell 44 in step S85. As shown in Figure 26(A), let's assume that voxels 32X and 32Y are identified as difference cells 44. In this case, as shown in Figure 26(B), the area of ​​voxel 32X and the area of ​​voxel 32Y are divided, and multiple smaller voxels 32 are set. In Figure 26, the area of ​​voxel 32X and the area of ​​voxel 32Y are each divided into four parts. Then, in the area of ​​voxel 32X, four voxels 32X1, voxel 32X2, voxel 32X3, and voxel 32X4 are set. Similarly, in the area of ​​voxel 32Y, four voxels 32Y1, voxel 32Y2, voxel 32Y3, and voxel 32Y4 are set.

[0085] Furthermore, the voxel setting unit 212, for areas where small-sized voxels 32 are set, if not all voxels 32 within the default-sized area are identified as difference cells 44, reverts the default-sized area back to a single voxel. In other words, the voxel setting unit 212 sets each area where a small-sized voxel 32 is set from the areas of each voxel 32 defined in the voxel definition 31 as a target area. Then, if not all voxels 32 within the target area are identified as difference cells 44, the voxel setting unit 212 sets that area to a single voxel 32. For example, suppose a voxel is set as shown in Figure 26(B). In this case, the area where voxel 32X was originally set and the area where voxel 32Y was originally set become areas where a smaller size voxel 32 is set. Then, suppose voxel 32Y1 and voxel 32Y2 are identified as difference cells 44, as shown in Figure 27(A). In this case, as shown in Figure 27(B), in the area where voxel 32X was originally set, not all voxels 32 were identified as difference cells 44, so they are returned to a single voxel 32X.

[0086] Subsequently, when new point cloud data 41 is acquired in step S81, processing is performed using the voxels 32 set in step S87.

[0087] ***Effects of Embodiment 8*** As described above, the detection device 10 according to Embodiment 8 divides the voxel 32 identified as the difference cell 44 and sets smaller size voxels 32. This allows for more precise identification of the presence or absence of differences in the portion identified as the difference cell 44. As a result, it becomes possible to more precisely identify the position of the object. Furthermore, for the parts that were not identified as difference cells 44, the presence or absence of differences can be identified in a coarser area by returning them to a single voxel. As a result, the number of targets to be identified as difference cells 44 can be reduced, thereby suppressing the computational load.

[0088] ***Other configurations*** <Variation 3> In Embodiment 1, each functional component was implemented in software. However, in Modification 3, each functional component may be implemented in hardware. The differences between this Modification 3 and Embodiment 1 will be explained below.

[0089] When each functional component is implemented in hardware, the detection device 10 includes an electronic circuit 15 instead of a processor 11, memory 12, and storage 13. The electronic circuit 15 is a dedicated circuit that implements the functions of each functional component, as well as the functions of the memory 12 and storage 13.

[0090] The electronic circuits 15 can include single circuits, complex circuits, programmed processors, parallel programmed processors, logic ICs, GAs, ASICs, and FPGAs. ASIC stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be implemented in a single electronic circuit 15, or each functional component may be implemented by distributing them across multiple electronic circuits 15.

[0091] <Modification 4> As a fourth variation, some of the functional components may be implemented in hardware, while others may be implemented in software.

[0092] The processor 11, memory 12, storage 13, and electronic circuit 15 are collectively referred to as the processing circuit. In other words, the function of each functional component is realized by the processing circuit.

[0093] Furthermore, the term "part" in the above explanation may be replaced with "circuit," "process," "procedure," "processing," or "processing circuit."

[0094] The various aspects of this disclosure are summarized below as an appendix. (Note 1) A point cloud acquisition unit repeatedly acquires point cloud data obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection point, A voxelization unit generates voxel data corresponding to the target point cloud data by taking each of the point cloud data repeatedly acquired by the point cloud acquisition unit as the target point, and associating each point of the target point cloud data with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, For each of the plurality of voxels, a difference identification unit identifies difference cells that have a difference between the voxel data generated by the voxelization unit and the reference data for the target point cloud data, based on the presence or absence of associated points. An accumulation processing unit identifies voxels as target cells among the clusters obtained by clustering the difference cells identified by the difference identification unit from each point cloud data acquired during the detection period, in which clusters exceeding a certain percentage overlap. A detection device equipped with the following features. (Note 2) The storage processing unit identifies, in addition to the voxels in the overlapping region, the voxels in the region included in the cluster containing the overlapping region as the target cells. The detection device described in Appendix 1. (Note 3) The detection device further includes, A detection unit detects an object based on the target cell identified by the storage processing unit. A detection device as described in Appendix 1 or 2, comprising the following features. (Note 4) The detection device further includes, A completed area setting unit sets a portion of the target area identified as the target object by the detection unit as a detected area until a reference time has elapsed. Equipped with, The difference identification unit identifies difference cells that have a difference between the voxel data and the reference data for voxels that constitute the area of ​​the target region excluding the detected region set by the completed region setting unit. The detection device described in Appendix 3. (Note 5) The difference identification unit identifies the voxel data generated from each point cloud data as the target voxel data, and for each of the plurality of voxels, it identifies the cells that have a difference between the target voxel data and the reference data, and that do not have a difference between the target voxel data and the previous frame data, which is voxel data generated from point cloud data from a specific time earlier, as the difference cells for the target voxel data. A detection device as described in any one of the items 1 to 4 in the appendix. (Note 6) The detection device further includes, A voxel setting unit that divides the target area into default-sized areas and sets the plurality of voxels, wherein the voxels identified as difference cells by the difference identification unit are divided into multiple parts to reduce their size. A detection device as described in any one of the appendices 1 to 5, comprising: (Note 7) If not all voxels within the default-sized area are identified as difference cells, the voxel setting unit will revert the default-sized area back to a single voxel. The detection device described in Appendix 6. (Note 8) The storage processing unit transfers the target cell to a specific type of object detection device. The detection device described in Appendix 1 or 2. (Note 9) The computer repeatedly acquires point cloud data obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection points. The computer generates voxel data corresponding to the target point cloud data by taking each of the repeatedly acquired point cloud data as the target point data, and associating each point of the target point cloud data with the voxel containing the position of the target point among the multiple voxels obtained by dividing the target region. The computer identifies difference cells for each of the multiple voxels, based on the presence or absence of associated points, between the voxel data generated for the target point cloud data and the reference data, A detection method in which a computer clusters the difference cells identified from each point cloud data acquired during the detection period, and identifies voxels in overlapping regions where a certain percentage or more of the clusters overlap as target cells. (Note 10) A point cloud acquisition process that repeatedly acquires point cloud data obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection point, A voxelization process is performed to generate voxel data corresponding to the target point cloud data by assigning each of the point cloud data repeatedly acquired by the point cloud acquisition process to the target point, and associating each point of the target point cloud data with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, and For each of the aforementioned multiple voxels, a difference identification process is performed to identify difference cells that have a difference between the voxel data generated by the voxelization process and the reference data, based on the presence or absence of associated points for the target point cloud data. An accumulation process is performed to identify target cells as voxels in overlapping regions where clusters exceeding a certain percentage overlap, among the clusters obtained by clustering the difference cells identified by the difference identification process from each point cloud data acquired during the detection period. A detection program that makes a computer function as a detection device to perform this task.

[0095] The embodiments and variations of this disclosure have been described above. Some of these embodiments and variations may be implemented in combination. Alternatively, some or all of them may be implemented in part. However, this disclosure is not limited to the embodiments and variations described above, and various modifications are possible as needed. [Explanation of Symbols]

[0096] 10 Detection device, 11 Processor, 12 Memory, 13 Storage, 14 Communication interface, 21 Point cloud acquisition unit, 22 Voxelization unit, 23 Difference identification unit, 24 Detection unit, 25 Storage processing unit, 26 Position acquisition unit, 27 Type determination unit, 28 Trajectory identification unit, 29 Notification unit, 30 Mask processing unit, 31 Voxel definition, 32 Voxel, 33 Target area, 34 Mask definition, 40 Optical sensor, 41 Point cloud data, 42 Voxel data, 43 Reference data, 44 Difference cell, 45 Cluster, 46 Target cell, 47 Cluster.

Claims

1. A point cloud acquisition unit repeatedly acquires point cloud data obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection point, A voxelization unit generates voxel data corresponding to the target point cloud data by taking each of the point cloud data repeatedly acquired by the point cloud acquisition unit as the target point, and associating each point of the target point cloud data with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, For each of the plurality of voxels, a difference identification unit identifies difference cells that have a difference between the voxel data generated by the voxelization unit and the reference data for the target point cloud data, based on the presence or absence of associated points. An accumulation processing unit identifies voxels as target cells among the clusters obtained by clustering the difference cells identified by the difference identification unit from each point cloud data acquired during the detection period, in which clusters exceeding a certain percentage overlap. A detection device equipped with the following features.

2. The storage processing unit identifies, in addition to the voxels in the overlapping region, the voxels in the region included in the cluster containing the overlapping region as the target cells. The detection device according to claim 1.

3. The detection device further, A detection unit detects an object based on the target cell identified by the storage processing unit. The detection device according to claim 1, comprising:

4. The detection device further, A completed area setting unit sets a portion of the target area identified as the target object by the detection unit as a detected area until a reference time has elapsed. Equipped with, The difference identification unit identifies difference cells that have a difference between the voxel data and the reference data for voxels that constitute the area of ​​the target region excluding the detected region set by the completed region setting unit. The detection device according to claim 3.

5. The difference identification unit identifies the voxel data generated from each point cloud data as the target voxel data, and for each of the plurality of voxels, it identifies the cells that have a difference between the target voxel data and the reference data, and that do not have a difference between the target voxel data and the previous frame data, which is voxel data generated from point cloud data from a specific time earlier, as the difference cells for the target voxel data. The detection device according to claim 1.

6. The detection device further, A voxel setting unit that divides the target area into default-sized areas and sets the plurality of voxels, wherein the voxels identified as difference cells by the difference identification unit are divided into multiple parts to reduce their size. The detection device according to claim 1, comprising:

7. If not all voxels within the default-sized area are identified as difference cells, the voxel setting unit will revert the default-sized area back to a single voxel. The detection device according to claim 6.

8. The storage processing unit transfers the target cell to a specific type of object detection device. The detection device according to claim 1.

9. The computer repeatedly acquires point cloud data obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection points. The computer generates voxel data corresponding to the target point cloud data by taking each of the repeatedly acquired point cloud data as the target point data, and associating each point of the target point cloud data with the voxel containing the position of the target point among the multiple voxels obtained by dividing the target region. The computer identifies difference cells for each of the multiple voxels, based on the presence or absence of associated points, between the voxel data generated for the target point cloud data and the reference data, A detection method in which a computer clusters the difference cells identified from each point cloud data acquired during the detection period, and identifies voxels in overlapping regions where a certain percentage or more of the clusters overlap as target cells.

10. A point cloud acquisition process that repeatedly acquires point cloud data obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection point, A voxelization process is performed to generate voxel data corresponding to the target point cloud data by assigning each of the point cloud data repeatedly acquired by the point cloud acquisition process to the target point, and associating each point of the target point cloud data with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, and For each of the aforementioned multiple voxels, a difference identification process is performed to identify difference cells that have a difference between the voxel data generated by the voxelization process and the reference data, based on the presence or absence of associated points for the target point cloud data. An accumulation processing process is performed to identify target cells as voxels in overlapping regions where clusters exceeding a certain percentage overlap, among the clusters obtained by clustering the difference cells identified by the difference identification process from each point cloud data acquired during the detection period. A detection program that makes a computer function as a detection device to perform this task.

Images