Background data management device, background data management method, and background data management program

Abstract

This enables the preparation of appropriate background point cloud data. [Solution] The point cloud acquisition unit 21 acquires point cloud data containing data from multiple points obtained by irradiating a target area with illumination light and receiving reflected light reflected at reflection points, and records the acquired point cloud data together with identification information for that point cloud data. The reference background generation unit 22 generates a single point cloud data as background point cloud data, which contains the data of each point included in the multiple point cloud data acquired by the point cloud acquisition unit 21 at different times.

Description

Technical Field

[0001] The present disclosure relates to a technique for detecting an object using point cloud data.

Background Art

[0002] There is a technique for detecting an object such as a replacement object using a visible light camera. When detecting an object using a visible light camera, it is not possible to appropriately detect the object in an environment where the amount of light is insufficient, such as at night, or in an environment where direct sunlight shines into the visible light camera and flare occurs.

[0003] There is an optical sensor such as LiDAR that collects point cloud data by irradiating irradiation light and receiving the reflected light reflected at the reflection points. LiDAR is an abbreviation for Light Detection And Ranging. In an optical sensor, it is possible to obtain point cloud data even in an environment where the amount of light is insufficient, such as at night, or in an environment where direct sunlight shines into the visible light camera and flare occurs. Therefore, detecting an object based on the point cloud data obtained using an optical sensor has been studied.

[0004] Patent Document 1 describes comparing background point cloud data held in a memory with current point cloud data and identifying that some object has appeared when a difference occurs.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Due to the effects of wind and vibration, the position of vegetation or structures changes over time. When detecting objects by comparing background point cloud data with current point cloud data, as in Patent Document 1, there is a risk of false detection, where an object is mistakenly identified as having appeared at a location where this change has occurred. This disclosure aims to enable the preparation of appropriate background point cloud data. [Means for solving the problem]

[0007] The background data management device related to this disclosure is: A point cloud acquisition unit acquires point cloud data containing data from multiple points obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection points, and records the acquired point cloud data together with identification information for that point cloud data. A reference background generation unit generates a single point cloud data set as background point cloud data, which includes the data of each point in multiple point cloud data sets acquired at different times by the point cloud acquisition unit. It is equipped with. [Effects of the Invention]

[0008] In this disclosure, the background point cloud data is defined as a point cloud data set containing the data of each point included in multiple point cloud data sets recorded together with identification information. This makes it possible to obtain appropriate background point cloud data by removing points from the point cloud data that satisfy certain conditions. As a result, when detecting an object by comparing the background point cloud data with the current point cloud data, it becomes possible to obtain appropriate detection results. [Brief explanation of the drawing]

[0009] [Figure 1] Configuration diagram of the background data management device 10 according to Embodiment 1. [Figure 2] An explanatory diagram of the voxel definition 31 according to Embodiment 1. [Figure 3] Flowchart of the background generation process according to Embodiment 1. [Figure 4] An explanatory diagram of the reference background 42 related to Embodiment 1. [Figure 5] Explanatory drawing of the voxelization method according to Embodiment 1. [Figure 6] Flowchart of the background update process according to Embodiment 1. [Figure 7] Explanatory drawing of the difference identification process according to Embodiment 1. [Figure 8] Explanatory drawing of the filtering process according to Embodiment 1. [Figure 9] Explanatory drawing of the background candidate extraction process according to Embodiment 1. [Figure 10] Configuration diagram of the background data management device 10 according to Modification 2. [Figure 11] Flowchart of the background update process according to Modification 2. [Figure 12] Configuration diagram of the background data management device 10 according to Modification 3. [Figure 13] Explanatory drawing of the display content according to Modification 3. [Figure 14] Explanatory drawing of the display content according to Modification 4. [Figure 15] Explanatory drawing of the background candidate extraction process according to Modification 5. [Figure 16] Explanatory drawing of the background candidate extraction process according to Modification 5. [Figure 17] Configuration diagram of the background data management device 1​​​​​​​​​​​​​​​​​​​​​​​​​​​​The background data management device 10 is a computer. The background data management device 10 comprises hardware including a processor 11, memory 12, storage 13, and a communication interface 14. The processor 11 is connected to and controls the other hardware via signal lines.

[0011] Processor 11 is an IC that performs processing. IC stands for Integrated Circuit. Specific examples of processor 11 include CPU, DSP, and GPU. CPU stands for Central Processing Unit. DSP stands for Digital Signal Processor. GPU stands for Graphics Processing Unit.

[0012] Memory 12 is a storage device that temporarily stores data. Specific examples of memory 12 include SRAM and DRAM. SRAM stands for Static Random Access Memory. DRAM stands for Dynamic Random Access Memory.

[0013] Storage 13 is a storage device for storing data. Specific examples of storage 13 include HDDs and SSDs. HDD stands for Hard Disk Drive. SSD stands for Solid State Drive. Storage 13 may also be a portable recording medium such as an SD® memory card, CompactFlash®, NAND flash, flexible disk, optical disk, compact disk, Blu-ray® disc, or DVD. SD stands for Secure Digital. DVD stands for Digital Versatile Disk.

[0014] Communication interface 14 is an interface for communicating with external devices. Specific examples of communication interface 14 include Ethernet®, USB, and HDMI® ports. USB stands for Universal Serial Bus. HDMI stands for High-Definition Multimedia Interface.

[0015] The background data management device 10 is connected to the optical sensor 40 via a communication interface 14. The optical sensor 40 is a device that acquires point cloud data 41 by irradiating a target area with light and receiving the reflected light reflected at the reflection points. The position of each point in the point cloud data 41 is determined from the irradiation angle of the irradiated light and the time from when the irradiated light is irradiated until the reflected light is received.

[0016] The background data management device 10 comprises, as functional components, a point cloud acquisition unit 21, a reference background generation unit 22, a voxelization unit 23, a difference identification unit 24, a background candidate extraction unit 25, and a background update unit 26. The functions of each functional component of the background data management device 10 are implemented by software. Storage 13 stores programs that implement the functions of each functional component of the background data management device 10. These programs are loaded into memory 12 by the processor 11 and executed by the processor 11. This enables the implementation of the functions of each functional component of the background data management device 10.

[0017] Storage 13 stores voxel definitions 31, reference backgrounds 42, etc. Memory stores the latest background 43, etc.

[0018] In Figure 1, only one processor 11 was shown. However, there may be multiple processors 11, and multiple processors 11 may work together to execute programs that implement each function.

[0019] ***Explanation of operation*** Referring to Figures 2 to 9, the operation of the background data management device 10 according to Embodiment 1 will be explained. The operation procedure of the background data management device 10 according to Embodiment 1 corresponds to the background data management method according to Embodiment 1. Furthermore, the program that implements the operation of the background data management device 10 according to Embodiment 1 corresponds to the background data management program according to Embodiment 1.

[0020] The operation of the background data management device 10 includes a background generation process and a background update process. The background generation process is the process of generating a reference background 42. The background generation process is executed once before the object detection process is executed. The background update process is the process of updating the reference background 42 to generate the latest background 43. The background update process is executed repeatedly after the background generation process has been executed. The background update process is executed repeatedly, for example, in parallel with the execution of the object detection process. Here, the reference background 42 is the default background point cloud data that is loaded only once when the system starts up. The reference background 42 becomes outdated over time and differs in appearance from the actual point cloud data, so it needs to be updated. The latest background 43 is the background point cloud data at the most recent time if updates are performed while the system is running.

[0021] Referring to Figure 2, the voxel definition 31 according to Embodiment 1 will be explained. Voxel definition 31 is data that defines voxels 32. Voxels 32 are three-dimensional regions obtained by dividing the target region 33. The size of voxels 32 can be arbitrarily set according to their position in the target region 33. In Embodiment 1, larger voxels 32 are defined as the distance from the light sensor 40 increases.

[0022] Referring to Figure 3, the background generation process according to Embodiment 1 will be explained. (Step S11: Point cloud acquisition process) The point cloud acquisition unit 21 acquires point cloud data 41 collected by the optical sensor 40. Here, the point cloud acquisition unit 21 repeatedly acquires point cloud data 41 for a certain period of time at a reference timing that does not contain any unwanted reflections. The time at which the point cloud acquisition unit 21 acquires point cloud data 41 does not need to be consecutive; it may be acquired at any interval. The point cloud acquisition unit 21 records the acquired point cloud data 41 in the memory 12 along with the identification information of the point cloud data 41. Here, the point cloud acquisition unit 21 uses the time the point cloud data 41 was acquired as the identification information of the point cloud data 41. Here, the reference timing is the timing when there is no object to be detected, and is the timing determined by the person managing the background data management device 10.

[0023] (Step S12: Base background generation process) The reference background generation unit 22 generates a single point cloud data set containing the data of each point in the multiple point cloud data sets 41 acquired at different times in step S11, and uses this data as the reference background 42, which is the background point cloud data. In other words, as shown in Figure 4, the reference background 42 is composed of multiple point cloud data sets 41 with set identification information (time), and is background point cloud data containing the data of each point in the multiple point cloud data sets 41. The reference background generation unit 22 writes the reference background 42 to the storage 13.

[0024] (Step S13: Voxelization process) The voxelization unit 23 voxels the reference background 42 generated in step S12 to generate background voxel data 44. In other words, the voxelization unit 23 voxels a single point cloud data containing the data of each point included in multiple point cloud data 41 to generate background voxel data 44. The voxelization unit 23 writes the background voxel data 44 to the storage 13, associating it with the reference background 42.

[0025] The method for creating voxels will be explained in detail with reference to Figure 5. The voxelization unit 23 sets each point in the point cloud data to be voxelized as a target point. In the process of step S13, the point cloud data to be voxelized is the background point cloud data generated in step S12. The voxelization unit 23 associates the target point with the voxel 32 that contains the location of the target point among the multiple voxels 32 obtained by dividing the target region 33. This generates voxel data corresponding to the point cloud data to be voxelized.

[0026] Referring to Figure 6, the background update process according to Embodiment 1 will be explained. (Step S21: Background acquisition process) The reference background generation unit 22 reads the reference background 42 and background voxel data 44 from the storage 13 and writes them to the memory 12.

[0027] (Step S22: Point cloud acquisition process) The point cloud acquisition unit 21 acquires point cloud data 41 collected by the optical sensor 40. Here, the point cloud acquisition unit 21 acquires point cloud data 41 for one time period. For example, the point cloud acquisition unit 21 acquires the most recently acquired point cloud data 41 for one time period. Here, the point cloud data 41 for one time period refers to the point cloud data acquired at intervals corresponding to the acquisition frame rate of the optical sensor 40. For example, if the frame rate is 1 fps, it will be the point cloud data acquired from the optical sensor 40 at 1-second intervals. fps is an abbreviation for frames per second.

[0028] (Step S23: Voxelization process) The voxelization unit 23 voxels the new point cloud data, which is the point cloud data 41 acquired in step S22, to generate new voxel data 45. The voxelization unit 23 writes the new voxel data 45 to the memory 12. Furthermore, if the background point cloud data is updated in the most recently executed background update process (step S27 described later), the voxelization unit 23 also voxels the latest background 43 to generate new background voxel data 44. The voxelization unit 23 updates the background voxel data 44 recorded in the memory 12 with the new background voxel data 44.

[0029] (Step S24: Difference Identification Process) The difference identification unit 24 identifies difference cells 46, which are voxels 32 that have a difference between the new voxel data 45 generated in step S23 and the background voxel data 44 of the latest background point cloud data, from among the multiple voxels 32 set by dividing the target region 33. Specifically, the difference identification unit 24 sets each of the multiple voxels 32 as the target voxel 32. The difference identification unit 24 determines whether there is a difference between the new voxel data 45 and the background voxel data 44 based on whether or not there is a corresponding point for the target voxel 32, that is, based on the presence or absence of a point in the voxel 32. If there is a difference, the difference identification unit 24 identifies that voxel 32 as the difference cell 46. Here, as shown in Figure 7, the difference identification unit 24 identifies a target voxel 32 as a difference cell 46 if no points are associated with it in the background voxel data 44, but points are associated with it in the new voxel data 45. In other words, the difference identification unit 24 does not identify a voxel 32 that has points associated with it in the background voxel data 44 as a difference cell 46, even if the number of associated points is different.

[0030] (Step S25: Filtering process) As shown in Figure 8, the difference identification unit 24 excludes voxels 32 from the difference cells 46 identified in step S24 that have a number of associated points equal to or greater than the standard number. As a result, only voxels with a number of associated points less than the standard number are identified as difference cells. It is assumed that the position of vegetation or structures due to wind or vibration does not change significantly in a short period of time. Therefore, it is assumed that the number of points associated with the difference cell 46 will be small. On the other hand, if the number of points associated with the difference cell 46 is large, it is possible that a person or other object has passed in front of the light sensor 40. In other words, it is possible that an object in the foreground was reflected rather than the background changing. Therefore, the difference identification unit 24 excludes voxels 32 with a number of associated points equal to or greater than the standard number from the difference cell 46. This is expected to improve the robustness of the background.

[0031] (Step S26: Background candidate extraction process) The background candidate extraction unit 25 identifies neighboring cells 47 from the difference cells 46 from which voxels 32 with a number of associated points equal to or greater than the reference number have been excluded in step S25. Specifically, as shown in Figure 9, the background candidate extraction unit 25 identifies neighboring cells 47 from among the difference cells 46, specifically voxels 32 whose distance from the voxel 32 containing the point cloud in the background voxel data 44 is less than the reference distance. The background candidate extraction unit 25 then identifies the data of the points associated with the neighboring cells 47 as additional point data 48. In Figure 9, the background candidate extraction unit 25 identifies neighboring cells 47 from among the difference cells 46, specifically voxels 32 adjacent to voxels 32 containing the point cloud in the background voxel data 44. This prevents voxels 32 containing the point cloud from being included in the background voxel data 44 with gaps between them. Alternatively, among the difference cells 46, the neighboring cells 47 may be defined as the range of voxels 32 in the background voxel data 44 that extends from the voxel 32 containing the point cloud by a distance less than a reference distance. In this case as well, it is possible to ensure that the voxels 32 containing the point cloud are not included in the background voxel data 44 with gaps in between.

[0032] (Step S27: Background update process) The background update unit 26 updates the background point cloud data by adding the additional point data 48 identified in step S26 to the background point cloud data. The updated background point cloud data is the latest background 43. The background update unit 26 writes the latest background 43 to memory 12. If the latest background 43 is already recorded in memory 12, the background update unit 26 updates the latest background 43 recorded in memory 12 with the updated latest background 43.

[0033] Subsequently, the process returns to step S22, and new point cloud data 41 is acquired.

[0034] ***Effects of Embodiment 1*** As described above, the background data management device 10 according to Embodiment 1 adds data of voxel points that are different from the newly acquired point cloud data 41 and are close in distance from points included in the background point cloud data to the background point cloud data. This makes it possible to obtain the latest background 43, which is background point cloud data that takes into account changes in the position of vegetation or structures. As a result, when detecting objects by comparing the latest background 43 with the current point cloud data, it becomes possible to obtain appropriate detection results.

[0035] The background data management device 10 according to Embodiment 1 is capable of generating and updating the latest background 43 in parallel with the execution of the detection process. Therefore, it is possible to update the latest background 43 without stopping the detection process.

[0036] The background data management device 10 according to Embodiment 1 adds only minute differences in the vicinity of points in the latest background point cloud data to the background point cloud data. Therefore, it does not add the target object that is originally to be detected, which appears at a certain distance away from structures, etc., to the background. In other words, only point clouds originating from structures, etc., can be added to the background data, thereby suppressing detection omissions.

[0037] Here, the process of adding point data to the reference background 42 to generate the latest background 43, and adding point data to the latest background 43 to update the latest background 43, is only possible because we are using 3D point cloud data 41. In other words, there is a technique that uses 2D image data to detect objects by comparing background image data with current image data. However, this technique does not allow for updating the background image data by adding point data. This is because, when using 2D image data, it is not possible to add pixels from later-obtained image data to the background image data. For example, it is conceivable to set the average value of the pixel values ​​of the later-obtained image data and the pixel values ​​of the background image data as the pixel values ​​in the background point cloud data. However, it is not possible to add pixels from later-obtained image data while maintaining the information of the background image data. Therefore, the idea of ​​generating the latest background 43 and updating the latest background 43 was not present in technologies using 2D background image data.

[0038] Furthermore, the background data management device 10 according to Embodiment 1 downsamples the point cloud data 41 by voxelizing it. This reduces the processing load on subsequent processing. In addition, quantizing to the level of voxels 32 has the effect of making the point cloud more resistant to minute fluctuations.

[0039] Furthermore, when generating a reference background 42, the background data management device 10 according to Embodiment 1 generates a single point cloud data containing the data of each point included in multiple point cloud data 41 as the reference background 42, which is the background point cloud data, and then voxels it. In other words, it does not voxelize each of the multiple point cloud data 41 and then integrate them. If each of the multiple point cloud data 41 were voxelized and then integrated, it would require the extra step of discarding overlapping parts during integration, but this extra step is avoided by generating the reference background 42 and then voxelizing it.

[0040] ***Other configurations*** <Example 1> In Embodiment 1, a reference background 42 is set in advance, then the latest background 43 is generated, and the latest background 43 is updated. If the system is restarted and the process in Figure 6 is executed again, the reference background 42 that was set in advance in step S21 will be read again. When the background update unit 26 writes the latest background 43 to the memory 12, it may update the reference background 42 recorded in the storage 13 with the latest background 43. As a result, when the system is restarted and the process in Figure 6 is executed again, the latest background 43 will be read out as the reference background 42. Alternatively, the latest background 43 may be stored in storage 13 as a separate file from the base background 43.

[0041] <Modification 2> The background data management device 10 may identify voxels that differ between the voxel data for the latest background 43 obtained in step S27 and the voxel data for the reference background 42 read in step S21, and may notify the user if the number of differing voxels is greater than or equal to a threshold number. This allows the user to be informed that the point cloud that should serve as the background may have changed significantly from the reference background 42. Upon receiving the notification, the user can decide whether it is necessary to regenerate the reference background 42. For example, if the user decides that it is necessary to regenerate the reference background 42, they can either re-execute the background generation process shown in Figure 3, or update the reference background 42 with the latest background 43.

[0042] In this case, as shown in Figure 10, the background data management device 10 includes a notification unit 27 as a functional component. As shown in Figure 11, when the latest background 43 is obtained in step S28, the latest background 43 is voxelized in step S28. The method of voxelization is the same as the process in step S23. In step S29, voxels that have differences between the new voxel data 45 of the latest background 43 and the background voxel data 44 are identified. The method of identifying difference cells is the same as the process in step S24. Then, in step S30, the notification unit 27 notifies if the number of difference cells between the new voxel data 45 of the latest background 43 and the background voxel data 44 is greater than or equal to a threshold number.

[0043] <Variation 3> The background data management device 10 may display the difference between the latest background 43 obtained in step S27 and the reference background 42 read in step S21. This makes it easier for the user to understand the parts that have been added as background.

[0044] In this case, as shown in Figure 12, the background data management device 10 includes a display unit 28 as a functional component. For example, as shown in Figure 13, the display unit 28 displays the data of points included in the reference background 42 as white circles, and the data of the difference, i.e., the data of points that are not in the reference background 42 but are in the latest background 43, as black circles. If an update date and time is selected, the display unit 28 may display data based on the latest background 43 at that time. In other words, the display unit 28 may display the data of points in the reference background 42 at that time as white circles, and the data of points in the latest background 43 at that time as black circles. For example, by switching the display to a time series, it is possible to understand how points have been added to the latest background 43.

[0045] In this case, as shown in Figure 13, the display unit 28 may also display photographic data obtained by converting the point data of the latest background 43 into two-dimensional image data. This is because the situation is difficult to understand from the point data alone. Although not shown in Figure 12, the background data management device 10 is also connected via a communication interface 14 to a camera that acquires images in conjunction with the light sensor 40. By using the camera linked to the light sensor 40, the background data management device 10 acquires photographic data of the area that matches the latest background 43, and the display unit 28 displays the corresponding portion of the photographic data as shown in Figure 13.

[0046] <Modification 4> In the modified example 3, the background data management device 10 may display the differences after clustering them, instead of simply displaying the differences. That is, the display unit 28 clusters the data of points that are not in the reference background 42 but are in the latest background 43. Then, as shown in Figure 14, the display unit 28 displays a rectangular parallelepiped for each cluster that contains the data of the points included in that cluster. In this case, the coordinates of the cluster may also be displayed. The coordinates of the cluster are the (x,y,z) coordinates of the center of the cluster, and the width in the x direction, y direction, and z direction.

[0047] The background data management device 10 may set the rectangular area, which is the cluster area, as a mask area, which is an area where objects are not detected. This is because the cluster area is an area where point data is likely to be added to the background, and if it is included as a detection target, it is an area where false detections are likely to occur in the future.

[0048] <Modification 5> In Embodiment 1, the data of points in neighboring cells 47 was added to the background point cloud data to generate or update the latest background 43. The data of points in voxels 32 surrounding neighboring cells 47 may be newly added to the background point cloud data as time progresses. For example, if vegetation has grown and the data of points in neighboring cells 47 has been added to the background, the vegetation may grow further, and it may become necessary to add the data of points in voxels 32 surrounding neighboring cells 47 to the background as well. Anticipating this possibility, the background candidate extraction unit 25 may identify the voxels 32 surrounding the neighboring cell 47 as extended cells, and identify not only the data of points associated with the neighboring cell 47 but also the data of points associated with the extended cells as additional point data 48. For example, as shown in Figure 15, when V=1, the background candidate extraction unit 25 identifies the voxels 32 surrounding the neighboring cell 47 as extended cells, and identifies the data of points associated with the neighboring cell 47 and the extended cells as additional point data 48. When V=2, as shown in Figure 16, the voxels 32 surrounding the neighboring cell 47 are treated as extended cells up to two voxels 32. If V is the base number, the voxels surrounding the neighboring cell 47 by the base number of voxels become extended cells.

[0049] Embodiment 2. Embodiment 2 differs from Embodiment 1 in that it removes point data and updates the background point cloud data. Embodiment 2 explains these differences, while omitting explanations for identical points.

[0050] ***Explanation of the structure*** Referring to Figure 17, the configuration of the background data management device 10 according to Embodiment 2 will be described. The background data management device 10 differs from the background data management device 10 shown in Figure 1 in that the background candidate extraction unit 25 includes an additional point extraction unit 251 and a deletion point extraction unit 252.

[0051] ***Explanation of operation*** Referring to Figure 18, the background update process according to Embodiment 2 will be described. The process from step S31 to step S34 is the same as the process from step S21 to step S24 in Figure 6.

[0052] (Step S35: Additional point extraction process) The additional point extraction unit 251 extracts additional point data 48, which is data for points to be added to the background point cloud data, based on the difference cell 46 identified in step S34. Specifically, the additional point extraction unit 251 extracts the additional point data 48 by executing steps S25 and S26 in Figure 6.

[0053] (Step S36: Extraction of deletion points) The deletion point extraction unit 252 extracts deletion point data 49, which are the data of points to be deleted from the data of points included in the background point cloud data. Specifically, the deletion point extraction unit 252 extracts the data of points included in the point cloud data 41 that satisfies the conditions from the point cloud data 41 that formed the basis of the background point cloud data, as deletion point data.

[0054] The conditions are determined by factors such as the time the point cloud data 41 was acquired and the degree of deviation between it and the most recently acquired point cloud data 41. If the condition is the time the point cloud data 41 was acquired, for example, the deletion point extraction unit 252 extracts the data of points included in the point cloud data 41 whose acquisition time is earlier than or equal to the reference time as deletion point data 49. Alternatively, the deletion point extraction unit 252 extracts the data of points included in the point cloud data 41 whose acquisition time is within the target period as deletion point data 49. If the condition is based on the degree of deviation from the most recently acquired point cloud data 41, the deletion point extraction unit 252 sets each of the point cloud data 41 that formed the basis of the background point cloud data as the target point cloud data 41. The deletion point extraction unit 252 calculates the degree of deviation as the number of voxels 32 that differ between the voxel data generated for the target point cloud data 41 and the voxel data generated for the most recently acquired point cloud data 41. The deletion point extraction unit 252 extracts the data of points included in the point cloud data 41 whose degree of deviation is equal to or greater than a certain number as deletion point data 49. The condition can be set as a single condition, to satisfy any of multiple conditions, or to satisfy all of multiple conditions.

[0055] (Step S37: Background update process) The background update unit 26 adds the additional point data 48 extracted in step S35 to the background point cloud data and deletes the deletion point data 49 extracted in step S36 from the background point cloud data. As a result, the background update unit 26 updates the background point cloud data. The updated background point cloud data is the latest background 43. The background update unit 26 writes the latest background 43 to the memory 12. If the latest background 43 is already recorded in the memory 12, the background update unit 26 updates the latest background 43 recorded in the memory 12 with the updated latest background 43.

[0056] Subsequently, the process returns to step S32, and new point cloud data 41 is acquired.

[0057] In this case, the processes in steps S35 and S36 were executed as a series of processes. Therefore, the processes in steps S35 and S36 were executed at the same time interval. However, the processes in steps S35 and S36 may be executed at different time intervals. For example, the process in step S36 may be executed at a longer time interval than the process in step S35.

[0058] ***Effects of Embodiment 2*** As described above, the background data management device 10 according to Embodiment 2 deletes point data from the background point cloud data based on the acquisition time of the point cloud data 41 and the degree of deviation from the most recently acquired point cloud data 41. This makes it possible to obtain the latest background 43, which is background point cloud data from which point data that does not match the current situation has been removed. As a result, when detecting an object by comparing the latest background 43 with the current point cloud data, it becomes possible to obtain an appropriate detection result.

[0059] Here, as shown in Figure 4, the reference background 42 consists of multiple pairs of identification information for the point cloud data 41 (the time of acquisition in Figure 4) and the point cloud data 41. The latest background 43 also has the same structure as the reference background 42, although at least one of the additions or deletions of point cloud data 41 has been made. Because the reference background 42 and the latest background 43 have this configuration, it is possible to delete point cloud data 41 that meet the conditions from the background point cloud data. In particular, since the time the point cloud data 41 was acquired is used as the identification information for the point cloud data 41, it is possible to extract the point data 49 to be deleted based on the time the point cloud data 41 was acquired.

[0060] Referring to Figures 19 and 20, the effects obtained by the background data management device 10 according to Embodiment 2 will be specifically explained. Here, the deletion point extraction unit 252 extracts data of points that satisfy both of the following two conditions as deletion point data 49. Condition 1: The data must be from a point included in the point cloud data 41, acquired at a time earlier than the reference time. Condition 2: The data must be from a point cloud data 41 that is the basis for the background point cloud data, and the number of voxels 32 that differ when compared as voxel data with the most recently acquired point cloud data 41 must be equal to or greater than a certain threshold.

[0061] Figure 19 shows the state of background point cloud data when construction is carried out first, followed by snowfall. Here, construction causes a permanent background change; that is, after construction is completed, the background change remains. Snowfall causes a temporary background change; that is, snowfall causes a background change, but the background returns to its original state when the snow melts. As construction progresses, the background gradually changes. Therefore, point cloud data 41 is added to the background point cloud data (point cloud data 41(1)(2)(3)). When snowfall begins, the background gradually changes as the snow accumulates. Therefore, point cloud data 41 is added to the background point cloud data (point cloud data 41(4)(5)(6)). Subsequently, as the snow begins to melt, the background gradually changes as the accumulated snow decreases. Therefore, point cloud data 41 is added to the background point cloud data (point cloud data 41(7)(8)). On the other hand, point cloud data 41 that satisfies both condition 1 and condition 2 is removed from the background point cloud data. Here, the point cloud data 41 that satisfies condition 1 are point cloud data 41(1),(2), and(3). Also, the point cloud data 41 that satisfies condition 2 are point cloud data 41(1) and(2). Of the point cloud data 41(1),(2), and(3) that satisfies condition 1, the point cloud data 41 that satisfies condition 2 are point cloud data 41(1) and(2). Therefore, point cloud data 41(1) and(2) that satisfy both condition 1 and condition 2 are removed from the background point cloud data. Note that point cloud data 41(3) does not satisfy condition 2 because the background shown on the vertical axis of Figure 19 is close to the current state and there is little difference.

[0062] Figure 20 shows the state after time has passed since Figure 19. In Figure 20, due to the passage of time, point cloud data 41(4)(5)(6)(7)(8) also satisfies condition 1. Assume that the point cloud data 41 that satisfies condition 2 is (4)(5)(6)(7)(8). At this time, point cloud data 41(4)(5)(6)(7)(8) is deleted from the background point cloud data.

[0063] Thus, the background data management device 10 according to Embodiment 2 can handle background changes due to various factors by adding and deleting point cloud data 41 to the background point cloud data.

[0064] ***Other configurations*** <Variation 6> The reference background generation unit 22 may generate multiple patterns of reference backgrounds 42. For example, the reference background generation unit 22 may generate reference backgrounds 42 for each period, such as by season, month, or time of day. Then, at the start of each period, the background update process may be restarted from the beginning to obtain the reference background 42 for that period. This allows the reference background 42 appropriate for that period to be used as the initial background point cloud data. For example, since the condition of plants and trees changes significantly depending on the season, it is effective to use a standard background 42 that is appropriate for the season.

[0065] <Example 7> The background data management device 10 may generate or update the latest background 43 by adding the data of points in the background portion of the point cloud data 41 in the event of a false detection to the background point cloud data. Specifically, suppose a false detection occurs in a detection process that is being executed in parallel. In this case, the additional point extraction unit 251 adds the point cloud data 41 used in the detection process, excluding the portion designated as the foreground, to the background point cloud data when the false detection occurs. This helps to suppress false detections caused by background changes similar to those that occurred when false detections occurred.

[0066] <Differentiation Example 8> Similar to the third modification, the background data management device 10 may display the difference between the latest background 43 obtained in step S27 and the reference background 42 read in step S21. In this case, as shown in Figure 21, the display unit 28 may display a delete button. When the delete button is pressed, the display unit 28 deletes the data of the points in the point cloud data 41 that were added at the selected update date and time from the background point cloud data.

[0067] Furthermore, as shown in Figure 22, the display unit 28 may also display the number of points in the background point cloud data for each time period. This makes it possible to understand the time-series changes in the number of points in the background point cloud data.

[0068] <Modification 9> In the embodiments described above, each functional component was implemented in software. However, in Modification 9, each functional component may be implemented in hardware. The differences between this Modification 9 and the embodiments described above will be explained.

[0069] When each functional component is implemented in hardware, the background data management device 10 includes an electronic circuit 15 instead of the processor 11, memory 12, and storage 13. The electronic circuit 15 is a dedicated circuit that implements the functions of each functional component, as well as the functions of the memory 12 and storage 13.

[0070] Electronic circuits 15 can include single circuits, complex circuits, programmed processors, parallel programmed processors, logic ICs, GAs, ASICs, and FPGAs. GA stands for Gate Array. ASIC stands for Application Specific Integrated Circuit. FPGA stands for Field-Programmable Gate Array. Each functional component may be implemented in a single electronic circuit 15, or each functional component may be implemented by distributing them across multiple electronic circuits 15.

[0071] <Variation 10> As a variation 10, some of the functional components may be implemented in hardware, while other functional components may be implemented in software.

[0072] The processor 11, memory 12, storage 13, and electronic circuit 15 are collectively referred to as the processing circuit. In other words, the function of each functional component is realized by the processing circuit.

[0073] Furthermore, the term "part" in the above explanation may be replaced with "circuit," "process," "procedure," "processing," or "processing circuit."

[0074] The various aspects of this disclosure are summarized below as an appendix. (Note 1) A point cloud acquisition unit acquires point cloud data containing data from multiple points obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection points, and records the acquired point cloud data together with identification information for that point cloud data. A reference background generation unit generates a single point cloud data set as background point cloud data, which includes the data of each point in multiple point cloud data sets acquired at different times by the point cloud acquisition unit. A background data management device equipped with the following features. (Note 2) The background data management device further, A deletion point extraction unit extracts deletion point data, which is data of points included in the point cloud data that satisfies the conditions from the point cloud data that formed the basis of the background point cloud data. A background update unit deletes the deletion point data extracted by the deletion point extraction unit from the background point cloud data. A background data management device as described in Appendix 1, comprising the above. (Note 3) The point cloud acquisition unit records the time the point cloud data was acquired as the identification information. The deletion point extraction unit extracts data of points included in the point cloud data whose acquired time satisfies the conditions as the deletion point data. The background data management device described in Appendix 2. (Note 4) The deletion point extraction unit extracts data of points included in the point cloud data whose acquired time is earlier than or equal to the reference time as the deletion point data. The background data management device described in Appendix 3. (Note 5) The deletion point extraction unit extracts data of points included in the point cloud data whose acquired time falls within the target period as the deletion point data. The background data management device described in Appendix 3. (Note 6) The background data management device further, The system includes a voxelization unit that generates voxel data corresponding to the point cloud data by associating each point in the point cloud data acquired by the point cloud acquisition unit with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, with the target point being the target point. The deletion point extraction unit extracts data of points included in the point cloud data that formed the basis of the background point cloud data, where the number of voxels that differ between the voxel data generated by the voxelization unit for that point cloud data and the voxel data generated by the voxelization unit for the newly acquired point cloud data is equal to or greater than a certain threshold, as the deletion point data. A background data management device as described in any one of the items 2 to 5 of the appendix. (Note 7) The background data management device further, An additional point extraction unit extracts additional point data, which is data of points to be added to the background point cloud data, based on the difference between the point cloud data newly acquired by the point cloud acquisition unit and the background point cloud data generated by the reference background generation unit. A background update unit adds the additional point data extracted by the additional point extraction unit to the background point cloud data. A background data management device as described in any one of the appendices 1 to 6, comprising: (Note 8) The background data management device further, A display unit that displays data for each point included in the background point cloud data, and displays data for points included in the point cloud data corresponding to the specified identification information in a distinguished manner. A background data management device as described in any one of the appendices 1 to 7, comprising: (Note 9) The computer acquires point cloud data containing data from multiple points obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection points, and records the acquired point cloud data along with its identification information. A background data management method in which a computer generates a single point cloud data set containing the data of each point in multiple point cloud data sets acquired at different times, and uses this data as background point cloud data. (Note 10) A point cloud acquisition process that obtains point cloud data containing data from multiple points obtained by irradiating a target area with light and receiving the reflected light reflected at reflection points, and records the acquired point cloud data along with identification information for that point cloud data, A reference background generation process generates a single point cloud data set containing the data of each point included in multiple point cloud data sets acquired at different times by the aforementioned point cloud acquisition process, which is then used as background point cloud data. A background data management program that makes a computer function as a background data update device.

[0075] The embodiments and variations of this disclosure have been described above. Some of these embodiments and variations may be implemented in combination. Alternatively, some or all of them may be implemented in part. However, this disclosure is not limited to the embodiments and variations described above, and various modifications are possible as needed. [Explanation of symbols]

[0076] 10 Background data management device, 11 Processor, 12 Memory, 13 Storage, 14 Communication interface, 21 Point cloud acquisition unit, 22 Reference background generation unit, 23 Voxelization unit, 24 Difference identification unit, 25 Background candidate extraction unit, 251 Additional point extraction unit, 252 Deleted point extraction unit, 26 Background update unit, 27 Notification unit, 28 Display unit, 31 Voxel definition, 32 Voxel, 33 Target area, 40 Light sensor, 41 Point cloud data, 42 Reference background, 43 Latest background, 44 Background voxel data, 45 Voxel data, 46 Difference cell, 47 Neighboring cell, 48 Additional point data, 49 Deleted point data.

Claims

1. A point cloud acquisition unit acquires point cloud data containing data from multiple points obtained by irradiating a target area with light and receiving the reflected light reflected at the reflection points, and records the acquired point cloud data together with identification information for that point cloud data. A reference background generation unit generates a single point cloud data set as background point cloud data, which includes the data of each point included in multiple point cloud data sets acquired at different times by the point cloud acquisition unit. A deletion point extraction unit extracts deletion point data, which is data of points included in the point cloud data that satisfies the conditions from the point cloud data that formed the basis of the background point cloud data. A background update unit that deletes the deleted point data extracted by the deleted point extraction unit from the background point cloud data, A voxelization unit generates voxel data corresponding to the point cloud data by associating each point of the point cloud data acquired by the point cloud acquisition unit with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, and using each point of the point cloud data acquired by the point cloud acquisition unit as the target point, and Equipped with, The deletion point extraction unit is a background data management device that extracts data of points included in point cloud data that is the basis of the background point cloud data, where the number of voxels that differ between the voxel data generated by the voxelization unit for that point cloud data and the voxel data generated by the voxelization unit for newly acquired point cloud data by the point cloud acquisition unit is equal to or greater than a standard number, as the deletion point data.

2. The point cloud acquisition unit records the time the point cloud data was acquired as the identification information. The deletion point extraction unit extracts data of points included in the point cloud data whose acquired time satisfies the conditions as the deletion point data. The background data management device according to claim 1.

3. The deletion point extraction unit extracts data of points included in the point cloud data whose acquired time is earlier than or equal to the reference time as the deletion point data. The background data management device according to claim 2.

4. The deletion point extraction unit extracts data of points included in the point cloud data whose acquired time falls within the target period as the deletion point data. The background data management device according to claim 2.

5. The background data management device further, An additional point extraction unit extracts additional point data, which is data of points to be added to the background point cloud data, based on the difference between the point cloud data newly acquired by the point cloud acquisition unit and the background point cloud data generated by the reference background generation unit. A background update unit adds the additional point data extracted by the additional point extraction unit to the background point cloud data. A background data management device according to claim 1, comprising:

6. The background data management device further, A display unit that displays data for each point included in the background point cloud data, and displays data for points included in the point cloud data corresponding to the specified identification information in a distinguished manner. A background data management device according to claim 1, comprising:

7. The computer acquires point cloud data containing data from multiple points obtained by irradiating a target area with light and receiving the reflected light reflected at reflection points, and records the acquired point cloud data along with its identification information. The computer generates a single point cloud data set containing the data of each point in multiple point cloud data sets acquired at different times, and uses this as background point cloud data. The computer extracts the data of points to be deleted from the point cloud data that formed the basis of the background point cloud data, which are points that satisfy the conditions. The computer deletes the deleted point data from the background point cloud data. The computer generates voxel data corresponding to the point cloud data by associating each point in the point cloud data with a voxel containing the location of the target point among a plurality of voxels obtained by dividing the target region, A background data management method in which a computer extracts data of points included in point cloud data that is the basis for the background point cloud data, where the number of voxels that differ between the voxel data generated for the point cloud data and the voxel data generated for the newly acquired point cloud data is equal to or greater than a certain threshold, as the deleted point data.

8. A point cloud acquisition process that obtains point cloud data containing data from multiple points obtained by irradiating a target area with light and receiving the reflected light reflected at reflection points, and records the acquired point cloud data together with identification information for that point cloud data, A reference background generation process generates a single point cloud data set containing the data of each point included in multiple point cloud data sets acquired at different times by the aforementioned point cloud acquisition process, as background point cloud data. A deletion point extraction process extracts deletion point data, which is the data of points included in the point cloud data that satisfies the conditions from the point cloud data that formed the basis of the aforementioned background point cloud data. A background update process that deletes the deleted point data extracted by the deletion point extraction process from the background point cloud data, A voxelization process generates voxel data corresponding to the point cloud data by associating each point in the point cloud data acquired by the point cloud acquisition process with the voxel containing the position of the target point among a plurality of voxels obtained by dividing the target region, and using each point in the point cloud data acquired by the point cloud acquisition process as the target point, and The computer functions as a background data update device to perform this task. The background data management program extracts, as the deleted point data, the data of points included in the point cloud data that formed the basis of the background point cloud data, where the number of voxels that differ between the voxel data generated by the voxelization process for that point cloud data and the voxel data generated by the voxelization process for the newly acquired point cloud data is equal to or greater than a certain number.

Images