Monitoring device, monitoring method, and program

Abstract

To provide a monitoring device, monitoring method, and program that can recognize objects existing on a background spread across a three-dimensional space. [Solution] The monitoring device according to this disclosure comprises: a division unit that divides a three-dimensional space representing point cloud data into at least one divided space; an observation unit that observes the feature quantities of a first point cloud data included in the divided space during a reference period and observes the feature quantities of a second point cloud data included in the divided space during an observation period; and a determination unit that determines a first divided space to be in an abnormal state if the time during which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the time during which the difference exceeds a first threshold, wherein the first divided space is included in at least one divided space.

Description

Technical Field

[0006] , ,

[0005] , , ,

[0001] The present disclosure relates to a monitoring device, a monitoring method, and a program.

Background Art

[0002] Towards the realization of automatic driving of vehicles, a traffic control system that utilizes information detected by sensors installed on the road side has been studied. As driving support realized by the traffic control system, detection of foreign objects or fallen objects on the road is assumed. Specifically, when the traffic control system detects a foreign object or a fallen object on the road, it provides the detected information to vehicles traveling on the road.

[0003] Patent Document 1 discloses a configuration example of a monitoring device capable of recognizing an object. The monitoring device disclosed in Patent Document 1 distinguishes and recognizes an object and the background of the object from the measurement results of a three-dimensional laser scanner that measures a monitoring area.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In a traffic control system, it is required to detect an object in a three-dimensional space. However, in Patent Document 1, since the monitoring device is characterized by distinguishing between the background, which is two-dimensional data, and the object, there is a problem that an object existing on a background extending in a three-dimensional space cannot be accurately recognized.

[0006] One object of the present disclosure is to provide a monitoring device, a monitoring method, and a program capable of recognizing an object existing on a background extending in a three-dimensional space.

Means for Solving the Problems

[0007] The monitoring device according to this disclosure includes: a division unit that divides a three-dimensional space representing point cloud data into at least one divided space; an observation unit that observes the feature quantities of a first point cloud data included in the divided space during a reference period and observes the feature quantities of a second point cloud data included in the divided space during an observation period; and a determination unit that determines a first divided space as abnormal if the time during which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the time during which the difference exceeds a first threshold, wherein the first divided space is included in the at least one divided space.

[0008] The monitoring method relating to this disclosure divides a three-dimensional space representing point cloud data into at least one divided space, observes the feature quantities of the first point cloud data included in the divided space during a reference period, observes the feature quantities of the second point cloud data included in the divided space during an observation period, and determines that the first divided space included in the at least one divided space is in an abnormal state if the time during which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the second threshold.

[0009] The program relating to this disclosure causes a computer to perform the following actions: divide a three-dimensional space representing point cloud data into at least one divided space; observe the feature quantities of the first point cloud data included in the divided space during a reference period; observe the feature quantities of the second point cloud data included in the divided space during an observation period; and determine that the first divided space included in the at least one divided space is in an abnormal state if the time during which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the second threshold. [Effects of the Invention]

[0010] This disclosure provides a monitoring device, monitoring method, and program that can recognize objects present on a background in a three-dimensional space. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows an example of the configuration of a monitoring device. [Figure 2] Figure 2 shows the flow of the abnormality detection process performed in the monitoring device. [Figure 3] Figure 3 shows the monitoring device and a vehicle traveling on the road. [Figure 4] Figure 4 shows an example of the configuration of the monitoring device. [Figure 5] Figure 5 shows a diagram of voxels set up in three-dimensional space. [Figure 6] Figure 6 shows a diagram of voxels set up in three-dimensional space. [Figure 7] Figure 7 shows an overview of the reference model. [Figure 8] Figure 8 shows the changes that occurred in voxel Vi during the observation period. [Figure 9] Figure 9 shows the flow of the process for generating the reference model. [Figure 10] Figure 10 shows the flow of the judgment process during the observation period. [Figure 11] Figure 11 is a block diagram showing an example of the configuration of a monitoring device. [Modes for carrying out the invention]

[0012] (Embodiment 1) Figure 1 shows an example configuration of the monitoring device 10. The monitoring device 10 may be, for example, a device that detects foreign objects on a road by monitoring the road on which a vehicle is traveling. Foreign objects may include objects that have fallen from a moving vehicle. The monitoring device 10 may also be a computer device that operates by having a processor execute a program stored in memory.

[0013] The monitoring device 10 includes a division unit 11, an observation unit 12, and a determination unit 13. The division unit 11, the observation unit 12, and the determination unit 13 may be software or modules whose processing is performed by a processor executing a program stored in memory. Alternatively, the division unit 11, the observation unit 12, and the determination unit 13 may be hardware such as circuits or chips. Figure 1 shows an example in which the monitoring device 10 has a division unit 11, an observation unit 12, and a determination unit 13, but the division unit 11, the observation unit 12, and the determination unit 13 may be distributed and arranged across multiple devices. In this case, each device may send and receive information via a network. In other words, a monitoring system may be constructed by distributing the division unit 11, the observation unit 12, and the determination unit 13 across multiple devices.

[0014] The division unit 11 divides the three-dimensional space representing the point cloud data into at least one divided space. Point cloud data is a set of points having three-dimensional information. The three-dimensional information may be coordinates on the X, Y, and Z axes representing the three-dimensional space. Point cloud data may be generated using a sensor. For example, point cloud data may be generated using a sensor that uses LiDAR (Light Detection and Ranging). Alternatively, point cloud data may be generated by matching feature points of multiple image data obtained by photographing the same object from multiple locations. The generation of point cloud data using multiple image data may be performed, for example, using SfM (Structure from Motion). Image data may be generated by an imaging device used as a sensor. The three-dimensional space representing the point cloud data may be rephrased as the three-dimensional space containing the point cloud data.

[0015] At least one partitioned space constitutes a three-dimensional space. That is, a three-dimensional space is composed of multiple partitioned spaces. The three-dimensional space may be a region measured by a sensor. That is, the three-dimensional space may be a space containing an object from which point cloud data is generated. The coordinates used in the three-dimensional space may be, for example, world coordinates or sensor-specific coordinates.

[0016] Each divided space that constitutes the three-dimensional space may have the same size or different sizes. Having the same size may mean that the divided spaces have the same shape and the same volume. The shape of the divided space may be, for example, a rectangular parallelepiped or a cube, or other polygons. Each divided space may be arranged so as not to overlap with other divided spaces, or may be arranged so as to overlap with other divided spaces in part of the space.

[0017] The dividing unit 11 may, for example, determine the coordinates of the vertices of each divided space. Alternatively, the dividing unit 11 may determine the coordinates of the center of the divided space and the distance from the center of the divided space to the boundary of the divided space. Further, the dividing unit 11 may save, record, store, etc. the determined information such as the coordinates of the vertices of the divided space in the memory of the monitoring device 10.

[0018] The observing unit 12 observes the feature amount of the first point cloud data included in the divided space during the reference period. Further, the observing unit 12 observes the feature amount of the second point cloud data included in the divided space during the observation period.

[0019] The reference period is, for example, a period for observing the three-dimensional space in a normal state where no abnormal state occurs in the three-dimensional space. The reference period may be, for example, rephrased as a normal period, a reference period, etc. The reference period may be, for example, a period for observing a road where no running vehicle exists. That is, the reference period may be a period in which the change in the shape indicated by the point cloud data is small. That the change in the shape indicated by the point cloud data is small may mean that the variation in the number of point cloud data, the variation in position, etc. fall within a predetermined range. On the other hand, the observation period is a period for observing the three-dimensional space in order to determine whether an abnormal state has occurred in the three-dimensional space. The observation period may be rephrased as a monitoring period, etc.

[0020] The features of the first and second point cloud data may, for example, be information about the number of points included in the partitioned space. Alternatively, the features of the first and second point cloud data may be information about the brightness of each point. The brightness of each point may be rephrased as the brightness associated with each point. Alternatively, the features of the first and second point cloud data may be information about the normal vector of each point.

[0021] Observing the features of the first and second point cloud data can be rephrased as measuring, calculating, or computing the features of the first and second point cloud data.

[0022] The determination unit 13 determines whether the time during which the difference between the features of the second point cloud data and the features of the first point cloud data exceeds the first threshold is longer than the time during which the difference exceeds the first threshold. The first threshold is used to determine whether a change has occurred in the observed segmented space during the observation period. The first threshold is a value that can be compared with the features of the first and second point cloud data. The second threshold is used to determine whether the observed change is continuing. The second threshold indicates the time or period during which the observed change continues.

[0023] Furthermore, the determination unit 13 determines that a first partitioned space is in an abnormal state if the time during which the difference exceeds the first threshold is longer than the time during which the difference exceeds the second threshold. In other words, the determination unit 13 identifies the first partitioned space as being in an abnormal state among the at least one partitioned space that constitutes the three-dimensional space.

[0024] Figure 2 shows the flow of the abnormality determination process performed in the monitoring device 10. First, the division unit 11 divides the three-dimensional space representing the point cloud data into at least one divided space (S11). Next, the observation unit 12 observes the feature quantities of the first point cloud data included in the divided space during the reference period (S12). Next, the observation unit 12 observes the feature quantities of the second point cloud data included in the divided space during the observation period (S13). Next, the determination unit 13 determines that the first divided space is in an abnormal state if the time during which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds the first threshold is longer than the second threshold (S14).

[0025] As explained above, the monitoring device 10 determines whether each divided space is in an abnormal state by comparing the feature quantities of the point cloud data during the reference period and the observation period in each divided space that constitutes the three-dimensional space. In this way, the monitoring device 10 can determine which space in the three-dimensional space is in an abnormal state.

[0026] (Embodiment 2) Next, the process or operation according to Embodiment 2 of this disclosure will be described. In Embodiment 2, the process by which the monitoring device 40 monitors the road and detects foreign objects on the road will be described.

[0027] Figure 3 shows a monitoring device 40 and vehicles 21-23 traveling on a road. The monitoring device 40 is assumed to be monitoring a predetermined section of the road. Figure 3 shows a scenario in which luggage 30 has fallen from vehicle 22. The monitoring device 40 may detect that luggage 30 has fallen from vehicle 22 or that 30 is present on the road, and may send a message to vehicles traveling within the section monitored by the monitoring device 40 notifying them that luggage 30 is present on the road. For example, the monitoring device 40 may send the message via a mobile network. Alternatively, the monitoring device 40 may send the message using wireless LAN (Local Area Network), Bluetooth®, or other communication standards.

[0028] Figure 4 shows an example of the configuration of the monitoring device 40. The monitoring device 40 is configured with a communication unit 41 added to the monitoring device 10 shown in Figure 1. In the following, detailed explanations of the components of the monitoring device 40 that have the same functions or processes as those of the monitoring device 10 will be omitted.

[0029] The division unit 11 of the monitoring device 40 sets voxels as divided spaces in the space being monitored. For example, the division unit 11 may set multiple voxels in the space being monitored, as shown in Figure 5. Alternatively, the division unit 11 may set voxels in the space where points that identify the surface of an object, generated using LiDAR or the like, exist, as shown in Figure 6. In other words, the division unit 11 may set multiple voxels in the space being monitored before generating point cloud data using LiDAR or the like, as shown in Figure 5. Alternatively, the division unit 11 may set voxels in the space where points exist after generating point cloud data using LiDAR or the like, as shown in Figure 6. Each voxel may be set so as not to overlap with other voxels, as shown in Figures 5 and 6, or it may be set to overlap with other voxels in some space.

[0030] The division unit 11 may set cubes of the same size as voxels, as shown in Figures 5 and 6. Alternatively, the division unit 11 may set cubes or rectangular prisms of different sizes as voxels. For example, the division unit 11 may make voxels closer to the monitoring device 10 larger than voxels further away from the monitoring device 10. In other words, the division unit 11 may make voxels smaller as it moves away from the monitoring device 10. Alternatively, the division unit 11 may make voxels smaller as it moves away from the sensor measuring the point cloud data.

[0031] When generating point cloud data using LiDAR, the spacing between points in the point cloud data widens as you move away from the LiDAR because the beam width between the emitted beams increases. In other words, the point cloud data contained within a single voxel becomes sparser as you move away from the LiDAR. Therefore, to increase the density of point cloud data contained within a voxel, the voxel size may be reduced as you move further away from the LiDAR. By increasing the density of point cloud data contained within a voxel, the accuracy of object estimation can be improved.

[0032] The observation unit 12 may include a sensor that generates point cloud data. For example, the observation unit 12 may include a LiDAR device as a sensor. That is, the LiDAR device may be mounted on the monitoring device 10. In this case, the observation unit 12 generates point cloud data. Alternatively, the observation unit 12 may be connected via a network to a LiDAR device installed at a different location from the monitoring device 10. In this case, the observation unit 12 acquires the point cloud data generated by the LiDAR device via the network. The observation unit 12 generates or acquires point cloud data indicating objects present in the space monitored by the monitoring device 10. In the following explanation, the observation unit 12 will be described as acquiring point cloud data generated by the LiDAR device via a network or the like.

[0033] A LiDAR device may also be called a laser scanner. A LiDAR device measures the distance to an object, for example, using the Time of Flight (ToF) method, and generates point cloud data that represents the object's shape. The point cloud data is a collection of points representing the object's surface. Points may be identified using three-dimensional coordinates in a given space.

[0034] The observation unit 12 may acquire point cloud data at predetermined intervals, for example. The predetermined interval may be, for example, an interval of 0.1 seconds, an interval of 0.5 seconds, etc., and the numerical value indicating the interval is not limited to 0.1 seconds or 0.5 seconds.

[0035] The observation unit 12 may continue to acquire point cloud data during the reference period. Based on the point cloud data acquired during the reference period, the observation unit 12 generates a reference model, which is a reference model for detecting foreign objects. The reference model may also be called a background model. The reference period is assumed to be sufficiently longer than the interval at which point cloud data is acquired.

[0036] Figure 7 shows an overview of the reference model. Figure 7 shows the change in the number of points in voxel Vi, which is an arbitrary voxel included in the space monitored by the monitoring device 10. For example, suppose the space monitored by the monitoring device 10 includes voxels V1 to Vn (1 ≤ i ≤ n, where i and n are positive integers). The number of points shown in Figure 7 may represent the number of points at the time the observation unit 12 acquired the point cloud data, that is, the number of points for each predetermined period. Alternatively, the number of points shown in Figure 6 may be the moving average value at a certain time. Specifically, the number of points shown in Figure 6 may be the average value of the number of points within a predetermined period, such as within the past 1 second at a certain time. The observation unit 12 generates a reference model for each of voxels V1 to Vn.

[0037] The voxel Vi used to generate the reference model is basically a region containing stationary objects. Alternatively, the voxel Vi used to generate the reference model may be a region containing objects whose shape changes little over time. Here, the LiDAR device generates point cloud data by receiving the reflected beam of the emitted beam. However, the LiDAR device does not always receive the reflected beam due to the influence of atmospheric conditions, climate, etc. In other words, the LiDAR device receives the reflected beam of the emitted beam at certain times and does not receive it at other times. Therefore, as shown in Figure 7, the number of points contained in voxel Vi varies depending on the timing of point cloud data acquisition.

[0038] The observation unit 12 may calculate a value indicating a predetermined statistical spread as statistical data regarding the number of points contained in voxel Vi, which varies over time. The mean and standard deviation of the number of points may be calculated as values ​​indicating a predetermined statistical spread. Generating a reference model for voxel Vi may mean determining the mean and standard deviation of the number of points in voxel Vi.

[0039] Furthermore, the observation unit 12 may continue to acquire point cloud data during the observation period. The determination unit 13 determines whether or not an abnormality has occurred on the road based on the comparison result between the point cloud data acquired during the observation period and the reference model.

[0040] For example, the determination unit 13 determines that a change has occurred in voxel Vi if the number of points included in the acquired point cloud data deviates from the average number of points in the reference model by a standard deviation × m (where m is a positive value). For example, if voxel Vi represents a part of a road, determining that a change has occurred in voxel Vi may be equivalent to determining that a change has occurred on the road. Here, the observation unit 12 may use a predetermined fixed value instead of the standard deviation. Furthermore, the determination unit 13 determines whether the occurred change exceeds a predetermined duration ct1. If the occurred change exceeds a predetermined duration ct1, the determination unit 13 determines that the corresponding voxel is in an abnormal state.

[0041] Figure 8 shows that the changes occurring in voxel Vi continued beyond duration ct1 during the observation period. The observation period is assumed to be the period from tn onward.

[0042] A state in which a change has occurred in voxel Vi is a state in which a new object is present in voxel Vi or it is presumed that a new object has appeared in voxel Vi. Specifically, if a change occurs in voxel Vi that indicates a certain point on the road, it is assumed that a vehicle has passed over voxel Vi or that cargo has fallen from a vehicle. In this case, if a vehicle has passed over voxel Vi, the change that occurred in voxel Vi will return to its original state after a predetermined period of time. On the other hand, if a fallen object is present in voxel Vi, the change that occurred in voxel Vi will continue even after the predetermined period of time has elapsed. Therefore, if the change that occurred in voxel Vi continues beyond the duration ct1, the determination unit 13 may determine that a fallen object is present in voxel Vi.

[0043] The communication unit 41 transmits the determination result from the determination unit 13 to a communication device located in a predetermined area. The determination result may include information such as the location and size of the fallen object. The predetermined area may be, for example, the monitoring area of ​​the monitoring device 40. The communication device may be, for example, a wireless communication device that enables communication via a mobile network or other communication standards. The communication device may be, for example, a portable terminal held by a person in the vehicle, or a device mounted in the vehicle. The device mounted in the vehicle may work in conjunction with the car navigation system to display the received determination result on a screen that displays instructions for the car navigation system.

[0044] Figure 9 shows the flow of the reference model generation process in the observation unit 12. First, the division unit 11 sets voxels as division spaces in the monitored space (S21). Here, it is assumed that the monitoring device 40 acquires point cloud data at predetermined intervals. In this case, the division unit 11 may set voxels in the space where the point cloud data exists. For example, the division unit 11 may have information about all voxels that can be set in the monitored space in advance. The information about voxels may include the position of the voxel, the size of the voxel, etc. Setting a voxel can be rephrased as activating the voxel, determining the voxel for creating the reference model, etc. The division unit 11 may set voxels to include the position where the point cloud data exists. For example, the division unit 11 may set voxels to include a space of a predetermined size from the center of the point cloud data.

[0045] Next, the observation unit 12 identifies the number of points in each voxel (S22). Identifying the number of points can also be rephrased as counting the number of points or calculating the number of points. The observation unit 12 identifies the number of points in each voxel set in step S21.

[0046] Next, the observation unit 12 determines whether the reference period has ended (S23). The reference period may be predetermined. Alternatively, the reference period may be changed by a user operating the monitoring device 10 by inputting information.

[0047] If the observation unit 12 determines that the reference period has not ended, the process from step S21 onwards is repeated. In step S21, the division unit 11 may add voxels in the space where new point cloud data exists to the voxels that have been set up so far.

[0048] In step S23, if the observation unit 12 determines that the reference period has ended, it generates a baseline model based on the number of points identified in step S22 (S24). Generating a baseline model may also involve generating statistical data regarding the number of points. A baseline model is generated for each voxel. The statistical data may, for example, be the mean and standard deviation of the number of points contained in voxel Vi, which varies over time.

[0049] Step S23 explained the calculation of the average number of points in the point cloud data acquired at predetermined intervals, but the statistical data is not limited to such values. For example, the number of points may be the average number of points within a predetermined period, such as within the past n (where n is a positive integer) seconds (moving average).

[0050] Figure 10 shows the flow of the determination process during the observation period. First, the division unit 11 sets up voxels as division spaces in the space to be observed or measured, similar to step S21 (S31). Next, the observation unit 12 identifies the number of points in the voxel (S32). The number of points may be the number of points included in the point cloud data acquired at predetermined intervals, or it may be the average value of the number of points within a predetermined period, such as within the past n seconds.

[0051] Next, the determination unit 13 determines whether or not there are voxels in which a change has occurred (S33). For example, the determination unit 13 may determine that a change has occurred from the reference model if the number of points in the voxel identified in step S32 deviates from the average number of points in the reference model by a standard deviation × m (where m is a positive value).

[0052] If the determination unit 13 determines that a voxel has changed, it determines whether or not there is a voxel whose duration of change exceeds a threshold (S34). The duration of the change may be, for example, the duration of the state in which the number of points in the voxel deviates from the average number of points in the reference model by a standard deviation × m (where m is a positive value). The threshold may be a predetermined duration ct1.

[0053] The determination unit 13 determines that if there is a voxel whose duration of change exceeds a threshold, a foreign object is present in that voxel (S35). The foreign object may be, for example, a fallen object from a vehicle traveling on a road.

[0054] If it is determined in step S33 that no voxels have undergone a change, and if it is determined in step S34 that no voxels exist whose duration of change exceeds a threshold, the process from step S31 onwards is repeated. Furthermore, if the number of times the process from step S31 onwards has been repeated reaches a predetermined upper limit, the determination process may be terminated.

[0055] As explained above, the monitoring device 10 detects changes in the number of points representing an object in each voxel, which is a divided space that constitutes the three-dimensional space. This allows the monitoring device 10 to detect newly generated objects on the road that extends in the three-dimensional space. Furthermore, the monitoring device 10 detects whether the duration of the changed state exceeds a threshold. This allows the monitoring device 10 to determine whether the newly generated object corresponds to a foreign object on the road. As a result, the monitoring device 10 can detect objects that have fallen from vehicles traveling on the road that extends in the three-dimensional space.

[0056] (Embodiment 3) Next, we will explain the features of the point cloud data observed by the observation unit 12. In Embodiment 2, we described an example in which the observation unit 12 generates a reference model based on the number of points included in the point cloud data, but in Embodiment 3, we will describe an example in which features other than the number of points are used.

[0057] For example, the brightness associated with each point may be used as a feature of the point cloud data. Point cloud data is generated when a LiDAR receives the reflected beam of a beam emitted towards an object. At this time, the LiDAR measures the brightness of the beam. The brightness of the beam varies depending on, for example, the material of the object to which the beam is reflected, the angle at which the beam is reflected by the object, etc.

[0058] Therefore, the observation unit 12 may generate a reference model using the brightness of each point contained in voxel Vi. For example, the observation unit 12 may change the number of points on the vertical axis shown in Figure 7 to brightness and generate a reference model based on the variation in brightness of each point contained in voxel Vi during the reference period. Specifically, the observation unit 12 may calculate the average value and standard deviation of the brightness of the points accumulated over time. Alternatively, the observation unit 12 may show the average value of the brightness of multiple points contained in voxel Vi on the vertical axis shown in Figure 7. In this case, the observation unit 12 may use the average value of the brightness of multiple points contained in voxel Vi at predetermined intervals to use the average value of brightness during the reference period. Alternatively, the observation unit 12 may show the maximum value, minimum value, or median value, etc., of the brightness of multiple points contained in voxel Vi as the brightness of voxel Vi on the vertical axis of Figure 7.

[0059] For example, since the road surface and foreign objects are made of different materials, the variation in brightness in a voxel Vi without foreign objects will differ from the variation in brightness in a voxel Vi with foreign objects. Therefore, the determination unit 13 may determine that a change has occurred in a voxel Vi if the brightness of the points included in the point cloud data acquired during the observation period deviates from the average brightness in the reference model by a standard deviation × m.

[0060] Furthermore, examples of feature quantities for point cloud data other than the number of points and brightness will be explained. For example, the direction of the normal vector of each point may be used as a feature quantity for point cloud data. For example, the observation unit 12 calculates the normal vector of each point contained in voxel Vi and identifies the direction of the normal vector of each point. The direction of the normal vector may be indicated using the angle between the normal vector and a reference surface, such as the ground surface.

[0061] The observation unit 12 may change the number of points on the vertical axis shown in Figure 7 to angles and generate a reference model based on the variation in the angles of each point contained in voxel Vi during the reference period. Specifically, the observation unit 12 may calculate the average value and standard deviation of the angles of the normal vectors of the points accumulated over time. Alternatively, the observation unit 12 may show the average value of the angles of the normal vectors of multiple points contained in voxel Vi on the vertical axis shown in Figure 7. In this case, the observation unit 12 may use the average value of the angles of multiple points contained in voxel Vi at predetermined intervals to use the average value of the angles during the reference period. Alternatively, the observation unit 12 may show the maximum value, minimum value, median value, etc., of the angles of multiple points contained in voxel Vi as the angle of voxel Vi on the vertical axis of Figure 7.

[0062] For example, the direction of the normal vector differs between a point representing the road surface and a point representing a foreign object. Therefore, the determination unit 13 may determine that a change has occurred in voxel Vi if the angle of the normal vector of a point included in the point cloud data acquired during the observation period deviates by a standard deviation × m from the average angle in the reference model.

[0063] As explained above, the observation unit 12 can use various values ​​as features in addition to the number of points. As a result, the observation unit 12 can appropriately select measurable features to detect abnormal conditions. Alternatively, the observation unit 12 may combine multiple features to detect abnormal conditions. This improves the accuracy of abnormal condition detection compared to the case where only one feature is used to detect abnormal conditions.

[0064] Figure 11 is a block diagram showing an example configuration of monitoring devices 10 and 40 (hereinafter referred to as monitoring device 10, etc.). Referring to Figure 11, monitoring device 10, etc. includes a network interface 1201, a processor 1202, and memory 1203. The network interface 1201 may be used to communicate with network nodes. The network interface 1201 may include, for example, a network interface card (NIC) compliant with the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0065] The processor 1202 reads and executes software (computer programs) from memory 1203 to perform the processing of the monitoring device 10, etc., as described using a flowchart. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0066] Memory 1203 is composed of a combination of volatile and non-volatile memory. Memory 1203 may also include storage located away from the processor 1202. In this case, the processor 1202 may access memory 1203 via an I / O (Input / Output) interface, which is not shown.

[0067] In the example shown in Figure 11, memory 1203 is used to store a group of software modules. The processor 1202 can read these software modules from memory 1203 and execute them, thereby enabling the monitoring device 10 and other systems to perform processing.

[0068] As explained using Figure 11, each processor in the monitoring device 10, etc., executes one or more programs that include a set of instructions for causing the computer to perform the algorithm described in the diagram.

[0069] In the examples described above, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically or otherwise propagating signals.

[0070] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0071] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate.

[0072] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A division unit that divides the 3D space representing point cloud data into at least one divided space, An observation unit that observes the feature quantities of the first point cloud data included in the divided space during the reference period, and observes the feature quantities of the second point cloud data included in the divided space during the observation period. The system includes a determination unit that determines a first divided space to be in an abnormal state if the time for which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the time for which the difference exceeds a first threshold, The first divided space is a monitoring device included in the at least one divided space. (Note 2) The aforementioned feature quantities are The monitoring device according to claim 1, which is a value relating to the number of points included in the divided space. (Note 3) The first threshold is, This value represents a predetermined statistical spread of the number of points included in the divided space with respect to the passage of time during the aforementioned reference period. The determination unit, The monitoring device according to claim 2, wherein, during the observation period, the first divided space is determined to be in an abnormal state if the time for which the number of points exceeds a value indicating a predetermined statistical spread is longer than the second threshold. (Note 4) The aforementioned feature quantities are The monitoring device according to claim 1, wherein the value relates to the brightness indicated by a point included in the divided space. (Note 5) The first threshold is, This value represents a predetermined statistical spread of the brightness of points included in the divided space with respect to the passage of time during the aforementioned reference period. The determination unit, The monitoring device according to claim 4, wherein, during the observation period, the first divided space is determined to be in an abnormal state if the time for which the brightness of a point exceeds a value indicating a predetermined statistical spread is longer than the second threshold. (Note 6) The aforementioned feature quantities are The monitoring device according to claim 1, wherein the direction is the normal vector of a point included in the divided space. (Note 7) The first threshold is, This value represents a predetermined statistical spread of the angle with respect to a predetermined direction of the normal vector of a point included in the divided space with respect to the passage of time during the aforementioned reference period. The determination unit, The monitoring device according to claim 6, wherein the first divided space is determined to be in an abnormal state if the time for which the value indicating the angle of a point with respect to the predetermined direction during the observation period exceeds the value indicating the predetermined statistical spread is longer than the second threshold. (Note 8) The monitoring device according to any one of claims 1 to 6, wherein the size of the divided space changes according to the distance from the observation unit. (Note 9) The monitoring device according to any one of claims 1 to 8, further comprising a communication unit that transmits a determination result indicating an abnormal state to a communication device located in a predetermined area. (Note 10) The 3D space representing the point cloud data is divided into at least one partitioned space, Observe the feature quantities of the first point cloud data included in the partitioned space during the reference period, Observe the feature quantities of the second point cloud data included in the partitioned space during the observation period, A monitoring method in which a first segmented space is determined to be in an abnormal state if the time over which the difference between the features of the second point cloud data and the features of the first point cloud data exceeds a first threshold is longer than the time over which the difference exceeds a first threshold, and the first segmented space is included in at least one of the segmented spaces. (Note 11) The 3D space representing the point cloud data is divided into at least one partitioned space, Observe the feature quantities of the first point cloud data included in the partitioned space during the reference period, Observe the feature quantities of the second point cloud data included in the partitioned space during the observation period, A program that causes a computer to perform the following actions: determine that a first partitioned space, which is included in at least one of the partitioned spaces, is in an abnormal state, where the time over which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the time over which the difference exceeds a first threshold.

[0073] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 9 that are dependent on Appendice 1 may also be dependent on Appendices 10 and 11 in the same way as those described in Appendices 2 to 9. Some or all of the elements described in any appendice may be applicable to various hardware, software, recording means, systems, and methods for recording software. [Explanation of Symbols]

[0074] 10 Monitoring equipment 11 Division 12 Observation Unit 13 Judgment section 21 vehicles 22 vehicles 23 vehicles 30 pieces of luggage 40 Monitoring equipment 41 Communications Department

Claims

1. A division unit that divides the three-dimensional space representing the point cloud data into at least one divided space, An observation unit that observes the feature quantities of the first point cloud data included in the divided space during the reference period, and observes the feature quantities of the second point cloud data included in the divided space during the observation period. The system includes a determination unit that determines a first divided space to be in an abnormal state if the time for which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the time for which the difference exceeds a first threshold, The first divided space is a monitoring device included in the at least one divided space.

2. The aforementioned feature quantities are The monitoring device according to claim 1, which is a value relating to the number of points included in the divided space.

3. The first threshold is, This value represents a predetermined statistical spread of the number of points included in the divided space with respect to the passage of time during the aforementioned reference period. The determination unit, The monitoring device according to claim 2, wherein, during the observation period, the first divided space is determined to be in an abnormal state if the time for which the number of points exceeds a value indicating a predetermined statistical spread is longer than the second threshold.

4. The aforementioned feature quantities are The monitoring device according to claim 1, wherein the value relates to the brightness indicated by a point included in the divided space.

5. The first threshold is, This value represents a predetermined statistical spread of the brightness of points included in the divided space with respect to the passage of time during the aforementioned reference period. The determination unit, The monitoring device according to claim 4, wherein, during the observation period, the first divided space is determined to be in an abnormal state if the time for which the brightness of a point exceeds a value indicating a predetermined statistical spread is longer than the second threshold.

6. The aforementioned feature quantities are The monitoring device according to claim 1, wherein the direction is the normal vector of a point included in the divided space.

7. The first threshold is, This value represents a predetermined statistical spread of the angle with respect to a predetermined direction of the normal vector of a point included in the divided space with respect to the passage of time during the aforementioned reference period. The determination unit, The monitoring device according to claim 6, wherein the first divided space is determined to be in an abnormal state if the time for which the value indicating the angle of a point with respect to the predetermined direction during the observation period exceeds the value indicating the predetermined statistical spread is longer than the second threshold.

8. The monitoring device according to any one of claims 1 to 6, wherein the size of the divided space changes according to the distance from the observation unit.

9. The three-dimensional space representing the point cloud data is divided into at least one partitioned space, Observe the feature quantities of the first point cloud data included in the partitioned space during the reference period, Observe the feature quantities of the second point cloud data included in the partitioned space during the observation period, A monitoring method in which a first segmented space is determined to be in an abnormal state if the time during which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the time during which the difference exceeds a first threshold, and the first segmented space is included in at least one of the segmented spaces.

10. The three-dimensional space representing the point cloud data is divided into at least one partitioned space, Observe the feature quantities of the first point cloud data included in the partitioned space during the reference period, Observe the feature quantities of the second point cloud data included in the partitioned space during the observation period, A program that causes a computer to perform the following actions: determine that a first partitioned space, which is included in at least one of the partitioned spaces, is in an abnormal state, where the time over which the difference between the feature quantities of the second point cloud data and the feature quantities of the first point cloud data exceeds a first threshold is longer than the time over which the difference exceeds a first threshold.

Images