Method for Detecting Installation Abnormality of Sensing Device, and Sensing Device for Implementing the Same

Abstract

Disclosed is a sensing device including a sensor unit that acquires a time-dependent point cloud related to a three-dimensional space using a LiDAR sensor, a memory, and a processor that executes one or more commands stored in the memory to determine a static object area in a three-dimensional space based on the acquired time-dependent point cloud related to the three-dimensional space, and determines an installation abnormality of the sensing device based on a static point cloud having a predetermined time difference among the time-dependent static point clouds corresponding to the determined static object area.

Description

Technical Field

[0001] The present invention relates to a method for detecting an abnormal installation of a sensing device and a sensing device for performing the same.

Background Art

[0002] With the development of sensing technologies such as LiDAR (light detection and ranging) technology, recently, in various industrial technology fields, advanced control functions integrated with sensing technologies have been utilized, and for this purpose, various sensing devices have been provided.

[0003] LiDAR is a technology that emits light to an object in a three-dimensional space and then receives the reflected light to obtain information related to the three-dimensional space from it. LiDAR, like a camera image sensor, cannot recognize color, but can sense objects far away and has excellent spatial resolution.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Relates to a method for detecting an abnormal installation of a sensing device based on a change in a point cloud obtained by using a LiDAR sensor without a separate additional device when a change occurs in the installation state of the sensing device, and a sensing device for performing the same.

Means for Solving the Problems

[0005] A method for detecting an abnormal installation of a sensing device according to a first aspect includes: obtaining a time-series point cloud related to a three-dimensional space by using a LiDAR sensor; determining a static object region in the three-dimensional space based on the obtained time-series point cloud related to the three-dimensional space; and determining whether there is an abnormal installation of the sensing device based on static point clouds having a predetermined time difference among the time-series static point clouds corresponding to the determined static object region.

[0006] The computer-readable recording medium according to the second aspect stores a program for causing a computer to execute commands including a command to acquire time-sequential point clouds related to a three-dimensional space using a LiDAR sensor, a command to determine a static object region of the three-dimensional space based on the acquired time-sequential point clouds related to the three-dimensional space, and a command to determine an installation abnormality of a sensing device based on static point clouds having a predetermined time difference among the time-sequential static point clouds corresponding to the determined static object region.

[0007] The sensing device according to the third aspect includes a sensor unit that acquires time-sequential point clouds related to a three-dimensional space using a LiDAR sensor, a memory that stores one or more commands, and a processor that executes the one or more commands to determine a static object region of the three-dimensional space based on the acquired time-sequential point clouds related to the three-dimensional space, and determines an installation abnormality of the sensing device based on static point clouds having a predetermined time difference among the time-sequential static point clouds corresponding to the determined static object region.

Brief Description of Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0009] Hereinafter, various embodiments will be described in detail with reference to the drawings. In order to more clearly explain the features of the present embodiment, detailed descriptions of matters well-known to those having ordinary knowledge in the technical field to which the following present embodiment pertains will be omitted.

[0010] In addition, in this specification, when a certain configuration is said to be "connected" to another configuration, it includes not only the case where it is "directly connected", but also the case where it is "connected with another configuration interposed therebetween". Further, when a certain configuration is said to "include" another configuration, it means that, unless otherwise stated to the contrary, it does not exclude still other configurations, and may further include the still other configurations.

[0011] Also, terms including ordinal numbers such as "first" or "second" used in this specification can be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

[0012] The present embodiment relates to a method of detecting an installation abnormality of a sensing device and a sensing device for performing the same. For matters well-known to those having ordinary knowledge in the technical field to which the following embodiments pertain, detailed descriptions will be omitted.

[0013] FIG. 1 is a drawing for explaining a state in which the sensing device 100 is provided and a state in which a change has occurred in the installation state of the sensing device 100.

[0014] The sensing device 100 is a device capable of acquiring point cloud data as spatial information related to a three-dimensional space, and may include at least one sensor. The sensing device 100 can emit light into the three-dimensional space and acquire a point cloud related to the three-dimensional space based on the light received as a response thereto. The sensing device 100 can be provided indoors or outdoors in a space corresponding to the region of interest or as wide a space as possible. For example, as shown in FIG. 1, the sensing device 100 can be fixedly provided on a structure.

[0015] The sensing device 100 includes a LiDAR (light detection and ranging) sensor as a 3D sensor for sensing a three-dimensional space and can acquire volumetric point cloud data. The sensing device 100 may further include various types of sensors such as a radar sensor, an infrared video sensor, and a camera if necessary. The sensing device 100 can use a plurality of sensors of the same type or a combination of different types of sensors in consideration of the sensing ranges of the various types of sensors and the types of data that can be acquired.

[0016] The LiDAR sensor cannot sense the color of an object in a three-dimensional space but can sense the shape, size, and position. A multi-channel LiDAR sensor capable of collecting information related to the three-dimensional space is suitable for fields where the general shape, size, and volume of an object can be utilized although the color and texture of the object cannot be sensed.

[0017] As shown in FIG. 1, when a physical impact is applied to the sensing device 100 provided on the structure, or when the support or connection part of the structure where the sensing device 100 is provided is aged and can no longer support the sensing device 100, a change may occur in the installation state of the sensing device 100. If a change occurs in the installation position or angle of the sensing device 100, a change will occur in the point cloud related to the three-dimensional space acquired by the sensing device 100. When using the point cloud related to the three-dimensional space acquired by the sensing device 100 for device control or providing a predetermined service, if the change in the installation state of the sensing device 100 cannot be known, using the point cloud related to the three-dimensional space acquired from the sensing device 100 may result in over-controlling the device or being unable to provide the service.

[0018] In order to detect a change in the installation state of the sensing device 100, an inertial sensor or the like may be further installed. However, if it is impossible or difficult to further install a device such as an inertial sensor in the sensing device 100, or if the installation state changes little by little over a long period of time, it is difficult to detect an abnormal installation of the sensing device 100.

[0019] Hereinafter, when a change occurs in the installation state of the sensing device 100, a method for detecting an abnormal installation of the sensing device 100 based on a change in the point cloud acquired using a LiDAR sensor without a separate additional device, and the sensing device 100 that performs the method will be described.

[0020] FIG. 2 is a drawing for explaining the configuration and operation of the sensing device 100 according to an embodiment.

[0021] Referring to FIG. 2, the sensing device 100 according to one embodiment may also include a memory 110, a processor 120, a sensor unit 130, and a communication interface 140. Those having ordinary knowledge in the technical field related to this embodiment will be able to know that, in addition to the components illustrated in FIG. 2, other general-purpose components may also be further included.

[0022] The memory 110 can store software and / or programs. The memory 110 can store instruction words executable by the processor 120.

[0023] The processor 120 can access the data stored in the memory 110, utilize it, or store new data in the memory 110. The processor 120 can execute the instruction words stored in the memory 110. The processor 120 can execute the computer programs installed in the sensing device 100. Also, the processor 120 can store and execute an externally received computer program or application in the memory 110. The processor 120 can execute at least one processing module and perform predetermined operations. For example, the processor 120 can execute or control a processing module that executes a program for detecting an abnormal installation of the sensing device 100. The processor 120 can control other components included in the sensing device 100 to perform operations corresponding to the execution results such as instruction words or computer programs.

[0024] The sensor unit 130 may also include at least one sensor for sensing a three-dimensional space. The sensor unit 130 may also include a light emitting unit that emits light into the three-dimensional space and a light receiving unit that receives light, and may further include a dedicated processor that acquires a point cloud related to the three-dimensional space based on the intensity of the light received by the light receiving unit. The sensor unit 130 can acquire a time-series point cloud related to the three-dimensional space in order to track an object located in the three-dimensional space within the sensing range.

[0025] The sensor unit 130 can also be a LiDAR sensor, includes at least one three-dimensional LiDAR sensor, and can acquire data related to a space within a predetermined range. Depending on the environment, the sensor unit 130 can further include various types of sensors such as a radar sensor, an infrared imaging sensor, and an ultrasonic sensor. For example, the sensor unit 130 further includes at least one of a radar sensor and an ultrasonic sensor, and can acquire data related to a blind spot area that cannot be sensed by the LiDAR sensor or a proximity space within a predetermined distance from the sensing device 100.

[0026] The communication interface 140 can perform wired or wireless communication with other devices or networks. To this end, the communication interface 140 can include a communication module that supports at least one of various wired or wireless communication methods. For example, it can include a communication module that performs short-range communication such as RFID (radio frequency identification), NFC (near field communication), Bluetooth (registered trademark), various types of wireless communication; or wired communication using a coaxial cable or an optical cable. The communication interface 140 is connected to a device located outside the sensing device 100 and can transmit and receive signals or data. The sensing device 100 can communicate with the administrator's terminal device through the communication interface 140, or can be connected to an external server that utilizes the time-series point cloud related to the three-dimensional space acquired from the sensing device 100 for device control or provides a predetermined service.

[0027] One example of the processor 120 with the foregoing configuration can utilize a LiDAR sensor to obtain a point cloud related to a three-dimensional space by executing one or more instruction words stored in the memory 110, and can identify an object in the three-dimensional space based on the obtained point cloud related to the three-dimensional space. When the sensing device 100 supports an object classification function using an object classification model, the processor 120 can apply the point cloud related to the three-dimensional space to the object classification model or cluster the point cloud related to the three-dimensional space to identify static objects such as the ground or buildings and dynamic objects such as animals.

[0028] When the sensing device 100 continuously senses the three-dimensional space to monitor the three-dimensional space, the processor 120 can utilize the LiDAR sensor to obtain a time-series point cloud related to the three-dimensional space. The processor 120 can sense an installation abnormality of the sensing device 100 in a manner described below based on the obtained time-series point cloud related to the three-dimensional space.

[0029] The processor 120 can determine a static object region of the three-dimensional space based on the obtained time-series point cloud related to the three-dimensional space by executing one or more instruction words stored in the memory 110. The processor 120 can detect an object in the three-dimensional space based on the obtained point cloud related to the three-dimensional space and generate a spatial information map related to the three-dimensional space. At this time, the processor 120 can classify the objects in the three-dimensional space into static objects and dynamic objects. In the spatial information map generated based on the time-series point cloud related to the three-dimensional space, the dynamic object is detected and then disappears or moves its position within the spatial information map, while the static object is continuously detected at the same position in the spatial information map.

[0030] Hereinafter, with reference to FIGS. 3 to 5, a method for distinguishing a dynamic point cloud corresponding to a dynamic object and a static point cloud corresponding to a static object in the spatial information map and a method for determining a static object region in the spatial information map will be described.

[0031] FIG. 3 is a drawing for explaining a dynamic point cloud, a static point cloud, and a static object region in the spatial information map generated by the sensing device 100.

[0032] Referring to FIG. 3, a spatial information map generated based on the time-sequential point cloud related to the three-dimensional space obtained by using the LiDAR sensor is displayed. The spatial information map in FIG. 3 is obtained by providing the sensing device 100 at a position where the crossroads where the vehicle moves can be observed, and the sensing device 100 uses the LiDAR sensor, and can be used to monitor the traffic information of the crossroads, which is the area of interest.

[0033] Looking at the spatial information map in FIG. 3, point clouds corresponding to moving and stopped vehicles and pedestrians can be confirmed by the traffic signals at the crossroads. Since vehicles and pedestrians move, they correspond to dynamic objects, and the positions of the dynamic point clouds corresponding to such dynamic objects are changed in the spatial information map and are not detected continuously in the same area for a certain period or more. Note that traffic infrastructure, roads, buildings, etc. at the crossroads do not move, so they correspond to static objects, and the static point clouds corresponding to such static objects can be continuously detected in the same area of the spatial information map for a certain period or more.

[0034] When point clouds are continuously detected in a specific area corresponding to specific coordinates for a certain period or more, the point clouds can be classified as static point clouds corresponding to static objects. Note that in a specific area, if the point clouds are detected for less than a certain period or if the point clouds are generated and then disappear, the point clouds can be classified as dynamic point clouds corresponding to dynamic objects.

[0035] Therefore, based on the period during which point clouds related to the same area of the spatial information map are continuously detected, the processor 120 can distinguish the point clouds into a dynamic point cloud corresponding to a dynamic object and a static point cloud corresponding to a static object. The processor 120 can determine at least a part or all of them as a static object area by removing the dynamic point cloud corresponding to the dynamic object from the entire point cloud corresponding to the three-dimensional space, or by extracting only the static point cloud corresponding to the static object from the entire point cloud corresponding to the three-dimensional space. That is, the processor 120 can determine the static object area based on the continuity of the point clouds related to the same area in the frame of the spatial information map.

[0036] Based on the period during which point clouds within a unit area of corresponding positions are continuously detected between frames of the spatial information map generated from the time-sequential point clouds related to the three-dimensional space acquired from the sensor unit 110, the processor 120 can determine the static object area. The processor 120 can determine the static object area formed by the static point cloud in the reference frame at a specific time of the spatial information map. For example, the processor 120 can determine the static object area formed by the static point cloud in the frame of the spatial information map corresponding to a certain time after a predetermined period required for the sensing device 100 to be provided and to distinguish between the dynamic point cloud and the static point cloud has passed.

[0037] Based on the period during which point clouds within a unit area of corresponding positions between frames of the spatial information map are continuously detected with the number of points equal to or more than the minimum detection threshold, the processor 120 can determine the static object area. For the same area, even if the point cloud is continuously detected and the point cloud contains noise, only when the number of points in the continuously detected point cloud maintains the number of points equal to or more than the minimum detection threshold, the cases classified as static point clouds due to noise are filtered.

[0038] The minimum detection threshold can be set to an appropriate value according to the environment and weather conditions in which the sensing device 100 is provided. At this time, the minimum detection threshold can be directly input by the user, or the sensing device 100 can receive information related to the external environment and weather conditions and be automatically set to an appropriate value according to the received information.

[0039] Note that the processor 120 can also determine the static object area of the three-dimensional space based on the point cloud of the specified area among the time-series point clouds related to the three-dimensional space acquired from the sensor unit 110. Instead of using the entire time-series point cloud related to the acquired three-dimensional space, if a reference area where almost no dynamic point cloud occurs is specified, the sensing device 100 can determine the static individual area of the three-dimensional space based on the point cloud of the specified area. The reference area can be specified by the user, or the sensing device 100 can automatically specify it to the area determined to correspond to the static point cloud of the static object corresponding to an appropriate height and size using a trained model.

[0040] On the other hand, the processor 120 can generate a normalized map indicating spatial information, such as a voxel map or a depth map, from the time-series point cloud related to the acquired three-dimensional space. For example, when the spatial information map is a voxel map, the processor 120 can distinguish between static voxels and dynamic voxels in each frame of the voxel map and determine the static object area composed of static voxels from the reference frame at a specific time point of the voxel map.

[0041] FIG. 4 is a drawing for explaining a voxel map and a voxel corresponding to a unit area of the voxel map.

[0042] Referring to FIG. 4, a voxel map corresponding to the three-dimensional space monitored by the sensing device 100 is shown. The voxel map is composed of a voxel array, and the voxel array can be composed of voxels. The voxel corresponds to a unit area of the voxel map, and each voxel may also contain a point cloud corresponding to the three-dimensional space. At this time, if the point cloud existing in the voxel is a dynamic point cloud, the voxel is called a dynamic voxel, and if the point cloud existing in the voxel is a static point cloud, it can be called a static voxel.

[0043] FIG. 5 is a drawing for explaining a process of distinguishing between a static voxel and a dynamic voxel.

[0044] In each frame of the voxel map, the processor 120 can distinguish between a static voxel in which the period during which the point cloud in the voxel is continuously detected is equal to or greater than a threshold value, and a dynamic voxel in which the period during which the point cloud in the voxel is continuously detected is less than the threshold value. In a specific voxel of the voxel map, when the point cloud in the voxel is continuously detected for a certain period or longer, since the point cloud corresponds to a static point cloud of a static object, the voxel can be classified as a static voxel. On the contrary, in a specific voxel of the voxel map, when the point cloud in the voxel is detected for less than a certain period or when the point cloud is generated and then disappears, since the point cloud corresponds to a dynamic point cloud of a dynamic object, the voxel can be classified as a dynamic voxel.

[0045] Referring to Fig. 5, for two voxels (the first voxel (Voxel 1) and the second voxel (Voxel 2)) that make up the voxel map, the point cloud existing within the voxel can be checked from time point T - 10 to time point T. It can be known that for the first voxel (Voxel 1), the point cloud is continuously detected within the voxel from time point T - 10 to before time point T - 1. Note that for the second voxel (Voxel 2), it can be confirmed that there is a point cloud at time point T - 9, there is no point cloud from time point T - 8 to time point T - 2, and there is a point cloud again at time point T - 1. Since the point cloud consisting of at least two points is continuously detected in the first voxel (Voxel 1) from time point T - 10 to before time point T - 1, it can be said that it is a static voxel, and since the second voxel (Voxel 2) is only temporarily detected at time points T - 9 and T - 1, it can be said that it is a dynamic voxel.

[0046] Based on the period during which the point cloud related to the same voxel in the voxel map is continuously detected, the processor 120 can classify the voxel where the point cloud exists into a dynamic voxel and a static voxel. The processor 120 can determine at least a part or all of them as a static object region by removing the dynamic voxels from all the voxels of the voxel map or by extracting only the static voxels from all the voxels of the voxel map. That is, the processor 120 can determine the static object region based on the continuity of the point cloud related to the voxels at the same position in the frame of the voxel map.

[0047] Based on the period during which the point cloud within the voxels at positions corresponding to each other between the frames of the voxel map generated from the time - series point cloud related to the three - dimensional space acquired from the sensor unit 110 is continuously detected, the processor 120 can determine the static object region. The processor 120 can determine the static object region composed of static voxels from the reference frame at a specific time point of the voxel map.

[0048] Fig. 6 is a drawing for explaining the process of determining the installation abnormality of the sensing device.

[0049] Based on the static point clouds over time corresponding to the static object region that have a predetermined time difference, the processor 120 can determine whether there is an abnormal installation of the sensing device 100. The static object region can be determined from a reference frame at any point in time when it is confirmed that the sensing device 100 is properly installed. For example, it can be determined from the frame of the spatial information map at a certain past point in time after the sensing device 100 is properly installed on the structure.

[0050] Among the static point clouds over time corresponding to the determined static object region, the processor 120 can extract the first static point cloud at the first point in time and the second static point cloud at the second point in time that has a predetermined time difference from the first point in time. The processor 120 can determine the presence or absence of an abnormal installation of the sensing device 100 based on the difference between the first static point cloud and the second static point cloud.

[0051] Referring to FIG. 6, at the first point in time when the sensing device 100 was properly installed and at the second point in time when a change occurred in the installation state of the sensing device 100, the first static point cloud and the second static point cloud corresponding to the static object region are shown hatched. The processor 120 can compare the first static point cloud corresponding to the static object region at the first point in time with the second static point cloud corresponding to the static object region at the second point in time.

[0052] For example, the processor 120 can compare the ratio of the area of the first static point group related to the static object area with the ratio of the area of the second static point group related to the static object area to determine whether there is an installation abnormality of the sensing device 100. When the time point at which the static object area is determined is set as the first time point, as shown in FIG. 6, all of the first static point groups at the first time point are detected identically within the static object area, while it can be known that the second static point groups at the second time point are detected only in a part of the static object area. In FIG. 6, the time point at which the static object area is determined is defined as an arbitrary past time point when the sensing device 100 is normally installed, but it is not limited thereto. Even if the time point at which the static object area is determined is set as the current time point when a change has occurred in the installation state of the sensing device 100, by comparing the first static point group at the current time point with the second static point group at the past time point, it is possible to determine whether there is an installation abnormality of the sensing device 100.

[0053] As another example, when generating a voxel map from the point groups over time related to the three-dimensional space acquired by the sensing device 100, the processor 120 compares, in the voxel map, the ratio of the number of voxels constituting the first static point group at the first time point to the total number of voxels constituting the determined static object area with the ratio of the number of voxels constituting the second static point group at the second time point to determine whether there is an installation abnormality of the sensing device 100.

[0054] Note that the processor 120 can, by executing one or more instruction words, use the point group related to the three-dimensional space acquired by the sensing device 100 via the communication interface 140 to utilize the installation abnormality determination result of the sensing device 100 for device control or transmit it to a server that provides a predetermined service.

[0055] FIG. 7 is a drawing for explaining the configuration and operation of the server 200 according to an embodiment. The server 200 can be replaced by a computer device, an operator, a console device, or the like.

[0056] As described above, the sensing device 100 may be equipped with a processor that acquires a point cloud over time as spatial information related to a three-dimensional space, detects static and dynamic objects on the three-dimensional space, and tracks the detected objects, but is not limited thereto. A server 200 that utilizes the point cloud over time related to the three-dimensional space acquired by the sensing device 100 for device control or provides a predetermined service receives the point cloud, and the server 200 can process a series of processes for detecting static and dynamic objects on the three-dimensional space and tracking the detected objects. The server 200 may be implemented by a technology such as a cloud computer. The server 200 can perform high-speed data communication with the sensing device 100.

[0057] Referring to FIG. 7, the server 200 may also include a memory 210, a processor 220, and a communication interface 230. A person having ordinary knowledge in the technical field related to the present embodiment will be able to know that other general-purpose components may be further included in addition to the components illustrated in FIG. 7. Even if the content described above for the sensing device 100 is omitted hereinafter, the same-named components of the server 200 can be applied as they are.

[0058] Each component in the block diagram of FIG. 7 may be separated, added, or omitted depending on the implementation method of the server 200. That is, depending on the implementation method, one component may be subdivided into two or more components, or two or more components may be combined into one component, and some components may be further added or removed.

[0059] The memory 210 can store instruction words executable by the processor 220. The memory 210 can store software or programs.

[0060] Processor 220 can execute the instruction words stored in memory 210. Processor 220 can perform overall control of server 200. Processor 220 can obtain the information and requests received via communication interface 230, and can store the received information in a storage (not shown). Also, processor 220 can process the received information. For example, processor 220 can perform a processing action for obtaining information to be utilized for device control, information to be used for providing a predetermined service, or for managing the received information from the information received from sensing device 100, and can store it in a storage (not shown). Also, processor 220 can use the data or information stored in a storage (not shown) as a response to the requests obtained from the administrator's terminal, and can transmit information corresponding to the requests to the administrator's terminal via communication interface 230.

[0061] Communication interface 230 can perform wired or wireless communication with other devices or networks. Communication interface 230 is connected to a device located outside server 200, and can transmit and receive signals or data. Server 200 can communicate with sensing device 100 via communication interface 230, or can also be connected to other servers connected to a network.

[0062] Storage (not shown) can store various software and information necessary for server 200 to utilize for device control or to provide a predetermined service. For example, storage (not shown) can store programs, applications executed on server 200, and various data or information used for a predetermined service.

[0063] Server 200 can be composed of a load balancing server and a functional server that provides a predetermined service. Server 200 can be composed of a plurality of servers separated by function, or can also be a server in an integrated form.

[0064] With the foregoing configuration, the server 200 can acquire a time-series point cloud related to a three-dimensional space from the sensing device 100 via the communication interface 230, or receive the result of detecting an installation abnormality of the sensing device 100. If the server 200 is connected to a plurality of sensing devices 100 and receives the result of detecting an installation abnormality from any one of the sensing devices 100, the server 200 can block or delete the reception of information and data received from the sensing device 100 where the installation abnormality is detected.

[0065] FIG. 8 is a flowchart for explaining a method of detecting an installation abnormality of the sensing device 100 according to an embodiment. In the above, for the content overlapping with that described for the sensing device 100, the detailed description thereof will be omitted hereinafter.

[0066] In step 810, the sensing device 100 can acquire a time-series point cloud related to a three-dimensional space by using a LiDAR sensor. The sensing device 100 can continuously acquire a point cloud related to a three-dimensional space.

[0067] In step 820, the sensing device 100 can determine a static object area in the three-dimensional space based on the acquired time-series point cloud related to the three-dimensional space. The sensing device 100 can generate a spatial information map from the acquired time-series point cloud related to the three-dimensional space. The spatial information map can also be a normalized map showing spatial information, such as a voxel map or a depth map. The sensing device 100 can determine the static object area based on the period during which point clouds in a unit area at corresponding positions are continuously detected between frames of the generated spatial information map. The sensing device 100 can determine the static object area based on the period during which point clouds in a unit area at corresponding positions are continuously detected with the number of points being equal to or greater than a minimum detection threshold between frames of the spatial information map.

[0068] For example, when the spatial information map is a voxel map, in each frame of the voxel map, the sensing device 100 classifies static voxels where the period during which the point cloud within the voxel is continuously detected is equal to or greater than a threshold value, and dynamic voxels where the period is less than the threshold value. In the reference frame at a specific time point of the voxel map, a static object region composed of static voxels can be determined.

[0069] Note that the sensing device 100 can also determine the static object region of the three-dimensional space based on the point cloud in the specified region among the time-sequential point clouds related to the acquired three-dimensional space. Instead of using the entire time-sequential point cloud related to the acquired three-dimensional space, if a reference region where almost no dynamic point cloud occurs is specified, the sensing device 100 can determine the static individual region of the three-dimensional space based on the point cloud in the specified region. The reference region can be specified by the user or automatically specified by the sensing device 100 as a region determined to be a region corresponding to a static point cloud of a static object corresponding to an appropriate height and size using a learned model.

[0070] In step 830, the sensing device 100 can determine whether there is an installation abnormality of the sensing device 100 based on the static point clouds with a predetermined time difference among the time-sequential static point clouds corresponding to the determined static object region. This will be described in detail below with reference to FIG. 9.

[0071] FIG. 9 is a detailed flowchart for explaining the process of determining the installation abnormality of the sensing device 100. The 830 step in FIG. 8 described above will be described in detail.

[0072] In step 910, the sensing device 100 can extract a first static point cloud at a first time point and a second static point cloud at a second time point having a predetermined time difference from among the time-sequential static point clouds corresponding to the determined static object region.

[0073] In step 920, the sensing device 100 can determine whether there is an installation abnormality of the sensing device 100 based on the difference between the first static point cloud and the second static point cloud.

[0074] For example, the sensing device 100 can compare the ratio of the area of the first static point group related to the determined static object area with the ratio of the area of the second static point group, and determine whether there is an installation abnormality of the sensing device 100.

[0075] As another example, when the sensing device 100 generates a voxel map from the point groups over time related to the three-dimensional space it has acquired, in the voxel map, the ratio of the number of voxels constituting the first static point group related to the total number of voxels constituting the determined static object area is compared with the ratio of the number of voxels constituting the second static point group, and it is possible to determine whether there is an installation abnormality of the sensing device 100.

[0076] For the sake of convenience of explanation, it has been described on the premise that the sensing device 100 is provided on a stationary structure. However, the sensing device 100 can be provided not only on stationary structures but also on moving objects. For example, when the sensing device 100 is provided on a vehicle or a drone, the three-dimensional space changes due to the movement of the vehicle or the drone, so the static objects in the three-dimensional space can also have their positions changed in the spatial information map. However, when the vehicle or the drone repeatedly passes through the same space, it is possible to acquire a point group related to the same space based on the position information of the vehicle or the drone. As another example, in order to monitor a wider space, when the structure on which the sensing device 100 is provided rotates periodically at a predetermined angle, the sensing device 100 can acquire the point group related to the three-dimensional space acquired at the same angle as the point group related to the same space.

[0077] Each of the foregoing embodiments can be provided in the form of a computer program or an application stored in a medium in order to execute a predetermined step of performing a method for detecting an installation abnormality of the sensing device 100. In other words, each of the foregoing embodiments can be provided in the form of a computer program or an application stored in a medium that causes at least one processor of the sensing device 100 to perform a predetermined step of performing the installation abnormality detection method.

[0078] The foregoing embodiments may be embodied in the form of a computer-readable recording medium storing instructions and data executable by a computer or a processor. At least one of the instructions and the data is stored in the form of program code and, when executed by the processor, can generate a predetermined program module and perform a predetermined operation. Such a computer-readable recording medium may be a ROM (read-only memory), a RAM (random access memory), a flash memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-RLTHs, BD-REs, magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, SSDs (solid state drives), and any device that can store instructions or software, related data, data files, and data structures and provide the instructions or software, related data, data files, and data structures to the processor or computer so that the processor or computer can execute the instructions.

[0079] As described above, the present embodiments have been described centering around the embodiments. Those having ordinary knowledge in the technical field to which the disclosed embodiments belong will understand that the disclosed embodiments can be embodied in a modified form without departing from the essential characteristics. Therefore, the disclosed embodiments should be considered from an illustrative perspective rather than a limiting perspective. The scope of the invention is shown not in the description of the foregoing embodiments but in the claims, and all differences within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims

1. Obtaining a time-series point cloud related to a three-dimensional space using a LiDAR sensor; Determining a static object region of the three-dimensional space based on the obtained time-series point cloud related to the three-dimensional space; Extracting a first static point cloud at a first time point and a second static point cloud at a second time point having a predetermined time difference from among the time-series static point clouds corresponding to the determined static object region; Determining whether there is an installation abnormality of the sensing device based on the difference between the first static point cloud and the second static point cloud; and The step of determining whether there is an installation abnormality of the sensing device compares the ratio of the area of the first static point cloud related to the determined static object region with the ratio of the area of the second static point cloud, and determines whether there is an installation abnormality of the sensing device, a method for detecting an installation abnormality of a sensing device.

2. The step of determining the static object region of the three-dimensional space determines the static object region based on a period during which point clouds within a unit region at corresponding positions between frames of a spatial information map generated from the obtained time-series point cloud related to the three-dimensional space are continuously detected. The method for detecting an installation abnormality of a sensing device according to Claim 1.

3. The step of determining the static object region of the three-dimensional space determines the static object region based on a period during which point clouds within a unit region at corresponding positions between frames of the spatial information map are continuously detected by the number of points that is equal to or greater than a minimum detection threshold. The method for detecting an installation abnormality of a sensing device according to Claim 2.

4. The step of determining the static object region of the three-dimensional space wherein the spatial information map is a voxel map, and in each frame of the voxel map, a static voxel in which the period during which point clouds within the voxel are continuously detected is equal to or greater than a threshold value and a dynamic voxel in which the period is less than the threshold value are distinguished, and the static object region formed by the static voxels is determined with reference to a frame at a specific time point of the voxel map. The method for detecting an installation abnormality of a sensing device according to Claim 2.

5. The step of determining the static object region of the three-dimensional space determines the static object region of the three-dimensional space based on the point cloud in a designated region among the obtained time-series point cloud related to the three-dimensional space. The method for detecting an installation abnormality of a sensing device according to Claim 1. Step of determining a static object region of the three-dimensional space based on the time-sequential point cloud related to the three-dimensional space obtained above; Step of extracting a first static point cloud at a first time point and a second static point cloud at a second time point having a predetermined time difference from the first time point, from among the time-sequential static point clouds corresponding to the determined static object region; Step of determining whether there is an installation abnormality of the sensing device based on the difference between the first static point cloud and the second static point cloud, including; The step of determining whether there is an installation abnormality of the sensing device is; In a voxel map generated from the time-sequential point cloud related to the three-dimensional space obtained above, compare the ratio of the number of voxels constituting the first static point cloud related to the total number of voxels constituting the determined static object region, and the ratio of the number of voxels constituting the second static point cloud, and determine whether there is an installation abnormality of the sensing device. A method for detecting an installation abnormality of a sensing device. A command to obtain a time-sequential point cloud related to a three-dimensional space using a LiDAR sensor; A command to determine a static object region of the three-dimensional space based on the time-sequential point cloud related to the three-dimensional space obtained above; A command to extract a first static point cloud at a first time point and a second static point cloud at a second time point having a predetermined time difference from the first time point, from among the time-sequential static point clouds corresponding to the determined static object region; A command to determine whether there is an installation abnormality of the sensing device based on the difference between the first static point cloud and the second static point cloud, including; The command to determine whether there is an installation abnormality of the sensing device is; Compare the ratio of the area of the first static point cloud related to the determined static object region and the ratio of the area of the second static point cloud, and determine whether there is an installation abnormality of the sensing device. A computer-readable recording medium storing a program for causing a computer to execute. A sensor unit that uses a LiDAR sensor to obtain a time-sequential point cloud related to a three-dimensional space; A memory that stores one or more commands; ​ ​ ​ By executing the one or more instruction words, based on the time-series point group related to the acquired three-dimensional space, a static object region of the three-dimensional space is determined, and among the time-series static point groups corresponding to the determined static object region, a first static point group at a first time point and a second static point group at a second time point having a predetermined time difference from the first time point are extracted, and based on the difference between the first static point group and the second static point group, a processor for determining whether there is an installation abnormality of the sensing device, is included. By executing the one or more instruction words, the processor A sensing device that compares the ratio of the area of the first static point group related to the determined static object region with the ratio of the area of the second static point group, and determines whether there is an installation abnormality of the sensing device.

9. By executing the one or more instruction words, the processor The sensing device according to claim 8, wherein the static object region is determined based on a period during which point groups are continuously detected in a unit region at corresponding positions between frames of a space information map generated from the time-series point group related to the acquired three-dimensional space.

10. By executing the one or more instruction words, the processor The sensing device according to claim 9, wherein the static object region is determined based on a period during which point groups are continuously detected in a unit region at corresponding positions between frames of the space information map with a number of points equal to or greater than a minimum detection threshold.

11. By executing the one or more instruction words, the processor The sensing device according to claim 9, wherein the space information map is a voxel map, and in each frame of the voxel map, static voxels in which the period during which point groups in the voxels are continuously detected is equal to or greater than a threshold value and dynamic voxels in which the period is less than the threshold value are distinguished, and the static object region formed by the static voxels is determined in a reference frame at a specific time point of the voxel map.

12. By executing the one or more instruction words, the processor The sensing device according to claim 8, wherein the static object region of the three-dimensional space is determined based on the point group in a designated region among the time-series point group related to the acquired three-dimensional space. A sensor unit that uses a LiDAR sensor to acquire a time-series point group related to a three-dimensional space, A memory that stores one or more instruction words By executing the one or more instruction words, based on the time-series point cloud related to the obtained three-dimensional space, a static object region of the three-dimensional space is determined, and among the time-series static point clouds corresponding to the determined static object region, a first static point cloud at a first time point and a second static point cloud at a second time point having a predetermined time difference from the first time point are extracted, and based on the difference between the first static point cloud and the second static point cloud, a processor for determining whether there is an installation abnormality of the sensing device is included. By executing the one or more instruction words, the processor In a voxel map generated from the time-series point cloud related to the obtained three-dimensional space, compares the ratio of the number of voxels constituting the first static point cloud related to the number of all voxels constituting the determined static object region with the ratio of the number of voxels constituting the second static point cloud, and determines whether there is an installation abnormality of the sensing device. A sensing device.

14. Instructions for obtaining a time-series point cloud related to a three-dimensional space using a LiDAR sensor, Instructions for determining a static object region of the three-dimensional space based on the obtained time-series point cloud related to the three-dimensional space, Instructions for extracting, among the time-series static point clouds corresponding to the determined static object region, a first static point cloud at a first time point and a second static point cloud at a second time point having a predetermined time difference from the first time point, Instructions for determining whether there is an installation abnormality of the sensing device based on the difference between the first static point cloud and the second static point cloud, are included. The instructions for determining whether there is an installation abnormality of the sensing device In a voxel map generated from the time-series point cloud related to the obtained three-dimensional space, compares the ratio of the number of voxels constituting the first static point cloud related to the number of all voxels constituting the determined static object region with the ratio of the number of voxels constituting the second static point cloud, and determines whether there is an installation abnormality of the sensing device. A computer-readable recording medium storing a program for causing a computer to execute.

Images