Obstacle detection device, obstacle detection method, and program

Abstract

The obstacle detection device (2000) is configured to: obtain a position dataset (20) including a plurality of position data (22) indicating three-dimensional positions of points at a target location (10); detect a first subset from the position dataset (20) including the plurality of position data (22) collectively representing the ground of the target location (10); identify a vertical position of the ground based on vertical positions indicated by the plurality of position data (22) in the first subset; detect one or more objects from the position dataset (20) to generate an obstacle candidate set; calculate, for each object in the obstacle candidate set, a height of the object based on the vertical position of the object and the vertical position of the ground; and exclude from the obstacle candidate set any object whose height is greater than a height threshold.

Description

[Technical Field]

[0001] The present disclosure generally relates to an obstacle detection device, an obstacle detection method, and a non-transitory computer-readable storage medium. [Background technology]

[0002] A computer system has been developed that detects obstacles using a set of position data (e.g., images or depth images derived from LiDAR) indicating the positions of multiple points at a certain location. Patent Document 1 discloses a problem that it is difficult for a cleaning robot to detect small objects, and provides a technology that enables the cleaning robot to detect small objects using a 3D point cloud. Patent Document 2 discloses a method for detecting obstacles on a road using a 3D point cloud. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] European Patent Application Publication No. 3825903 [Patent Document 2] US Patent Application Publication No. 2020 / 0012869 Summary of the Invention [Problem to be solved by the invention]

[0004] Regarding Patent Document 1, it is clear that the cleaning robot must detect any obstacle that impedes its movement. Therefore, the cleaning robot is required to detect not only small objects but also objects other than small objects as obstacles. The method disclosed in Patent Document 2 does not take into account the height of the obstacle. The purpose of the present disclosure is to provide a novel technique for detecting obstacles using a set of position data. [Means for solving the problem]

[0005] The present disclosure provides an obstacle detection apparatus that includes at least one memory configured to store instructions and at least one processor. At least one processor is configured to execute instructions to obtain a position dataset including a plurality of position data indicating three-dimensional positions of points at a target location; detect from the position dataset a first subset including a plurality of the position data collectively representing a ground surface of the target location; identify a vertical position of the ground surface based on vertical positions indicated by the plurality of position data in the first subset; detect one or more objects from the position dataset to generate an obstacle candidate set; calculate, for each object in the obstacle candidate set, a height of the object based on the vertical position of the object and the vertical position of the ground surface; and exclude from the obstacle candidate set any object whose height is greater than a height threshold.

[0006] The present disclosure further provides an obstacle detection method, the method including: acquiring a position dataset including a plurality of position data indicating three-dimensional positions of points of a target location; detecting a first subset from the position dataset including a plurality of the position data collectively representing a ground surface of the target location; identifying a vertical position of the ground surface based on vertical positions indicated by the plurality of position data in the first subset; detecting one or more objects from the position dataset to generate an obstacle candidate set; calculating, for each object in the obstacle candidate set, a height of the object based on the vertical position of the object and the vertical position of the ground surface; and excluding from the obstacle candidate set any object whose height is greater than a height threshold.

[0007] The present disclosure further provides a non-transitory computer-readable storage medium storing a program that causes a computer to: acquire a position dataset including a plurality of position data indicating three-dimensional positions of points at a target location; detect from the position dataset a first subset including a plurality of the position data collectively representing a ground surface of the target location; identify a vertical position of the ground surface based on vertical positions indicated by the plurality of position data in the first subset; detect one or more objects from the position dataset to generate an obstacle candidate set; calculate, for each object in the obstacle candidate set, a height of the object based on the vertical position of the object and the vertical position of the ground surface; and exclude from the obstacle candidate set any object whose height is greater than a height threshold. [Effects of the Invention]

[0008] According to the present disclosure, a novel technique for detecting obstacles using a set of position data is provided. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an overview of an obstacle detection device according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of a functional configuration of an obstacle detection device according to a first embodiment. [Figure 3] 1 is a block diagram showing an example of a hardware configuration of a computer that realizes an obstacle detection device according to a first embodiment. [Figure 4] 4 is a flowchart showing an example of a processing flow of the obstacle detection device of the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of dividing the ground into sections based on section heights. [Figure 6] FIG. 10 is a diagram showing an example of dividing the ground into sections in a plan view using a predetermined window. DETAILED DESCRIPTION OF THE INVENTION

[0010] Embodiments according to the present disclosure will be described below with reference to the drawings. The same elements are assigned the same reference numerals throughout the drawings, and redundant descriptions will be omitted as necessary. Furthermore, unless otherwise specified, predefined information (e.g., predetermined values or predetermined threshold values) is pre-stored in a storage unit accessible by a computer that uses the information. In the present disclosure, the storage unit may be implemented by one or more storage devices such as a hard disk, a solid-state drive (SSD), or a random-access memory (RAM).

[0011] Embodiment 1 <Summary> Fig. 1 shows an overview of an obstacle detection device 2000 according to embodiment 1. Note that Fig. 1 does not limit the operation of the obstacle detection device 2000, but merely shows one example of the operation that the obstacle detection device 2000 can perform.

[0012] The obstacle detection device 2000 is a device configured to detect one or more obstacles in the target location 10 that tend to be easily out of human sight. Such obstacles are small objects whose height is equal to or less than a predetermined threshold (e.g., 10 cm, 50 cm, 1 m, etc.). An example of a small object may be a step or a pipe located on the ground of the target location 10. Because small objects tend to be out of human sight and may cause accidents (e.g., falls), it is preferable to make such small objects more easily noticeable to humans. Hereinafter, candidate obstacles to be detected may be referred to as "candidate obstacles." On the other hand, objects that are not so small (e.g., walls, buildings, or machines) are easily within human sight and can be easily noticed by humans without the intervention of a computer system.

[0013] Therefore, the obstacle detection device 2000 treats objects on the ground of the target location 10 whose height is greater than a predetermined threshold as outliers and excludes them from obstacle candidates. To this end, the obstacle detection device 2000 can operate as follows.

[0014] The obstacle detection device 2000 acquires a position dataset 20. The position dataset 20 includes a plurality of position data 22. The position data 22 indicate three-dimensional (3D) positions of points in the target location 10. In some implementations, the position dataset 20 may be referred to as a "3D point cloud."

[0015] The obstacle detection device 2000 detects the ground of the target location 10 from the position dataset 20. Specifically, the obstacle detection device 2000 detects a first subset from the position dataset 20. The first subset is a subset of the position dataset 20 and includes a plurality of position data 22 that collectively represent the ground of the target location 10.

[0016] The ground of the target location 10 is the ground on which people move in the target location 10. The ground may be located inside a building or outside a building. In the former case, the ground may be the floor of an indoor facility, such as a factory or office. In the latter case, the ground may be the ground of an outdoor facility, such as a substation.

[0017] The obstacle detection device 2000 determines the vertical position (or height) of the ground. In some implementations, the obstacle detection device 2000 calculates a statistical value of the vertical positions (e.g., z coordinates indicated by the position data 22) of the position data 22 included in the first subset as the vertical position of the ground. Hereinafter, the vertical position of the ground may be referred to as "ground level."

[0018] The obstacle detection device 2000 detects objects from the position dataset 20 to generate a set of obstacle candidates called an "obstacle candidate set." Specifically, the obstacle detection device 2000 detects one or more second subsets from the position dataset 20. The second subset is a subset of the position dataset 20 and includes multiple position data 22 that collectively represent an object. The obstacle candidate set may be a collection of the second subsets.

[0019] The obstacle detection device 2000 calculates the height of each object in the obstacle candidate set based on the ground level and the vertical position of the object. For example, the obstacle detection device 2000 may calculate the difference between the vertical position of the object and the ground level as the height of the object. Note that the vertical position of the object may be the highest vertical position of all the position data 22 in the second subset corresponding to the object.

[0020] For each object in the obstacle candidate set, the obstacle detection device 2000 determines whether the height of the object is greater than a predetermined threshold called the "height threshold." If the height of the object is determined to be greater than the height threshold, the obstacle detection device 2000 removes the object from the obstacle candidate set. This means that an object whose height is determined to be greater than the height threshold is treated as an outlier. On the other hand, if the height of the object is determined to be equal to or less than the height threshold, the obstacle detection device 2000 does not remove the object from the obstacle candidate set.

[0021] <Examples of effects> In some locations, such as factories and substations, it is important to ensure the safety of workers from obstacles that could cause accidents, so it is useful to provide alerts to workers when they are approaching an obstacle, for example.

[0022] However, workers can easily notice tall objects such as walls and machines and do not need alerts to notice them. Furthermore, if even tall objects are treated as obstacles, workers may receive too many alerts, and their work may be distracted by the alerts. Therefore, in some cases, it may be preferable and effective to treat only small objects as obstacles.

[0023] As described above, the obstacle detection device 2000 excludes objects whose height is greater than the height threshold from the obstacle candidate set. As a result of this exclusion, all objects included in the obstacle candidate set are equal to or less than the height threshold. By performing obstacle detection on this obstacle candidate set, it is possible to detect obstacles whose height is equal to or less than the height threshold. Furthermore, it is possible to treat only small objects as targets for alerts, thereby providing people with effective alerts while preventing them from being distracted by less effective alerts.

[0024] The obstacle detection device 2000 also determines the vertical height of the ground at the target location 10 based on the position data set 20, and calculates the height of the object. Therefore, the obstacle detection device 2000 does not need to be installed at the height of the sensor (e.g., LiDAR device or depth camera) used to generate the position data set 20.

[0025] The obstacle detection device 2000 will be described in more detail below.

[0026] <Example of functional configuration> 2 is a block diagram showing an example of the functional configuration of the obstacle detection device 2000 according to embodiment 1. The obstacle detection device 2000 includes an acquisition unit 2020, a ground level identification unit 2040, an object detection unit 2060, and an outlier removal unit 2080.

[0027] The acquisition unit 2020 acquires the position dataset 20. The ground level identification unit 2040 detects the ground of the target location 10 from the position dataset 20 and identifies the ground level. The object detection unit 2060 detects one or more objects from the position dataset 20 and generates an obstacle candidate set. The outlier removal unit 2080 calculates the height of each object and determines whether the height of the object is greater than a height threshold. The outlier removal unit 2080 removes objects whose height is determined to be greater than the height threshold from the obstacle candidate set.

[0028] <Example of hardware configuration> The obstacle detection device 2000 may be realized by one or more computers. Each of the one or more computers may be a dedicated computer manufactured for implementing the obstacle detection device 2000, or may be a general-purpose computer such as a personal computer (PC), a server machine, or a mobile device.

[0029] The obstacle detection device 2000 may be realized by installing an application on a computer. The application is realized by a program that causes the computer to function as the obstacle detection device 2000. In other words, the program is an implementation of each functional unit of the obstacle detection device 2000. There are various methods for acquiring the program. For example, the program can be acquired from a storage medium (such as a DVD disk or USB memory) on which the program is pre-stored. In another example, the program can be acquired by downloading it from a server machine that manages the storage medium on which the program is pre-stored.

[0030] 3 is a block diagram showing an example of the hardware configuration of a computer 1000 that realizes the obstacle detection device 2000 of embodiment 1. In FIG. 3, the computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output (I / O) interface 1100, and a network interface 1120.

[0031] The bus 1020 is a data transmission channel through which the processor 1040, memory 1060, storage device 1080, input / output interface 1100, and network interface 1120 transmit and receive data to and from each other. The processor 1040 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), or a digital signal processor (DSP). The memory 1060 is a main storage element such as a random access memory (RAM) or a read-only memory (ROM). The storage device 1080 is an auxiliary storage element such as a hard disk, a solid-state drive (SSD), or a memory card. The input / output interface 1100 is an interface between the computer 1000 and peripheral devices such as a keyboard, a mouse, or a display device. The network interface 1120 is an interface between the computer 1000 and a network. The network may be a local area network (LAN) or a wide area network (WAN). The storage device 1080 may store the above-mentioned programs. The processor 1040 executes a program to realize each functional unit of the obstacle detection device 2000.

[0032] The hardware configuration of the computer 1000 is not limited to that shown in Fig. 3. For example, as described above, the obstacle detection device 2000 may be realized by a plurality of computers. In this case, the computers may be connected to each other via a network.

[0033] <Process flow> 4 is a flowchart showing an example of the processing flow of the obstacle detection device 2000 according to the first embodiment. The acquisition unit 2020 acquires the position data set 20 (S102). The ground level identification unit 2040 detects the ground of the target location 10 from the position data set 20 (S104). The ground level identification unit 2040 identifies the ground level (S106). The object detection unit 2060 detects one or more objects from the position data set 20 and generates an obstacle candidate set (S108).

[0034] Steps S110 to S118 constitute a loop process L1 that is executed for each object in the obstacle candidate set. In step S110, the obstacle detection device 2000 determines whether or not loop process L1 has been executed for all objects in the obstacle candidate set. If loop process L1 has been executed for all objects in the obstacle candidate set, the detection device 2000 ends loop process L1. This ends the processing shown in FIG. 4. On the other hand, if the detection device 2000 has not yet executed loop process L1 for all objects in the obstacle candidate set, it selects one of the objects for which loop process L1 has not been executed. The object selected here is referred to as "object i."

[0035] The outlier removal unit 2080 calculates the height of object i (S112). The outlier removal unit 2080 determines whether the height of object i is greater than the height threshold (S114). If the height of the object is greater than the height threshold (S114: Yes), the outlier removal unit 2080 removes object i from the obstacle candidate set (S116). On the other hand, if the height of the object is equal to or less than the height threshold (S114: No), object i is not removed from the object candidate set. Step S118 marks the end of the current iteration of loop processing L1. This causes step S110 to be executed again.

[0036] When the repeated execution of the loop process L1 is completed, the obstacle detection device 2000 can obtain a set of obstacle candidates that includes only objects whose height is equal to or less than the height threshold.

[0037] The processing flow performed by the obstacle detection device 2000 is not limited to that shown in Fig. 4. For example, the detection of the ground surface (S104) and the determination of the ground level (S106) may be performed after or in parallel with the detection of an object (S108).

[0038] <Acquisition of location dataset 20: S102> The acquisition unit 2020 acquires the position dataset 20 (S102). There are various methods for acquiring the position dataset 20. In some embodiments, the acquisition unit 2020 can receive the position dataset 20 transmitted from another computer (such as a computer that generates the position dataset 20). In other embodiments, the position dataset 20 may be pre-stored in a storage unit that the acquisition unit 2020 accesses. In this case, the acquisition unit 2020 reads the position dataset 20 from the storage unit.

[0039] There are various well-known methods for generating a set of data indicating the location of a place, such as a 3D point cloud, and any one of these methods can be used to generate the location dataset 20. For example, the location dataset 20 can be generated by measuring multiple points in the location of interest 10 using a sensor device (e.g., a LiDAR device or a depth camera) that measures the distance from the device to multiple points in the scene.

[0040] <Ground detection: S104> The ground level identification unit 2040 detects the ground surface of the target location 10 from the position data set 20 (S104). Specifically, as described above, the ground level identification unit 2040 detects, as a first subset, a set of position data 22 that collectively represent the ground surface of the target location 10 from the position data set 20. There are various known methods for detecting a subset of position data that collectively represent the ground surface from a set of position data (e.g., a 3D point cloud), and any of these methods can be applied to the ground level identification unit 2040.

[0041] For example, the ground level determination unit 2040 can divide the position data set 20 into a plurality of subsets each including a plurality of position data 22 that collectively represent regions of the target location 10 by performing segmentation on the position data set 20. Here, the region of the target location 10 is a part of the target location 10 existing in the real world, which is collectively represented by a plurality of position data 22 corresponding to that region. Next, the ground level determination unit 2040 selects one of the subsets that satisfies a predetermined characteristic of the ground as the first subset.

[0042] The predetermined characteristic of the ground may include: (1) the normal direction of the region represented by the subset is substantially parallel to the vertical direction, and (2) the size of the region represented by the subset is the largest among all regions having characteristic (1). Note that there are various known methods for obtaining the normal vector of the region represented by the set of position data, and any of these methods can be applied to the ground level determination unit 2040.

[0043] It should also be noted that whether the normal direction of the region is substantially parallel to the vertical direction may be determined based on the angle A formed by the normal direction of the region and the vertical direction. Specifically, when |A| < Th1, it is determined that the normal direction of the region is substantially parallel to the vertical direction. Th1 is a predetermined threshold value and can be set to a value sufficiently close to 0. On the other hand, if |A| >= Th1, it is determined that the normal direction of the region is not substantially parallel to the vertical direction.

[0044] It should also be noted that the normal direction of the region can be calculated using the set of position data 22 that collectively represents the region. For example, the ground level determination unit 2040 can determine a plane that fits the set of position data 22 that collectively represents the region, and calculate the normal direction of that plane as the normal direction of the region.

[0045] In the above example, one of the subsets that meets the predetermined characteristics of the ground is selected as the first subset. However, in another example, the ground level identification unit 2040 may select two or more subsets that meet the predetermined characteristics of the ground, merge them into one, and provide it as the first subset. In this case, the predetermined characteristics of the ground may include (1) the normal direction of the area represented by the subset is approximately parallel to the vertical direction, and (2) the size of the area represented by the subset is larger than a predetermined value. One or more subsets that meet these characteristics are selected to belong to the first subset. In this case, the one or more subsets are merged into one and provided as the first subset, i.e., the ground.

[0046] In another example, the ground level identification unit 2040 performs semantic segmentation on the location data 22. In this case, each subset of the location data set 20 that collectively represent an area of the location of interest 10 is assigned one of several predefined classes. This allows the ground level identification unit 2040 to determine the subset to which the class "ground" is assigned as the first subset.

[0047] It should be noted that various methods are known for performing semantic segmentation on a set of position data, any of which may be applied to the ground level identification unit 2040. For example, the ground level identification unit 2040 may include a machine learning based model, such as a neural network, configured to take the set of position data 22 as input and trained to identify a class to which a region represented by the input data belongs.

[0048] <Ground level determination: S106> The ground level determination unit 2040 determines the ground level (i.e., the vertical position of the ground) (S106). In some embodiments, the ground level determination unit 2040 may calculate a statistical value of the vertical position of the position data 22 included in the first subset (i.e., the set of position data 22 that collectively represent the ground of the target location 10, as described above). It should be noted that there are various types of statistical values that can be used to calculate the ground level, for example, average, maximum, minimum, or mode.

[0049] If the average is taken as the type of statistic, the ground level can be calculated as follows: formula 1

number

[0050] The ground surface of the target location 10 may include two or more areas with sufficiently different levels. Therefore, the ground level specifying unit 2040 may specify two or more ground levels of the ground surface of the target location 10. Specifically, the ground level specifying unit 2040 may divide the ground surface of the target location 10 into two or more sections and determine the ground level for each section.

[0051] When the ground is divided into two or more sections, the ground level identification unit 2040 divides the first subset into two or more subsets that collectively represent the section of the ground. Then, for each section, the ground level identification unit 2040 may calculate a statistical value of the vertical position of the position data 22 included in the subset of the first subset that collectively represents the section. When an average is adopted as the type of statistical value, the ground level of the section can be calculated as follows: formula 2

number

[0052] There are various ways to divide the ground into sections, some examples of which are described below.

[0053] <<Example 1>> To divide the ground surface into segments, the ground level determination unit 2040 can use predetermined heights of the segments, called "segment heights," which specify the height of a segment, i.e., the difference between the lowest and highest vertical positions within a single segment.

[0054] The level of the lowest-level position data p_l included in the first subset is defined as v(p_l), and the level of the highest-level position data p_h included in the first subset is defined as v(p_h). Furthermore, if the section height is defined as S, the ground level identification unit 2040 can divide the range of vertical positions of the ground [v(p_l), v(p_h)] by an interval S to obtain multiple ranges of vertical positions {R1:[v(p_l), v(p_l)+S], R2:[v(p_l)+S, v(p_l)+2S],..., Rn:[v(p_l)+(n-1)*S, v(p_h)]}, where n is the number of obtained ranges. In some embodiments, the interval S may be defined based on a height threshold, for example, S=α*HT (0<α<1), where HT represents the height threshold.

[0055] Figure 5 shows an example of dividing the ground into sections based on section height. In Figure 5, the range of vertical positions on the ground is divided into four ranges: [v(p_l), v(p_l)+S], [v(p_l)+S, v(p_l)+2S], [v(p_l)+2S, v(p_l)+3S], and [v(p_l)+3S, v(p_h)]. Therefore, the ground is divided into four sections.

[0056] The ground level identification unit 2040 divides the first subset into sections by assigning to each of the position data 22 of the first subset one of a range of vertical positions that includes the vertical position of the position data 22.

[0057] <<Example 2>> The ground level identification unit 2040 can divide the ground into sections in a planar view using a predetermined window. For example, the predetermined window may be preset to have a rectangular shape with a predetermined x-direction length W and y-direction length H. In this case, the ground is divided into two or more sections each having a rectangular shape with an x-direction length W and a y-direction length H in a planar view.

[0058] An example of dividing a ground plane into monoscopic sections using a predetermined window is shown in Figure 6. In Figure 6, the ground plane 30 is divided into 5x6 sections using a predetermined window 40.

[0059] <<Example 3>> The ground level identifyr 2040 may divide the ground surface into segments based on their class (or type), e.g., flat segments, upslope segments, downslope segments, etc. To do so, the ground level identifyr 2040 may perform semantic segmentation on a first subset to divide them into two or more subsets based on the class of the segments. As mentioned above, there are various well-known methods for performing semantic segmentation on a set of position data, and the ground level identifyr 2040 may apply one of these methods to perform semantic segmentation to divide the ground surface into segments.

[0060] <Object detection: S108> The object detection unit 2060 detects one or more objects from the position data set 20 (S108). Specifically, the object detection unit 2060 detects one or more subsets (i.e., second subsets) of the position data set 20, and generates an obstacle candidate set including the detected second subsets. As described above, the second subset includes a plurality of position data 22 that collectively represent objects other than the ground. There are various methods for detecting objects from a set of position data, and one of them can be applied to the object detection unit 2060.

[0061] The obstacle detection device 2000 may detect objects from the entire position dataset 20, or may detect objects from a portion of the position dataset 20. In the latter case, the obstacle detection device 2000 may identify a portion of the position dataset 20 as a target for object detection. In some cases, the target location 10 includes one or more areas that are inaccessible to people (e.g., areas occupied by machinery). Small objects located in these areas may not need to be detected for the purpose of providing an alert to a person in response to the person's approaching them.

[0062] Therefore, the object detection unit 2060 may identify a partial area of the target location 10 called a "target area" as a target for object detection. For example, the object detection unit 2060 may identify the range of a planar view area that a person can enter. The range of this area may be defined by a pair of an x-direction range [x_min, x_max] and a y-direction range [y_min, y_max]. In this case, the object detection unit 2060 extracts from the position dataset 20 all position data 22 whose x position is between x_min and x_max and whose y position is between y_min and y_max, thereby obtaining a set of position data 22 included in the target area. The object detection unit 2060 then detects objects from this set of position data 22.

[0063] The target area may be defined based on the extent of the ground of the target location 10 in plan view. For example, the minimum x position of the position data 22 included in the first subset is defined as x_min, and the maximum x position is defined as x_max. Similarly, the minimum y position of the position data 22 included in the first subset is defined as y_min, and the maximum y position is defined as y_max.

[0064] The object detection unit 2060 may add a buffer to the target region. Assume that the minimum x position and maximum x position of the position data 22 included in the first subset are x1 and x2, respectively. In this case, the object detection unit 2060 may set the buffer size in the x direction to b1 and set x1-b1 and x2+b1 to x_min and x_max, respectively. Similarly, the minimum y position of the position data 22 included in the first subset to y1 and the maximum y position to y2. In this case, the object detection unit 2060 may set the buffer size in the y direction to b2 and set y1-b2 and y2+b2 to y_min and y_max, respectively.

[0065] <Exclusion of outliers: S112, S114, S116> The outlier removal unit 2080 removes one or more outliers from the obstacle candidate set. Specifically, for each object in the obstacle candidate set, the outlier removal unit 2080 calculates the height of the object (S112), determines whether the height of the object is greater than a height threshold (S114), and, if it determines that the height of the object is greater than the height threshold (S116), removes the object (more specifically, a second subset corresponding to the object) from the obstacle candidate set.

[0066] The height of an object can be represented by the absolute value of the difference between the vertical position of the object and the ground level. Let the vertical position of the object be Vo and the ground level be Vg. In this case, the height of the object is the absolute value of the difference between Vo and Vg, which can be represented as |Vo - Vg|. If Vo > Vg, the object is a convex object, such as a step or a pipe. On the other hand, if Vo < Vg, the object can be a concave object, such as a hole or a drainage channel. Note that when Vo < Vg, the height of the object can also be referred to as the "depth" of the object.

[0067] As described above, the ground level may be specified for two or more sections of the ground. In this case, the outlier-removed external part 2080 calculates the height of the object based on the vertical position of the section corresponding to the object. Specifically, for each object, the outlier-removed external part 2080 identifies the section corresponding to that object. The section corresponding to the object is the section where the object is located. Then, the outlier-removed external part 2080 calculates the height of the object based on the vertical position of the identified section and the vertical position of the object.

[0068] <Obstacle Detection> After excluding outliers from the set of obstacle candidates, the obstacle detection device 2000 can determine whether each object in the set of obstacle candidates is an obstacle. Before determining whether an object is an obstacle, non-small objects (i.e., objects with a height greater than the height threshold) are excluded from the set of obstacle candidates, so that non-small objects can be prevented from being detected as obstacles.

[0069] There are various methods for determining whether an object is an obstacle. In some embodiments, the obstacle detection device 2000 can calculate the normal direction of each object and determine whether the normal direction of the object is substantially perpendicular to the vertical direction. If it is determined that the normal direction of the object is substantially perpendicular to the vertical direction, the obstacle detection device 2000 determines that the object is an obstacle. On the other hand, if the obstacle detection device 2000 determines that the normal direction of the object is not substantially perpendicular to the vertical direction, it determines that the object is not an obstacle.

[0070] Whether the normal direction of the object is substantially perpendicular to the vertical direction may be determined based on the angle B formed by the normal direction of the object and the vertical direction. Specifically, if |B - 90°| < Th2, it is determined that the normal direction of the object is substantially perpendicular to the vertical direction. Th2 is a predetermined threshold value and can be set to a value sufficiently close to 0. On the other hand, when |B - 90°| >= Th2, it is determined that the normal direction of the object is not substantially perpendicular to the vertical direction.

[0071] Here, the normal direction of the object can be calculated using the second subset corresponding to the object. For example, the object detection device 2000 can determine the plane of the object based on the distribution of the position data 22 within the second subset corresponding to the object, and calculate the normal direction of the plane as the normal direction of the object.

[0072] <Output from the obstacle detection device> The obstacle detection device 2000 may output information called "output information" related to the result of the processing performed by the obstacle detection device 2000. In some embodiments, the output information can indicate the obstacles detected by the obstacle detection device 2000. Specifically, for each object determined to be an obstacle, the output information can include the second subset corresponding to the object.

[0073] Alternatively or additionally, the output information may include a set of obstacle candidates from which outliers have already been removed (i.e., objects with a height greater than the height threshold). In this case, the determination of whether each object in the set of obstacle candidates is an obstacle may be performed by another device that acquires the output information, rather than by the obstacle detection device 2000.

[0074] There can be various ways to output the output information. For example, the obstacle detection device 2000 may store the output information in the storage unit. In another example, the obstacle detection device 2000 may transmit the output information to any device. In another example, the obstacle detection device 2000 may output the output information to the display device so that the content of the output information is displayed on the display device.

[0075] <Application example of obstacle detection> The detection result of the obstacle may be used to provide an alert to a person approaching the obstacle, and the device that provides this alert will be referred to as the "alert device" hereinafter.

[0076] The alert device can track the position of the target person by repeatedly acquiring information called "person position information" that indicates the target person's current position. The alert device also acquires information called "obstacle information" that indicates the position of each obstacle detected by the obstacle detection device 2000.

[0077] In response to the acquisition of the person position information, the alert device determines whether the subject is approaching an obstacle based on the obstacle information and the time-series person position information. For example, the alert device may identify the subject's moving direction based on the time-series person position information. If there is an obstacle in this identified direction from the subject's current position and the distance between the obstacle's position and the subject's current position is less than a predetermined threshold, the alert device determines that the subject is approaching an obstacle. Otherwise, the alert device determines that the subject is not approaching an obstacle.

[0078] If it is determined that the subject is approaching an obstacle, the alert device provides alert information to the subject. The alert information may be information that helps the subject avoid the approaching obstacle. For example, the alert information may indicate the location of the obstacle, the type of the obstacle (e.g., steps or pipes), or the height of the obstacle.

[0079] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs, CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and RAMs). The program may also be provided to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can provide the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0080] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the arrangements and details of the present disclosure within the scope of the present invention.

[0081] All or part of the above-described embodiments can also be described as, but not limited to, the following supplementary notes. <Additional Notes> (Appendix 1) An obstacle detection device, at least one memory configured to store instructions; at least one processor, wherein the at least one processor executes the instructions to obtaining a location dataset including a plurality of location data indicating three-dimensional positions of points of interest; detecting a first subset from the location data set, the first subset including a plurality of the location data collectively representing a ground surface of the location of interest; determining a vertical position of the ground based on vertical positions indicated by the plurality of position data in the first subset; detecting one or more objects from the position dataset to generate a set of obstacle candidates; For each object in the set of obstacle candidates, calculate a height of the object based on the vertical position of the object and the vertical position of the ground; The obstacle detection apparatus is configured to exclude from the set of obstacle candidates those objects whose height is greater than a height threshold. (Appendix 2) 2. The obstacle detection device of claim 1, wherein the determining of the vertical position of the ground includes calculating a statistical value of the vertical positions indicated by the plurality of position data in the first subset as the vertical position of the ground. (Appendix 3) The determination of the vertical position of the ground surface comprises: dividing the ground surface into two or more sections; 3. The obstacle detection device according to claim 1, further comprising: calculating, for each section of the ground, the vertical position of the section based on the vertical positions indicated by the plurality of position data collectively representing the section. (Appendix 4) The division of the ground surface comprises: identifying a lowest vertical position and a highest vertical position indicated by the plurality of position data in the first subset; Dividing the range between the lowest vertical position and the highest vertical position into two or more height ranges at predetermined intervals; and dividing the first subset into two or more subsets representing different sections of the ground based on the two or more height ranges. (Appendix 5) 5. The obstacle detection device of claim 4, wherein the predetermined interval is proportional to the height threshold. (Appendix 6) The detecting of the one or more objects includes: Identifying a partial area of the target location based on an area of the ground in a planar view; and detecting the one or more objects from a plurality of sets of position data indicating positions included in the partial region. (Appendix 7) acquiring a location dataset including a plurality of location data indicating three-dimensional positions of points of interest; detecting a first subset from the location data set, the first subset including a plurality of the location data collectively representing a ground surface of the location of interest; determining a vertical position of the ground based on vertical positions indicated by the plurality of position data in the first subset; detecting one or more objects from the position dataset to generate a set of obstacle candidates; For each object in the set of obstacle candidates, calculating a height of the object based on a vertical position of the object and the vertical position of the ground; and removing from the set of obstacle candidates those objects whose height is greater than a height threshold. (Appendix 8) 8. The obstacle detection method of claim 7, wherein the determining of the vertical position of the ground includes calculating a statistical value of the vertical positions indicated by the plurality of position data in the first subset as the vertical position of the ground. (Appendix 9) The determination of the vertical position of the ground surface comprises: dividing the ground surface into two or more sections; 9. The obstacle detection method of claim 7 or 8, further comprising: calculating, for each section of the ground, the vertical position of the section based on the vertical positions indicated by the plurality of position data collectively representing the section. (Appendix 10) The division of the ground surface comprises: identifying a lowest vertical position and a highest vertical position indicated by the plurality of position data in the first subset; Dividing the range between the lowest vertical position and the highest vertical position into two or more height ranges at predetermined intervals; and dividing the first subset into two or more subsets representing different sections of the ground based on the two or more height ranges. (Appendix 11) 11. The obstacle detection method of claim 10, wherein the predetermined interval is proportional to the height threshold. (Appendix 12) The detecting of the one or more objects includes: Identifying a partial area of the target location based on an area of the ground in a planar view; 9. The obstacle detection method according to claim 7 or 8, further comprising detecting the one or more objects from a plurality of sets of position data indicating positions included in the partial region. (Appendix 13) acquiring a location dataset including a plurality of location data indicating three-dimensional positions of points of interest; detecting a first subset from the location data set, the first subset including a plurality of the location data collectively representing a ground surface of the location of interest; determining a vertical position of the ground based on vertical positions indicated by the plurality of position data in the first subset; detecting one or more objects from the position dataset to generate a set of obstacle candidates; For each object in the set of obstacle candidates, calculating a height of the object based on a vertical position of the object and the vertical position of the ground; and excluding from the obstacle candidate set any object whose height is greater than a height threshold. (Appendix 14) 14. The storage medium of claim 13, wherein the determining the vertical position of the ground includes calculating a statistical value of the vertical positions indicated by the plurality of position data in the first subset as the vertical position of the ground. (Appendix 15) The determination of the vertical position of the ground surface comprises: dividing the ground surface into two or more sections; 15. The storage medium of claim 13, further comprising: for each section of the ground, calculating the vertical position of the section based on the vertical positions indicated by the plurality of position data collectively representing the section. (Appendix 16) The division of the ground surface comprises: identifying a lowest vertical position and a highest vertical position indicated by the plurality of position data in the first subset; Dividing the range between the lowest vertical position and the highest vertical position into two or more height ranges at predetermined intervals; and dividing the first subset into two or more subsets representing different sections of the ground based on the two or more height ranges. (Appendix 17) 17. The storage medium of claim 16, wherein the predetermined interval is proportional to the height threshold. (Appendix 18) The detecting of the one or more objects includes: Identifying a partial area of the target location based on an area of the ground in a planar view; Detecting the one or more objects from a plurality of sets of position data indicating positions included in the partial region. [Explanation of symbols]

[0082] 10 Target Locations 20 Location Dataset 22 Location Data 30 ground 40 Predefined Window 1000 computers 1020 Bus 1040 processor 1060 memory 1080 storage device 1100 Input / Output Interface 1120 Network Interface 2000 Obstacle Detection Device 2020 Acquisition Department 2040 Ground level identification part 2060 Object detection unit 2080 Outlier removal section

Claims

1. An obstacle detection device, at least one memory configured to store instructions; at least one processor, wherein the at least one processor executes the instructions to obtaining a location dataset including a plurality of location data indicating three-dimensional positions of points of interest; detecting a first subset from the location data set, the first subset including a plurality of the location data collectively representing a ground surface of the location of interest; determining a vertical position of the ground based on vertical positions indicated by the plurality of position data in the first subset; detecting one or more objects from the position dataset to generate a set of obstacle candidates; For each object in the set of obstacle candidates, calculate a height of the object based on the vertical position of the object and the vertical position of the ground; The obstacle detection apparatus is configured to exclude from the set of obstacle candidates those objects whose height is greater than a height threshold.

2. 2. The obstacle detection device according to claim 1, wherein the determining of the vertical position of the ground surface includes calculating a statistical value of vertical positions indicated by the plurality of position data in the first subset as the vertical position of the ground surface.

3. The determination of the vertical position of the ground surface comprises: dividing the ground surface into two or more sections; 3. The obstacle detection device according to claim 1, further comprising: calculating, for each section of the ground, the vertical position of the section based on the vertical positions indicated by the plurality of position data collectively representing the section.

4. The division of the ground surface comprises: identifying a lowest vertical position and a highest vertical position indicated by the plurality of position data in the first subset; Dividing the range between the lowest vertical position and the highest vertical position into two or more height ranges at predetermined intervals; and dividing the first subset into two or more subsets representing different sections of the ground surface based on the two or more height ranges.

5. 5. The obstacle detection device according to claim 4, wherein the predetermined interval is proportional to the height threshold.

6. The detecting of the one or more objects comprises: Identifying a partial area of the target location based on an area of the ground in a planar view; 3. The obstacle detection device according to claim 1, further comprising: detecting the one or more objects from a plurality of sets of position data indicating positions included in the partial region.

7. acquiring a position dataset including a plurality of position data indicating three-dimensional positions of points of the target location; detecting a first subset from the location data set, the first subset including a plurality of the location data collectively representing a ground surface of the location of interest; determining a vertical position of the ground based on vertical positions indicated by the plurality of position data in the first subset; detecting one or more objects from the position dataset to generate a set of obstacle candidates; For each object in the set of obstacle candidates, calculating a height of the object based on a vertical position of the object and the vertical position of the ground; and removing from the set of obstacle candidates those objects whose height is greater than a height threshold.

8. 8. The obstacle detection method according to claim 7, wherein the determining the vertical position of the ground surface includes calculating a statistical value of vertical positions indicated by the plurality of position data in the first subset as the vertical position of the ground surface.

9. The determination of the vertical position of the ground surface comprises: dividing the ground surface into two or more sections; 9. The obstacle detection method according to claim 7, further comprising: calculating, for each section of the ground, the vertical position of the section based on the vertical positions indicated by the plurality of position data collectively representing the section.

10. The division of the ground surface comprises: identifying a lowest vertical position and a highest vertical position indicated by the plurality of position data in the first subset; Dividing the range between the lowest vertical position and the highest vertical position into two or more height ranges at predetermined intervals; and dividing the first subset into two or more subsets representing different sections of the ground surface based on the two or more height ranges.

11. The method of claim 10, wherein the predetermined interval is proportional to the height threshold.

12. The detecting of the one or more objects comprises: Identifying a partial area of the target location based on an area of the ground in a planar view; 9. The obstacle detection method according to claim 7, further comprising: detecting the one or more objects from a plurality of sets of position data indicating positions included in the partial region.

13. acquiring a position dataset including a plurality of position data indicating three-dimensional positions of points of the target location; detecting a first subset from the location data set, the first subset including a plurality of the location data collectively representing a ground surface of the location of interest; determining a vertical position of the ground based on vertical positions indicated by the plurality of position data in the first subset; detecting one or more objects from the position dataset to generate a set of obstacle candidates; For each object in the set of obstacle candidates, calculating a height of the object based on a vertical position of the object and the vertical position of the ground; and excluding from the obstacle candidate set any object whose height is greater than a height threshold.

14. 14. The storage medium of claim 13, wherein the determining the vertical position of the ground surface includes calculating a statistical value of vertical positions indicated by the plurality of position data in the first subset as the vertical position of the ground surface.

15. The determination of the vertical position of the ground surface comprises: dividing the ground surface into two or more sections; 15. The storage medium according to claim 13, further comprising: calculating, for each section of the ground, the vertical position of the section based on the vertical positions indicated by the plurality of position data collectively representing the section.

16. The division of the ground surface comprises: identifying a lowest vertical position and a highest vertical position indicated by the plurality of position data in the first subset; Dividing the range between the lowest vertical position and the highest vertical position into two or more height ranges at predetermined intervals; and dividing the first subset into two or more subsets that represent different sections of the ground surface based on the two or more height ranges.

17. 17. The storage medium of claim 16, wherein the predetermined interval is proportional to the height threshold.

18. The detecting of the one or more objects comprises: Identifying a partial area of the target location based on an area of the ground in a planar view; Detecting the one or more objects from a plurality of sets of position data indicating positions included in the partial region.

Images