Obstacle detection device, obstacle detection method, and program

Abstract

An obstacle detection device (2000) acquires a three-dimensional map (40) representing a three-dimensional position of each of a plurality of locations in a facility (10). The obstacle detection device (2000) detects target surfaces from the three-dimensional map (40), which are surfaces for which a determination is to be made as to whether or not the surfaces are obstacle areas. The obstacle detection device (2000) detects, as an obstacle area, a target surface among the detected target surfaces that satisfies prescribed obstacle conditions. The obstacle conditions include an essential condition that is satisfied when the size of the angle formed between the target surface and a horizontal plane is at least equal to a first threshold value.

Description

[Technical field]

[0001] The present disclosure relates to a technique for detecting an obstacle. [Background technology]

[0002] Technologies for detecting obstacles have been developed. For example, Patent Document 1 discloses a technology in which a self-propelled traveling device is moved around a residence to identify walking obstacles in the residence, thereby allowing a manager or the like to grasp each walking obstacle in the residence. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-192040 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the invention of Patent Document 1, whether or not an object is an obstacle is determined based on the position and size of the object. The present invention has been made in consideration of such problems, and one of its objectives is to provide a new technique for detecting obstacles. [Means for solving the problem]

[0005] The obstacle detection device disclosed herein includes an acquisition unit that acquires a 3D map showing the 3D positions of a plurality of points in a facility, a surface detection unit that detects a target surface from the 3D map as a surface to be determined as an obstacle area, and an obstacle detection unit that detects, as the obstacle area, a target surface that satisfies a predetermined obstacle condition among the detected target surfaces. The obstacle condition includes a necessary condition that is satisfied when the magnitude of an angle formed by the target surface and a horizontal plane is equal to or larger than a first threshold value.

[0006] The obstacle detection method disclosed herein is executed by a computer. The obstacle detection method includes an acquisition step of acquiring a 3D map showing the 3D positions of a plurality of points in a facility, a surface detection step of detecting a target surface from the 3D map, which is a surface to be determined as an obstacle area, and an obstacle detection step of detecting, as the obstacle area, the target surface that satisfies a predetermined obstacle condition among the detected target surfaces. The obstacle condition includes a necessary condition that is satisfied when the magnitude of an angle formed between the target surface and a horizontal plane is equal to or larger than a first threshold value.

[0007] The computer-readable medium of the present disclosure stores a program that causes a computer to execute the obstacle detection method of the present disclosure. Effect of the Invention

[0008] According to the present disclosure, a new technique for detecting an obstacle is provided. [Brief description of the drawings]

[0009] [Figure 1] 3 is a diagram illustrating an example of an outline of the operation of the obstacle detection device of the first embodiment. FIG. [Diagram 2] 1 is a block diagram illustrating a functional configuration of an obstacle detection device according to a first embodiment. [Diagram 3] FIG. 2 is a block diagram illustrating a hardware configuration of a computer that realizes the obstacle detection device. [Figure 4] 4 is a flowchart illustrating a flow of processing executed by the obstacle detection device of the first embodiment. [Diagram 5] 2 is a block diagram illustrating a functional configuration of an obstacle detection device having an output unit according to the first embodiment. FIG. [Figure 6] 11 is a diagram illustrating an example of an outline of the operation of the obstacle detection device of the second embodiment. FIG. [Figure 7] FIG. 11 is a block diagram illustrating a functional configuration of an obstacle detection device according to a second embodiment. [Figure 8] 10 is a flowchart illustrating a process executed by an obstacle detection device according to a second embodiment. [Figure 9] FIG. 1 is a first diagram illustrating a detection range. [Figure 10] FIG. 2 is a second diagram illustrating the detection range. [Figure 11] FIG. 3 is a third diagram illustrating the detection range. [Figure 12] 11 is a block diagram illustrating a functional configuration of an obstacle detection device having an output unit according to a second embodiment. FIG. [Figure 13] FIG. 11 is a first diagram illustrating output information. [Figure 14] FIG. 2 is a second diagram illustrating output information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are given the same reference numerals, and duplicated explanations are omitted as necessary for clarity of explanation. Furthermore, unless otherwise specified, predetermined values ​​such as predetermined values ​​and threshold values ​​are stored in advance in a storage device accessible from a device that uses the values. Furthermore, unless otherwise specified, the storage unit is composed of one or more arbitrary number of storage devices.

[0011] [Embodiment 1] <Summary> Fig. 1 is a diagram illustrating an example of an outline of the operation of the obstacle detection device 2000 of the embodiment 1. Here, Fig. 1 is a diagram for facilitating understanding of the outline of the obstacle detection device 2000, and the operation of the obstacle detection device 2000 is not limited to that shown in Fig. 1.

[0012] The obstacle detection device 2000 detects an obstacle 30 that impedes the movement of a user 20 in a facility 10. The facility 10 may be an outdoor facility or an indoor facility. An outdoor facility 10 is, for example, a substation. An indoor facility 10 is, for example, the inside of a factory. The user 20 is a person who uses the obstacle detection device 2000 in the facility 10, for example, a worker performing work in the facility 10.

[0013] In order to detect the obstacle 30, the obstacle detection device 2000 uses a 3D map 40, which is data representing a 3D map of the facility 10. The 3D map 40 represents the 3D positions (3D coordinates) of each of a plurality of locations in the facility 10. The 3D map 40 is realized, for example, by point cloud data representing the 3D positions of the plurality of locations in the facility 10. The point cloud data has point data for each of the plurality of locations. The point data indicates the 3D coordinates of the corresponding location. The point cloud data is obtained, for example, by scanning the plurality of locations in the facility 10 using LiDAR (light detection and ranging) or a depth camera. However, as described later, the 3D map 40 is not limited to point cloud data.

[0014] The obstacle detection device 2000 detects one or more target surfaces, which are surfaces to be determined as to whether or not they represent an obstacle area, from the 3D map 40. An obstacle area is an area that represents a part or the whole of an obstacle 30. Furthermore, the obstacle detection device 2000 detects a target surface that represents an obstacle area from among the detected target surfaces.

[0015] Here, the obstacle detection device 2000 detects, as an obstacle area, an object surface that satisfies a predetermined obstacle condition among object surfaces detected from the 3D map 40. The obstacle condition is a condition that is satisfied when the object surface is an area that represents a part or all of the obstacle 30 (i.e., an obstacle area). The obstacle condition includes at least a condition that is satisfied when the magnitude of the angle between the object surface and the horizontal plane is equal to or greater than a predetermined threshold (hereinafter, a required condition). This threshold is preferably set to a value close to 90 degrees (for example, a value between 60 degrees and 90 degrees).

[0016] <Examples of effects> In facilities such as substations and factories, it is undesirable for workers to get injured by tripping over or bumping into obstacles. For example, the number of such injuries can be used as an evaluation index for a company regarding the safety of its work. In addition, if a worker is unable to work for a while due to injury, human resources will decrease. Therefore, it is important to prevent injuries caused by obstacles.

[0017] Therefore, the obstacle detection device 2000 detects a target surface from the 3D map 40, and detects an obstacle area, which is a surface that satisfies an obstacle condition, from the detected target surface. As described above, the obstacle condition includes at least a necessary condition that is satisfied when the magnitude of the angle between the target surface and a horizontal plane is equal to or larger than a threshold value.

[0018] Conceptually speaking, this prerequisite is met when the target surface is not a surface on which one lies but is a surface on which one stands to some extent. Compared to a surface on which one lies, such a standing surface has a higher probability of the user 20 tripping and falling on the surface, or of the user 20 bumping into the surface and getting injured. For example, if there is a step at the destination of the user 20, the user 20 may trip on the step and fall. On the other hand, if there is a gentle slope at the destination of the user 20, it can be said that the probability of the user 20 getting injured on the slope is low.

[0019] According to the obstacle detection device 2000, a new technique for detecting an obstacle is provided, which detects the surface on which the user is standing as an obstacle area. This also enables the user 20 to easily grasp the presence of an obstacle 30 that requires attention. This makes it possible to prevent injuries caused by the obstacle 30 in the facility 10.

[0020] The obstacle detection device 2000 of this embodiment will be described in more detail below.

[0021] <Example of functional configuration> 2 is a block diagram illustrating a functional configuration of the obstacle detection device 2000 of the first embodiment. The obstacle detection device 2000 includes an acquisition unit 2020, a surface detection unit 2040, and an obstacle detection unit 2060. The acquisition unit 2020 acquires a 3D map 40. The surface detection unit 2040 detects an object surface from the 3D map 40. The obstacle detection unit 2060 detects an object surface that satisfies an obstacle condition as an obstacle area.

[0022] <Example of hardware configuration> Each functional component of the obstacle detection device 2000 may be realized by hardware that realizes each functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). A case in which each functional component of the obstacle detection device 2000 is realized by a combination of hardware and software will be further described below.

[0023] 3 is a block diagram illustrating an example of a hardware configuration of a computer 500 that realizes the obstacle detection device 2000. The computer 500 is any computer. For example, the computer 500 is a stationary computer such as a PC (Personal Computer) or a server machine. In addition, for example, the computer 500 is a portable computer such as a smartphone or a tablet terminal. The portable computer is, for example, a user terminal carried by the user 20. The computer 500 may be a dedicated computer designed to realize the obstacle detection device 2000, or may be a general-purpose computer.

[0024] For example, by installing a specific application in the computer 500, each function of the obstacle detection device 2000 is realized by the computer 500. The application is configured with a program for realizing each functional component of the obstacle detection device 2000. The method of acquiring the program is arbitrary. For example, the program can be acquired from a storage medium (such as a DVD disk or a USB memory) in which the program is stored. Alternatively, the program can be acquired by downloading the program from a server device that manages the storage device in which the program is stored.

[0025] The computer 500 has a bus 502, a processor 504, a memory 506, a storage device 508, an input / output interface 510, and a network interface 512. The bus 502 is a data transmission path for the processor 504, the memory 506, the storage device 508, the input / output interface 510, and the network interface 512 to transmit and receive data to and from each other. However, the method of connecting the processor 504 and other components to each other is not limited to a bus connection.

[0026] The processor 504 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 506 is a main storage device realized using a random access memory (RAM) or the like. The storage device 508 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, or a read only memory (ROM) or the like.

[0027] The input / output interface 510 is an interface for connecting the computer 500 to an input / output device. For example, the input / output interface 510 is connected to an input device such as a keyboard and an output device such as a display device.

[0028] The network interface 512 is an interface for connecting the computer 500 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0029] The storage device 508 stores a program (a program for realizing the above-mentioned application) for realizing each functional component of the obstacle detection device 2000. The processor 504 reads this program into the memory 506 and executes it to realize each functional component of the obstacle detection device 2000.

[0030] The obstacle detection device 2000 may be realized by one computer 500, or may be realized by multiple computers 500. In the latter case, the configurations of the computers 500 do not need to be the same, and may be different from each other.

[0031] <Processing flow> Fig. 4 is a flowchart illustrating a process flow executed by the obstacle detection device 2000 of the embodiment 1. Note that the process flow shown in Fig. 4 is an example, and the process flow executed by the obstacle detection device 2000 is not limited to that shown in Fig. 4.

[0032] The acquisition unit 2020 acquires the 3D map 40 (S102). The surface detection unit 2040 detects a target surface from the 3D map 40 (S104). Steps S106 to S112 constitute a loop process L1. The loop process L1 is executed for each target surface.

[0033] In S106, the obstacle detection device 2000 determines whether or not the loop process L1 has been executed for all target faces. If the loop process L1 has already been executed for all target faces, the process in FIG. 4 ends. On the other hand, if there are target faces that have not yet been the target of the loop process L1, the obstacle detection device 2000 selects one of them. The target face selected here is referred to as target face i. After target face i is selected, S108 is executed.

[0034] The obstacle detection unit 2060 judges whether the target surface i satisfies the obstacle condition (S108). If the target surface i satisfies the obstacle condition (S108: YES), the obstacle detection unit 2060 detects the target surface i as an obstacle area (S110). Since S112 is the end of the loop process L1, S106 is executed next.

[0035] <About 3D Map 40> The three-dimensional map 40 is generated by measuring the three-dimensional position of each of the multiple locations in the facility 10 using various sensors. The sensor used to generate the three-dimensional map 40 is, for example, a distance measuring device capable of measuring the distance between the sensor and a measurement target. The distance measuring device measures the three-dimensional position of each of the multiple locations in real space using, for example, electromagnetic waves such as laser light. Specifically, the distance measuring device emits electromagnetic waves in multiple different directions, and receives reflected waves of each electromagnetic wave that are the electromagnetic waves reflected by an object. Then, the distance measuring device generates point data representing the three-dimensional position of the location that reflected the electromagnetic wave from the relationship between the emitted electromagnetic wave and the reflected wave. Point cloud data is generated as a collection of point data obtained for each of the multiple locations in the facility 10. Examples of such distance measuring devices include LiDAR (light detection and ranging) and depth cameras.

[0036] The sensor is not limited to a distance measuring device. For example, a camera that generates a two-dimensional captured image may be used as the sensor. In this case, for example, by applying SFM (Structure from Motion) to a plurality of captured images including the facility 10, the three-dimensional coordinates of each of a plurality of points in the facility 10 are identified, and a three-dimensional map 40 consisting of a collection of point data indicating the identified three-dimensional coordinates is generated. In this case, the plurality of captured images are obtained by capturing images of the facility 10 from different directions. Note that existing technology can be used for the technology of generating three-dimensional data of an imaged object using SFM.

[0037] The three-dimensional map 40 is not limited to the point cloud data described above as long as it has data (point data) that indicates the three-dimensional position of each of the multiple locations of the facility 10. For example, the three-dimensional map 40 may be composed of mesh data, surface data, or the like.

[0038] The sensor may be used in a fixed state at a specific position, or may be used while moving from that position. In the former case, measurement is performed using a sensor in a fixed state using, for example, a tripod. Alternatively, for example, a worker may remain at a specific position while holding or wearing the sensor and perform measurement using the sensor. After that, if measurement is to be performed at another position, the same operation is performed at the other position. In this way, measurement can be performed at each of a plurality of positions within the facility 10.

[0039] When the sensor is used while moving, for example, a worker holding or wearing the sensor moves through the facility 10 to measure the facility 10. In this case, the sensor is provided on a smartphone held by the worker, a wearable device worn by the worker, or the like. Alternatively, for example, a moving body (e.g., a flying or running robot) to which a sensor is attached may move through the facility 10 to measure the facility 10. By having the sensor repeatedly perform measurements while moving in this manner, measurements can be taken at multiple locations within the facility 10.

[0040] It should be noted that multiple measurements may be taken not only by changing the position of the sensor, but also by changing the measurement direction at the same position.

[0041] When measurements are performed at multiple positions and in multiple measurement directions, the measurement results obtained from each measurement are integrated to generate the 3D map 40. Note that existing technology can be used as a technology for integrating the measurement results performed at multiple positions and in multiple measurement directions to generate one piece of 3D data.

[0042] Here, the 3D map 40 may be data directly representing the measurement results of the distance measuring device or the results of the SFM, or may be data that has been subjected to a predetermined processing process on these data. The predetermined processing process may be, for example, a process of applying coordinate conversion to each point data so that the position of a specific point in the real space becomes the origin of the coordinate space of the 3D map 40, or a downsampling process that reduces the amount of point data by thinning it out at a predetermined distance interval.

[0043] Here, the 3D map 40 may reflect the results of measurements performed in real time (at the time when the obstacle detection device 2000 is being used). For example, assume that a user 20 of the obstacle detection device 2000 moves through the facility 10 while performing measurements with a sensor. In this case, the results of this measurement are reflected in the 3D map 40 at all times. Hereinafter, a device that updates the 3D map 40 is referred to as a map update device. For example, the map update device is realized by the obstacle detection device 2000, a user terminal carried by the user 20 of the obstacle detection device 2000, an arbitrary server device, or the like.

[0044] The map updating device detects three-dimensional data not included in the three-dimensional map 40 by comparing three-dimensional data obtained by measurements performed in real time with the three-dimensional map 40. For example, three-dimensional data representing objects temporarily placed in the facility 10 or objects newly installed in the facility 10 after the three-dimensional map 40 is generated is detected as three-dimensional data not included in the three-dimensional map 40. The map updating device updates the three-dimensional map 40 so as to include the three-dimensional data detected in this manner.

[0045] <Acquisition of 3D map 40: S102> The acquisition unit 2020 acquires the 3D map 40 (S102). There are various methods for the acquisition unit 2020 to acquire the 3D map 40. For example, the 3D map 40 is stored in advance in a storage unit accessible from the obstacle detection device 2000. The acquisition unit 2020 acquires the 3D map 40 by accessing this storage unit.

[0046] Alternatively, for example, the 3D map 40 may be input to the obstacle detection device 2000 in response to a user operation. For example, the user 20 connects a portable storage unit (such as a memory card) in which the 3D map 40 is stored to the obstacle detection device 2000, and inputs the 3D map 40 from the storage unit to the obstacle detection device 2000.

[0047] Alternatively, for example, the acquisition unit 2020 may acquire the 3D map 40 by receiving the 3D map 40 transmitted from another device. For example, the other device is a device that generates the 3D map 40.

[0048] <Detection of target surface: S104> The surface detection unit 2040 detects a target surface from the 3D map 40 (S104). A method for detecting a target surface will be described below.

[0049] For example, the surface detection unit 2040 clusters each piece of point data shown on the 3D map 40 for each piece of point data that represents the same surface. Then, the surface detection unit 2040 treats the surface represented by each cluster as a target surface. In other words, multiple pieces of point data included in one cluster are treated as a group of point data that constitute one target surface.

[0050] There are various methods for clustering point data for each same surface. For example, the surface detection unit 2040 calculates the normal vector of each point data included in the 3D map 40. Here, the normal vector of the point data can be calculated as the normal vector of a triangular surface formed by the point data and two other point data adjacent to the point data. Note that, when the 3D map 40 is mesh data, the normal vector of the point data can be calculated as the normal vector of any one of the surfaces including the point data.

[0051] The surface detection unit 2040 clusters the point data based on the direction of the normal vector. Specifically, when the difference between the direction of the normal vector of a certain point data and the direction of the normal vector of other point data that is close to the point data (for example, the distance is less than a predetermined threshold value), the surface detection unit 2040 places these point data in the same cluster. This process makes it possible to capture multiple small surfaces that are close to each other and have similar orientations as one large surface.

[0052] Here, the 3D map 40 may contain objects with curved surfaces, such as pipes and tanks. For example, the surface detection unit 2040 may detect cylindrical regions by performing cylindrical fitting, and treat the side surfaces of the cylindrical regions as target surfaces. Here, the cylindrical fitting can be realized using an algorithm such as RANSAC.

[0053] When performing cylindrical fitting, the surface detection unit 2040 detects an area in the 3D map 40 that is highly likely to have a cylindrical shape, and performs cylindrical fitting on that area. This can be achieved, for example, by using semantic segmentation. Specifically, the surface detection unit 2040 performs semantic segmentation on point data included in the 3D map 40, thereby clustering the point data for each piece of point data representing the same object. As a result, the point data is divided into clusters for each piece of point data representing the same object. In addition, the type of object represented by each cluster (wall, tank, pipe, steel structure, etc.) is identified.

[0054] Furthermore, for each cluster of a type (such as a tank or a pipe) that is the target of cylindrical fitting, the surface detection unit 2040 performs cylindrical fitting on point data included in the cluster to identify a target surface corresponding to the cluster. Note that the type to be the target of cylindrical fitting is determined in advance.

[0055] The surface detection unit 2040 may use both a method of clustering point data based on normal vectors and a method of using cylindrical fitting. In this case, for example, the surface detection unit 2040 first identifies an area to be subjected to cylindrical fitting, and performs cylindrical fitting on the area to detect the side of the cylindrical area as a target surface. After that, the surface detection unit 2040 performs clustering using normal vectors on each piece of point data that is not included in the cylindrical area among the point data included in the 3D map 40, thereby detecting the remaining target surfaces.

[0056] Alternatively, for example, the surface detection unit 2040 may divide the point data into clusters for the same object by semantic segmentation, and then detect the target surface for each object. In this case, the surface detection unit 2040 first applies semantic segmentation to the point data included in the 3D map 40 to divide the point data representing the same object into clusters. Next, the surface detection unit 2040 detects the target surface using cylindrical fitting for the cluster of the object type that is the target of cylindrical fitting. Furthermore, for each cluster of the other types of objects, the surface detection unit 2040 performs clustering using the normal vector described above for multiple point data included in the same cluster. As a result, multiple target surfaces are detected from the cluster of the same object.

[0057] Alternatively, for example, when the 3D map 40 is composed of point cloud data, the surface detection unit 2040 may detect the target surface by converting the point cloud data into mesh data. As a result, data representing a plurality of surfaces is obtained from the point cloud data. For example, the surface detection unit 2040 may treat each surface obtained by meshing as a target surface. However, in order to treat a surface of a certain size as a target surface, the surface detection unit 2040 may downsample the point cloud data and then convert it into mesh data.

[0058] In the above description, the target surface is detected using each point data included in the 3D map 40. That is, the target surface is detected for the entire area represented by the 3D map 40. However, the surface detection unit 2040 may detect the target surface for a part of the area in the 3D map 40 (in other words, using a part of the point data included in the 3D map 40). The part of the area may be specified by the user 20 or may be determined by the obstacle detection device 2000. An example of a case in which the detection range of the target surface is determined by the obstacle detection device 2000 is described in the second embodiment described later.

[0059] Furthermore, when measurements are performed on the facility 10 in real time, as described above, three-dimensional data that is not included in the three-dimensional map 40 may be obtained. The surface detection unit 2040 may further detect a target surface from the three-dimensional data thus obtained.

[0060] For example, the surface detection unit 2040 identifies three-dimensional data representing an object not shown in the three-dimensional map 40 by taking the difference between three-dimensional data obtained by measurements performed in real time and the three-dimensional map 40. In this way, the three-dimensional data representing an object not shown in the three-dimensional map 40 among the three-dimensional data obtained by measurements performed in real time is called temporary three-dimensional data. The temporary three-dimensional data is, for example, three-dimensional data representing an object such as equipment or a cable temporarily placed in the facility 10. The surface detection unit 2040 detects the target surface from the temporary three-dimensional data in a manner similar to the method of detecting the target surface from the three-dimensional map 40 described above.

[0061] It is preferable that the obstacle areas detected from the 3D map 40 and the obstacle areas detected from the temporary 3D data are distinguishable from each other. Therefore, for example, when outputting information about obstacle areas as described later, it is possible to determine whether each obstacle area was detected from the 3D map 40 or the temporary 3D data.

[0062] <Obstacle area detection: S108, S110> The obstacle detection unit 2060 detects an object surface that satisfies the obstacle condition as an obstacle area (S108, S110). As described above, the obstacle condition includes at least a necessary condition that is satisfied when the angle between the object surface and the horizontal plane is equal to or greater than a threshold value.

[0063] The size of the angle between the target surface and the horizontal plane can be grasped, for example, by using the normal vector of the target surface. Specifically, the closer the direction of the normal vector of the target surface is to the horizontal direction, the larger the angle between the target surface and the horizontal plane becomes.

[0064] Therefore, for example, the obstacle detection unit 2060 may use a condition that "the angle between the normal direction of the target surface and the horizontal direction is equal to or smaller than a threshold value" as a specific necessary condition. This threshold value is preferably set to a value close to 0 degrees (for example, a value between 0 degrees and 30 degrees). In this case, the obstacle detection unit 2060 calculates the normal vector of the target surface, and calculates the angle between the calculated normal vector and the horizontal direction. The obstacle detection unit 2060 determines whether or not the magnitude of the calculated angle is equal to or smaller than a threshold value. A target surface whose calculated angle is equal to or smaller than the threshold value is a target surface that satisfies the necessary condition. Here, an existing method can be used to calculate the normal vector of the surface.

[0065] In addition, there are cases where the target surface is a curved surface, such as when the side surface of a cylindrical region is used as the target surface. In this case, for example, the normal line of the target surface that is closest to the horizontal direction is used to determine whether the essential condition is satisfied. In other words, for the normal line that is closest to the horizontal direction, if the angle between the normal line direction and the horizontal direction is equal to or smaller than a threshold value, it is determined that the essential condition is satisfied.

[0066] The obstacle conditions may further include conditions other than the above-mentioned essential conditions. Hereinafter, among the conditions included in the obstacle conditions, the conditions other than the essential conditions will be referred to as additional conditions.

[0067] For example, the additional condition is determined so that a target surface that is relatively low in height from the floor surface is detected as an obstacle area. Since an object that is low in height from the floor surface is unlikely to be in the field of view of the user 20, it is considered that the object is likely to cause a fall.

[0068] Therefore, for example, a condition that "the height of the target surface from the floor surface is equal to or less than a threshold value" is defined as an additional condition. In this case, the obstacle detection unit 2060 judges whether or not both the essential condition that "the angle between the target surface and the horizontal plane is equal to or more than a threshold value" and the additional condition that "the height of the target surface from the floor surface is equal to or less than a threshold value" are satisfied for each target surface. Then, for a target surface that satisfies both of these, it is determined that the obstacle condition is satisfied. On the other hand, for a target surface that does not satisfy either one or both of the essential condition and the additional condition, it is determined that the obstacle condition is not satisfied.

[0069] The floor surface here means the floor surface on which the object including the target surface is located. The facility 10 may have multiple floors and scaffolding in various locations. In such cases, the "height from the floor surface of the target surface" refers to the height from the floor surface of the floor or scaffolding on which the object including the target surface is placed.

[0070] Therefore, for example, the obstacle detection unit 2060 first identifies the floor surface on which the object including the target surface is located by using the 3D map 40. Then, the obstacle detection unit 2060 calculates the height of the target surface from the floor surface by subtracting the z coordinate of the floor surface from the maximum z coordinate of the z coordinates of each point included in the target surface. However, the method of calculating the height of the target surface from the floor surface is not limited to this method.

[0071] Other conditions that can be used as additional conditions include, for example, "located at a location where the height from the floor surface is equal to or less than a threshold value." For example, it is considered that the probability that the user 20 will collide with an object that is installed at a position 2 m or higher above the floor surface is low. Therefore, in this way, target surfaces that are located at a high position where the user 20 will not collide with them are not treated as obstacle areas.

[0072] The obstacle condition may include multiple additional conditions. In this case, a target surface that satisfies the essential condition and all the additional conditions is detected as an obstacle region.

[0073] <Detection result output> When an obstacle area is detected, the obstacle detection device 2000 preferably outputs information related to the detected obstacle area. The information output here is called output information. Hereinafter, the functional configuration unit that outputs the output information is called an output unit.

[0074] 5 is a block diagram illustrating a functional configuration of the obstacle detection device 2000 having an output unit. The output unit 2080 outputs output information 80. The output destination of the output information 80 is, for example, a user terminal used by the user 20. By outputting the output information 80 to the user terminal, the user 20 can easily recognize the presence of the obstacle 30. Therefore, for example, when the user 20 moves around the facility 10, the safety of the user 20 when using the facility 10 is improved. Note that the output destination of the output information 80 is arbitrary and is not limited to the user terminal.

[0075] Various information is indicated by the output information 80. For example, the output information 80 indicates the type (step, equipment, desk, cable, etc.), size, or position of the obstacle 30 represented by the obstacle area. Furthermore, in the case where an obstacle area is also detected from temporary three-dimensional data as described above, the output information 80 may indicate, for each obstacle area, whether it was detected from the three-dimensional map 40 or the temporary three-dimensional data.

[0076] There are various modes for outputting the output information 80. For example, the output unit 2080 transmits the output information 80 to an arbitrary device such as a user terminal. Alternatively, for example, the output unit 2080 may store the output information 80 in an arbitrary storage unit. Alternatively, for example, the output unit 2080 may display the output information 80 on an arbitrary display device.

[0077] [Embodiment 2] <Summary> Fig. 6 is a diagram illustrating an example of an outline of the operation of the obstacle detection device 2000 of the embodiment 2. Here, Fig. 6 is a diagram for facilitating understanding of the outline of the obstacle detection device 2000, and the operation of the obstacle detection device 2000 is not limited to that shown in Fig. 6.

[0078] The obstacle detection device 2000 of the second embodiment detects an obstacle area from a predetermined range around the user 20 when the user 20 moves through the facility 10. The predetermined range within which an obstacle area is detected is called a detection range.

[0079] More specifically, the obstacle detection device 2000 acquires position information 50 indicating the position of the user 20. For example, the position information 50 indicates the position of a user terminal carried by the user 20. The user terminal is, for example, a smartphone, a tablet terminal, or a wearable terminal (such as a glasses-type terminal or a watch-type terminal). However, as will be described later, the position indicated by the position information 50 is not limited to the position of the user terminal.

[0080] The obstacle detection device 2000 determines a detection range based on the position of the user 20 represented by the position information 50. Furthermore, the obstacle detection device 2000 detects an object surface from the detection range. Then, the obstacle detection device 2000 detects an object surface that satisfies an obstacle condition as an obstacle area.

[0081] <Examples of effects> The obstacle detection device 2000 of the second embodiment detects an obstacle area existing within a detection range around a user 20 (such as a worker performing work in the facility 10) by using the three-dimensional map 40 and the position information 50. In this way, the user 20 can easily recognize the obstacles 30 existing around the user 20. Therefore, it is possible to prevent the user 20 from being injured by the obstacles 30 in the facility 10.

[0082] The obstacle detection device 2000 of this embodiment will be described in more detail below.

[0083] <Example of functional configuration> 7 is a block diagram illustrating a functional configuration of an obstacle detection device 2000 according to the second embodiment. The acquisition unit 2020 according to the second embodiment further acquires position information 50. The obstacle detection device 2000 according to the second embodiment further includes a detection range determination unit 2100. The detection range determination unit 2100 determines a detection range using the 3D map 40 and the position information 50. The surface detection unit 2040 according to the second embodiment detects a target surface from the detection range.

[0084] <Example of hardware configuration> The hardware configuration of the obstacle detection device 2000 of the second embodiment is similar to that of the obstacle detection device 2000 of the first embodiment, and is shown in, for example, Fig. 3. However, the storage device 508 of the second embodiment stores programs for realizing each function of the obstacle detection device 2000 of the second embodiment.

[0085] <Processing flow> Fig. 8 is a flowchart illustrating a process flow executed by the obstacle detection device 2000 of the embodiment 2. Note that the process flow shown in Fig. 8 is an example, and the process flow executed by the obstacle detection device 2000 is not limited to that shown in Fig. 8.

[0086] The acquisition unit 2020 acquires the 3D map 40 (S202). Steps S204 to S220 constitute a loop process L2. The loop process L2 is repeatedly executed until a predetermined end condition is satisfied.

[0087] In S204, the obstacle detection device 2000 determines whether or not a termination condition is satisfied. If the termination condition is satisfied, the process in Fig. 8 ends. On the other hand, if the termination condition is not satisfied, S206 is executed next.

[0088] The acquisition unit 2020 acquires the position information 50 (S206). The detection range determination unit 2100 determines the detection range (S208) using the 3D map 40 and the position information 50. The surface detection unit 2040 detects a target surface from within the detection range (S210).

[0089] S212 to S218 constitute a loop process L3. The loop process L3 is executed for each target surface. In S212, the obstacle detection device 2000 determines whether or not the loop process L3 has been executed for all target surfaces. If the loop process L3 has already been executed for all target surfaces, S220 is executed next. Since S220 is the end of the loop process L2, S204 is executed next.

[0090] On the other hand, if there are target faces that have not yet been subjected to the loop process L3, the obstacle detection device 2000 selects one of them. The target face selected here is referred to as target face i. After target face i is selected, S214 is executed.

[0091] The obstacle detection unit 2060 determines whether the target surface i satisfies the obstacle condition (S214). If the target surface i satisfies the obstacle condition (S214: YES), the obstacle detection unit 2060 detects the target surface i as an obstacle area (S216). Since S218 is the end of the loop process L3, S212 is executed next.

[0092] In this way, by repeatedly performing the process of “acquiring position information 50 and detecting an obstacle area from a detection range determined based on the acquired position information 50”, it is possible to determine whether or not an obstacle 30 is present around the user 20 after the user 20 moves, depending on the user's movement.

[0093] Here, various conditions can be used as the termination condition for the loop process L2. For example, the termination condition is that "a specified input operation is performed." Another example of the termination condition is that "the user 20 goes outside the range shown on the 3D map 40." For example, if the 3D map 40 shows the entire facility 10, this termination condition is satisfied when the user 20 goes outside the facility 10.

[0094] <About location information 50> The location information 50 is information that represents the location of the user 20. Here, various information can be adopted as the information that represents the location of the user 20. For example, the location information 50 represents the location of the user terminal of the user 20. Here, various existing technologies can be used as a technology for identifying the location of a specific terminal in a specific place (here, in the facility 10). For example, the location of the user terminal can be identified by detecting the location of a location sensor such as a GPS (Global Positioning System) sensor provided in the user terminal. In addition, for example, the location of the user terminal may be identified based on communication between the user terminal and a beacon or RFID sensor provided in the facility 10.

[0095] Alternatively, the position of the user terminal may be specified by using measurement data obtained from a sensor provided in the user terminal. The measurement data used to specify the position of the user terminal may be, for example, a two-dimensional captured image obtained from a two-dimensional camera, or three-dimensional data obtained from a depth camera or LiDAR. Note that existing technology may be used to specify the position where the measurement was performed from the measurement data.

[0096] When using captured images as measurement data, for example, captured images obtained by capturing images at various combinations of the position and imaging direction of the facility 10 are prepared in advance as reference images. Then, by identifying an image that matches the captured image obtained from the user terminal from among the reference images, the position corresponding to the reference image that matches the captured image can be identified as the position of the user 20.

[0097] When three-dimensional data is used as the measurement data, for example, three-dimensional data that matches the three-dimensional data obtained from the user terminal is detected from the three-dimensional map 40. The three-dimensional data detected in this manner represents the three-dimensional area measured by the sensor. In addition, the measurement direction can be identified based on the measurement data. Then, the position of the sensor (i.e., the position of the user terminal) can be identified based on the measured three-dimensional area and the measurement direction.

[0098] Alternatively, for example, the user terminal may be provided with a self-location estimation function such as that employed in autonomous mobile robots, and the position of the user terminal may be identified by this self-location estimation.

[0099] A user terminal does not have to be used to identify the location of the user 20. For example, the location of the user 20 may be identified by detecting the user 20 using a monitoring sensor (such as a camera) provided in the facility 10.

[0100] The generation of the position information 50 may be performed by the obstacle detection device 2000, or may be performed by a device (such as a user terminal or a server device) other than the obstacle detection device 2000. The device that generates the position information 50 acquires information (e.g., measurement data) necessary to identify the position of the user 20, and generates the position information 50 by identifying the position of the user 20 using the acquired information.

[0101] <Getting location information 50: S206> The acquiring unit 2020 acquires the location information 50 (S206). The method of acquiring the location information 50 is arbitrary. For example, the acquiring unit 2020 acquires the location information 50 by receiving the location information 50 transmitted from a device that generated the location information 50. Alternatively, for example, the acquiring unit 2020 may acquire the location information 50 by accessing a storage unit in which the location information 50 is stored.

[0102] <Detection range determination: S208> The detection range determination unit 2100 determines the detection range by using the 3D map 40 and the position information 50 (S208). To this end, the detection range determination unit 2100 first identifies the position of the user 20 on the 3D map 40 (the position on the 3D map 40 corresponding to the position of the user 20 represented by the position information 50). Here, it is assumed that the coordinate system of the 3D map 40 and the coordinate system of the coordinates of the user 20 represented by the position information 50 are different from each other. In this case, the detection range determination unit 2100 identifies the position of the user 20 on the 3D map 40 by converting the coordinates of the user 20 represented by the position information 50 into coordinates on the coordinate system of the 3D map 40. Note that the relationship between the coordinate system of the 3D map 40 and the coordinate system of the coordinates of the user 20 represented by the position information 50 is determined in advance.

[0103] Here, a method of converting coordinates in one coordinate system into coordinates in another coordinate system can use an existing method. For example, a transformation matrix for converting coordinates from the coordinate system of the user 20 represented by the position information 50 into the coordinate system of the 3D map 40 is determined in advance. In this case, the detection range determination unit 2100 calculates coordinates representing the position of the user 20 on the 3D map 40 by applying this transformation matrix to the coordinates of the user 20 shown in the position information 50.

[0104] On the other hand, it is assumed that the coordinate system of the 3D map 40 and the coordinate system of the coordinates of the user 20 indicated by the position information 50 are the same. In this case, the position of the user 20 on the 3D map 40 is represented by the coordinates of the user 20 represented by the position information 50. Therefore, the detection range determination unit 2100 can use the position of the user 20 indicated in the position information 50 as the position of the user 20 on the 3D map 40 as it is.

[0105] After the position of the user 20 on the 3D map 40 is identified, the detection range determination unit 2100 determines the detection range on the 3D map 40 based on the position of the user 20 on the 3D map 40. The detection range is a range around the user 20, which is a range within which an obstacle area is to be detected.

[0106] There are various methods for determining the detection range. For example, the detection range determination unit 2100 determines a range of a predetermined shape and a predetermined size centered on the position of the user 20 as the detection range. The detection range is determined, for example, as a range on a two-dimensional plane obtained by viewing the three-dimensional map 40 in a planar view. Note that the planar view referred to here may be a view of the three-dimensional map 40 looking vertically downward, or a view of the floor surface on which the user 20 is located in the opposite direction to the normal vector of the map. Note that the floor surface referred to here also includes the ground. In other words, when the facility 10 is an outdoor facility, the floor surface on which the user 20 is located means the ground surface on which the user 20 is located.

[0107] The shape of the detection range may be any shape, such as a circle or a rectangle. The shape of the detection range may be fixed in advance, or may be changeable by an input operation. Similarly, the size of the detection range may be fixed in advance, or may be changeable by an input operation.

[0108] 9 is a first diagram illustrating an example of a detection range. A detection range 60 is a range of a radius d centered on the position 25 of the user 20 in a plan view.

[0109] The detection range may be determined based on the direction of the user 20 (the direction of gaze or the direction of movement). For example, the detection range is determined as a range of a predetermined angle with the direction of the user 20 as the reference direction. FIG. 10 is a second diagram illustrating an example of the detection range. In FIG. 10, the direction of the user 20 is represented by an arrow 70. The detection range 60 is determined as a sector-shaped range with a radius of d in a plan view and a range of ±θ° with the direction of the user 20 as the reference.

[0110] In this way, by narrowing the range in which obstacles 30 are detected based on the direction of user 20, it is possible to detect objects that are likely to hinder the movement of user 20 while reducing the time and computer resources required for the detection.

[0111] Here, various techniques can be used to detect the gaze direction and movement direction of the user 20. For example, the measurement direction of a sensor provided in the user terminal used by the user 20 can be identified, and the measurement direction can be treated as the gaze direction and movement direction of the user. Here, existing techniques can be used to identify the measurement direction based on measurement data obtained from the sensor. In addition, for example, the movement direction of the user 20 may be identified from the time change in the position of the user 20 indicated by each of the multiple position information 50 acquired so far.

[0112] The detection range may be determined so that the area is wider in a direction closer to the direction of the user 20. Fig. 11 is a third diagram illustrating an example of the detection range. In the example of Fig. 11, the distance from the user 20 to the boundary of the detection range is longer in a direction closer to the direction of the user 20. In this way, it is possible to detect the obstacle 30 in a wider range in a direction where the user 20 is more likely to move.

[0113] The detection range may be determined not only in the horizontal direction but also in the vertical direction. As another example, the detection range determination unit 2100 may determine, as the detection range, a range above, below, or both above and below the position of the user 20, the distance from the position of the user 20 being equal to or less than a predetermined value. This predetermined value may be the same above and below, or may be different. As another example, when the facility 10 has multiple floors, it is preferable that the detection range determination unit 2100 identifies the floor on which the user 20 is located and limits the detection range to only that floor.

[0114] <Detection of target surface: S210> The surface detection unit 2040 detects a target surface from the detection range 60 (S210). Specifically, the surface detection unit 2040 detects the target surface by using point data included in the detection range 60, among the point data shown on the 3D map 40. Note that the method of detecting the target surface by using the point data is as described in the first embodiment.

[0115] <Obstacle area detection: S214, S216> The obstacle detection unit 2060 determines whether each object surface detected in S210 satisfies an obstacle condition, and detects the object surface that satisfies the obstacle condition as an obstacle region (S214, S216). The method of detecting the object surface that satisfies the obstacle condition as an obstacle region is as described in the first embodiment.

[0116] <Detection result output> When an obstacle area is detected, the obstacle detection device 2000 of the second embodiment may output the output information 80, similar to the obstacle detection device 2000 of the first embodiment. In this case, the obstacle detection device 2000 of the second embodiment has the output unit 2080 described in the first embodiment. FIG. 12 is a diagram illustrating an example of the functional configuration of the obstacle detection device 2000 of the second embodiment having the output information 80.

[0117] Various information is indicated by the output information 80 of the embodiment 2. For example, like the output information 80 of the embodiment 1, the output information 80 of the embodiment 2 indicates the type (step, equipment, desk, cable, etc.), size, position, etc. of the obstacle 30 represented by the obstacle region.

[0118] Alternatively, for example, the output information 80 may represent a warning to inform the user 20 of the presence of an obstacle 30. In this case, it is preferable that the output information 80 includes a message representing the warning. FIG. 13 is a first diagram illustrating the output information 80. FIG. 13 shows a message 90 representing the contents of the output information 80. The message 90 indicates the position of the obstacle (diagonally forward to the left), the size of the obstacle (height: approximately 20 cm), and the type of the obstacle (step). The message 90 is displayed, for example, on a display device of a user terminal carried by the user 20.

[0119] The output information 80 may further include a three-dimensional map of the periphery of the obstacle area. FIG. 14 is a second diagram illustrating the output information 80. FIG. 14 shows a screen 100 representing the contents of the output information 80. The screen 100 displays a portion of the three-dimensional map 40 that is included in the field of view of the user 20. In addition, the obstacle area 130 is highlighted on the screen 100. Specifically, the obstacle area 130 is shaded with diagonal lines. In addition, the screen 100 displays a message 110 for notifying the user that a cable exists as the obstacle 30, and an arrow 120 indicating the direction in which the obstacle 30 is located. Note that it is preferable that the range of the three-dimensional map 40 displayed on the screen 100 is changed in accordance with a change in the field of view of the user 20 (a change in the position or direction of the user 20).

[0120] There are various methods for determining the portion of the 3D map 40 included in the field of view of the user 20. For example, the output unit 2080 treats the portion of the 3D map 40 included in a predetermined range that represents the field of view when looking in the direction of movement of the user 20 from the position of the user 20 as the portion included in the field of view of the user 20. In addition, for example, when the user 20 uses a camera (hereinafter, a user camera) provided on a user terminal, the output unit 2080 may treat the portion of the 3D map 40 included in the imaging range of the user camera or a predetermined range therein as the portion of the 3D map 40 included in the field of view of the user 20.

[0121] Here, it is assumed that the scenery in the field of view of the user 20 is captured using the user camera. In this case, the image obtained from the user camera may be displayed on the screen 100 instead of the 3D map 40. In this case, the output unit 2080 superimposes the various displays shown in FIG. 14 on the image obtained from the user camera. Specifically, the output unit 2080 identifies an area corresponding to an obstacle area detected from the detection range from the image, and highlights the area. In addition, the output unit 2080 superimposes a message 110 and an arrow 120 indicating the direction of the area corresponding to the obstacle area on the image of the user camera.

[0122] Also, assume that the user 20 is wearing a glasses-type device, and the glasses-type device is configured to display any information on a transparent lens that functions as a display device, thereby superimposing the information on the actual scenery seen through the lens. In this case, the output unit 2080 superimposes information about the obstacle 30 on the surrounding scenery seen by the user 20 through the glasses-type device. Specifically, the output unit 2080 identifies an area of ​​the lens that corresponds to an obstacle area, and displays an image indicating that the area is an obstacle area (for example, an image of a specific color covering the obstacle area, etc.) on the area. Also, the output unit 2080 displays a message 110 and an arrow 120 on the lens.

[0123] The output information 80 is not limited to visual information. For example, the output information 80 may be auditory information. That is, the output unit 2080 may output a message indicating a warning or a message indicating the position of an obstacle as a voice message. In this case, the user 20 can recognize the obstacle by listening to the voice message output from a speaker provided in the user terminal or the facility 10.

[0124] There are various ways to output the output information 80. For example, the output unit 2080 transmits the output information 80 to a user terminal. Alternatively, for example, the output unit 2080 may output the output information 80 to a storage unit accessible from the user terminal. In this case, the user terminal acquires the output information 80 by accessing this storage unit. Alternatively, for example, when the output information 80 is a voice message, the output information 80 may be output to a speaker provided in the facility 10, so that the speaker outputs the voice message. Here, when a plurality of speakers are provided in the facility 10, it is preferable that, for example, the output unit 2080 identifies the speaker closest to the position of the user 20 and outputs the output information 80 to the speaker.

[0125] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0126] In the above example, the program includes a set of instructions (or software code) for making the computer perform one or more functions described in the embodiment when the program is loaded into the computer. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagating signals.

[0127] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) an acquisition unit that acquires a 3D map representing a 3D position of each of a plurality of points in the facility; a surface detection unit that detects a target surface from the three-dimensional map, the target surface being a surface to be determined as being an obstacle area; an obstacle detection unit that detects, from among the detected object surfaces, the object surface that satisfies a predetermined obstacle condition as the obstacle region; An obstacle detection device, wherein the obstacle condition includes a necessary condition that is satisfied when the magnitude of an angle formed between the target surface and a horizontal plane is equal to or greater than a first threshold value. (Appendix 2) The obstacle detection device according to claim 1, wherein the obstacle condition further includes a condition that a height of the target surface from a floor surface is equal to or less than a second threshold value. (Appendix 3) the three-dimensional map has point data indicating a three-dimensional position of each of a plurality of points of the facility; 3. The obstacle detection device according to claim 1, wherein the surface detection unit generates a plurality of clusters by clustering the point data, and detects, for each cluster, the target surface constituted by the point data contained in that cluster. (Appendix 4) The obstacle detection device described in Appendix 3, wherein the surface detection unit calculates a normal direction for each of the multiple point data, and classifies the multiple point data whose difference in normal direction is less than or equal to a threshold and whose distance between them is less than or equal to a third threshold into the same cluster. (Appendix 5) 5. The obstacle detection device according to claim 3, wherein the surface detection unit detects a plurality of point data representing a cylindrical area by applying cylindrical fitting to the point data included in the three-dimensional map, and detects a side surface of the cylinder represented by the area as the target surface. (Appendix 6) The surface detection unit applying semantic segmentation to the plurality of point data included in the three-dimensional map, thereby classifying the point data representing the same object into the clusters; 6. The obstacle detection device according to claim 3, wherein for each cluster representing the same object, one or more target surfaces are detected from that cluster. (Appendix 7) 7. The obstacle detection device described in claim 6, wherein a cylindrical area constituted by the plurality of point data included in the cluster is detected by performing cylindrical fitting on the plurality of point data classified into the cluster of a cylindrical object, and the side surface of the cylinder represented by the area is detected as the target surface. (Appendix 8) The acquisition unit acquires location information representing a location of a user, a detection range determination unit that uses the three-dimensional map and the position information to determine a detection range in which the obstacle area is to be detected from an area around the user's position on the three-dimensional map, The obstacle detection device according to any one of claims 1 to 7, wherein the surface detection unit detects the target surface from the detection range. (Appendix 9) 9. An obstacle detection device according to any one of claims 1 to 8, comprising an output unit that outputs output information indicating information related to the detected obstacle area. (Appendix 10) The obstacle detection device described in Appendix 9, wherein the output information includes a display highlighting the obstacle area, a display showing the position or direction of the obstacle area, a display showing the type of obstacle represented by the obstacle area, or a screen including two or more of these. (Appendix 11) 1. A computer implemented method for obstacle detection, comprising: acquiring a three-dimensional map representing a three-dimensional position of each of a plurality of points of the facility; a surface detection step of detecting a target surface from the three-dimensional map, the target surface being a surface to be determined as being an obstacle area; an obstacle detection step of detecting, as the obstacle region, the object surface that satisfies a predetermined obstacle condition among the detected object surfaces; The obstacle detection method, wherein the obstacle condition includes a necessary condition that is satisfied when the magnitude of an angle between the target surface and a horizontal plane is equal to or greater than a first threshold value. (Appendix 12) The obstacle detection method according to claim 11, wherein the obstacle condition further includes a condition that a height of the target surface from a floor surface is equal to or less than a second threshold value. (Appendix 13) the three-dimensional map has point data indicating a three-dimensional position of each of a plurality of points of the facility; 13. The obstacle detection method according to claim 11, wherein in the surface detection step, the point data is clustered to generate a plurality of clusters, and for each cluster, the target surface constituted by the point data included in that cluster is detected. (Appendix 14) The obstacle detection method described in Appendix 13, wherein in the surface detection step, a normal direction is calculated for each of the multiple point data, and multiple point data whose difference in normal direction is equal to or less than a threshold and whose distance between them is equal to or less than a third threshold are classified into the same cluster. (Appendix 15) 15. The obstacle detection method according to claim 13, wherein in the surface detection step, a plurality of point data representing a cylindrical area are detected by applying cylindrical fitting to the point data included in the three-dimensional map, and a side surface of the cylinder represented by the area is detected as the target surface. (Appendix 16) In the surface detection step, applying semantic segmentation to the plurality of point data included in the three-dimensional map, thereby classifying the point data representing the same object into the clusters; 16. The method of any one of claims 13 to 15, further comprising detecting, for each cluster representing the same object, one or more target surfaces from that cluster. (Appendix 17) 17. The obstacle detection method according to claim 16, further comprising: detecting a cylindrical area constituted by the plurality of point data included in the cluster by performing cylindrical fitting on the plurality of point data classified into the cluster of a cylindrical object; and detecting a side surface of the cylinder represented by the area as the target surface. (Appendix 18) In the acquiring step, location information representing a location of the user is acquired; a detection range determination step of determining a detection range in which the obstacle area is to be detected from an area around the user's position on the three-dimensional map by using the three-dimensional map and the position information; 18. The obstacle detection method according to claim 11, wherein in the surface detection step, the target surface is detected from the detection range. (Appendix 19) 19. The obstacle detection method according to any one of claims 11 to 18, comprising an output step of outputting output information indicative of information regarding the detected obstacle region. (Appendix 20) The obstacle detection method described in Appendix 19, wherein the output information includes a display highlighting the obstacle area, a display showing the position or direction of the obstacle area, a display showing the type of obstacle represented by the obstacle area, or a screen including two or more of these. (Appendix 21) On the computer, acquiring a three-dimensional map representing a three-dimensional position of each of a plurality of points of the facility; a surface detection step of detecting a target surface from the three-dimensional map, the target surface being a surface to be determined as being an obstacle area; an obstacle detection step of detecting, as the obstacle area, the object surface that satisfies a predetermined obstacle condition among the detected object surfaces; The obstacle condition includes a prerequisite condition that is met if the magnitude of an angle between the target surface and a horizontal plane is greater than or equal to a first threshold. (Appendix 22) 22. The computer-readable medium of claim 21, wherein the obstacle condition further includes a condition that a height of the target surface from a floor surface is less than or equal to a second threshold. (Appendix 23) the three-dimensional map has point data indicating a three-dimensional position of each of a plurality of points of the facility; 23. The computer-readable medium of claim 21, wherein in the surface detection step, the point data is clustered to generate a plurality of clusters, and for each cluster, the target surface constituted by the point data included in that cluster is detected. (Appendix 24) 24. The computer-readable medium of claim 23, wherein in the surface detection step, a normal direction is calculated for each of the plurality of point data, and the plurality of point data whose difference in normal direction is less than or equal to a threshold and whose distance between each other is less than or equal to a third threshold is classified into the same cluster. (Appendix 25) 25. The computer-readable medium of claim 23, wherein in the surface detection step, a plurality of point data representing a cylindrical region is detected by applying cylindrical fitting to the point data included in the three-dimensional map, and a side surface of the cylinder represented by the region is detected as the target surface. (Appendix 26) In the surface detection step, applying semantic segmentation to the plurality of point data included in the three-dimensional map, thereby classifying the point data representing the same object into the clusters; 26. The computer-readable medium of any one of claims 23 to 25, further comprising detecting, for each cluster representing the same object, one or more target faces from that cluster. (Appendix 27) The computer-readable medium of claim 26, further comprising: performing cylindrical fitting on the plurality of point data classified into the cluster of a cylindrical object to detect a cylindrical region formed by the plurality of point data included in the cluster; and detecting the side of the cylinder represented by the region as the target surface. (Appendix 28) In the acquiring step, location information representing a location of the user is acquired; a detection range determination step of determining a detection range in which the obstacle area is to be detected from an area around the user's position on the three-dimensional map by using the three-dimensional map and the position information; 28. The computer-readable medium of any one of claims 21 to 27, wherein in the surface detection step, the target surface is detected from the detection range. (Appendix 29) 29. The computer-readable medium of any one of claims 21 to 28, comprising an output step of outputting output information indicative of information regarding the detected obstacle region. (Appendix 30) 30. The computer-readable medium of claim 29, wherein the output information includes a display highlighting the obstacle region, a display indicating the position or direction of the obstacle region, a display indicating the type of obstacle represented by the obstacle region, or a screen including two or more of these. [Explanation of symbols]

[0128] 10 Facilities 20 users 25 positions 30 Obstacles 40 3D Map 50 Location information 60 Detection Range 70 Arrow 80 Output Information 90 Messages 100 screens 110 Messages 120 Arrow 130 Obstacle Area 500 Computers 502 Bus 504 Processor 506 Memory 508 Storage Devices 510 Input / Output Interface 512 Network Interface 2000 Obstacle detection device 2020 Acquisition Department 2040 Surface detection unit 2060 Obstacle detection unit 2080 Output section 2100 Detection range determination unit

Claims

1. an acquisition unit that acquires a three-dimensional map representing a three-dimensional position of each of a plurality of points in the facility; a surface detection unit that detects a target surface from the three-dimensional map, the target surface being a surface to be determined as being an obstacle area; an obstacle detection unit that detects, from among the detected object surfaces, the object surface that satisfies a predetermined obstacle condition as the obstacle region; An obstacle detection device, wherein the obstacle condition includes a necessary condition that is satisfied when the magnitude of an angle formed between the target surface and a horizontal plane is equal to or greater than a first threshold value.

2. The obstacle detection device according to claim 1 , wherein the obstacle condition further includes a condition that a height of the target surface from a floor surface is equal to or less than a second threshold value.

3. the three-dimensional map has point data indicating a three-dimensional position of each of a plurality of points of the facility; 3 . The obstacle detection device according to claim 1 , wherein the surface detection unit generates a plurality of clusters by clustering the point data, and detects, for each cluster, the target surface constituted by the point data included in that cluster.

4. 4. The obstacle detection device according to claim 3, wherein the surface detection unit calculates a normal direction for each of the plurality of point data, and classifies the plurality of point data whose difference in normal direction is equal to or less than a threshold and whose distance between them is equal to or less than a third threshold into the same cluster.

5. 5. The obstacle detection device according to claim 3, wherein the surface detection unit detects a plurality of point data representing a cylindrical area by applying cylindrical fitting to the point data included in the three-dimensional map, and detects a side surface of the cylinder represented by the area as the target surface.

6. The surface detection unit applying semantic segmentation to the plurality of point data included in the three-dimensional map, thereby classifying the point data representing the same object into the clusters; The obstacle detection apparatus according to claim 3 , further comprising: detecting, for each of the clusters representing the same object, one or more target surfaces from the cluster.

7. 7. The obstacle detection device according to claim 6, further comprising: a cylindrical fitting process for the plurality of point data classified into the cluster of a cylindrical object, to detect a cylindrical region constituted by the plurality of point data included in the cluster, and a side surface of the cylinder represented by the region is detected as the target surface.

8. The acquisition unit acquires location information representing a location of a user, a detection range determination unit that determines a detection range in which the obstacle area is to be detected from an area around the user's position on the three-dimensional map by using the three-dimensional map and the position information, The obstacle detection device according to claim 1 , wherein the surface detection unit detects the target surface from within the detection range.

9. 1. A computer implemented method for obstacle detection, comprising: An acquisition step of acquiring a three-dimensional map representing a three-dimensional position of each of a plurality of points of the facility; a surface detection step of detecting a target surface from the three-dimensional map, the target surface being a surface to be determined as being an obstacle area; an obstacle detection step of detecting, as the obstacle region, the object surface that satisfies a predetermined obstacle condition among the detected object surfaces; The obstacle detection method, wherein the obstacle condition includes a necessary condition that is satisfied when the magnitude of an angle between the target surface and a horizontal plane is equal to or greater than a first threshold value.

10. On the computer, An acquisition step of acquiring a three-dimensional map representing a three-dimensional position of each of a plurality of points of the facility; a surface detection step of detecting a target surface from the three-dimensional map, the target surface being a surface to be determined as being an obstacle area; an obstacle detection step of detecting, as the obstacle region, the object surface that satisfies a predetermined obstacle condition among the detected object surfaces; The obstacle condition includes a necessary condition that is satisfied when the magnitude of an angle between the target surface and a horizontal plane is equal to or greater than a first threshold value.

Images