Obstacle proximity detection device

Abstract

This obstacle proximity detection device comprises: an acquisition unit that sequentially acquires three-dimensional point cloud data; an identification unit that identifies, from the point cloud data, a three-dimensional object model representing an object and a three-dimensional obstacle model representing an obstacle; a detection unit that detects proximity between the object and the obstacle on the basis of the distance between the object model and the obstacle model; and an alarm unit that issues an alarm when proximity between the object and the obstacle is detected.

Description

Obstacle proximity detection device

[0001] The disclosed technology relates to an obstacle proximity detection device.

[0002] A mobile mapping system (MMS) has been known that uses a vehicle-mounted 3D laser scanner to create a 3D model of an outdoor structure. The 3D laser scanner is a device for measuring a 3D point cloud that represents the 3D coordinates of the surface of the structure, such as a light detection and ranging (LiDAR) sensor.

[0003] For example, Patent Document 1 discloses a technique for obtaining good results in recall and precision even when the point cloud is sparse (when the vehicle speed is high) by creating a point cloud and scan lines in a space where no three-dimensional point cloud exists and then creating a three-dimensional model. Furthermore, Patent Document 2 discloses a technique for detecting an object from a point cloud contained in a three-dimensional space corresponding to a specified region in each of a plurality of two-dimensional images.

[0004] JP 2017-156179 A JP 2018-097588 A

[0005] When carrying out construction work to erect a new utility pole, which is an example of an object, the utility pole is moved during the construction work. In this case, it is undesirable for the moving utility pole to come into contact with other obstacles. Therefore, there is a need for a technology that uses point clouds measured by a 3D laser scanner to detect the proximity of an object to an obstacle during construction in real time and to issue an alert when the object is approaching an obstacle.

[0006] However, with conventional technology, it is difficult to detect objects in real time because it takes time to create a three-dimensional model. Furthermore, when a target area is specified in advance, it is necessary to acquire a number of point clouds sufficient to detect objects within the target area, making it difficult to accurately detect both moving objects and fixed obstacles. Therefore, with conventional technology, it is difficult to detect the proximity of objects and obstacles in real time.

[0007] The disclosed technology has been made in consideration of the above points, and aims to provide an obstacle proximity detection device that can detect the proximity of a moving object to an obstacle in real time.

[0008] An aspect of the present disclosure is an obstacle proximity detection device that includes an acquisition unit that sequentially acquires three-dimensional point cloud data, an identification unit that identifies a three-dimensional object model representing an object and a three-dimensional obstacle model representing an obstacle from the point cloud data, a detection unit that detects proximity between the object and the obstacle based on the distance between the object model and the obstacle model, and an alarm unit that issues an alarm when proximity between the object and the obstacle is detected.

[0009] According to the disclosed technology, the proximity of a moving object to an obstacle can be detected in real time.

[0010] 1 is a schematic diagram showing an example of the configuration of an obstacle proximity detection system. FIG. 2 is a block diagram showing an example of the hardware configuration of an obstacle proximity detection device. FIG. 3 is a block diagram showing an example of the functional configuration of an obstacle proximity detection device. FIG. 4 is a diagram for explaining a cylinder model identification process. FIG. 5 is a diagram for explaining a cylinder model identification process. FIG. 6 is a diagram for explaining a method for determining the identity of cylinder models. FIG. 7 is a diagram for explaining a method for determining the identity of cylinder models. FIG. 8 is a diagram for explaining a wall surface model identification process. FIG. 9 is a diagram for explaining a method for determining the identity of wall surface models. FIG. 10 is a diagram for explaining a method for determining the identity of wall surface models. FIG. 11 is a diagram for explaining a method for determining the identity of wall surface models. FIG. 12 is a diagram for explaining a method for determining the identity of wall surface models. FIG. 13 is a diagram for explaining a cable model identification process. FIG. 14 is a diagram for explaining a method for determining the identity of cable models. FIG. 15 is a diagram for explaining a method for calculating the distance between cylinder models. FIG. 16 is a diagram for explaining a method for calculating the distance between cylinder models. FIG. 17 is a diagram for explaining a method for calculating the distance between cylinder models. FIG. 18 is a diagram for explaining a method for calculating the distance between cylinder models. 1 is a diagram for explaining a method for calculating the distance between a cylinder model and a wall surface model. FIG. 2 is a diagram for explaining a method for calculating the distance between a cylinder model and a wall surface model. FIG. 3 is a diagram for explaining a method for calculating the distance between a cylinder model and a wall surface model. FIG. 4 is a diagram for explaining a method for calculating the distance between a cylinder model and a wall surface model. FIG. 5 is a diagram for explaining a method for calculating the distance between a cylinder model and a cable model. FIG. 6 is a flowchart showing the flow of obstacle proximity detection processing in a first operation mode. FIG. 7 is a flowchart showing the flow of pre-processing in a second operation mode. FIG. 8 is a flowchart showing the flow of obstacle proximity detection processing in a second operation mode. FIG. 9 is a flowchart showing the flow of cylinder model identification processing. FIG. 10 is a flowchart showing the flow of wall surface model identification processing. FIG. 11 is a flowchart showing the flow of cable model identification processing.

[0011] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that in each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional proportions in the drawings are exaggerated for the convenience of explanation and may differ from the actual proportions.

[0012] [Overview of the embodiment] Fig. 1 is a schematic diagram showing an example of the configuration of an obstacle proximity detection system to which an obstacle proximity detection device 10 according to the present embodiment is applied. The obstacle proximity detection system includes the obstacle proximity detection device 10 and a three-dimensional laser scanner 20. Hereinafter, as shown in Fig. 1, with the position of the three-dimensional laser scanner 20 as the reference, the east-west direction is defined as the X axis, the north-south direction as the Y axis, and altitude as the Z axis.

[0013] The 3D laser scanner 20 sequentially measures 3D point cloud data of outdoor structures such as utility poles, walls, and cables, and outputs the point cloud data to the obstacle proximity detection device 10. The point cloud data represents the 3D coordinates of the surface of the outdoor structure.

[0014] The three-dimensional laser scanner 20 may be, for example, a LiDAR (Light Detection and Ranging) sensor. A LiDAR sensor irradiates an object with laser light and measures the distance to the object based on the time it takes for the reflected light to be received or the phase change between the emitted light and the received light. In a LiDAR sensor, multiple laser light emitters are arranged vertically, and each emitter performs horizontal scanning (rotation) to measure the three-dimensional coordinates of the object. The LiDAR sensor can also measure the reflection intensity at each coordinate on the object surface using the light intensity ratio between the emitted light and the received light.

[0015] When carrying out construction work to erect a new utility pole, which is an example of an object, the utility pole is moved during the construction work. In this case, it is undesirable for the moving utility pole to come into contact with other obstacles such as existing utility poles, walls, and cables.

[0016] Therefore, the obstacle proximity detection device 10 detects the proximity of a moving object to an obstacle in real time using three-dimensional point cloud data measured by the three-dimensional laser scanner 20. Furthermore, by issuing an alarm when proximity is detected, the obstacle proximity detection device 10 contributes to preventing contact between the moving object and other obstacles. The obstacle proximity detection device 10 will be described in detail below.

[0017] [Configuration of Obstacle Proximity Detection Device] First, a hardware configuration of the obstacle proximity detection device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the obstacle proximity detection device 10. As shown in Fig. 2, the obstacle proximity detection device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication I / F (Interface) 17. Each component is connected to each other via a bus 18 so as to be able to communicate with each other.

[0018] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads programs from the ROM 12 or the storage 14 and executes the programs using the RAM 13 as a work area. The CPU 11 controls the above components and performs various arithmetic processing in accordance with the programs stored in the ROM 12 or the storage 14. In this embodiment, an obstacle proximity detection program is stored in the ROM 12 or the storage 14.

[0019] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured by a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

[0020] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to input various information to the device itself. The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may also function as the input unit 15 by employing a touch panel system.

[0021] The communication I / F 17 is an interface for communication between the device itself and other external devices. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI (Fiber Distributed Data Interface) or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.

[0022] The obstacle proximity detection device 10 according to this embodiment is implemented by a general-purpose computer device such as a server computer or a personal computer (PC).

[0023] Next, the functional configuration of the obstacle proximity detection device 10 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the functional configuration of the obstacle proximity detection device 10. Each functional configuration is realized by the CPU 11 reading out an obstacle proximity detection program stored in the ROM 12 or the storage 14, expanding it in the RAM 13, and executing it.

[0024] 3, the obstacle proximity detection device 10 is configured with processing units including an acquisition unit 30, an identification unit 32, a switching unit 38, a detection unit 34, and an alarm transmission unit 36. Note that each processing unit may be configured as a device.

[0025] The obstacle proximity detection device 10 also includes a data storage unit 40 that stores various data necessary for executing the obstacle proximity detection process. The data storage unit 40 stores, for example, point cloud data measured by the 3D laser scanner 20 and 3D models of identified objects and obstacles. The data storage unit 40 is an example of a storage unit of the present disclosure.

[0026] The acquisition unit 30 sequentially acquires three-dimensional point cloud data. For example, the acquisition unit 30 may acquire point cloud data from the three-dimensional laser scanner 20, or may acquire point cloud data stored in the data storage unit 40. The acquisition unit 30 may acquire one frame of three-dimensional point cloud measured by the three-dimensional laser scanner 20 as point cloud data for one obstacle proximity detection process, or may acquire a superposition of three-dimensional point clouds for multiple frames as point cloud data for one obstacle proximity detection process.

[0027] The identification unit 32 identifies a three-dimensional object model representing the object and a three-dimensional obstacle model representing the obstacle from the point cloud data. For example, assume that the object is a moving cylinder (e.g., a utility pole) and the obstacles are an existing cylinder (e.g., a utility pole), a wall, and a cable. In this case, the identification unit 32 identifies a three-dimensional cylinder model as the object model and identifies a three-dimensional cylinder model, a three-dimensional wall model, and a three-dimensional cable model as the obstacle models. Details of how each model is identified will be described later.

[0028] The detection unit 34 detects the proximity of the object and the obstacle based on the distance between the object model and the obstacle model. Specifically, the detection unit 34 determines that the object and the obstacle are in proximity when the distance between the object model and the obstacle model is less than a predetermined threshold. Details of a method for calculating the distance between the object model and the obstacle model will be described later.

[0029] Here, when the identification unit 32 generates three-dimensional models of each of the cylinder, the wall, and the cable, in other words, the identification unit 32 further identifies the type of obstacle model. In this case, the detection unit 34 may use a threshold value that is predetermined for each type of obstacle model when detecting proximity. That is, the detection unit 34 may detect proximity between the object and the obstacle when the distance between the object model and the obstacle model is less than a threshold value corresponding to the identified type of obstacle model.

[0030] The detection unit 34 may also detect the proximity of point cloud data that has not been identified as an object or an obstacle, i.e., the remaining point cloud data that has not been identified as a utility pole, a wall, or a cable, to the object model. That is, the detection unit 34 may detect the proximity of an object represented by each of the remaining point clouds to the object based on the shortest distance between each of the remaining point clouds and the object model. In this case, the detection unit 34 may detect proximity using a threshold different from that used for the cylinder, the wall, and the cable. According to this embodiment, the time required for obstacle proximity detection processing can be reduced by 3D modeling the cylinder, the wall, and the cable, while the obstacle proximity detection processing can also be performed for other objects, thereby improving accuracy.

[0031] The alarm issuing unit 36 ​​issues an alarm when proximity between an object and an obstacle is detected. The alarm issuing format is not particularly limited as long as it is recognizable by the user. For example, the alarm may be issued by displaying on the display unit 16 or by audio output to a speaker (not shown). Alternatively, the alarm may be issued to a pre-registered smartphone or the like owned by the user. The alarm issuing unit 36 ​​may also issue an alarm in a different format depending on the type of obstacle whose proximity is detected. The alarm issuing conditions and format may be arbitrarily set by the user.

[0032] The obstacle proximity detection device 10 repeatedly performs the above-described processes of acquiring point cloud data, identifying an object model and an obstacle model, detecting proximity, and issuing an alarm over time.

[0033] Incidentally, when existing cylinders (e.g., utility poles), walls, and cables are used as obstacles, these obstacles do not move, so the obstacle models once identified can be reused for subsequent processing. Therefore, the identification unit 32 may store the identified obstacle models from the point cloud data acquired in advance in the data storage unit 40.

[0034] In this case, the detection unit 34 can calculate the distance using the object model identified from the sequentially acquired point cloud data and the obstacle model stored in the data storage unit 40. By using the obstacle model stored in the data storage unit 40 in this way, at least a part of the process of identifying the obstacle model can be omitted, thereby reducing the processing time.

[0035] Specifically, the identification unit 32 may determine whether the generated obstacle model is at least partially identical to an identified obstacle model stored in the data storage unit 40. If the generated obstacle model is not identical to the identified obstacle model stored in the data storage unit 40, the identification unit 32 may store the generated obstacle model in the data storage unit 40.

[0036] On the other hand, if the generated obstacle model is at least partially identical to the identified obstacle model stored in the data storage unit 40, the two obstacle models may be updated (i.e., merged or replaced) into a single obstacle model that includes both of them. For example, the larger of the generated obstacle model and the identified obstacle model may be stored in the data storage unit 40, and the other obstacle model may be discarded.

[0037] Depending on the point cloud data, the generated obstacle model may only partially represent the actual obstacle or may contain errors. The identification unit 32 repeatedly updates the obstacle model stored in the data storage unit 40, thereby improving the recall rate and precision rate of the obstacle model. This contributes to improving the accuracy of obstacle proximity detection.

[0038] The storage of the obstacle model in the data storage unit 40 may be completed in advance before the obstacle proximity detection process is started, or may be performed sequentially during the obstacle proximity detection process. Completing the storage in advance means measuring point cloud data in advance when the object is stationary and not moving, or when the object and heavy construction equipment are not present, and identifying the obstacle model from the point cloud data. In this case, obstacles that exist in areas hidden from the 3D laser scanner 20 during construction (e.g., areas behind the object and heavy construction equipment) can also be modeled in advance. Therefore, the proximity of obstacles in hidden areas can also be detected with high accuracy.

[0039] On the other hand, when the obstacle proximity detection process is performed sequentially, obstacle models are gradually registered in the data storage unit 40 each time point cloud data is acquired. This has the advantage that there is no need to measure point cloud data in advance. Furthermore, as the obstacle proximity detection process progresses, the time required to identify the obstacle model can be reduced.

[0040] Furthermore, there may be situations where it is preferable to use the obstacle model stored in the data storage unit 40, and situations where it is preferable to re-specify the obstacle model each time. Therefore, the switching unit 38 may be capable of switching between a first operation mode in which proximity is detected using an obstacle model specified each time point cloud data is sequentially acquired, and a second operation mode in which proximity is detected using the obstacle model stored in the data storage unit 40. The switching between the first operation mode and the second operation mode may be performed at any timing by the user via the input unit 15, or may be predetermined by a setting file or the like.

[0041] In the first operation mode, the identification unit 32 identifies an object model and an obstacle model each time point cloud data is acquired sequentially, and in the second operation mode, the identification unit 32 excludes point clouds corresponding to obstacle models already stored in the data storage unit 40 and identifies an object model from the remaining point clouds.

[0042] Next, specific examples of the method for identifying the cylinder model, wall model, and cable model by the identification unit 32 will be described.

[0043] (Identification of a cylindrical model) Figures 4 and 5 are diagrams for explaining the process of identifying a cylindrical model from point cloud data. Figure 4 is a side view showing a state in which a utility pole P, which is a cylinder, is being gripped by heavy equipment M. Figure 5 is a top view showing the result of detecting a circle from the point cloud representing the utility pole P, as viewed from above. The black circles in Figures 4 and 5 represent multiple three-dimensional point data included in the point cloud data.

[0044] As shown in Fig. 4, the identification unit 32 divides the point cloud into groups G of any size or any number along any axis. In the example of Fig. 4, the center line of the utility pole P shown by the dashed line is used as the axis.

[0045] Next, the identification unit 32 extracts a group G having a high point cloud density from all of the obtained groups G. For example, the identification unit 32 may extract a group G having a point cloud density equal to or greater than a predetermined threshold from all of the groups G, or may extract a group G having a relatively high point cloud density (for example, a predetermined percentage of the highest density) from all of the groups G.

[0046] Next, the identification unit 32 determines whether or not a planar circular shape as shown in Fig. 5 is detected for each of the extracted groups G (hereinafter, simply referred to as "circle detection"). For circle detection, for example, a known RANSAC (Random Sample Consensus) process or the like is used.

[0047] Next, the identification unit 32 identifies the center of each detected circle and overlaps the circles so that the centers overlap. At this time, the identification unit 32 may overlap the circles only when the circles are aligned in a substantially straight line, for example, when the deviation of the center positions of the circles is less than a predetermined threshold. Furthermore, the identification unit 32 may overlap the circles only when the sizes of the circles are substantially the same, for example, when the difference in area or diameter of the circles is less than a predetermined threshold.

[0048] Next, the identification unit 32 determines whether the overlapping circles correspond to a cylinder based on the height, inclination, etc. of the overlapping circles. For example, the identification unit 32 may determine that the overlapping circles correspond to a cylinder when at least one of the number and height of the overlapping circles is equal to or greater than a threshold. Furthermore, for example, the identification unit 32 may determine that the overlapping circles correspond to a cylinder when the inclination of the line connecting the centers of the overlapping circles is within a predetermined range.

[0049] In this way, the three-dimensional coordinates of the cylinder, specifically the position, inclination, and length of the central axis of the cylinder, and the diameter of the cylinder, are identified from the point cloud data. Based on this information, the identification unit 32 generates a three-dimensional cylindrical model representing the utility pole P. Hereinafter, the line segment passing through the central axis of the cylinder from one end to the other end will be referred to as the "center line."

[0050] To perform circle detection with high accuracy, it is preferable that the angle of the center line of the target utility pole P be tilted from 0 degrees (upright state) to approximately 45 degrees with respect to the Z axis. If the center line of the utility pole P is tilted too much with respect to the Z axis, the point cloud data obtained by measuring the utility pole P will no longer be circular, and the accuracy of circle detection will decrease. For example, even if point cloud data of a utility pole P lying down is measured, the resulting point cloud will be linear, and it may not be possible to detect a circle.

[0051] Furthermore, if the target cylinder is, for example, a utility pole P under construction, the direction and angle of the utility pole P will change depending on whether it is placed on the ground, held and suspended, or freestanding, making it unclear which axis to use when dividing it into groups G. While it is possible to search for an appropriate axis using a brute force approach, processing time can be reduced by inputting the angle of the utility pole P in advance and limiting the axes to those corresponding to that angle.

[0052] One example of a method for inputting the angle of utility pole P is to visually determine the angle of utility pole P relative to the 3D laser scanner 20 and input it manually. For example, the angle of utility pole P with respect to the Z axis can be calculated by inputting the distance from the 3D laser scanner 20 to the top of utility pole P and the distance from the 3D laser scanner 20 to the bottom of utility pole P. Since it is difficult to manually input the tilt direction of utility pole P, circle detection may be performed while changing the direction at regular intervals along the calculated angle with respect to the Z axis. In this case, it is also possible to extract only point clouds that are likely to correspond to utility pole P, depending on the distance from the 3D laser scanner 20, the size of the point cloud, etc.

[0053] Another example of a method for inputting the angle of the utility pole P is to attach an object made of a highly reflective material or structure to at least one of the utility pole P and the heavy equipment M and use it to identify the axis. For example, if the 3D laser scanner 20 is a LiDAR sensor, it can also measure reflection intensity, allowing highly reflective objects to be extracted from the point cloud. Therefore, if two or more highly reflective objects are attached to the grip of the heavy equipment M, the position of the axis can be identified. The identification unit 32 may use an axis connecting point clouds representing two highly reflective objects extracted based on reflection intensity as the axis of the utility pole and perform circle detection. Alternatively, an object with a unique shape that is easy to extract from point cloud data may be used instead of a highly reflective object.

[0054] Furthermore, the process of extracting groups G according to density is performed in the same way for each of the groups G. The process of detecting circles is performed in the same way for each of the groups G extracted according to density. Therefore, if these processes are multithreaded, the overall processing speed can be increased. For example, the well-known OpenMP (Open Multi-Processing) or the like is used for multithreading.

[0055] The identification unit 32 may also have a function of identifying a cylindrical model from each of the point cloud data measured at different times and determining the identity of those cylindrical models. Basically, two cylindrical models can be determined to be identical if their three-dimensional coordinates match, but there are cases where they do not match completely because, for example, part of the cylinder is temporarily hidden from the view of the 3D laser scanner 20 or it shakes slightly. Therefore, the identification unit 32 determines the identity of the cylindrical models using the following method.

[0056] 6 and 7 are diagrams illustrating a method for determining the identity of two cylindrical models. FIG. 6 shows a first cylindrical model MP1, and FIG. 7 shows a second cylindrical model MP2. Here, the center line L1 of the first cylindrical model MP1 is assumed to be longer than the center line L2 of the second cylindrical model MP2. The identification unit 32 determines that the first cylindrical model MP1 and the second cylindrical model MP2 are identical when the following two conditions are satisfied:

[0057] The first condition is that the angle θ between the unit vector e1 (see FIG. 6) of the center line L1 of the first cylindrical model MP1 and the unit vector e2 (see FIG. 7) of the center line L2 of the second cylindrical model MP2 is less than a predetermined threshold value.

[0058] The second condition is that the center line L2 of the second cylindrical model MP2 or its extension line is located within a predetermined range Re from the center point Q1b at one end of the first cylindrical model MP1. In the example of Fig. 7, a point Q2e on the extension line of the center line L2 of the second cylindrical model MP2 in the Z coordinate of one end of the first cylindrical model MP1 is located within the range Re, thereby satisfying the second condition.

[0059] In the second operation mode (a mode in which an obstacle model stored in the data storage unit 40 is used), of the two cylindrical models determined to be identical in the above manner, the identification unit 32 stores the cylindrical model with the longer center line in the data storage unit 40. In the examples of Fig. 6 and Fig. 7 , the identification unit 32 stores the first cylindrical model MP1 with the longer center line L1 in the data storage unit 40, and discards the second cylindrical model MP2.

[0060] In this embodiment, as an example, both the object and the obstacle include a cylinder. Therefore, the identification unit 32 determines whether the identified cylinder model represents an object (e.g., a moving utility pole) or an obstacle (e.g., an existing utility pole). Specifically, the identification unit 32 determines whether the identified cylinder model is an object model or an obstacle model based on the movement distances of two cylinder models identified from point cloud data measured at different times.

[0061] For example, suppose the two cylindrical models are a first cylindrical model MP1 shown in Fig. 6 and a second cylindrical model MP2 shown in Fig. 7. In this case, the distance between a center point Q1b at one end of the first cylindrical model MP1 and a point Q2e on an extension line of the center line L2 of the second cylindrical model MP2 in the Z coordinate of one end of the first cylindrical model MP1 can be found as the movement distance of the cylindrical model.

[0062] The identification unit 32 may identify the first cylinder model MP1 and the second cylinder model MP2 as obstacle models that are not moving when the distance between point Q1b and point Q2e is less than a first threshold value (e.g., 5 mm). Furthermore, the identification unit 32 may identify the first cylinder model MP1 and the second cylinder model MP2 as object models that are moving when the distance between point Q1b and point Q2e is equal to or greater than the first threshold value (e.g., 5 mm) and less than a second threshold value (e.g., 50 mm) that is greater than the first threshold value. Furthermore, the identification unit 32 may identify the first cylinder model MP1 and the second cylinder model MP2 as models that represent different cylinders when the distance between point Q1b and point Q2e is equal to or greater than a second threshold value (e.g., 50 mm).

[0063] (Identification of Wall Surface Model) Fig. 8 is a diagram for explaining the process of identifying a wall surface model from point cloud data. Fig. 8 is a diagram showing a point cloud representing a wall surface W and a point cloud representing a wall surface Wy obtained by rotating the wall surface W. The black circles in Fig. 8 represent multiple three-dimensional point data included in the point cloud data.

[0064] The identification unit 32 clusters the point cloud data according to voxels of an arbitrary size, point density, etc., and divides the data into a plurality of clusters. Next, the identification unit 32 determines whether or not a plane (wall surface W) as shown in Fig. 8 is detected for each cluster (hereinafter, also simply referred to as "plane detection"). For example, the well-known RANSAC (Random Sample Consensus) process is used for plane detection.

[0065] Next, if the angle formed between the normal vector n of the detected wall surface W and the Z axis is within a predetermined range, the identification unit 32 identifies the detected plane as the wall surface W. For example, if the angle formed between the normal vector n and the Z axis is within a range of 85 degrees to 95 degrees, the identification unit 32 may identify the plane as the wall surface W.

[0066] In this way, the three-dimensional coordinates of the plane, specifically the position, size, and orientation of the wall surface W, are identified from the point cloud data. The identification unit 32 generates a three-dimensional wall surface model representing the wall surface W based on this information.

[0067] The identifying unit 32 may also have a function of identifying wall surface models from each of the point cloud data measured at different times and determining the identity of these wall surface models. For example, the identifying unit 32 may determine the identity of the wall surface models by the following method.

[0068] The identification unit 32 compares, for identity, a wall surface having a normal vector whose angle with the normal vector n of the reference wall surface W is less than a predetermined threshold, i.e., a normal vector in approximately the same direction. The reference wall surface W is, for example, a wall surface included in the current point cloud data, and the comparison wall surface is, for example, a wall surface represented by a wall surface model already stored in the data storage unit 40.

[0069] 8 to 13 are diagrams for explaining a method for determining the identity of wall surface models. Fig. 9 is a diagram in which a point cloud representing the wall surface Wy (see Fig. 8) after rotation is projected onto the XZ plane. The black circles in Fig. 9 represent multiple pieces of three-dimensional point data included in the point cloud data.

[0070] First, as shown in Fig. 8 , the identification unit 32 rotates the wall surface W so that the normal vector n of the identified wall surface W is aligned with the Y axis. That is, the normal vector ny of the wall surface Wy after rotation is aligned with the Y axis. Next, as shown in Fig. 9 , the identification unit 32 obtains a rectangular area consisting of the maximum value xmax and minimum value xmin in the X axis direction and the maximum value zmax and minimum value zmin in the Z axis direction of the point cloud included in the wall surface Wy.

[0071] Similarly, the wall surfaces represented by the wall surface models to be compared are also rotated so that their normal vectors are aligned with the Y axis. In other words, by aligning the wall surfaces in the same direction (Y axis direction), it becomes easier to determine their identity. Note that the direction in which the wall surfaces are aligned is not limited to the Y axis direction, and may be any direction.

[0072] 10 to 13 are diagrams showing the relationship in the XZ plane between a wall surface Wy obtained by rotating the reference wall surface W and a wall surface Wb obtained by rotating the comparison wall surface.

[0073] In Fig. 10, the wall surface Wy and the wall surface Wb do not overlap at all. Also, in Fig. 11, the portion of the wall surface Wy that overlaps with the wall surface Wb is a portion of the wall surface Wy, and the portion of the wall surface Wb that overlaps with the wall surface Wy is a portion of the wall surface Wb. In these cases, the identification unit 32 determines that the wall surface Wy and the wall surface Wb are different wall surfaces. For example, the identification unit 32 may determine that the wall surface Wy and the wall surface Wb are different wall surfaces if the area of ​​the overlapping portion of the wall surface Wb and the wall surface Wy is less than a predetermined threshold value.

[0074] 12 , the portion of the wall surface Wy that overlaps with the wall surface Wb is a majority of the wall surface Wy, and the portion of the wall surface Wb that overlaps with the wall surface Wy is a majority of the wall surface Wb. In this case, the identification unit 32 determines that the wall surface Wy and the wall surface Wb are the same wall surface. For example, the identification unit 32 may determine that the wall surface Wb and the wall surface Wy are the same wall surface if the area of ​​the overlapping portion of the wall surface Wb and the wall surface Wy is equal to or greater than a predetermined threshold. In this case, when storing wall surface models in the data storage unit 40, the wall surface model corresponding to the larger wall surface Wy is stored, and the wall surface model corresponding to the wall surface Wb is discarded.

[0075] 13 , the portion of the wall surface Wy that overlaps with the wall surface Wb is a majority of the wall surface Wy, and the portion of the wall surface Wb that overlaps with the wall surface Wy is a portion of the wall surface Wb. In this case, the identification unit 32 determines that the wall surface Wy and the wall surface Wb are the same wall surface. For example, the identification unit 32 may determine that the wall surface Wb and the wall surface Wy are the same wall surface if the ratio of the area of ​​the overlapping portion of the wall surface Wb and the wall surface Wy to the area of ​​either the wall surface Wb or the wall surface Wy is equal to or greater than a predetermined threshold. In this case, when storing wall surface models in the data storage unit 40, the wall surface model corresponding to the larger wall surface Wb is stored, and the wall surface model corresponding to the wall surface Wy is discarded.

[0076] (Identification of Cable Model) Fig. 14 is a diagram for explaining the process of identifying a cable model from point cloud data. Fig. 14 is a diagram showing point clouds representing cables C1, C2, and another object N. The black and white circles in Fig. 14 represent multiple pieces of three-dimensional point data included in the point cloud data.

[0077] The identification unit 32 clusters the point cloud data according to voxels of an arbitrary size, point density, etc., and divides the data into a plurality of clusters. Next, the identification unit 32 determines whether a representative value of reflection intensity (e.g., average, median, maximum, minimum, etc.) for each cluster is equal to or less than an arbitrary threshold value.

[0078] The identification unit 32 identifies clusters with reflection intensities below a threshold as representing cables. This is based on the idea that the reflection intensity of cables is lower than that of other objects. The reflection intensity is represented by the ratio of the amount of emitted light to the amount of received light measured by, for example, a three-dimensional laser scanner 20 (LiDAR sensor), and the threshold is, for example, 5. As an example, in FIG. 14 , points with reflection intensities below 5 are shown as black circles, and points with reflection intensities of 5 or greater are shown as white circles.

[0079] In this way, the three-dimensional coordinates of the cable, specifically the position, length, and orientation of the cable, are identified from the point cloud data. The identification unit 32 generates a three-dimensional cable model based on this information. The cable model is represented by a catenary curve. On the other hand, the identification unit 32 identifies a cluster whose reflection intensity is equal to or greater than an arbitrary threshold as another object N and does not perform cable modeling. In this way, the cable model can be easily identified by using the reflection intensity.

[0080] The identifying unit 32 may also have a function of identifying a cable model from each of the point cloud data measured at different times and determining the identity of the cable models. For example, the identifying unit 32 may determine the identity of the cable models by the following method.

[0081] Fig. 15 is a diagram illustrating a method for determining the identity of a cable model, showing a cable model MC and a tangent line Lc that extends in the extension direction of the cable model MC and has a vertex Qc of the cable model MC as a contact point.

[0082] The identification unit 32 determines that the two cable models MC are identical if the distance between the coordinates of the vertices Qc of the two cable models MC whose identity is to be determined is less than a predetermined threshold value and the angle formed by the direction vectors ec of the respective tangents Lc is less than a predetermined threshold value.

[0083] In the second operating mode (a mode that uses the obstacle model stored in the data storage unit 40), the identification unit 32 stores the longer cable model in the data storage unit 40 and discards the shorter cable model.

[0084] The above-described identification processes for the cylinder model, wall model, and cable model can be realized using a general-purpose library such as PCL (Point Cloud Library). For example, in the identification process for the cylinder model, libraries for circle extraction and clustering can be applied. This reduces the effort required to create a new dictionary for the identification process for each model.

[0085] Note that, as a pre-processing step, the identification unit 32 may extract point clouds to be modeled from the point cloud data. For example, the identification unit 32 may delete point clouds whose distances from the measurement points of the 3D laser scanner 20 are equal to or greater than a given threshold (i.e., point clouds far from the 3D laser scanner 20). For example, the identification unit 32 may also delete point clouds corresponding to previously identified obstacle models. Specifically, the identification unit 32 may identify a three-dimensional area surrounding a previously identified obstacle model, and exclude point clouds included in that three-dimensional area. Furthermore, point clouds within a given distance from a point cloud corresponding to a previously identified obstacle model may also be excluded.

[0086] Furthermore, parameters for each process can be set arbitrarily. For example, the threshold value for the deviation of the center positions when the detected circles are superimposed may be set larger as the angle of the center line of the cylinder with respect to the Z axis (the stronger the degree of inclination).

[0087] Next, a specific example of a method for calculating the distance D between the object model and the obstacle model will be described with reference to Figures 16 to 25. As described above, the object model is the cylindrical model MPo, and the obstacle models are the cylindrical model MPe, the wall model MW, and the cable model MC.

[0088] (Cylinder Model and Cylinder Model) A method for calculating the distance D when the obstacle model is a cylinder model MPe will be described with reference to Fig. 16 and Fig. 17. Fig. 16 and Fig. 17 are diagrams showing the distance D between a cylinder model MPo as an object model and a cylinder model MPe as an obstacle model.

[0089] The detection unit 34 calculates the line segment Lm that minimizes the distance between the center line Lo of the cylindrical model MPo and the center line Le of the cylindrical model MPe. At this time, the calculation formula for the distance D is changed depending on the positional relationship between the center line Lo, the center line Le, and the line segment Lm.

[0090] 16, a case will be described in which the line segment Lm intersects with the center line Lo of the cylindrical model MPo midway and the line segment Lm intersects with the center line Le of the cylindrical model MPe midway. In this case, the detection unit 34 determines the distance D between the cylindrical models MPo and MPe as the value obtained by subtracting the radius ro of the cylindrical model MPo and the radius re of the cylindrical model MPe from the line segment Lm, as shown in the following equation (1-1): D=Lm-ro-re (1-1)

[0091] Next, a case where the line segment Lm intersects with the center line Lo of the cylindrical model MPo at the endpoint Qot. That is, one end of the line segment Lm is the endpoint Qot, and the other end is the endpoint or an intermediate point of the center line Le of the cylindrical model MPe. As an example, FIG. 17 illustrates an example where one end of the line segment Lm is the endpoint Qot, and the other end is the endpoint of the center line Le of the cylindrical model MPe. The angle between the line segment Lm and the perpendicular to the center line Le of the cylindrical model MPe is defined as θ. In this case, the detection unit 34 determines the distance D between the cylindrical model MPo and the cylindrical model MPe as the value obtained by subtracting the radius ro of the cylindrical model MPo and the value obtained by dividing the radius re of the cylindrical model MPe by cos θ from the line segment Lm, as shown in the following equation (1-2). The radius ro is subtracted from the line segment Lm to provide a margin of error, since it is difficult to accurately identify the position of the endpoint Qot. In Figure 17, the position where the distance from the end point Qot is the radius ro is shown by a dotted line. Note that if the line segment Lm intersects with the center line Le of the cylindrical model MPe not at the end point but at some point, θ is 0. D = Lm - ro - re / cos θ (1-2)

[0092] Conversely, when the line segment Lm intersects with the cylindrical model MPe at an end point of the center line Le, the models can be swapped in the calculation method using the above formula (1-2). That is, one end of the line segment Lm is the end point of the center line Le of the cylindrical model MPe, and the other end is the end point or an intermediate point of the center line Lo of the cylindrical model MPo. The angle between the line segment Lm and the perpendicular to the center line Lo of the cylindrical model MPo is defined as θ. In this case, the detection unit 34 determines the distance D between the cylindrical model MPo and the cylindrical model MPe as the value obtained by subtracting the radius re of the cylindrical model MPe and the value obtained by dividing the radius ro of the cylindrical model MPo by cos θ from the line segment Lm, as shown in the following formula (1-3). Note that when the line segment Lm intersects with the cylindrical model MPo not at the end point Qot but intermediately, θ is 0. D = Lm - re - ro / cos θ (1-3)

[0093] Furthermore, when the line segment Lm intersects with the center line Lo of the cylindrical model MPo at an end point Qot, and when the line segment Lm intersects with the center line Le of the cylindrical model MPe at an end point, the distance D may be calculated using the following calculation method instead of the calculation method using equation (1-2) or equation (1-3). In this case, the detection unit 34 calculates the distance D between the cylindrical models MPo and MPe as the value obtained by subtracting the radius re of the cylindrical model MPe and the radius ro of the cylindrical model MPo from the line segment Lm, as shown in the following equation (1-4). In this case, the calculation can be made simpler than the calculation method using equation (1-2) or equation (1-3). D=Lm-re-ro (1-4)

[0094] (Cylinder Model and Wall Model) A method for calculating the distance D when the obstacle model is a wall model MW will be described with reference to Figures 18 to 23. Figures 18 to 23 are diagrams showing the distance D between a cylinder model MPo as an object model and a wall model MW as an obstacle model.

[0095] The detection unit 34 calculates the line segment Lm that minimizes the distance between the center line Lo of the cylinder model MPo and the wall surface model MW. In Figures 18 to 20, the line segment Lm intersects with the center line Lo of the cylinder model MPo midway. In Figures 21 and 22, the line segment Lm intersects with the end point Qot of the center line Lo of the cylinder model MPo. In this case, the calculation formula for the distance D is changed depending on the positional relationship between the center line Lo, the wall surface model MW, and the line segment Lm.

[0096] 18 is a diagram showing a case where the center line Lo of the cylindrical model MPo is located in the normal direction (Y-axis direction) of the wall surface model MW. In this case, the line segment Lm is the shortest distance in the normal direction from the wall surface model MW to the center line Lo. The angle between the line segment Lm and the normal line of the wall surface model MW is defined as θ. As shown in the following equation (2-1), the detection unit 34 determines the distance D between the cylindrical model MPo and the wall surface model MW as the value obtained by subtracting the value obtained by dividing the radius ro of the cylindrical model MPo by cos θ from the line segment Lm. D = Lm - ro / cos θ (2-1) Note that in the example of FIG. 18, θ is 0, so D = Lm - ro.

[0097] 19 is a diagram showing a case where the center line Lo of the cylindrical model MPo is not positioned in the normal direction (Y-axis direction) of the wall surface model MW. In this case, the line segment Lm is the shortest distance from the side of the wall surface model MW to the center line Lo. The angle between the line segment Lm and the normal line of the wall surface model MW is defined as θ. As shown in the following formula (2-2), the detection unit 34 determines the distance D between the cylindrical model MPo and the wall surface model MW as the value obtained by subtracting the value obtained by dividing the radius ro of the cylindrical model MPo by cos θ from the line segment Lm. D = Lm - ro / cos θ (2-2)

[0098] Similarly, Figure 20 is a diagram showing a case where the center line Lo of the cylindrical model MPo is not positioned in the normal direction (Y-axis direction) of the wall surface model MW. In this case, the line segment Lm is the shortest distance from the vertex of the wall surface model MW to the center line Lo. The angle between the line segment Lm and the normal line of the wall surface model MW is set to θ. As shown in the following equation (2-3), the detection unit 34 determines the distance D between the cylindrical model MPo and the wall surface model MW as the value obtained by subtracting the value obtained by dividing the radius ro of the cylindrical model MPo by cos θ from the line segment Lm. D = Lm - ro / cos θ (2-3)

[0099] 21 is a diagram showing a case where the end point Qot of the center line Lo of the cylindrical model MPo is located in the normal direction (Y-axis direction) of the wall surface model MW. In this case, the line segment Lm is the shortest distance in the normal direction from the wall surface model MW to the end point Qot. As shown in the following equation (2-4), the detection unit 34 determines the distance D between the cylindrical model MPo and the wall surface model MW as the value obtained by subtracting the radius ro of the cylindrical model MPo from the line segment Lm. D = Lm - ro (2-4)

[0100] 22 is a diagram showing a case where the end point Qot of the center line Lo of the cylindrical model MPo is not located in the normal direction (Y-axis direction) of the wall surface model MW. In this case, the line segment Lm is the shortest distance from the side of the wall surface model MW to the end point Qot. As shown in the following equation (2-5), the detection unit 34 determines the distance D between the cylindrical model MPo and the wall surface model MW as the value obtained by subtracting the radius ro of the cylindrical model MPo from the line segment Lm. D = Lm - ro (2-5)

[0101] Similarly, Figure 23 is a diagram showing a case where the end point Qot of the center line Lo of the cylindrical model MPo is not located in the normal direction (Y-axis direction) of the wall surface model MW. In this case, the line segment Lm is the shortest distance from the vertex of the wall surface model MW to the end point Qot. As shown in the following equation (2-6), the detection unit 34 determines the distance D between the cylindrical model MPo and the wall surface model MW as the value obtained by subtracting the radius ro of the cylindrical model MPo from the line segment Lm. D = Lm - ro (2-6)

[0102] (Cylinder Model and Cable Model) A method for calculating the distance D when the obstacle model is a cable model MC will be described with reference to Fig. 24 and Fig. 25. Fig. 24 and Fig. 25 are diagrams showing the distance D between a cylinder model MPo as an object model and a cable model MC as an obstacle model, and also show a tangent line Lc that extends in the extension direction of the cable model MC and has a vertex Qc of the cable model MC as a contact point.

[0103] The detection unit 34 calculates the line segment Lm that minimizes the distance between the center line Lo of the cylinder model MPo and the cable model MC. At this time, the calculation formula for the distance D is changed depending on the positional relationship between the center line Lo, the cable model MC, and the line segment Lm.

[0104] 24 is a diagram showing a case where the line segment Lm intersects with the center line Lo of the cylindrical model MPo midway. As shown in the following equation (3-1), the detection unit 34 determines the distance D between the cylindrical model MPo and the cable model MC as a value obtained by subtracting the radius ro of the cylindrical model MPo from the line segment Lm. D=Lm-ro (3-1)

[0105] 25 is a diagram showing a case where the line segment Lm intersects with the center line Lo of the cylindrical model MPo at the end point Qot. The detection unit 34 determines the distance D between the cylindrical model MPo and the cable model MC as the line segment Lm, as shown in the following equation (3-2): D=Lm (3-2)

[0106] 24 and 25 illustrate an example in which one end point of the line segment Lm is the vertex Qc of the cable model MC, but this is not limitative. The line segment Lm can have any point on the cable model MC as its end point as long as the point is the shortest distance from the center line Lo of the cylindrical model MPo.

[0107] [Operation of Obstacle Proximity Detection Device] Next, the operation of the obstacle proximity detection device 10 according to this embodiment will be described. Fig. 26 is a flowchart showing an example of the flow of processing by an obstacle proximity detection program according to this embodiment. The CPU 11 reads out the obstacle proximity detection program stored in the ROM 12 or the storage 14, deploys it in the RAM 13, and executes it, thereby performing the obstacle proximity detection processing shown in Fig. 26. The obstacle proximity detection processing is executed, for example, when the 3D laser scanner 20 starts measuring point cloud data and outputting the data to the obstacle proximity detection device 10.

[0108] In step S100, the CPU 11, functioning as the acquisition unit 30, sequentially acquires three-dimensional point cloud data. In step S200, the CPU 11, functioning as the identification unit 32, executes a cylindrical model identification process using the point cloud data acquired in step S100 (see FIG. 29 ). In step S300, the CPU 11, functioning as the identification unit 32, executes a wall surface model identification process using the point cloud data acquired in step S100 (see FIG. 30 ). In step S400, the CPU 11, functioning as the identification unit 32, executes a cable model identification process using the point cloud data acquired in step S100 (see FIG. 31 ).

[0109] In step S102, the CPU 11, functioning as the detection unit 34, calculates the distance between the object model and the obstacle model identified in steps S200, S300, and S400. In step S104, the CPU 11, functioning as the detection unit 34, determines whether the distance calculated in step S102 is less than a predetermined threshold value, i.e., whether the object and the obstacle are close to each other.

[0110] If the distance is less than the threshold (if Y in step S104), the process proceeds to step S106. In step S106, the CPU 11 causes the alarm issuing unit 36 ​​to issue an alarm indicating that the object and the obstacle are in close proximity, and ends the obstacle proximity detection process. On the other hand, if the distance is equal to or greater than the threshold (if N in step S104), this means that the object and the obstacle are not in close proximity and are in a safe state, so step S106 is skipped and the obstacle proximity detection process ends.

[0111] The obstacle proximity detection device 10 according to the present embodiment monitors the proximity of an object and an obstacle over time, and therefore the obstacle proximity detection process is repeatedly executed until an instruction to stop the process is received.

[0112] 26 corresponds to the first operation mode (a mode in which proximity is detected using an obstacle model that is identified each time point cloud data is sequentially acquired). The obstacle proximity detection device 10 according to this embodiment is also configured to be switchable to the second operation mode (a mode in which proximity is detected using an obstacle model stored in the data storage unit 40), and the operation in the second operation mode will be described below.

[0113] 27 and 28 are flowcharts showing an example of the flow of processing by the obstacle proximity detection program according to this embodiment. FIG. 27 shows pre-processing for identifying an obstacle model in advance. FIG. 28 shows processing for detecting proximity using an obstacle model that has been identified in advance. The CPU 11 reads out the obstacle proximity detection program stored in the ROM 12 or the storage 14, deploys it in the RAM 13, and executes it, thereby performing the pre-processing shown in FIG. 27 and the obstacle proximity detection processing shown in FIG. 28. The pre-processing and the obstacle proximity detection processing are executed, for example, when the 3D laser scanner 20 starts measuring point cloud data and outputting the data to the obstacle proximity detection device 10.

[0114] In step S120, the CPU 11, functioning as the acquisition unit 30, sequentially acquires three-dimensional point cloud data. In step S200, the CPU 11, functioning as the identification unit 32, executes a cylindrical model identification process using the point cloud data acquired in step S120 (see FIG. 29 ). In step S300, the CPU 11, functioning as the identification unit 32, executes a wall surface model identification process using the point cloud data acquired in step S120 (see FIG. 30 ). In step S400, the CPU 11, functioning as the identification unit 32, executes a cable model identification process using the point cloud data acquired in step S120 (see FIG. 31 ).

[0115] In step S122, the CPU 11, functioning as the identification unit 32, determines whether or not a 3D model that is at least partially identical to the 3D models generated in steps S200, S300, and S400 is stored in the data storage unit 40. If no 3D models are stored (if step S122 is Y), the process proceeds to step S124. In step S124, the CPU 11, functioning as the identification unit 32, stores the 3D models generated in steps S200, S300, and S400 in the data storage unit 40, and ends the pre-processing. On the other hand, if the 3D models are stored (if step S122 is N), step S124 is skipped, and the pre-processing ends.

[0116] When measuring point cloud data for identifying an obstacle model in advance, temporary obstacles such as pedestrians and passing vehicles may be included. Therefore, the above pre-processing may be repeatedly executed until an instruction to stop is given.

[0117] In step S150, the CPU 11 functions as the acquisition unit 30 to sequentially acquire three-dimensional point cloud data. In step S151, the CPU 11 functions as the identification unit 32 to acquire the identified obstacle model stored in the data storage unit 40 in step S124. In step S200, the CPU 11 functions as the identification unit 32 to execute a cylindrical model identification process for identifying an object model using the point cloud data acquired in step S150 (see FIG. 29 ).

[0118] In step S152, the CPU 11, functioning as the detection unit 34, calculates the distance between the object model acquired in step S200 and the obstacle model acquired in step S151. In step S154, the CPU 11, functioning as the detection unit 34, determines whether the distance calculated in step S152 is less than a predetermined threshold value, i.e., whether the object and obstacle are close to each other.

[0119] If the distance is less than the threshold (if Y in step S154), the process proceeds to step S156. In step S156, the CPU 11 causes the alarm issuing unit 36 ​​to issue an alarm indicating that the object and the obstacle are in close proximity, and ends the obstacle proximity detection process. On the other hand, if the distance is equal to or greater than the threshold (if N in step S154), this means that the object and the obstacle are not in close proximity and are in a safe state, so step S156 is skipped and the obstacle proximity detection process ends.

[0120] The obstacle proximity detection device 10 according to the present embodiment monitors the proximity of an object and an obstacle over time, and therefore the obstacle proximity detection process is repeatedly executed until an instruction to stop the process is received.

[0121] 27 and 28 illustrate the process in which the obstacle model is identified in advance, but the present invention is not limited to this. The obstacle proximity detection device 10 according to the present embodiment may sequentially store the obstacle model identified during the obstacle proximity detection process in the data storage unit 40 and use the stored obstacle model in subsequent obstacle proximity detection processes. This process can be implemented by incorporating step S151 (see FIG. 28) after step S100 and step S122 and step S124 (see FIG. 27) after step S400 in the obstacle proximity detection process of FIG. 26.

[0122] 29, the cylindrical model identification process in step S200 will be described. In step S210, the CPU 11 functions as the identification unit 32 to extract a point cloud to be processed from the point cloud data acquired in step S100. For example, the identification unit 32 may delete a point cloud whose distance from the measurement point of the 3D laser scanner 20 is equal to or greater than a given threshold, or may delete a point cloud corresponding to an obstacle model that has been identified in the past.

[0123] In step S212, the CPU 11, functioning as the identification unit 32, divides the point cloud extracted in step S210 into groups of any size or any number along any axis. In step S214, the CPU 11, functioning as the identification unit 32, extracts groups with high point density from all the groups obtained in step S212. In step S216, the CPU 11, functioning as the identification unit 32, performs circle detection for each of the groups extracted in step S214.

[0124] In step S218, the CPU 11, as the identification unit 32, identifies the center of each circle detected in step S216 and overlaps the circles so that the centers overlap. In step S220, the CPU 11, as the identification unit 32, determines whether the overlapped circles correspond to a cylinder based on the height, inclination, etc. of the circles overlapped in step S218.

[0125] If it corresponds to a cylinder (if Y in step S220), the process proceeds to step S222. In step S222, the CPU 11, as the identification unit 32, generates a three-dimensional cylinder model based on the circle (cylinder) superimposed in step S218, and ends the cylinder model identification process. On the other hand, if it does not correspond to a cylinder (if N in step S220), the subsequent steps are skipped, and the cylinder model identification process ends.

[0126] The wall surface model identification process in step S300 will be described with reference to Figure 30. In step S310, the CPU 11, functioning as the identification unit 32, extracts a point cloud to be processed from the point cloud data acquired in step S100. For example, the identification unit 32 may delete a point cloud whose distance from the measurement point of the 3D laser scanner 20 is equal to or greater than an arbitrary threshold, or may delete a point cloud corresponding to an obstacle model that has been identified in the past.

[0127] In step S312, the CPU 11, functioning as the identification unit 32, performs clustering on the point cloud extracted in step S210 and divides it into a plurality of clusters. In step S314, the CPU 11, functioning as the identification unit 32, executes plane detection for each of the clusters obtained in step S312.

[0128] In step S316, the CPU 11, functioning as the identification unit 32, determines whether the angle between the normal vector of the plane detected in step S314 and the Z axis is within a predetermined range. The angle range is, for example, approximately 85 degrees to 95 degrees, that is, whether the plane is a wall surface that is approximately parallel to the Z axis.

[0129] If the angle is within the range (if Y in step S316), the process proceeds to step S318. In step S318, the CPU 11, functioning as the identification unit 32, generates a three-dimensional wall surface model based on the plane detected in step S314, and ends the wall surface model identification process. On the other hand, if the angle is not within the range (if N in step S316), the subsequent steps are skipped, and the wall surface model identification process ends.

[0130] The cable model identification process in step S400 will be described with reference to Figure 31. In step S410, the CPU 11, functioning as the identification unit 32, extracts a point cloud to be processed from the point cloud data acquired in step S100. For example, the identification unit 32 may delete a point cloud whose distance from the measurement point of the 3D laser scanner 20 is equal to or greater than an arbitrary threshold, or delete a point cloud corresponding to an obstacle model that has been identified in the past.

[0131] In step S412, the CPU 11, functioning as the identification unit 32, clusters the point cloud extracted in step S410 and divides it into a plurality of clusters. In step S414, the CPU 11, functioning as the identification unit 32, extracts clusters obtained in step S412 that have a reflection intensity less than a predetermined threshold. In step S416, the CPU 11, functioning as the identification unit 32, extracts catenary curves from the clusters extracted in step S414 to generate a three-dimensional cable model, and the cable model identification process ends.

[0132] Next, an example of the obstacle proximity detection device 10 will be described. First, point cloud data including a moving utility pole and a wall surface as an obstacle was acquired by a LiDAR sensor, and a cylindrical model and a wall surface model were identified by the obstacle proximity detection device 10. At this time, among the planes detected from the point cloud data, those whose angle between the normal vector and the Z axis was in the range of 85 degrees to 95 degrees were determined as wall surface models. Then, the distance between the utility pole and the wall surface was calculated based on the distance between the cylindrical model (object model) and the wall surface model (obstacle model).

[0133] As a comparative example, the distance between the utility pole and the wall surface was calculated using the distance between point clouds, which is a conventional method. Furthermore, the actual distance between the utility pole and the wall surface was measured. The calculated value for this example, the calculated value for the comparative example, and the actual measured value for the distance between the utility pole and the wall surface were as follows: Actual measured value: 1.37 m This example: 1.39 m Comparative example: 1.44 m

[0134] In this way, the obstacle proximity detection device 10 according to this embodiment was able to obtain good results that were closer to actual measurements. This is thought to be because, while conventional methods that use the distance between point clouds are prone to errors in sparse areas where no point clouds exist, this embodiment complements the sparse areas by using a three-dimensional model.

[0135] Furthermore, when calculating the distance between the target object and the obstacle, the calculation speed was 1 millisecond for one model. Thus, the obstacle proximity detection device 10 according to this embodiment can also achieve high-speed processing.

[0136] In addition, for cables that are difficult to extract from point cloud data, the point cloud of the cable could be extracted with high accuracy by limiting the reflection intensity obtained by the LiDAR sensor to 5 or less and limiting the extraction range to the work area including the cable.

[0137] As described above, the obstacle proximity detection device 10 according to this embodiment includes an acquisition unit 30 that sequentially acquires three-dimensional point cloud data, an identification unit 32 that identifies a three-dimensional object model representing the object and a three-dimensional obstacle model representing the obstacle from the point cloud data, a detection unit 34 that detects proximity between the object and the obstacle based on the distance between the object model and the obstacle model, and an alarm issuance unit 36 ​​that issues an alarm when proximity between the object and the obstacle is detected. In other words, because the distance between the object and the obstacle is calculated using the three-dimensional model, it is possible to detect the proximity of a moving object and the obstacle with higher accuracy in real time.

[0138] In the above embodiment, the identification unit 32 identifies the cylinder model, the wall model, and the cable model from the point cloud data. However, this is not limiting. For example, a user may manually label the cylinder point cloud, the wall point cloud, and the cable point cloud, and the identification unit 32 may generate each model based on the labeled point clouds. Furthermore, the processing by the identification unit 32 and manual labeling may be combined. For example, the user may manually label only point clouds that the identification unit 32 was unable to properly identify. Furthermore, for example, the identification process for the cylinder model and the wall model may be performed by the identification unit 32, and the cable model may be generated based on the manual labeling by the user.

[0139] In the above embodiment, the proximity between an object model and an obstacle model is detected using the distance D calculated based on the object model and the obstacle model, but the present invention is not limited to this. Here, the distance D is obtained by correcting the line segment Lm that minimizes the distance between the object model and the obstacle model with the radius ro, etc., as shown in equations (1-1) to (3-2), and this correction is intended to provide a margin for detecting proximity.

[0140] On the other hand, other methods can be used to provide a margin for detecting proximity. For example, instead of the distance between the object model and the obstacle model, a threshold value to be compared with the distance may be corrected to be larger. In this case, by comparing the line segment Lm calculated from the three-dimensional model with the corrected threshold value, proximity can be detected with a margin.

[0141] For example, in each of the specific examples of Figures 16 to 25, the corrected threshold value T1 may be calculated by adding the term subtracted from the line segment Lm in equations (1-1) to (3-2) to a predetermined threshold value T0. For example, in the example of Figure 16, the corrected threshold value T1 may be calculated using the following equation (4-1): T1 = T0 + ro + re (4-1)

[0142] In the above embodiment, the cylinder is a utility pole, but the present invention is not limited to this. Cylinders other than utility poles may also be used as targets. In the above embodiment, the target is a utility pole (cylinder model), and the obstacles are an existing utility pole (cylinder model), a wall (wall model), and a cable (cable model). However, the present invention is not limited to this. For example, a wall and a cable may be used as targets.

[0143] Furthermore, the obstacle proximity detection process executed by the CPU 11 after reading the obstacle proximity detection program in the above embodiment may be executed by various processors other than the CPU 11. Examples of such processors include a programmable logic device (PLD) (such as a field-programmable gate array (FPGA)) whose circuit configuration can be changed after manufacture, and a dedicated electrical circuit, such as an application-specific integrated circuit (ASIC), which is a processor having a circuit configuration specifically designed to execute specific processes. Furthermore, the obstacle proximity detection process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

[0144] In the above embodiment, the obstacle proximity detection program is described as being pre-stored (also referred to as "installed") in the ROM 12 or the storage 14, but the present invention is not limited to this. The obstacle proximity detection program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The obstacle proximity detection program may also be downloaded from an external device via a network.

[0145] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0146] The following additional notes are provided regarding the above-described embodiments.

[0147] (Supplementary Item 1) An obstacle proximity detection device including: a memory; and at least one processor connected to the memory, wherein the processor is configured to: sequentially acquire three-dimensional point cloud data; identify a three-dimensional object model representing an object and a three-dimensional obstacle model representing an obstacle from the point cloud data; detect proximity between the object and the obstacle based on the distance between the object model and the obstacle model; and issue an alarm when proximity between the object and the obstacle is detected.

[0148] (Supplementary Item 2) A non-transitory storage medium storing a program executable by a computer to execute an obstacle proximity detection process, wherein the obstacle proximity detection process sequentially acquires three-dimensional point cloud data, identifies a three-dimensional object model representing an object and a three-dimensional obstacle model representing an obstacle from the point cloud data, detects proximity between the object and the obstacle based on the distance between the object model and the obstacle model, and issues an alarm when proximity between the object and the obstacle is detected.

[0149] REFERENCE SIGNS LIST 10 Obstacle proximity detection device 20 Three-dimensional laser scanner 30 Acquisition unit 32 Identification unit 34 Detection unit 36 ​​Alarm unit 38 Switching unit 40 Data storage unit

Claims

1. An obstacle proximity detection device comprising: an acquisition unit that sequentially acquires three-dimensional point cloud data; a specification unit that specifies, from the point cloud data, a three-dimensional object model representing an object and a three-dimensional obstacle model representing an obstacle; a detection unit that detects proximity between the object and the obstacle based on a distance between the object model and the obstacle model; and a notification unit that issues an alarm when proximity between the object and the obstacle is detected.

2. The obstacle proximity detection device according to claim 1, wherein the specification unit stores the obstacle model specified from the point cloud data acquired in advance in a storage unit, and the detection unit calculates the distance using the object model specified from the sequentially acquired point cloud data and the obstacle model stored in the storage unit.

3. The obstacle proximity detection device according to claim 2, further comprising a switching unit that switches between a first operation mode in which the proximity is detected using the obstacle model specified each time the point cloud data is sequentially acquired, and a second operation mode in which the proximity is detected using the obstacle model stored in the storage unit.

4. The obstacle proximity detection device according to claim 1, wherein the specification unit further specifies a type of the obstacle model, and the detection unit detects proximity between the object and the obstacle when the distance is a threshold value predetermined for each type of the obstacle model and is less than the threshold value corresponding to the type of the specified obstacle model.

Images