Detection apparatus, detection method, and system

Abstract

To provide a detection device, a detection method and a system for further improving object detection accuracy.SOLUTION: In a system 1 including a point group sensor 10 and a detection device 30, the detection device includes a distance calculation unit that calculates a distance from the point group sensor to a specific position, a processing area setting unit that sets a processing area based on the distance calculated by the distance calculation unit and extracts point group data included in the processing area from point group data acquired by the point group sensor, and an object detection unit that detects an object based on the point group data included in the processing area extracted by the processing area setting unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a detection device, a detection method, and a system. [Background technology]

[0002] In recent years, there has been an increasing demand for technology to measure real space. For example, a 3D laser sensor that detects a reaction to active light (near-infrared light), such as a LiDAR (Laser Imaging Detection and Ranging) sensor, can be used for measuring real space.

[0003] Regarding a technology for measuring real space, for example, Patent Document 1 discloses a technology for detecting an object after separating point cloud data measured by a LiDAR sensor into a background region and a region where the object exists. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-219248 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology disclosed in Patent Document 1 uses background subtraction to improve object detection accuracy. However, in an environment where the background changes as work progresses, such as a construction site, the background subtraction may include changes in accordance with the progress of work, which could result in a decrease in object detection accuracy.

[0006] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a mechanism that can further improve the accuracy of object detection. [Means for solving the problem]

[0007] In order to solve the above problem, according to one aspect of the present invention, there is provided a detection device comprising: a distance calculation unit that calculates the distance from a point cloud sensor to a specific position; a processing range setting unit that sets a processing range based on the distance calculated by the distance calculation unit and extracts the point cloud data that is included in the processing range from the point cloud data acquired by the point cloud sensor; and an object detection unit that detects an object based on the point cloud data that is included in the processing range extracted by the processing range setting unit.

[0008] When a distance from the point cloud sensor to the specific position changes, the processing range setting unit may set the processing range based on the changed distance.

[0009] The processing range setting unit may set, as the processing range, a space whose distance from the point cloud sensor is shorter than the distance calculated by the distance calculation unit.

[0010] The specific position may be the position of the horizontal end of the space in which the object is to be detected by the object detection unit, the distance calculation unit may calculate the horizontal distance from the point cloud sensor to the specific position, and the processing range setting unit may set as the processing range a space whose horizontal distance from the point cloud sensor is shorter than the horizontal distance calculated by the distance calculation unit.

[0011] The specific location may be the location of a tunnel face.

[0012] The specific position may be a position on the ground, the distance calculation unit may calculate the vertical distance from the point cloud sensor to the specific position, and the processing range setting unit may set as the processing range a space whose vertical distance from the point cloud sensor is shorter than the vertical distance calculated by the distance calculation unit.

[0013] The processing range setting unit may set a plurality of processing ranges, each of which may be a mutually exclusive space in the horizontal direction and a space whose vertical distance from the point cloud sensor is shorter than the highest point on the ground in the vertical direction of the space.

[0014] In addition, in order to solve the above problem, according to another aspect of the present invention, there is provided a detection method executed by a computer, the detection method including: calculating a distance from a point cloud sensor to a specific position; setting a processing range based on the calculated distance; extracting the point cloud data included in the processing range from the point cloud data acquired by the point cloud sensor; and detecting an object based on the point cloud data included in the extracted processing range.

[0015] In addition, in order to solve the above-mentioned problems, according to another aspect of the present invention, there is provided a system comprising a point cloud sensor and a detection device, wherein the detection device has a distance calculation unit that calculates the distance from the point cloud sensor to a specific position, a processing range setting unit that sets a processing range based on the distance calculated by the distance calculation unit and extracts the point cloud data that is included in the processing range from the point cloud data acquired by the point cloud sensor, and an object detection unit that detects an object based on the point cloud data that is included in the processing range extracted by the processing range setting unit. [Effects of the Invention]

[0016] As described above, according to the present invention, a mechanism capable of further improving the accuracy of object detection is provided. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram illustrating an overview of a system 1 according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing an example of the configuration of a system 1 according to the present embodiment. [Figure 3] 10 is a flowchart showing an example of the flow of a detection process executed by the detection device 30 according to the present embodiment. [Figure 4] 10A and 10B are diagrams for explaining an example of setting a processing range R according to the present embodiment. [Figure 5] 10A and 10B are diagrams for explaining an example of setting a processing range R according to the present embodiment. [Figure 6] FIG. 2 is a block diagram showing an example of a hardware configuration of the information processing device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant explanations will be omitted.

[0019] <1. Overview> 1 is a diagram illustrating an overview of a system 1 according to an embodiment of the present invention. The system 1 according to this embodiment includes a point cloud sensor 10.

[0020] The point cloud sensor 10 is a sensor that measures a real space and acquires point cloud data. The point cloud data includes information indicating the three-dimensional positions of a plurality of points that make up the real space. The three-dimensional positions in this specification may be, for example, coordinates with the position of the point cloud sensor 10 as the origin. For example, the point cloud sensor 10 is a LiDAR sensor. Alternatively, the point cloud sensor 10 may be a laser scanner, a ToF (Time of Flight) sensor, a stereo camera, or the like.

[0021] As shown in Fig. 1, a point cloud sensor 10 is placed at a construction site where a worker P and a bulldozer M are working together to excavate a tunnel T. The face F of the tunnel T (i.e., the excavation surface) moves further in as the work progresses. For this reason, even if background subtraction is used as in the technology disclosed in Patent Document 1, it is difficult to accurately detect objects.

[0022] Therefore, the system 1 sets a processing range R and detects objects (for example, a worker P and a bulldozer M) from the point cloud data included in the processing range R. In particular, the system 1 calculates the distance L from the point cloud sensor 10 to the face F. HThe processing range R is automatically updated in accordance with changes in the distance R. With this configuration, the processing range R can be automatically updated in accordance with the progress of the work, making it possible to perform object detection at the construction site with higher accuracy.

[0023] More specifically, if the processing range R is not updated even when the tunnel T is excavated and the tunnel face F moves further back, it becomes difficult to detect objects in the excavated space. In this regard, the system 1 expands the processing range R to the farther side of the tunnel T as the tunnel face F of the tunnel T moves further back. This makes it possible for the system 1 to detect objects in the excavated space.

[0024] The system 1 can output various information based on the object detection results. For example, the system 1 can issue a warning when the distance between the detected objects is equal to or less than a predetermined threshold. This configuration can prevent accidents such as contact between the worker P and the bulldozer M.

[0025] <2.Configuration example> 2 is a block diagram showing an example of the configuration of the system 1 according to this embodiment. As shown in FIG. 2, the system 1 includes a point cloud sensor 10 and a detection device 30.

[0026] The configuration of the point cloud sensor 10 is as described above with reference to Fig. 1. The point cloud sensor 10 outputs the acquired point cloud data to the detection device 30 (particularly, the distance calculation unit 31 and the processing range setting unit 32).

[0027] The detection device 30 is an information processing device that detects an object based on point cloud data measured by the point cloud sensor 10. As shown in FIG. 2, the detection device 30 includes a distance calculation unit 31, a processing range setting unit 32, and an object detection unit 33.

[0028] (Distance calculation unit 31) The distance calculation unit 31 calculates the distance from the point cloud sensor 10 to the specific position. For example, the distance calculation unit 31 calculates the distance from the point cloud sensor 10 to the specific position based on the three-dimensional position of a point corresponding to the specific position among the point cloud data acquired by the point cloud sensor 10.

[0029] The specific position is the position of an edge of the space where an object is to be detected by the object detection unit 33. In particular, the specific position may be the position of an edge in the horizontal direction of the space where an object is to be detected by the object detection unit 33. In this case, the distance calculation unit 31 calculates the horizontal distance from the point cloud sensor 10 to the specific position. For example, the distance calculation unit 31 determines the point in the point cloud data obtained by the point cloud sensor 10 that has the longest horizontal distance from the point cloud sensor 10 as the specific position, and calculates the horizontal distance to that point as the horizontal distance from the point cloud sensor 10 to the specific position.

[0030] In the example shown in FIG. 1, the space in which the object detection unit 33 detects an object is the space inside the tunnel T, and the end in the horizontal direction is the face F. That is, the specific position is the position of the face F of the tunnel T. Therefore, the distance calculation unit 31 calculates the horizontal distance L from the point cloud sensor 10 to the face F of the tunnel T. H Calculate.

[0031] (Processing range setting unit 32) The processing range setting unit 32 sets a processing range R based on the distance calculated by the distance calculation unit 31. Then, the processing range setting unit 32 extracts point cloud data included in the processing range R from the point cloud data acquired by the point cloud sensor 10. The processing range R is a space where an object is to be detected by the object detection unit 33. With this configuration, for example, it is possible to extract only point cloud data included in the space inside the tunnel T, which is the space where an object is to be detected by the object detection unit 33.

[0032] The processing range setting unit 32 sets a space whose distance from the point cloud sensor 10 is shorter than the distance calculated by the distance calculation unit 31 as the processing range R. For example, the processing range setting unit 32 sets a space whose distance from the point cloud sensor 10 is shorter than the distance calculated by the distance calculation unit 31 as the processing range R. HA space that is closer in horizontal direction to the point cloud sensor 10 than the face F of the tunnel T is set as the processing range R. According to this configuration, it is possible to set the space that is closer to the point cloud sensor 10 in the horizontal direction than the face F of the tunnel T as the processing range R.

[0033] Here, when the distance from the point cloud sensor 10 to the specific position changes, the processing range setting unit 32 sets the processing range R based on the changed distance. For example, the processing range setting unit 32 sets the processing range R based on the horizontal distance L from the point cloud sensor 10 to the face F of the tunnel T. H When the horizontal distance L changes, H A space with a horizontal distance shorter than the distance between the tunnel face F and the tunnel T is set as the processing range R. With this configuration, it is possible to expand the processing range R toward the back of the tunnel T as the tunnel face F moves toward the back.

[0034] The processing range setting unit 32 outputs the point cloud data included in the extracted processing range R to the object detection unit 33.

[0035] (Object detection unit 33) The object detection unit 33 detects an object based on the point cloud data included in the processing range R extracted by the processing range setting unit 32. For example, the object detection unit 33 detects an object by applying various processes such as noise removal, clustering, and pattern matching to the point cloud data included in the processing range R.

[0036] Then, the object detection unit 33 outputs information indicating the detected object.

[0037] The above describes an example configuration of the system 1 according to this embodiment. According to the configuration described above, each time the tunnel T is excavated, the processing range R is updated so that the excavated space is included in the processing range R. This allows the object detection unit 33 to accurately detect objects present in the tunnel T, including objects present in the excavated space.

[0038] <3. Processing example> Hereinafter, an example of the flow of the detection process executed by the detection device 30 according to this embodiment will be described with reference to FIG.

[0039] FIG. 3 is a flowchart showing an example of the flow of the detection process executed by the detection device 30 according to this embodiment.

[0040] As shown in FIG. 3, first, the distance calculation unit 31 calculates the distance from the point cloud sensor 10 to the specific position (step S102).

[0041] Next, the processing range setting unit 32 sets the processing range R (step S104).

[0042] Next, the processing range setting unit 32 extracts point cloud data included in the processing range R from the point cloud data obtained by the point cloud sensor 10 (step S106).

[0043] Next, the object detection unit 33 detects an object based on the point cloud data included in the processing range R (step S108).

[0044] Thereafter, the detection device 30 determines whether or not to end the detection process (step S110).

[0045] If it is determined that the detection process should be continued (step S110: NO), the process returns to step S102.

[0046] If it is determined that the detection process should be ended (step S110: YES), the process ends.

[0047] <4. Modifications> In the above, the horizontal distance L from the point cloud sensor 10 to the end of the tunnel T in the direction in which the tunnel T is excavated (i.e., the depth direction) is H However, the processing range R may be set in the other directions in a similar manner.

[0048] As an example, the processing range R may be set based on the horizontal distance from the point cloud sensor 10 to the end of the tunnel T in the width direction. With this configuration, even if the width of the tunnel T changes, it is possible to accurately detect an object inside the tunnel T.

[0049] As another example, the processing range R may be set based on the vertical distance from the point cloud sensor 10 to the end of the tunnel T in the height direction. In this case, the specific position is the position of the end of the space in the height direction where the object is to be detected by the object detection unit 33. In terms of the direction of the ground (i.e., the vertical direction), the specific position is the position of the ground. The processing range R in the height direction will be described in detail with reference to FIG. 4.

[0050] 4 is a diagram for explaining an example of setting the processing range R according to this embodiment. As shown in FIG. 4, the distance calculation unit 31 calculates the vertical distance L from the point cloud sensor 10 to the position of the ground G. V For example, the distance calculation unit 31 determines the point in the point cloud data obtained by the point cloud sensor 10 that has the longest vertical distance from the point cloud sensor 10 as the specific position, and calculates the vertical distance to that point as the vertical distance L from the point cloud sensor 10 to the ground G. V Then, the processing range setting unit 32 calculates the vertical distance L calculated by the distance calculation unit 31 as V A space that is closer in vertical distance from the point cloud sensor 10 than the ground surface G of the tunnel T is set as the processing range R. With this configuration, it is possible to set the space that is closer to the point cloud sensor 10 in the vertical direction than the ground surface G of the tunnel T as the processing range R. As a result, it is possible to accurately detect objects that exist above the ground surface G.

[0051] Here, the ground of the tunnel T may be inclined. In this case, the processing range R may be set according to the inclination of the ground G. This point will be described in detail with reference to FIG. 5.

[0052] FIG. 5 is a diagram for explaining an example of setting a processing range R according to this embodiment. As shown in FIG. 5, the processing range setting unit 32 may set a plurality of processing ranges R (R1 to R3). Each of the plurality of processing ranges R is a space that is mutually exclusive in the horizontal direction, and is a space whose vertical distance from the point cloud sensor 10 is shorter than the highest point of the ground G in the vertical direction of the space. That is, as shown in FIG. 5, the processing ranges R1 to R3 are each a portion of the ground G that is within a vertical distance L from the point cloud sensor 10. V1 ~L V3 is set as the space closer to the point cloud sensor 10 in the vertical direction than the position where the distance between the processing ranges R and the point cloud sensor 10 is shortest. In this way, by setting a plurality of processing ranges R with different lower ends according to the inclination of the ground surface G, it is possible to minimize the space above the ground surface G that is not included in the processing range R. As a result, it is possible to accurately detect objects that exist above the inclined ground surface G.

[0053] Note that, with regard to the ceiling direction, the processing range R may also be set in the same manner as the method described with reference to Fig. 5. That is, the processing range setting unit 32 may set a plurality of processing ranges R, and each of the plurality of processing ranges R may be a space that is mutually exclusive in the horizontal direction and whose vertical distance from the point cloud sensor 10 is shorter than the lowest point of the ceiling in the counter-vertical direction of the space.

[0054] In addition, when the tunnel face F of the tunnel T is inclined, when the tunnel T is meandering, or when the tunnel T tapers, etc., the processing range R may be set in a manner similar to that described with reference to Figure 5 regarding horizontal inclination.

[0055] <5. Hardware configuration example> Next, the hardware configuration of an information processing device according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the hardware configuration of an information processing device according to this embodiment. Note that the information processing device 900 shown in Fig. 6 can realize, for example, the detection device 30 shown in Fig. 1. Information processing by the detection device 30 according to this embodiment is realized by cooperation between software and hardware described below.

[0056] As shown in FIG. 6, the information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, and a communication device 911.

[0057] The CPU 901 functions as an arithmetic processing unit and control unit, and controls the overall operation of the information processing device 900 in accordance with various programs. The CPU 901 may also be a microprocessor. The ROM 902 stores programs used by the CPU 901, calculation parameters, etc. The RAM 903 temporarily stores programs used in the execution of the CPU 901, and parameters that change as appropriate during the execution. These are interconnected by a host bus 904 that is composed of a CPU bus, etc. The CPU 901 may form, for example, the distance calculation unit 31, the processing range setting unit 32, and the object detection unit 33 shown in FIG. 1 .

[0058] The host bus 904 is connected to an external bus 906, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 905. It is not necessary to configure the host bus 904, bridge 905, and external bus 906 separately, and these functions may be implemented on a single bus.

[0059] The input device 908 is composed of input means for the user to input information, such as a mouse, keyboard, touch panel, button, microphone, switch, and lever, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. By operating this input device 908, the user who operates the information processing device 900 can input various data to the information processing device 900 and instruct the information processing device 900 to perform processing operations.

[0060] The output device 909 may include, for example, a display device that outputs visual information, such as a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, or a lamp. The output device 909 may include an audio output device that outputs auditory information, such as a speaker. The output device 909 may include a tactile presentation device that outputs tactile information, such as an eccentric motor. The output device 909 may, for example, output information indicating objects detected by the object detection unit 33, or output information that warns of collisions between detected objects.

[0061] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded on the storage medium. The storage device 910 is configured, for example, with an HDD (Hard Disk Drive). This storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data. The storage device 910 may store, for example, information indicating the processing range R set by the processing range setting unit 32.

[0062] The communication device 911 is, for example, a communication interface configured with a communication device for connecting to a network, etc. The communication device 911 may be compatible with either wireless communication or wired communication.

[0063] The above describes an example of a hardware configuration capable of realizing the functions of the information processing device 900 according to this embodiment. Each of the above components may be realized using general-purpose components, or may be realized by hardware specialized for the function of each component. Therefore, the hardware configuration used can be changed as appropriate depending on the technical level at the time of implementing this embodiment.

[0064] <6. Supplementary Information> Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modifications and alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention.

[0065] In the above embodiment, an example has been described in which the distance from the point cloud sensor 10 to the specific position is calculated based on point cloud data acquired by the point cloud sensor 10. However, the present invention is not limited to such an example. The distance from the point cloud sensor 10 to the specific position may be calculated based on other data acquired by another sensor. As an example, the distance from the point cloud sensor 10 to the specific position may be calculated based on data obtained by the point cloud sensor 10 and another sensor that measures the three-dimensional position of the specific position, and the relative positional relationship between the other sensor and the point cloud sensor 10. As another example, the distance from the point cloud sensor 10 to the specific position may be calculated based on geographical position information of the point cloud sensor 10 and the specific position acquired by a Global Navigation Satellite System (GNSS) or the like. As another example, a sensor that measures altitude may be placed at each point on the ground G, and the vertical distance from the point cloud sensor 10 to each point may be calculated based on the measured altitude of each point.

[0066] In the above embodiment, an example has been described in which the present invention is applied to a construction site where a tunnel T is being excavated, but the application of the present invention is not limited to such an example. As an example, the present invention may be applied to a road construction site. In that case, for example, a processing range R corresponding to the space from the point cloud sensor 10 to a restricting material such as a traffic cone or a fence may be set, and the processing range R may be updated according to the position of the restricting material that moves as the work progresses.

[0067] The series of processes performed by each device described herein may be implemented using software, hardware, or a combination of software and hardware. The software programs may be stored in advance, for example, on a recording medium (more specifically, a non-transitory computer-readable storage medium) internal or external to each device. Each program is loaded into a random access memory (RAM) and executed by a processing circuit such as a central processing unit (CPU). The recording medium may be, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory. The computer program may be distributed, for example, via a network without using a recording medium. The computer may be an application-specific integrated circuit (ASIC), a general-purpose processor that executes functions by loading a software program, or a computer on a server used in cloud computing. The series of processes performed by each device described herein may be centrally processed by a single computer or distributed across multiple computers. Furthermore, in each of the above embodiments, two or more communication means present in one device may be physically implemented on a single medium.

[0068] Note that each device described in this specification may be realized as a single device, or some or all of them may be realized as separate devices. As an example, the detection device 30 may be configured as a server on the cloud and connected to the point cloud sensor 10 via a network or the like. As another example, the point cloud sensor 10 and the detection device 30 may be configured as a single device.

[0069] Furthermore, the processes described herein using flowcharts or sequence diagrams do not necessarily have to be performed in the order shown. Some process steps may be performed in parallel. Furthermore, additional process steps may be employed, and some process steps may be omitted. [Explanation of symbols]

[0070] 1 System 10 Point Cloud Sensor 30 Detection device 31 Distance calculation unit 32 Processing range setting section 33 Object detection unit

Claims

1. a distance calculation unit that calculates a distance from the point cloud sensor to a specific position; a processing range setting unit that sets a processing range based on the distance calculated by the distance calculation unit and extracts the point cloud data included in the processing range from the point cloud data acquired by the point cloud sensor; an object detection unit that detects an object based on the point cloud data included in the processing range extracted by the processing range setting unit; A detection device comprising:

2. when a distance from the point cloud sensor to the specific position changes, the processing range setting unit sets the processing range based on the changed distance. The detection device according to claim 1 .

3. the processing range setting unit sets, as the processing range, a space whose distance from the point cloud sensor is shorter than the distance calculated by the distance calculation unit; The detection device according to claim 1 .

4. the specific position is a position of an end in a horizontal direction of a space in which an object is to be detected by the object detection unit, the distance calculation unit calculates a horizontal distance from the point cloud sensor to the specific position; the processing range setting unit sets, as the processing range, a space whose horizontal distance from the point cloud sensor is shorter than the horizontal distance calculated by the distance calculation unit. The detection device according to claim 3 .

5. The specific position is the position of the tunnel face, The detection device according to claim 4 .

6. the specific position is a position on the ground, the distance calculation unit calculates a vertical distance from the point cloud sensor to the specific position; the processing range setting unit sets, as the processing range, a space whose vertical distance from the point cloud sensor is shorter than the vertical distance calculated by the distance calculation unit. The detection device according to claim 1 .

7. the processing range setting unit sets a plurality of the processing ranges, each of the plurality of processing ranges is a space that is mutually exclusive in the horizontal direction, and is a space that is located at a shorter vertical distance from the point cloud sensor than the highest point on the ground in the vertical direction of the space; The detection device according to claim 6.

8. 1. A computer-implemented detection method comprising: Calculating a distance from the point cloud sensor to a specific position; setting a processing range based on the calculated distance, and extracting the point cloud data included in the processing range from the point cloud data acquired by the point cloud sensor; Detecting an object based on the point cloud data included in the extracted processing range; A detection method comprising:

9. A point cloud sensor and a detection device are provided, The detection device includes: a distance calculation unit that calculates a distance from the point cloud sensor to a specific position; a processing range setting unit that sets a processing range based on the distance calculated by the distance calculation unit and extracts the point cloud data included in the processing range from the point cloud data acquired by the point cloud sensor; an object detection unit that detects an object based on the point cloud data included in the processing range extracted by the processing range setting unit; A system having:

Images