Object detection device, system, method, and program

Abstract

To provide an object detection device and the like that can contribute to efficiently performing operations to set up a new no trespassing space.SOLUTION: An object detection device comprises: a no trespassing area designation unit configured to detect marks corresponding to a mode related to a preset mark in photographic data, create one or more no trespassing spaces according to a preset form on the basis of, the detected marks, and selects any of the created no trespassing spaces; and a detection processing unit configured to, based on a point group in the selected no trespassing space in the photographic data, detect a detection target object matching preset or newly set detection condition information.SELECTED DRAWING: Figure 18

Description

Technical Field

[0001] The present invention relates to an object detection device, a system, a method, and a program.

Background Art

[0002] In a work site, for the safety management of workers, 3D (dimension) cameras (such as 3D-LIDAR (Laser Imaging Detection and Ranging), ToF (Time of Flight) cameras, stereo cameras, 3D scanners, etc.) are utilized to set a restricted area (intrusion prohibited space) that prohibits the intrusion of people and objects in an area with risks, and a technique for monitoring the restricted area to detect the intrusion of an object exists (for example, see Patent Documents 1 to 5).

[0003] Patent Document 1 discloses an intrusion detection device that includes a portable sensor unit, an information acquisition unit that acquires new sensor observation information detected by the portable sensor unit from the portable sensor unit, an area setting unit that sets a new detection area within an observation area corresponding to the new sensor observation information, and an intrusion detection unit that detects the intrusion of a moving object into the new detection area. The area setting unit sets the new detection area based on a comparison between the new sensor observation information and past sensor observation information and past detection area information. In the intrusion detection device described in Patent Document 1, a 3D camera is installed so that a scene for which safety management is desired fits within the range that can be photographed by the 3D camera, and the area to be monitored is grasped as a "3D space", and one or more restricted areas are set. When setting the restricted area, the site is photographed with a 3D camera, the angle of view is adjusted, and the restricted area is set while referring to the photographed 3D point cloud data so that the scene for which safety management is desired fits within the angle of view of the 3D camera.

[0004] In Patent Document 2, as a technique for providing a simple means for designating an image to be projected onto a projection device, a projection device capable of projecting a predetermined range, and a photographing device capable of photographing the range that can be projected by the projection device are connected. Based on the image photographed by the photographing device, a figure recognition unit that recognizes a figure drawing operation by the user, and a projection control unit that instructs the projection device to project a figure based on the figure drawing operation are provided. In the projection image designation device described in Patent Document 2, the area to be monitored is set by performing an installation operation of three or more markers as a figure drawing operation by the user.

[0005] In Patent Document 3, as a technique for enabling the accurate positions of various objects such as workers, utility poles, overhead lines, and trees existing in the vicinity to be determined with reference to a construction machine, a laser scanner device, distance data acquisition means for acquiring distance data for each scan position in the peripheral space based on the reflected light in the peripheral space of the light periodically emitted into the peripheral space from the laser scanner device, distance calculation means for calculating the distance to the worker based on the received radio wave from the ID transmission means worn by the worker, object position detection means for detecting the position of an object in the peripheral space based on the distance data, worker position detection means for detecting the position of the worker based on the distance calculated by the distance calculation means and the position detected by the object position detection means, monitoring area setting means for setting a monitoring area, first alarm output means for outputting an alarm based on the fact that the object detected by the object position detection means exists in the monitoring area, and second alarm output means for outputting an alarm based on the fact that the worker detected by the worker position detection means exists in the monitoring area are provided. In the construction machine described in Patent Document 3, the monitoring area is dynamically set according to the turning range of the working device.

[0006] Patent Document 4 discloses a robot system including an imaging unit that acquires three-dimensional and two-dimensional images of a predetermined monitoring area as a technology for recognizing an object whose shape changes, an object extraction unit that extracts an area having pixel values within a predetermined range in the acquired three-dimensional image, an image search unit that searches for a reference image registered in advance according to the type of the object in the two-dimensional image, and a determination unit that determines the type of the object based on whether the reference image searched by the image search unit exists within the area extracted by the object extraction unit, a robot that works in a predetermined work area within the monitoring area, and a robot control unit that controls the robot. When the object recognition device recognizes that the object is a safety monitoring target, it detects the intrusion of the object into the work area, and when the robot control unit detects the intrusion of the object into the work area by the object recognition device, it stops the robot. In the robot system described in Patent Document 4, the monitoring area is set to include the work area of the robot installed at a predetermined position.

[0007] Patent Document 5 discloses a robot system for avoiding contact between a human and an industrial robot, including a robot, a sensor that detects the intrusion of a human into the peripheral area of the robot, and a control device that controls the robot and the sensor. The control device includes a robot control unit that controls the robot based on an operation program, an operation area prediction unit that sequentially predicts the operation area of the robot until a predetermined time based on the operation program, and a detection area control unit that effectively sets the detection result of the sensor in an area including the operation area predicted by the operation area prediction unit within the peripheral area and invalidates the detection result of the sensor in other peripheral areas. The robot control unit restricts the operation of the robot when the intrusion of a human is detected within the area where the detection result by the sensor is effectively set. In the robot system described in Patent Document 5, the area detectable by the sensor is set in the peripheral area of the robot installed at a predetermined position.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

[0009] The following analysis is provided by the inventor of the present application.

[0010] However, in the intrusion detection device described in Patent Document 1, since it is necessary to install a 3D camera inside the restricted entry area surrounded by the triangular cones, it is not possible to monitor multiple restricted entry areas with a single 3D camera, and there is no freedom in the installation position of the 3D camera.

[0011] Also, in railway construction, for example, in the case of double tracks, triple tracks, etc., the track to be constructed shifts according to the time zone according to the construction plan, and accordingly the restricted entry area also changes. However, in the intrusion detection device described in Patent Document 1, every time, fine tuning operations such as setting the restricted entry area while adjusting the space are required, which is time-consuming and it is difficult to immediately operate the intrusion detection for safety management. The same can be said for the projection image designation device described in Patent Document 2 that sets the area to be monitored by performing the marker installation operation.

[0012] In addition, the safety management of workers at the construction site is not limited to the ground, and there are also high-altitude operations such as on utility poles and towers, and it is necessary to set a restricted entry area for entry into the air. However, in the intrusion detection device described in Patent Document 1, it is not possible to set a restricted entry area for entry into the air by placing the triangular cones that serve as the marks of the restricted entry area.

[0013] In addition, in the intrusion detection device described in Patent Document 1, since a triangular cone serving as a mark for the restricted entry area is placed to set the area, the triangular cone may interfere with the work.

[0014] Furthermore, the construction machinery described in Patent Document 3 and the robot systems described in Patent Documents 4 and 5 only perform monitoring within a preset range centered on the construction machinery or the robot, and are not configured to set a new restricted entry space.

[0015] A main object of the present invention is to provide an object detection device, a system, a method, and a program that can contribute to efficiently performing an operation of setting a new restricted entry space.

Means for Solving the Problems

[0016] The object detection device according to the first aspect detects a mark corresponding to a form related to a preset mark in the captured data, creates a plurality of restricted entry spaces according to a preset form based on the detected mark, and, a restricted entry specifying unit configured to select any one of the plurality of created restricted entry spaces, space and a detection processing unit configured to detect a detection target object that matches preset or newly set detection condition information based on the point cloud in the selected restricted entry space in the captured data.

[0017] The object detection system according to the second aspect includes a three-dimensional sensor configured to output captured data obtained by capturing a predetermined space, and the object detection device according to the first aspect.

[0018] The object detection method according to the third aspect is an object detection method for detecting an object using hardware resources. In the captured data, a mark corresponding to a form related to a preset mark is detected, and a plurality of restricted entry spaces are created according to a preset form based on the detected mark. and,selecting any one of the created plurality of intrusion prohibited spaces; and detecting a detection target object that matches detection condition information set in advance or newly based on a point group within the selected intrusion prohibited space in the captured data.

[0019] The program according to the fourth viewpoint is a program for causing hardware resources to execute a process of detecting an object, and in the captured data, detects a mark corresponding to a form related to a preset mark, and creates a plurality of intrusion prohibited spaces according to a preset form based on the detected mark. and, causing the hardware resources to execute a process of selecting any one of the created plurality of intrusion prohibited spaces and a process of detecting a detection target object that matches detection condition information set in advance or newly based on a point group within the selected intrusion prohibited space in the captured data.

[0020] Note that the program can be recorded on a computer-readable storage medium. The storage medium can be a non-transient one such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium, etc. Also, in the present disclosure, it is also possible to embody it as a computer program product. The program is input into a computer device via an input device or from the outside via a communication interface, stored in a storage device, drives a processor according to a predetermined step or process, and can display the processing result including an intermediate state step by step via a display device as necessary, or communicate with the outside via a communication interface. A computer device for that purpose typically includes a processor, a storage device, an input device, a communication interface, and a display device that can be connected to each other by a bus as an example.

Advantages of the Invention

[0021] According to the first to fourth viewpoints, it is possible to contribute to efficiently performing the work of setting a new intrusion prohibited space.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Embodiments for Carrying out the Invention

[0023] Hereinafter, embodiments will be described with reference to the drawings. In the present application, when reference numerals are attached to the drawings, they are for assisting understanding only and are not intended to limit the illustrated modes. Further, the following embodiments are merely examples and do not limit the present invention. Also, the connection lines between blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional ones. The one-way arrow schematically shows the flow of the main signal (data) and does not exclude bidirectionality. Furthermore, in the circuit diagrams, block diagrams, internal configuration diagrams, connection diagrams, etc. shown in the present application disclosure, although not explicitly shown, input ports and output ports exist at the input ends and output ends of each connection line respectively. The same applies to the input / output interface. The program is executed via a computer device, and the computer device includes, for example, a processor, a storage device, an input device, a communication interface, and a display device as required. The computer device is configured to be able to communicate with devices inside or outside the device (including computers) via the communication interface, whether wired or wireless.

[0024] [Embodiment 1] The object detection system according to Embodiment 1 will be described with reference to the drawings. FIG. 1 is a block diagram schematically showing the configuration and one usage mode of the object detection system according to Embodiment 1. FIG. 2 is a block diagram schematically showing the configuration of the object detection device in the object detection system according to Embodiment 1.

[0025] The object detection system 1 is a system that detects a detection target object 11 existing in the intrusion prohibited space 31 using the three-dimensional sensor 300 (see FIG. 1). In the object detection system 1, the three-dimensional sensor 300, the object detection device 200, and the mobile communication terminal 500 are communicably connected via the network 400. In the object detection system 1, the object detection device 200 selectively detects the detection target object 11 existing in the intrusion prohibited space 31 based on the captured data (100 in FIG. 2) captured by the three-dimensional sensor 300. Here, the captured data 100 is three-dimensional point cloud data created by capturing with the three-dimensional sensor 300. The object detection system 1 has a function of recognizing a landmark (the line 14 in FIG. 1) and creating the intrusion prohibited space 31. The object detection system 1 has a function of displaying the detected detection target object 11 together with the segment 40a, for example, as illustrated in FIG. 3.

[0026] The 3D sensor 300 is a sensor that senses the surface of an object (including the detection target object 11, non-detection target object 12, line 14, ground, etc. in the imaging space (20 in FIG. 4)) and performs 3D imaging (see FIG. 1). The 3D sensor 300 is communicably connected to the object detection device 200 and the mobile communication terminal 500 via the network 400. The 3D sensor 300 captures the imaging space 20 to create imaging data (100 in FIG. 2), and outputs the created imaging data 100 toward the object detection device 200 and the mobile communication terminal 500. Note that the imaging data 100 is point cloud data (3D point cloud data) drawn in a point cloud (3D coordinates of a large number of points). The 3D sensor 300 can be selected according to environmental conditions such as the imaging distance, angle of view, indoor / outdoor, presence / absence of sunlight, etc. required for detecting the detection target object 11 and customer requirements. Also, products sold by various manufacturers can be used for the 3D sensor 300. As the 3D sensor 300, for example, a 3D sensor such as a stereo camera, a ToF (Time of Flight) camera, a 3D-LIDAR (Three Dimensions - Laser Imaging Detection and Ranging) can be used. The 3D sensor 300 is installed outside the intrusion prohibited space 31 created by the object detection device 200 or outside the area of the construction site. Thereby, a plurality of intrusion prohibited spaces can be set, and it is also possible to change the installation location and utilize it. The 3D sensor 300 is attached or installed at a position where the imaging space 20 can be imaged. If there is no suitable support (such as a wall or a pillar) for attachment or installation, it may be attached or installed using a tripod 301 as shown in FIG. 1. The 3D sensor 300 only needs to be at least one in the imaging space 20, but there may be a plurality in the imaging space 20. Also, when the imaging space 20 is vast and it is difficult to image with a single sensor, a plurality of 3D sensors 300 may be used. Note that when there are a plurality of 3D sensors 300 in the imaging space 20, they may be of different types or manufacturers. Also, when there are a plurality of 3D sensors 300 in the imaging space 20, the imaging data captured by each 3D sensor 300 may be synthesized by the object detection device 200.

[0027] Here, the imaging space 20 is a space in which an intrusion prohibited space (31 in FIG. 1) can be set, and is a space capable of detecting an object that has intruded into the intrusion prohibited space. Note that the intrusion prohibited space (31 in FIG. 1) is a space that prohibits the intrusion of an object.

[0028] The network 400 is a wired or wireless communication network that communicably connects between the three-dimensional sensor 300, the object detection device 200, and the mobile communication terminal 500. For the network 400, for example, communication networks such as PAN (Personal Area Network), LAN (Local Area Network), MAN (Metropolitan Area Network), WAN (Wide Area Network), and GAN (Global Area Network) can be used.

[0029] The mobile communication terminal 500 is a terminal capable of displaying information from the three-dimensional sensor 300 and the object detection device 200 (see FIG. 1). The mobile communication terminal 500 is used, for example, by users such as on-site supervisors and workers. As the mobile communication terminal 500, a portable communication terminal is used, and for example, a smartphone, a mobile phone, a tablet terminal, etc. can be used. The mobile communication terminal 500 is communicably connected to the object detection device 200 and the three-dimensional sensor 300 via the network 400. The mobile communication terminal 500 can receive and display the imaging data (100 in FIG. 2) from the three-dimensional sensor 300. The mobile communication terminal 500 can receive and display the result (110 in FIG. 2) and warning information (when received) from the object detection device 200.

[0030] The object detection device 200 is a device that creates a prohibited entry space 31 based on a predetermined landmark in the captured data 100 and selectively detects a detection target object 11 that has entered the created prohibited entry space 31 (see FIGS. 1 and 2). The object detection device 200 is used by a user such as a monitor. The object detection device 200 can use a device (computer device) having functional units (for example, a processor, a storage device, an input device, a communication interface, and a display device) that constitute a computer. As the object detection device 200, for example, hardware resources such as a computer, a personal computer, a notebook personal computer, and a tablet terminal can be used. The object detection device 200 has a time function. The object detection device 200 realizes a preprocessing unit 210, a prohibited entry space specifying unit 220, a detection processing unit 230, a result generation unit 240, and an interface unit 250 by executing a predetermined program (software).

[0031] The preprocessing unit 210 is a functional unit that performs predetermined preprocessing (here, format conversion and noise removal) on the captured data 100 (see FIG. 2). The preprocessing unit 210 acquires the captured data 100 from a sensor (300 in FIG. 1). The preprocessing unit 210 outputs the preprocessed captured data 100 to the prohibited entry space specifying unit 220, the detection processing unit 230, and the display unit 252 of the interface unit 250. The preprocessing unit 210 includes a format conversion unit 211 and a noise removal unit 212.

[0032] The format conversion unit 211 is a functional unit that converts the format of the captured data 100, which differs depending on the type of sensor (300 in FIG. 1), into a common format that can be commonly used in the object detection device 200 (see FIG. 2). The format conversion unit 211 acquires the captured data 100 from a sensor (300 in FIG. 1). The format conversion unit 211 outputs the captured data 100 converted into the common format to the noise removal unit 212.

[0033] The noise removal unit 212 is a functional unit that removes noise (point clouds unnecessary for detection) from the point clouds in the captured data 100 (see FIG. 2). The noise removal unit 212 acquires the captured data 100 from the format conversion unit 211. The noise removal unit 212 outputs the captured data 100 from which noise has been removed to the intrusion prohibition space specifying unit 220, the detection processing unit 230, and the display unit 252 of the interface unit 250. Examples of noise removal methods include smoothing processing, filtering (e.g., moving average filter processing, median filter processing, etc.), and outlier removal processing (e.g., outlier removal processing by chi-square test).

[0034] The intrusion prohibition space specifying unit 220 is a functional unit that specifies an intrusion prohibition space (31a in FIG. 8, 31b in FIG. 9) in the shooting space (20 in FIG. 4) of the shooting data 100 (see FIG. 2). The intrusion prohibition space specifying unit 220 acquires the pre-processed shooting data 100 from the pre-processing unit 210. The intrusion prohibition space specifying unit 220 detects marks (the lines 14a and 14b in FIG. 5) corresponding to a preset form (for example, a pair of parallel lines) related to the marks in the shooting space (20 in FIG. 4) of the shooting data 100, and creates detection lines (22a and 22b in FIG. 5) related to the marks. Based on the created detection lines (22a and 22b in FIG. 5), the intrusion prohibition space specifying unit 220 automatically creates one or more intrusion prohibition space candidates (30a and 30b in FIG. 6) according to a preset form. The intrusion prohibition space specifying unit 220 stores information related to the created intrusion prohibition space candidates (30a and 30b in FIG. 6). The intrusion prohibition space specifying unit 220 acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250. Based on the setting condition information (here, width, depth, height, angle), the intrusion prohibition space specifying unit 220 specifies or changes the setting conditions of the created intrusion prohibition space candidates (30a and 30b in FIG. 6) to create an intrusion prohibition space (31a in FIG. 7, 31b in FIG. 7). The intrusion prohibition space specifying unit 220 stores information related to the created intrusion prohibition space (31a in FIG. 7, 31b in FIG. 7). Note that when there is no setting condition information, the intrusion prohibition space specifying unit 220 can use the created intrusion prohibition space candidates (30a and 30b in FIG. 6) as the intrusion prohibition space as they are. Based on the setting condition information (here, setting time), the intrusion prohibition space specifying unit 220 outputs information related to the created intrusion prohibition space (31a in FIG. 8, 31b in FIG. 9) to the detection processing unit 230 and the result generation unit 240 during the set time (working time). The intrusion prohibition space specifying unit 220 includes a mark recognition unit 221, a space specifying unit 222, an angle specifying unit 223, and a time management unit 224.

[0035] The mark recognition unit 221 is a functional unit that, based on the pre-processed captured data 100 from the pre-processing unit 210, recognizes marks (such as lines 14a and 14b in FIG. 5) corresponding to a preset form (such as a pair of parallel lines in FIG. 5) related to the mark from within the capture space (20 in FIG. 4) and automatically creates intrusion prohibited space candidates (30a and 30b in FIG. 6) (see FIG. 2). The mark recognition unit 221 detects, based on the captured data 100, marks (such as lines 14a and 14b in FIG. 5) corresponding to a preset form (for example, a pair of parallel lines with respect to FIG. 4) in the capture space (20 in FIG. 4), and creates detection lines (such as 22a and 22b in FIG. 5) related to the marks. The mark recognition unit 221 automatically creates one or more intrusion prohibited space candidates (30a and 30b in FIG. 6) according to a preset form based on the created detection lines (such as 22a and 22b in FIG. 5). Thereby, the complexity of setting the intrusion prohibited space can be eliminated, and the customization of the intrusion prohibited space can be easily performed. The mark recognition unit 221 stores information related to the created intrusion prohibited space candidates (30a and 30b in FIG. 6). The mark recognition unit 221 outputs the information related to the created intrusion prohibited space candidates (30a and 30b in FIG. 6) to the space designating unit 222.

[0036] Here, as a landmark, for example, a continuous or intermittent structure or display existing as infrastructure such as a railway (14a, 14b in FIG. 4), an electric wire (17a, 17b in FIG. 12), a communication line, a lane, a pipeline, a guardrail, a utility pole, a guiding sign, a snow pole, etc. can be used. Also, as a landmark, for example, a triangular cone (see 15 in FIG. 10), a person (see 16 in FIG. 11), a pole, etc., which are installed by a person subsequently, may be used. Further, as a form related to a preset landmark, for example, a pair of parallel lines such as a railway, a suspension line such as an electric wire, a broken line or a continuous line such as a lane of a road, etc., a continuous or intermittent form (line, shape, etc.) existing as infrastructure can be used. Also, as a form related to a preset landmark, for example, a characteristic form of a landmark such as a triangular pyramid surface, a human-shaped surface, a cylindrical surface, etc. can be used. As a preset form, for example, a box-shaped (cuboid) space with a predetermined or arbitrary width, depth, and height including the created detection lines (22a, 22b in FIG. 5, 23 in FIG. 10, 24 in FIG. 11, 25a, 25b in FIG. 12), or a box-shaped (cuboid) space with a predetermined or arbitrary width, depth, and height at a predetermined distance from the created detection lines can be used. Note that the preset form is not limited to a box-shaped (cuboid) space, and may be a cylindrical shape, a polygonal prism shape, a spherical shape, etc.

[0037] The space specifying unit 222 is a functional unit that specifies prohibited entry spaces (31a, 31b in FIG. 7) in the captured data 100 (see FIG. 2). The space specifying unit 222 acquires the captured data 100 from the preprocessing unit 210, acquires information related to prohibited entry space candidates (30a, 30b in FIG. 6) from the landmark recognition unit 221, and acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250. The space specifying unit 222 stores the width, depth, and height among the acquired setting condition information. The space specifying unit 222 specifies the setting conditions of the acquired prohibited entry space candidates (30a, 30b in FIG. 6) based on the width, depth, and height among the acquired setting condition information, and creates prohibited entry spaces (31a, 31b in FIG. 7). Note that when there is no setting condition information, the space specifying unit 222 can use the acquired prohibited entry space candidates (30a, 30b in FIG. 6) as the prohibited entry spaces without newly creating prohibited entry spaces. The space specifying unit 222 stores information related to the created prohibited entry spaces (31a, 31b in FIG. 7). The space specifying unit 222 outputs the information related to the created prohibited entry spaces (31a, 31b in FIG. 7) to the angle specifying unit 223.

[0038] The angle specifying unit 223 is a functional unit that specifies the angle of the intrusion prevention space (31a, 31b in FIG. 7) with respect to the captured data 100 (see FIG. 2). The angle specifying unit 223 acquires the captured data 100 from the preprocessing unit 210, acquires information related to the intrusion prevention space (31a, 31b in FIG. 7) from the space specifying unit 222, and acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250. The angle specifying unit 223 stores the angle among the acquired setting condition information. The angle specifying unit 223 changes the angle of the intrusion prevention space 31 by rotating the point cloud of the captured data 100 (or the intrusion prevention spaces 31a, 31b from the space specifying unit 222) based on the angle among the acquired setting condition information. Note that when there is no setting condition information, the angle specifying unit 223 can keep the angle of the intrusion prevention space (31a, 31b in FIG. 7) acquired from the space specifying unit 222 without changing the angle of the intrusion prevention space 31. The angle specifying unit 223 stores information related to the intrusion prevention space (31a, 31b in FIG. 7) whose angle has been changed. The angle specifying unit 223 outputs information related to the intrusion prevention space (31a, 31b in FIG. 7) whose angle has been changed to the time management unit 224.

[0039] The time management unit 224 is a functional unit that sets the time (e.g., working hours: start time and end time) during which the intrusion prevention space (31a, 31b in FIG. 7) is valid for each intrusion prevention space (see FIG. 2). The time management unit 224 acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250. The time management unit 224 stores the setting time among the acquired setting condition information. Based on the setting time among the acquired setting condition information, the time management unit 224 selects the intrusion prevention space corresponding to the valid time, and outputs the information related to the selected intrusion prevention space to the detection processing unit 230. For example, based on the setting time, the time management unit 224 selects the intrusion prevention space 31a for the working hours from 23:00 to 1:29 as shown in FIG. 8, and outputs the information related to the intrusion prevention space 31a to the detection processing unit 230. As shown in FIG. 9, for the working hours from 1:30 to 3:29, the intrusion prevention space 31b can be selected and the information related to the intrusion prevention space 31b can be output to the detection processing unit 230. Thereby, the intrusion prevention space can be sequentially and automatically changed. When there is a change in the setting time of the setting condition information, the time management unit 224 changes the time zone for selecting the intrusion prevention spaces 31a and 31b accordingly. Thereby, depending on the progress of the construction work, it is also possible to advance, delay, or change the time for selecting the intrusion prevention space.

[0040] The detection processing unit 230 is a functional unit that performs detection processing of a detection target object (11 in FIG. 1) that matches the detection condition information based on the point cloud in the intrusion prohibited space (31 in FIG. 3) in the captured data 100 (see FIG. 2). The detection processing unit 230 acquires the captured data 100 from the preprocessing unit 210, acquires information related to the intrusion prohibited space 31 from the intrusion prohibited space designating unit 220, and acquires detection condition information (for example, height range, dimension range, volume range, detecting a moving object by checking in time series, information related to an exclusion area) from the operation unit 251 of the interface unit 250. The detection processing unit 230 not only automatically executes the detection processing periodically, but may also be arbitrarily manually executed by the operation of the operation unit 251 of the interface unit 250. The detection processing unit 230 outputs detection information (including detection conditions and information related to the detection processing) to the result generation unit 240. The detection processing unit 230 includes a condition specifying unit 231 and a detection unit 232.

[0041] The condition specifying unit 231 is a function for specifying detection conditions related to the detection target object 11 (see FIG. 2). The condition specifying unit 231 acquires and stores detection condition information (for example, height range, dimension range, volume range, detecting a moving object by checking in time series, exclusion area, etc.) from the operation unit 251 of the interface unit 250. The condition specifying unit 231 outputs the acquired detection conditions to the detection unit 232.

[0042] Here, examples of the detection condition information include a condition of detecting an object that is 90 cm or higher in height and not in the exclusion area, a condition of detecting an object that is moving and not in the exclusion area, and the like.

[0043] The detection unit 232 is a functional unit that selectively detects a detection target object (11 in FIG. 1) that matches the detection condition information (for example, height range, dimension range, volume range, detecting a moving object by checking in time series, exclusion area, etc.) based on the point cloud within the intrusion prohibited space (for example, 31 in FIG. 3) in the captured data 100. The detection unit 232 acquires the captured data 100 from the preprocessing unit 210, acquires information related to the intrusion prohibited space 31 from the intrusion prohibited space designating unit 220, and acquires the detection condition information from the condition designating unit 231. In the detection of the detection target object 11, for example, as shown in FIG. 3, first, the point cloud within the intrusion prohibited space 31 in the captured data (100 in FIG. 2) is deleted. Subsequently, one or a plurality of segments 40a are created that separate by the distance for each point cloud of the objects (detection target object 11, non-detection target object 12) in the intrusion prohibited space 31. Subsequently, based on the point cloud in the one or a plurality of created segments 40a, the detection target object 11 that matches the specified detection condition is detected. The detection unit 232 may label and hold the information related to the segment 40a of the detection target object 11 so that the detection target object 11 can be tracked. The detection unit 232 outputs detection information (including detection conditions, segment 40a) toward the result generation unit 240.

[0044] The result generation unit 240 is a functional unit that generates (or calculates) a result for notifying the administrator based on the information related to the intrusion prohibited space (31 in FIG. 3) and the detection information (including the detection conditions, segments (40a in FIG. 3)) (see FIG. 2). The result generation unit 240 acquires the information related to the intrusion prohibited space 31 from the intrusion prohibited space designating unit 220, and acquires the detection information (including the detection conditions, segments 40a) from the detection processing unit 230. The generated result can be the figure of the intrusion prohibited space (31 in FIG. 3) and the figure or specifications of the detection information (for example, the height, exclusion area, figure or height of the segment 40a in the detection conditions, etc.). The result generation unit 240 outputs the generated result toward the interface unit 250. The result generation unit 240 determines whether there is a segment 40a (for example, a labeled segment) of the detection target object 11 in the detection information, and when there is a segment 40a of the detection target object 11, it may output warning information toward the interface unit 250 together with the generated result. The result generation unit 240 includes a figure generation unit 241 and a specification calculation unit 242.

[0045] The figure generation unit 241 is a functional unit that generates a figure based on the information related to the intrusion prohibited space (31 in FIG. 3) and the detection information (including the detection conditions, segments (40a in FIG. 3)) (see FIG. 2). Examples of the generated figure include the figure of the intrusion prohibited space 31 and the figure of the detection information (for example, the figure of the exclusion area of the detection conditions, the figure of the segment 40a of the detection target object 11, etc.). The figure is generated considering the position. Also, in the generation of the figure, generation of some elements of the intrusion prohibited space 31 and the detection information can be omitted. Note that the figure generation unit 241 can be configured not to generate the figure of the segment 40a of the non-detection target object (12 in FIG. 7) in order to clarify the difference between the detection target object (11 in FIG. 1) and the non-detection target object (12 in FIG. 1). The figure generation unit 241 outputs the generated figure as a result toward the display unit 252 and the communication unit 253 of the interface unit 250.

[0046] The specification calculation unit 242 is a functional unit that calculates specifications (dimensions, volume, moving speed, etc.) based on detection information (including detection conditions, segments (40a in FIG. 3)) (see FIG. 2). Examples of the calculated specifications include the specifications of the detection information (for example, the height in the height range of the detection conditions, the height of segment 40a, etc.). The specification calculation unit 242 outputs the calculated specifications as a result to the display unit 252 and the communication unit 253 of the interface unit 250.

[0047] The interface unit 250 is a functional unit that exchanges information between the user and the object detection device 200 (see FIG. 2). The interface unit 250 includes an operation unit 251, a display unit 252, and a communication unit 253.

[0048] The operation unit 251 is a functional unit that receives user operations (see FIG. 2). The user outputs the data (for example, characters, numbers, positions, regions, etc.) input by operating the operation unit 251 (for example, keyboard operation, mouse click operation, touch panel tap operation, etc.) to the intrusion prohibited space specifying unit 220 or the detection processing unit 230. The operation unit 251 outputs information related to the conditions (width, depth, height, angle, setting time) input by the user's operation to the intrusion prohibited space specifying unit 220. The operation unit 251 outputs the detection conditions (for example, height range, dimension range, volume range, detecting a moving object by checking in time series, exclusion region, etc.) input by the user's operation to the detection processing unit 230. The operation unit 251 instructs the start and end of the overall processing in the functions of the object detection device 200 by the user's operation.

[0049] The display unit 252 is a functional unit that displays various types of information (see FIG. 2). The display unit 252 displays the pre-processed captured data 100 from the noise removal unit 212 of the pre-processing unit 210. The display unit 252 displays the result from the result generation unit 240 and the warning information (when acquired). As a way of displaying the warning information, for example, the background color can be changed or blinked. Regarding the displayed captured data 100 and results, they may be saved as screenshots, numerical data, etc. Note that the information displayed on the display unit 252 may be output as audio using a speaker, printed using a printer, or transmitted and output to other terminals.

[0050] The communication unit 253 is a functional unit that transmits and receives various types of information (see FIG. 2). The communication unit 253 transmits the result 110 from the result generation unit 240 and the warning information (when acquired) to the mobile communication terminal (500 in FIG. 1) via the network (400 in FIG. 1).

[0051] Next, an example of specifying an intrusion prohibited space using the object detection device in the object detection system according to Embodiment 1 will be described with reference to the drawings.

[0052] Here, FIGS. 4 to 9 show the procedure for specifying an intrusion prohibited space when using a line as a landmark. FIG. 10 shows an example of automatically creating candidates for an intrusion prohibited space when using a triangular cone as a landmark. FIG. 11 shows an example of automatically creating candidates for an intrusion prohibited space when using a person as a landmark. Further, FIG. 12 shows an example of automatically creating candidates for an intrusion prohibited space when using an electric wire as a landmark. FIG. 13 shows an example of automatically creating candidates for an intrusion prohibited space at a position a predetermined distance away from the detection line. Note that for the configuration parts of the object detection device, refer to FIGS. 1 and 2. Also, it is assumed that the object detection device 200 performs processing when the monitor operates the operation unit 251 of the interface unit 250.

[0053] FIG. 4 shows an image of the captured data 100 after preprocessing by the preprocessing unit 210 of the object detection device 200. In the imaging space 20 of the captured data 100, there is a non-detection target object 12.

[0054] FIG. 5 shows an image of the captured data 100 after the mark recognition unit 221 of the intrusion prohibition space specifying unit 220 of the object detection device 200 detects marks (tracks 14a, 14b) corresponding to a preset form (here, a pair of parallel lines) of the mark and creates detection lines 22a, 22b related to the mark.

[0055] FIG. 6 shows an image of the captured data 100 after the intrusion prohibition space specifying unit 220 of the object detection device 200 automatically creates corresponding intrusion prohibition space candidates 30a, 30b so as to include the detection lines 22a, 22b according to a preset form based on the created detection lines 22a, 22b.

[0056] FIG. 7 shows that in the space specifying unit 222 of the intrusion prohibition space specifying unit 220 of the object detection device 200, based on the width, depth, and height among the setting condition information, the conditions of the intrusion prohibition space candidates (30a, 30b in FIG. 6) (for the intrusion prohibition space candidate 30a, the coordinates (x 10 , y 10 , z 10 ) and the coordinates of its opposite pole (x 11 , y 11 , z 11 ), for the intrusion prohibition space candidate 30b, the coordinates (x 20 , y 20 , z 20 ) and the coordinates of its opposite pole (x 21 , y 21 , z 21 )) are specified to create the intrusion prohibition spaces 31a, 31b, and then shows an image of the captured data 100.

[0057] FIG. 8 shows an image of the captured data 100 after the intrusion prohibited space 31a is selected during the working hours from 23:00 to 1:29 based on the set time among the set condition information by the time management unit 224 of the intrusion prohibited space specifying unit 220 of the object detection device 200.

[0058] FIG. 9 shows an image of the captured data 100 after the intrusion prohibited space 31b is selected during the working hours from 1:30 to 3:29 based on the set time among the set condition information by the time management unit 224 of the intrusion prohibited space specifying unit 220 of the object detection device 200.

[0059] Here, for example, in railway construction, in the case of double tracks, triple tracks, etc., according to the construction plan, the track to be constructed shifts according to the time zone, so the intrusion prohibited space also changes accordingly. However, if the setting work associated with the switching of the intrusion prohibited space is manually performed every time, it is very time-consuming. Therefore, according to the automatic switching of the intrusion prohibited space automatically created as shown in FIGS. 8 and 9, the setting work associated with the switching of the intrusion prohibited space becomes unnecessary, and it becomes possible to automatically shift the created intrusion prohibited space together with the working hours.

[0060] FIG. 10 shows an image of the captured data 100 after automatically creating one or more intrusion prohibition space candidates 32 so as to include the detection line 23 based on the preset form, after the mark recognition unit 221 of the intrusion prohibition space designating unit 220 of the object detection device 200 detects a mark (triangle cone 15) corresponding to a form (here, a triangular vertical plane) related to a preset mark, creates a detection line 23 related to the mark. The subsequent creation and selection of the intrusion prohibition space are the same as in the cases of FIGS. 7 to 9. The triangle cone 15 serving as the mark has the characteristics of red / plastic, and when the triangle cone 15 is photographed by the 3D sensor 300, reflection intensity information can be acquired together with distance information. Therefore, since the triangle cone 15 can acquire reflection intensity information with a reflection intensity different from the surrounding point cloud information (size / shape), it is possible to automatically extract the triangle cone 15 as a mark from the captured captured data 100.

[0061] FIG. 11 shows an image of the captured data 100 after automatically creating an intrusion prohibition space candidate 33 so as to include the detection line 24 based on the preset form, after the mark recognition unit 221 of the intrusion prohibition space designating unit 220 of the object detection device 200 detects a mark (person 16) corresponding to a form (here, a human-shaped surface) related to a preset mark, creates a detection line 24 related to the mark. The subsequent creation and selection of the intrusion prohibition space are the same as in the cases of FIGS. 7 to 9.

[0062] For example, in close-range photography, the 3D sensor 300 can capture landmarks such as power lines and electric wires. However, when the shooting distance increases, the captured point cloud data becomes coarser, and there may be cases where the information of the landmarks cannot be clearly captured. In such cases, as shown in FIGS. 10 and 11, landmarks such as triangular cones and people can be utilized, and what appears in the point cloud data can be added as a landmark and used as a complementary method for setting the intrusion prohibited space. Note that after creating the intrusion prohibited space candidates, even if the landmarks (triangular cones and people) disappear, there is no problem because the intrusion prohibited space candidates are retained in the software. Also, in FIGS. 10 and 11, there are four triangular cones 15 and people 16 serving as landmarks, but three or more landmarks are sufficient.

[0063] FIG. 12 shows an image of the captured data 100 after automatically creating corresponding intrusion prohibited space candidates 34a, 34b so as to include the detection lines 25a, 25b according to a preset form, by detecting the landmarks (electric wires 17a, 17b) corresponding to the form (here, a catenary) related to the preset landmark in the landmark recognition unit 221 of the intrusion prohibited space specifying unit 220 of the object detection device 200 and creating the detection lines 25a, 25b related to the landmarks. Regarding the subsequent creation and selection of the intrusion prohibited space, it is the same as in the cases of FIGS. 7 to 9.

[0064] FIG. 13 shows an image of the captured data 100 after automatically creating intrusion prohibited space candidates 35a, 35b at a predetermined distance (height) from the detection line 26 according to a preset form, by detecting the landmark (triangular cone 18; other objects are also possible) corresponding to the form (here, a triangular pyramid surface) related to the preset landmark in the landmark recognition unit 221 of the intrusion prohibited space specifying unit 220 of the object detection device 200 and creating the detection line 26 related to the landmark. Regarding the subsequent creation and selection of the intrusion prohibited space, it is the same as in the cases of FIGS. 7 to 9. For example, since the height of the electric wire from the ground is often determined by the type such as low-voltage wire and high-voltage wire, in FIG. 13, the intrusion prohibited space candidates 35a, 35b are automatically created at a height corresponding to the type of the electric wire based on the landmark on the ground.

[0065] Next, the operation of the object detection device in the object detection system according to Embodiment 1 will be described with reference to the drawings. FIG. 14 is a flowchart diagram schematically showing the operation of the object detection device in the object detection system according to Embodiment 1. For the configurations of the object detection system and the object detection device, refer to FIGS. 1 and 2.

[0066] First, the preprocessing unit 210 of the object detection device 200 acquires the captured data 100 captured by the three-dimensional sensor 300 (step A1).

[0067] Next, the format conversion unit 211 of the preprocessing unit 210 converts the format of the acquired captured data 100 into a common format (step A2).

[0068] Next, the noise removal unit 212 of the preprocessing unit 210 removes the noise of the captured data 100 converted into the common format (step A3).

[0069] Next, the landmark recognition unit 221 of the intrusion prohibited space designating unit 220 detects landmarks (lines 14a, 14b in FIG. 5) corresponding to a preset form (for example, a pair of parallel lines in FIG. 4) in the shooting space (20 in FIG. 4) of the noise-removed captured data 100 (step A4).

[0070] Next, the landmark recognition unit 221 of the intrusion prohibited space designating unit 220 creates detection lines (22a, 22b in FIG. 5) related to the detected landmarks (lines 14a, 14b in FIG. 5) (step A5).

[0071] Next, the landmark recognition unit 221 of the intrusion prohibited space designating unit 220 automatically creates one or more intrusion prohibited space candidates (30a, 30b in FIG. 6) according to a preset form based on the created detection lines (22a, 22b in FIG. 5) (step A6).

[0072] Next, the intrusion prohibition space specifying unit 220 acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250 (step A7). If there is no setting condition information, step A7 is skipped.

[0073] Next, the space specifying unit 222 of the intrusion prohibition space specifying unit 220 specifies the setting conditions of the created intrusion prohibition space candidates (30a, 30b in FIG. 6) based on the width, depth, and height among the acquired setting condition information, and creates an intrusion prohibition space (31a, 31b in FIG. 7) (step A8). If the width, depth, or height among the setting condition information is missing, step A8 is skipped.

[0074] Next, the angle specifying unit 223 of the intrusion prohibition space specifying unit 220 changes the angle of the intrusion prohibition space (31a, 31b in FIG. 7) by rotating the point cloud of the captured data 100 (or the intrusion prohibition spaces 31a, 31b from the space specifying unit 222) based on the angle among the acquired setting condition information (step A9). If the angle among the setting condition information is missing, step A9 is skipped.

[0075] Next, the time management unit 224 of the intrusion prohibition space specifying unit 220 acquires setting condition information (width, depth, height, angle, setting time) from the operation unit 251 of the interface unit 250, and based on the setting time among the acquired setting condition information, selects the intrusion prohibition space (31a in FIG. 8) corresponding to the valid time (23:00 to 1:29 in FIG. 8) (step A10). If the setting time among the setting condition information is missing, either the intrusion prohibition space candidates (30a, 30b in FIG. 6) or the intrusion prohibition spaces (31a, 31b in FIG. 7) can be manually selected, or all of the intrusion prohibition space candidates (30a, 30b in FIG. 6) created in step A6 can be selected as the intrusion prohibition space, or all of the intrusion prohibition spaces (31a, 31b in FIG. 7) whose angles were changed in step A9 can be selected.

[0076] Next, the detection processing unit 230 of the object detection device 200 acquires detection condition information (for example, height range, dimension range, volume range, detecting a moving object by checking in time series, exclusion area, etc.) from the operation unit 251 of the interface unit 250 (step A11). If there is no detection condition information, step A11 is skipped.

[0077] Next, the detection processing unit 230 of the object detection device 200 uses the noise-removed captured data 100 and the selected intrusion prohibited space (31a in FIG. 8) to detect a detection target object 11 that matches the acquired detection condition information (for example, height range, dimension range, volume range, detecting a moving object by checking in time series, exclusion area, etc.) based on the point cloud in the intrusion prohibited space (31a in FIG. 8) in the captured data 100 (step A12). If there is no detection condition information, the preset initial detection condition information or the detection condition information acquired last time can be used. Also, in the detection of the detection target object 11, the point cloud in the intrusion prohibited space 31 in the captured data 100 is deleted, and a segment 40a is created to separate the point clouds of the objects (detection target object 11, non-detection target object 12) on the intrusion prohibited space 31 according to the distance for each point cloud. Then, based on the point cloud in each created segment 40a, the detection target object 11 that matches the specified detection condition is detected. When the detection target object 11 is detected, the segment 40a of the detection target object 11 is labeled. If the detection target object 11 is not detected in step A12, the process proceeds to step A13.

[0078] Next, the graphic generation unit 241 of the result generation unit 240 of the object detection device 200 uses the selected intrusion prohibited space (31a in FIG. 8) and the acquired detection information (particularly, the segment (40a in FIG. 3)) to generate the graphic of the intrusion prohibited space (31a in FIG. 8) and the detection information (step A13). Note that in the generation of the graphic, generation of some elements of the intrusion prohibited space (31a in FIG. 8) and the detection information can be omitted. Also, when generating the graphic, it is determined whether there is a segment 40a (with a label) of the detection target object 11 in the detection information, and if there is a segment 40a of the detection target object 11, warning information is generated. Also, in the generation of the graphic, after the graphic is generated, if there is no change in the intrusion prohibited space (31a in FIG. 8) and the detection information when newly generating the graphic, step A13 can be skipped while maintaining the graphic generated last time.

[0079] Next, the specification calculation unit 242 of the result generation unit 240 calculates specifications (dimensions, volume, moving speed, etc.) based on the detection information (detection conditions, segment 40a) (step A14). Note that in the calculation of the specifications, after the specifications are calculated, if there is no change in the detection information when newly calculating the specifications, step A14 can be skipped while maintaining the specifications calculated last time.

[0080] Finally, the display unit 252 of the interface unit 250 of the object detection device 200 displays the noise-removed photographed data 100, the generated graphic, and the calculated specifications (step A15). Also, when warning information is generated, the display unit 252 displays the warning information. When displaying, the communication unit 253 of the interface unit 250 transmits the displayed content (result) to the mobile communication terminal 500. Thereby, the mobile communication terminal 500 can also display the same content as that displayed on the display unit 252. After that, the flow ends and the next flow is executed.

[0081] According to Embodiment 1, since a new intrusion prohibited space candidate is automatically created by recognizing a predetermined mark in the photographed data 100, it is possible to contribute to efficiently performing the work of setting a new intrusion prohibited space.

[0082] [Embodiment 2] The object detection system according to Embodiment 2 will be described with reference to the drawings. FIG. 15 is a block diagram schematically showing the configuration of an object detection device in the object detection system according to Embodiment 2.

[0083] Embodiment 2 is a modification of Embodiment 1, and enables switching of the selection of the intrusion prohibition area by gesture instead of time. In the object detection device 200 in the object detection system according to Embodiment 2, a gesture recognition unit 260 is newly added, and in the intrusion prohibition space designating unit 220, a gesture instruction management unit 225 is provided instead of the time management unit (224 in FIG. 2) of Embodiment 1. Note that in the intrusion prohibition space designating unit 220, a configuration in which the time management unit (224 in FIG. 2) and the gesture instruction management unit 225 are provided side by side may also be adopted.

[0084] The gesture recognition unit 260 is a functional unit that recognizes the gesture (for example, predetermined body movements and hand movements) of a person (supervisor) in the captured data 100 and outputs selection instruction information (selection instruction or selection change instruction for the intrusion prohibition space) corresponding to the recognized gesture (see FIG. 15). The gesture recognition unit 260 acquires the captured data 100 pre-processed by the pre-processing unit 210. The gesture recognition unit 260 outputs selection instruction information corresponding to the recognized gesture toward the intrusion prohibition space designating unit 220. The gesture recognition unit 260 includes a gesture detection unit 261 and an instruction unit 262.

[0085] The gesture detection unit 261 is a functional unit that detects the gesture of the supervisor in the captured data 100 (see FIG. 15). The gesture detection unit 261 recognizes the gesture of the supervisor in the captured data 100 as a three-dimensional shape, and outputs information related to the recognized three-dimensional shape toward the instruction unit 262.

[0086] The instruction unit 262 is a functional unit that outputs selection instruction information related to an instruction to select or an instruction to change the selection of a prohibited space corresponding to the information related to the three-dimensional shape from the gesture detection unit 261 (see FIG. 15). The instruction unit 262 outputs the selection instruction information related to an instruction to select or an instruction to change the selection of a prohibited space to the gesture instruction management unit 225 of the prohibited space designation unit 220.

[0087] The gesture instruction management unit 225 is a functional unit that performs initial selection or selection change of a no-entry space (31a, 31b in FIG. 7) based on selection instruction information from the instruction unit 262 of the gesture recognition unit 260 (see FIG. 15). The gesture instruction management unit 225 acquires selection instruction information from the instruction unit 262 of the gesture recognition unit 260. The gesture instruction management unit 225 stores the acquired selection instruction information. The gesture instruction management unit 225 performs initial selection or selection change of a no-entry space (31a, 31b in FIG. 7) whose angle has been changed by the angle designation unit 223 based on the acquired selection instruction information. When there is no selection instruction information, the gesture instruction management unit 225 can manually select one of the entry-prohibited spaces (31a, 31b in FIG. 7) or can select all of the entry-prohibited spaces (31a, 31b in FIG. 7) whose angles have been changed, and can select one of the entry-prohibited spaces (31a, 31b in FIG. 7) according to time using the time management unit (224 in FIG. 2) of the first embodiment. The gesture instruction management unit 225 outputs information related to the selected entry-prohibited space to the detection processing unit 230 and the result generation unit 240.

[0088] Other configurations and operations in the second embodiment are similar to those of the object detection device in the object detection system according to the first embodiment.

[0089] According to the second embodiment, as in the first embodiment, new candidates for no-entry spaces are automatically created by recognizing specific landmarks in the photographed data 100, which contributes to efficient work of setting new no-entry spaces and enables no-entry spaces to be selected according to the gestures of the site supervisor who is familiar with the situation at the site.

[0090] [Embodiment 3] The object detection system according to Embodiment 3 will be described with reference to the drawings. FIG. 16 is a block diagram schematically showing the configuration of the object detection system according to Embodiment 3. FIG. 17 is a block diagram schematically showing the configuration of the object detection device in the object detection system according to Embodiment 3.

[0091] Embodiment 3 is a modification of Embodiment 1, and a function of adding location identification information for each automatically created intrusion prohibition space candidate is added. Embodiment 3 can be applied to Embodiment 2.

[0092] In the object detection system 1 according to Embodiment 3, a map information providing server 600 that provides map information to the object detection device 200 in response to a request from the object detection device 200 is added.

[0093] Also, in the object detection system 1 according to Embodiment 3, in the three-dimensional sensor 300, in addition to the three-dimensional sensor unit 310 that senses and photographs the surface of an object (including the detection target object 11, non-detection target object 12, track 14, ground, etc. in FIG. 4) within the photographing space (20 in FIG. 4), a position detection unit 320 that detects the position of the three-dimensional sensor 300 itself and an azimuth detection unit 330 that detects the azimuth of the line of sight of the three-dimensional sensor unit 310 are further provided (see FIG. 16).

[0094] The sensor unit 310 photographs the photographing space 20 to create photographing data (100 in FIG. 2), and outputs the created photographing data 100 toward the object detection device 200 and the mobile communication terminal 500 (see FIGS. 16 and 17). For the position detection unit 320, a GPS (Global Positioning System) receiver can be used. The position detection unit 320 outputs the detected position data 120 of the three-dimensional sensor 300 itself toward the object detection device 200. The azimuth detection unit 330 outputs the detected azimuth data 130 of the line of sight of the three-dimensional sensor unit 310 toward the object detection device 200.

[0095] Also, in the object detection system 1 according to Embodiment 3, a position specifying unit 226 that adds position specifying information to each of the automatically created intrusion prohibited space candidates is added to the intrusion prohibited space specifying unit 220 of the object detection device 200. The position specifying unit 226 acquires the captured data 100 from the preprocessing unit 210, acquires information related to the intrusion prohibited space candidates (30a and 30b in FIG. 6) from the landmark recognition unit 221, and acquires the position data 120 and the orientation data 130 from the three-dimensional sensor 300. Based on the acquired position data 120, the position specifying unit 226 acquires map information 140 of a predetermined range of the position related to the position data 120 from the map information providing server 600. Based on the acquired captured data 100, position data 120, orientation data 130, and map information 140, the position specifying unit 226 specifies the position for each of the intrusion prohibited space candidates (30a and 30b in FIG. 6), extracts the position specifying information related to the specified position from the map information 140, and adds (associates, links) the extracted position specifying information to the intrusion prohibited space candidates. The position specifying unit 226 outputs the information related to the intrusion prohibited space candidates to which the position specifying information is added toward the space specifying unit 222. The space specifying unit 222 creates an intrusion prohibited space using the information related to the intrusion prohibited space candidates (those to which the position specifying information is added) from the position specifying unit 226. As a result, it becomes easy to specify the position of the intrusion prohibited space.

[0096] In the specification of the position of the intrusion prohibited space candidate in the position specifying unit 226, based on the position data 120 and the orientation data 130 of the three-dimensional sensor 300 and the point group within the intrusion prohibited space candidate in the captured data 100, the position of the intrusion prohibited space candidate on the map information 140 can be specified. Also, in the acquisition of the position specifying information in the position specifying unit 226, based on the specified position of the intrusion prohibited space candidate, position specifying information related to the intrusion prohibited space candidate (for example, route information such as the Yamanote Line, the Chuo Line, the Sōbu Line, and the Saikyō Line if it is a railway near Shinjuku) can be acquired from the map information 140.

[0097] The other configurations and operations in Embodiment 3 are the same as those of the object detection device in the object detection system according to Embodiment 1.

[0098] According to Embodiment 3, similar to Embodiment 1, since a predetermined landmark in the captured data 100 is recognized to automatically create a new candidate for the intrusion prohibited space, it is possible to contribute to efficiently performing the work of setting a new intrusion prohibited space, and it becomes easy to specify the position of the intrusion prohibited space.

[0099] [Embodiment 4] The object detection system according to Embodiment 4 will be described with reference to the drawings. FIG. 18 is a block diagram schematically showing the configuration of the object detection system according to Embodiment 4. FIG. 19 is a block diagram schematically showing the configuration of the object detection device in the object detection system according to Embodiment 4.

[0100] Embodiment 4 is a modification of Embodiment 1. In order to make it easier to check the three-dimensional point cloud data obtained by photographing a landmark, in the object detection system 1, an RGB (Red Green Blue) camera 700 is installed together with the three-dimensional sensor 300 (see FIG. 18). Embodiment 4 can also be applied to Embodiments 2 and 3.

[0101] The RGB camera 700 is a camera that senses the color of an object in the imaging space (20 in FIG. 4) and performs two-dimensional imaging (see FIG. 18). The RGB camera 700 captures an image so as to include landmarks (14a and 14b in FIG. 4) in the imaging space (20 in FIG. 4) being captured by the three-dimensional sensor 300. The RGB camera 700 captures the imaging space (20 in FIG. 4) to create RGB data (150 in FIG. 19), and outputs the created RGB data 150 to the landmark recognition unit 221 of the intrusion prohibited space designation unit 220 of the object detection device 200.

[0102] Based on the captured data 100 and the RGB data 150, the mark recognition unit 221 detects a mark (the lines 14a and 14b in FIG. 5) corresponding to a preset form related to the mark (for example, a pair of parallel lines and the colors of the lines 14a and 14b in FIG. 4) in the captured space (20 in FIG. 4).

[0103] Other configurations and operations in Embodiment 4 are the same as those of the object detection device in the object detection system according to Embodiment 1.

[0104] According to Embodiment 4, similar to Embodiment 1, since a predetermined mark in the captured data 100 is recognized to automatically create a new intrusion prohibited space candidate, it can contribute to efficiently performing the work of setting a new intrusion prohibited space. At the same time, by adding RGB data 150 with a color visible to the naked eye in the real world to the captured data 100 (3D point cloud data) for sensor fusion (combining a plurality of sensors with different detection principles to enhance recognition), visibility is improved, and the recognition of the mark for setting the intrusion prohibited space can be enhanced.

[0105] [Embodiment 5] The object detection device according to Embodiment 5 will be described with reference to the drawings. FIG. 20 is a block diagram schematically showing the configuration of the object detection device according to Embodiment 5.

[0106] The object detection device 200 is a device that detects a predetermined object in the captured data 100. The object detection device 200 includes an intrusion prohibited space designating unit 220 and a detection processing unit 230.

[0107] The intrusion prohibited space designating unit 220 detects a mark corresponding to a preset form related to the mark in the captured data 100. The intrusion prohibited space designating unit 220 creates one or more intrusion prohibited spaces according to a preset form based on the detected mark. The intrusion prohibited space designating unit 220 selects one of the created intrusion prohibited spaces.

[0108] The detection processing unit 230 detects a detection target object that matches the detection condition information set in advance or newly, based on the point cloud within the selected intrusion prohibited space in the captured data 100.

[0109] According to Embodiment 5, since a new intrusion prohibited space is created by recognizing a predetermined mark in the captured data 100, it is possible to contribute to efficiently performing the work of setting a new intrusion prohibited space.

[0110] Note that the object detection device according to Embodiments 1 to 5 can be configured by so-called hardware resources (information processing device, computer), and a device having the configuration illustrated in FIG. 21 can be used. For example, the hardware resource 1000 includes a processor 1001, a memory 1002, a network interface 1003, etc., which are mutually connected by an internal bus 1004.

[0111] Note that the configuration shown in FIG. 21 is not intended to limit the hardware configuration of the hardware resource 1000. The hardware resource 1000 may include hardware not shown (for example, an input / output interface). Alternatively, the number of units such as the processor 1001 included in the device is not intended to be limited to the example of FIG. 21. For example, a plurality of processors 1001 may be included in the hardware resource 1000. For the processor 1001, for example, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), a GPU (Graphics Processing Unit), etc. can be used.

[0112] For the memory 1002, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. can be used.

[0113] For the network interface 1003, for example, a LAN (Local Area Network) card, a network adapter, a network interface card, etc. can be used.

[0114] The functions of the hardware resources 1000 are realized by the above-described processing modules. The processing modules are realized, for example, by the processor 1001 executing a program stored in the memory 1002. Also, the program can be downloaded via a network or updated using a storage medium storing the program. Further, the above processing module may be realized by a semiconductor chip. That is, the functions performed by the above processing module may be realized as long as software is executed in some hardware.

[0115] Some or all of the above embodiments may be described as follows in the appended claims, but are not limited thereto.

[0116] [Appended Claim 1] In the captured data, detect a mark corresponding to a form related to a preset mark, create one or more intrusion prevention spaces according to a preset form based on the detected mark, and an intrusion prevention area specifying unit configured to select any one of the created intrusion prevention spaces; A detection processing unit configured to detect a detection target object that matches detection condition information set in advance or newly based on the point cloud in the selected intrusion prevention space in the captured data; An object detection device comprising the above. [Appended Claim 2] Further comprising an operation unit that receives a user's operation, The intrusion prevention area specifying unit is In the captured data, a mark recognition unit configured to detect a mark corresponding to a form related to the preset mark and automatically create at least one intrusion prevention space candidate according to the preset form based on the detected mark; A space designating unit configured to create at least one prohibited entry space by designating the setting conditions of at least one candidate for a prohibited entry space based on the width, depth, and height included in the setting condition information input from the operation unit. A management unit configured to select any one of the at least one prohibited entry spaces based on a predetermined criterion. The object detection device according to Appendix 1, comprising the above. [Appendix 3] The object detection device according to Appendix 2, wherein the mark recognition unit detects a mark corresponding to the form related to the preset mark in the captured data, creates a detection line related to the detected mark, and automatically creates the at least one candidate for a prohibited entry space according to the preset form based on the created detection line. [Appendix 4] The object detection device according to Appendix 2 or 3, wherein the management unit is configured to select any one of the at least one prohibited entry spaces based on the selection information input from the operation unit as the predetermined criterion. [Appendix 5] The object detection device according to any one of Appendices 2 to 4, wherein the management unit is a time management unit configured to select any one of the at least one prohibited entry spaces corresponding to the set time based on the set time when the set time is included in the setting condition information as the predetermined criterion. [Appendix 6] The object detection device further includes a gesture recognition unit configured to recognize a person's gesture in the captured data and output selection instruction information corresponding to the recognized gesture. The object detection device according to any one of Appendices 2 to 5, wherein the management unit is a gesture instruction management unit configured to select any one of the at least one prohibited entry spaces corresponding to the selection instruction information based on the selection instruction information as the predetermined criterion. [Appendix 7] The intrusion prohibition area designating unit acquires position data and orientation data of a sensor that captures the captured data, and based on the acquired position data, acquires map information of a predetermined range of the position corresponding to the position data from a map information providing server. Based on the captured data, the position data, the orientation data, and the map information, the position of each intrusion prohibition space candidate is specified, position specifying information corresponding to the specified position is extracted from the map information, and the extracted position specifying information is added to the corresponding intrusion prohibition space candidate. The object detection device according to any one of Supplementary Notes 2 to 6 further includes a position specifying unit configured to output information related to the intrusion prohibition space candidate to which the position specifying information is added to the space designating unit. [Supplementary Note 8] The mark recognition unit acquires RGB data from an RGB camera configured to capture the captured data so as to include a mark in the captured data, and also uses the RGB data to detect a mark corresponding to a form related to the preset mark in the captured data. The object detection device according to any one of Supplementary Notes 2 to 7. [Supplementary Note 9] The form related to the preset mark is a continuous or intermittent form existing as infrastructure. The object detection device according to any one of Supplementary Notes 1 to 8. [Supplementary Note 10] A sensor configured to output captured data obtained by capturing a predetermined space, The object detection device according to any one of Supplementary Notes 1 to 9, An object detection system comprising: [Supplementary Note 11] An object detection method for detecting an object using hardware resources, In the captured data, detecting a mark corresponding to a form related to a preset mark, creating one or more intrusion prohibition spaces according to a preset form based on the detected mark, and selecting any one of the created intrusion prohibition spaces. Based on the point cloud within the selected intrusion prohibited space in the captured data, detecting a detection target object that matches detection condition information set in advance or newly; An object detection method including this. [Appendix 12] A program for causing hardware resources to execute a process of detecting an object, In the captured data, detecting a mark corresponding to a form related to a preset mark, creating one or more intrusion prohibited spaces according to a preset form based on the detected mark, and selecting any one of the created intrusion prohibited spaces; Based on the point cloud within the selected intrusion prohibited space in the captured data, detecting a detection target object that matches detection condition information set in advance or newly; A program for causing the hardware resources to execute this.

[0117] Note that each disclosure of the above patent documents is incorporated herein by reference and can be used as the basis or part of the present invention as necessary. Within the scope of the entire disclosure of the present invention (including the claims and drawings), further modifications and adjustments of the embodiments or examples can be made based on its basic technical idea. Also, within the scope of the entire disclosure of the present invention, various combinations or selections (including non-selections if necessary) of various disclosure elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. That is, the present invention naturally includes various deformations and corrections that those skilled in the art could make according to the entire disclosure including the claims and drawings and the technical idea. Also, regarding the numerical values and numerical ranges described in this application, even if not explicitly stated, any intermediate value, lower numerical value, and small range are considered to be described. Furthermore, each disclosure item of the above-cited documents is considered to be included in (belong to) the disclosure of this application as part of the disclosure of the present invention according to the spirit of the present invention and can be used in combination with the description items of this book, either in part or in whole, as necessary.

Explanation of Reference Signs

[0118] 1 Object detection system 11 Object to be detected 12 Non-detection target object 13 Train 14, 14a, 14b Track (mark) 15, 18 Cone (mark) 16 Person (mark) 17a, 17b Electric wire (mark) 20 Shooting space 22a, 22b, 23, 24, 25a, 25b, 26 Detection line 30, 30a, 30b, 32, 33 Prohibited intrusion space candidates 31, 31a, 31b Prohibited intrusion space 34a, 34b, 35a, 35b Prohibited intrusion space candidates 40a Segment 100 Shooting data 110 Result 120 Position data 130 Azimuth data 140 Map information 150 RGB data 200 Object detection device 210 Preprocessing unit 211 Format conversion unit 212 Noise removal unit 220 Prohibited intrusion space designation unit 221 Mark recognition unit 222 Space designation unit 223 Angle designation unit 224 Time management unit (management unit) 225 Gesture instruction management unit (management unit) 226 Position identification unit 230 Detection processing unit 231 Condition designation unit 232 Detection unit 240 Result generation unit 241 Graphic generation unit 242 Specification calculation unit 250 Interface unit 251 Operation unit 252 Display unit 253 Communication Unit 260 Gesture Recognition Unit 261 Gesture Detection Unit 262 Instruction Unit 300 3D Sensor 301 Tripod 310 3D Sensor Unit 320 Position Detection Unit 330 Orientation Detection Unit 400 Network 500 Mobile Communication Terminal 600 Map Information Providing Server 700 RGB Camera 1000 Hardware Resources 1001 Processor 1002 Memory 1003 Network Interface 1004 Internal Bus

Claims

1. In the captured data, a mark corresponding to a form related to a preset mark is detected, a plurality of intrusion prohibited spaces are created according to a preset form based on the detected mark, and an intrusion prohibited space designating unit configured to select any one of the plurality of created intrusion prohibited spaces; A detection processing unit configured to detect a detection target object that matches detection condition information set in advance or newly based on the point group in the selected intrusion prohibited space in the captured data; An object detection device comprising:

2. Further comprising an operation unit that receives a user's operation, The intrusion prohibited space designating unit: A mark recognition unit configured to detect a mark corresponding to a form related to the preset mark in the captured data and automatically create a plurality of intrusion prohibited space candidates according to the preset form based on the detected mark; A space designating unit configured to create a plurality of intrusion prohibited spaces by designating the setting conditions of the plurality of intrusion prohibited space candidates based on the width, depth, and height included in the setting condition information input from the operation unit; A management unit configured to select any one of the plurality of intrusion prohibited spaces based on a predetermined criterion; The object detection device according to claim 1, comprising:

3. The object detection device according to claim 2, wherein the mark recognition unit is configured to detect a mark corresponding to a form related to the preset mark in the captured data, create a detection line related to the detected mark, and automatically create the plurality of intrusion prohibited space candidates according to the preset form based on the created detection line.

4. The management unit is configured to select any one of the plurality of intrusion prohibited spaces based on the selection information input from the operation unit as the predetermined criterion, according to the object detection device of claim 2 or 3.

5. The management unit is a time management unit configured to select any one of the plurality of intrusion prohibited spaces corresponding to the set time from among the plurality of intrusion prohibited spaces based on the set time when the set time is included in the set condition information as the predetermined criterion, according to the object detection device of any one of claims 2 to 4.

6. The apparatus further includes a gesture recognition unit configured to recognize a person's gesture in the captured data and output selection instruction information corresponding to the recognized gesture. The management unit is a gesture instruction management unit configured to select any one of the plurality of intrusion prohibited spaces corresponding to the selection instruction information from among the plurality of intrusion prohibited spaces based on the selection instruction information as the predetermined criterion, according to the object detection device of any one of claims 2 to 5.

7. The intrusion prohibited space specifying unit further includes a position specifying unit configured to acquire position data and orientation data of a three-dimensional sensor that captures the captured data, acquire map information of a predetermined range of the position related to the acquired position data from a map information providing server based on the acquired position data, specify the position for each of the intrusion prohibited space candidates based on the captured data, the position data, the orientation data, and the map information, extract position specifying information related to the specified position from the map information, add the extracted position specifying information to the corresponding intrusion prohibited space candidate, and output information related to the intrusion prohibited space candidate to which the position specifying information is added to the space specifying unit, according to the object detection device of any one of claims 2 to 6.

8. The mark recognition unit acquires RGB data from an RGB camera that performs imaging so as to include a mark in the imaging data, and is configured to detect, in the imaging data, a mark corresponding to a form related to the preset mark using the RGB data as well. The object detection device according to any one of claims 2 to 7.

9. The form related to the preset mark is a continuous or intermittent form existing as infrastructure. The object detection device according to any one of claims 1 to 8.

10. A sensor configured to output imaging data obtained by imaging a predetermined space, The object detection device according to any one of claims 1 to 9, An object detection system comprising:

11. An object detection method for detecting an object using hardware resources, comprising: detecting, in the imaging data, a mark corresponding to a form related to a preset mark, creating a plurality of intrusion prohibited spaces according to a preset form based on the detected mark, and selecting any one of the created plurality of intrusion prohibited spaces; detecting a detection target object that matches preset or newly set detection condition information based on a point group in the selected intrusion prohibited space in the imaging data; An object detection method comprising:

12. A program for causing hardware resources to execute a process of detecting an object, comprising: detecting, in the imaging data, a mark corresponding to a form related to a preset mark, creating a plurality of intrusion prohibited spaces according to a preset form based on the detected mark, and selecting any one of the created plurality of intrusion prohibited spaces; detecting a detection target object that matches preset or newly set detection condition information based on a point group in the selected intrusion prohibited space in the imaging data; A program that causes to be executed on the hardware resources.

Images