Information processing apparatus, information processing method, and program

Abstract

To provide a technique that enables a user to easily recognize a critical dead angle area.SOLUTION: An information processing device comprises: distance information obtaining means that obtains distance information measured by a sensor; area information obtaining means that obtains information on a virtual three-dimensional protection area set in a region where the sensor can measure; dead angle calculating means that calculates a three-dimensional dead angle area which becomes a dead angle to the sensor, on the basis of the distance information; and display means that displays, on a virtual three-dimensional space, the protection area and the dead angle area.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In a scene where monitoring is required, such as a production site, a sensor is used to detect an object such as a human body. In order to reliably detect an object, it is required to install the sensor at an appropriate position and orientation, or to appropriately set the area to be monitored. However, it is not easy for the user to determine whether the installation position and orientation of the distance sensor are appropriate, or whether the area to be monitored is appropriately set.

[0003] Patent Document 1 discloses a system that uses a 3D sensor to monitor a work space for safety purposes. In this system, the three-dimensional space of the cone monitored by the 3D sensor is classified and displayed into a space marked as "unoccupied" where no obstacle is detected, a space marked as "occupied" where an object is detected, and a space marked as "unknown" that is a dead zone of the 3D sensor due to the detected object.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, it is not easy for the user to recognize whether the space marked as "unknown" in Patent Document 1 is a critical dead zone.

[0006] Therefore, an object of the present invention is to provide a technique that allows a user to easily recognize a critical dead zone.

Means for Solving the Problem

[0007] To achieve the above object, the present invention adopts the following configuration.

[0008] A first aspect of the present invention is a distance information acquisition means for acquiring distance information measured by a sensor, an area information acquisition means for acquiring information on a virtual three-dimensional protection area set within a measurable area of the sensor, a dead angle calculation means for calculating a three-dimensional dead angle area that is a dead angle of the sensor based on the distance information, and a display means for displaying the protection area and the dead angle area in a virtual three-dimensional space, and provides an information processing apparatus characterized by having the above.

[0009] The "sensor" is a sensor capable of measuring three-dimensional information, and as an example, it is a TOF (Time of Flight) sensor.

[0010] According to this configuration, based on the distance information measured by the sensor, a three-dimensional dead angle area that is a dead angle of the sensor is calculated, and the protection area and the dead angle area are displayed in a virtual three-dimensional space. As a result, the protection area and the dead angle area are visualized, and the user can visually confirm the relationship between the positions and sizes of the protection area and the dead angle area, so that a critical dead angle area can be easily recognized.

[0011] The display means may further display a point cloud based on the distance information in the virtual three-dimensional space. Each point constituting the point cloud corresponds to a point (measurement point) on the surface of an object whose distance is measured by the sensor. Therefore, by displaying the point cloud in a virtual three-dimensional space, the object (outer shape) existing within the measurable area can be represented. As a result, the positional and size relationships with the objects existing inside or near the protection area, and the dead angle areas caused by the objects can also be grasped.

[0012] The area information acquisition means may acquire a 3D model of an object existing within the measurable area, and the display means may further display the 3D model in the virtual three-dimensional space. The "3D model" is not data obtained by measurement with a sensor, but three-dimensional data that defines the outer shape of an object, such as CAD data. By performing display using such a 3D model, an object existing within the measurable area can be represented more precisely and accurately. Thereby, the positional and size relationships with objects existing inside or near the protection area, as well as dead angle areas caused by such objects, can also be grasped.

[0013] Further, a warning means for issuing a warning when at least a part of the dead angle area is within the protection area may be provided. When there is a dead angle of the sensor within the protection area, there is a risk that an intruder who has entered the protection area cannot be detected. Therefore, a state in which a dead angle area is included within the protection area is a critical problem. Thus, by issuing a warning as described above, the user can more easily recognize a critical dead angle area.

[0014] Further, the warning means may issue a danger warning when the size of the dead angle area within the protection area is equal to or greater than a threshold value. Thereby, when the dead angle area within the protection area is, for example, large enough for a person to enter, it is possible to inform the user that the sensor arrangement has a high degree of danger.

[0015] Further, the range of the protection area displayed in the virtual three-dimensional space may be changeable by a user operation. This facilitates appropriate setting of the protection area.

[0016] A working machine is installed within the measurable area, and the dead angle calculation means may calculate a dead angle area caused by the working machine. This is because there is a high possibility that a worker may approach near the working machine, and the dead angle area caused by the working machine is often critical.

[0017] Further, the working machine is a robot, and further includes a recognition unit that recognizes the operating range of the robot from the distance information measured by the sensor at a plurality of timings while moving the robot. The blind spot calculation unit calculates the blind spot area from the operating range of the robot, and the display unit may display the operating range by superimposing it on the virtual three-dimensional space. Thereby, the user can easily recognize the blind spot area as viewed from the sensor caused by the operation of the robot.

[0018] Further, when the blind spot area caused by the robot is within the protection area, an instruction transmission unit that transmits an instruction to change the operation range so that the blind spot area decreases may be further provided to the robot. Thereby, feedback of the measurement result by the sensor can be given to the robot.

[0019] Further, the recognition unit converts each of the distance information measured by the sensor at the plurality of timings into object range information indicating the range occupied by each object in the measurable area, and from the object range information, excluding the surrounding environment information indicating the range occupied by the objects other than the robot in the measurable area, generates actual operation information indicating the range occupied by the robot in the measurable area, and superimposes the actual operation information at the plurality of timings to calculate the operation range of the robot. Thereby, the determination of the operation range of the robot can be made in a short time.

[0020] Further, a setting unit that sets the surrounding environment information based on the distance information measured by the sensor in a state where the robot is stopped may be further provided. Thereby, the position of the object in the measurable area can be easily detected using the sensor.

[0021] The distance information measured by the sensor at the plurality of timings may be the distance information measured by the sensor while moving the robot with the same operation as during actual operation. Thereby, the same blind spot area as that generated during actual operation can be set.

[0022] The blind spot calculation means may further calculate the blind spot area based on the surrounding environment information. Thereby, the user can more easily recognize a critical blind spot area.

[0023] The robot further includes second blind spot calculation means for calculating a second blind spot area that can be a blind spot of the sensor from the maximum movable range of the robot, and the display means may further superimpose the second blind spot area on the virtual three-dimensional space in a state distinguishable from the blind spot area calculated from the operation range of the robot. Thereby, the user can grasp critical blind spot areas among the blind spot areas within the maximum movable range of the robot.

[0024] The robot further includes second blind spot calculation means for calculating a second blind spot area that can be a blind spot of the sensor from the maximum movable range of the robot, and the display means may switch between the blind spot area calculated from the operation range of the robot and the second blind spot area and superimpose and display them on the virtual three-dimensional space. Thereby, the user can grasp critical blind spot areas among the blind spot areas within the maximum movable range of the robot.

[0025] A second aspect of the present invention provides an information processing method, comprising: a step of acquiring distance information measured by a sensor; a step of acquiring information on a virtual three-dimensional protection area set within a measurable area of the sensor; a step of calculating a three-dimensional blind spot area that is a blind spot of the sensor based on the distance information; and a step of displaying the protection area and the blind spot area in a virtual three-dimensional space.

[0026] A third aspect of the present invention provides a program for causing a computer to execute each step of the above information processing method.

[0027] The present invention may be regarded as an information processing apparatus, a blind spot display apparatus, a blind spot confirmation apparatus, etc. having at least a part of the above means, or may be regarded as an object detection system, a monitoring system, etc. including these apparatuses and sensors. Further, the present invention may be regarded as an information processing method, a blind spot display method, a blind spot confirmation method, an object detection method, a monitoring method, a control method having at least a part of the above processing. Further, the present invention can also be regarded as a program for realizing such a method and a recording medium in which the program is non-temporarily recorded. Note that each of the above means or processing can be combined with each other as much as possible to constitute the present invention.

Effects of the Invention

[0028] According to the present invention, a user can easily recognize a critical blind spot area.

Brief Description of the Drawings

[0029]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0030] <Application Example> With reference to FIGS. 1 to 3, an application example of the information processing apparatus according to the present invention will be described. FIG. 1 is a block diagram of a monitoring system including an information processing apparatus 50 and a sensor 10 according to an embodiment of the present invention. FIG. 2 is a functional block diagram of a control unit 30. FIG. 3 is a schematic three-dimensional view of a site where the monitoring system is used.

[0031] As shown in FIG. 3, as a site where a three-dimensional distance sensor 10 (hereinafter referred to as sensor 10) that measures and outputs three-dimensional distance information is used, a production site where a working machine such as a robot 302 or a manufacturing apparatus (also called a "hazard source") produces while collaborating with an operator is assumed, and a place where object monitoring is required is envisioned. The monitoring of the object by the sensor 10 is performed in a virtual three-dimensional protection area 305 determined in consideration of the installation position and orientation of the sensor 10, the arrangement of the robot 302, and the safety distance, etc.

[0032] On the other hand, when the robot 302 is installed on the workbench 303 and moves on a moving path 306 which is a three-dimensional operation range during actual operation (in FIG. 3, only the area on the back side of the moving path 306 is shown when viewed from the sensor 10), a dead angle area 306a (FIG. 3) that becomes a dead angle of the sensor 10 is generated during actual operation. A dead angle area existing within the protection area 305 such as the dead angle area 306a becomes a critical dead angle area where the object cannot be detected by the sensor 10 despite the protection area 305. Therefore, the user needs to confirm the safety of such a critical dead angle area, but it is not easy for the user to recognize which area of the site is the critical dead angle area.

[0033] Therefore, in this embodiment, as shown in FIG. 3, the user installs the sensor 10 such that at least a part of the robot 302 enters the measurable region 301. In this state, the sensing unit 201 (FIG. 2) of the control unit 30 as the distance information acquisition means acquires the distance information (3D information) measured and output by the sensor 10. Then, the protection area setting unit 202 (FIG. 2) of the control unit 30 as the area setting means sets a virtual 3D protection area 305 within the measurable region 301 based on the installation position and orientation of the sensor 10, the arrangement of the robot 302, and the safety distance. The area information acquisition unit 203 (FIG. 2) of the control unit 30 as the area information acquisition means acquires the information of the protection area 305 and the 3D models of the respective objects (in the case of the example in FIG. 3, the robot 302, the workbench 303, and the shelf 304) existing in the measurable region 301 from the storage unit or an external computer. Further, the dead angle discrimination unit 204 (FIG. 2) of the control unit 30 as the dead angle calculation means calculates a 3D dead angle region (in the case of the example in FIG. 3, the dead angle regions 303a, 304a, 306a caused by the workbench 303, the movement path 306 of the robot 302, and the shelf 304 respectively) that is the dead angle of the sensor 10 based on the distance information measured by the sensor 10. The 3D space coordinate display unit 205 (FIG. 2) of the control unit 30 as the display means displays the protection area 305 and the dead angle region in a virtual 3D space. The 3D space coordinate display unit 205 may further display a point cloud based on the distance information measured by the sensor 10, the 3D model of the object acquired by the area information acquisition unit 203, etc. in the virtual 3D space together with the protection area 305 and the dead angle region.

[0034] Here, the warning unit 206 (FIG. 2) of the control unit 30 as the warning means may be configured to issue a warning when at least a part of the dead angle region is within the protection area 305.

[0035] As shown in FIG. 1, the sensor 10 includes a light emitting unit 41, a light receiving unit 42, and an arithmetic unit 43. The light emitting unit 41 emits light (for example, infrared light), and the light receiving unit 42 receives the reflected light. As an example, the sensor 10 acquires a distance image from the time of flight (TOF) of light. A TOF sensor is adopted. For example, an indirect TOF sensor that estimates the time difference from the phase difference between the projected light and the reflected light is adopted. The sensor 10 outputs the three-dimensional distance information of each position within the measurable region 301 as a measurement result. The measurement result is supplied to the control unit 30 via the sensor I / F 44 in the information processing apparatus 50. The sensor 10 is controlled by the control unit 30 via the sensor I / F 44.

[0036] The above application examples are for illustration to assist in understanding the present invention and are not intended to limit the interpretation of the present invention.

[0037] <Embodiment> Next, the configuration of the information processing apparatus 50 and each function of the control unit 30 in the embodiment of the present invention will be described in detail.

[0038] First, the configuration of the information processing apparatus 50 will be described with reference to FIG. 1. The information processing apparatus 50 includes a control unit 30, a sensor I / F 44, a display unit 34, an operation input unit 35, a storage unit 36, and a communication I / F 37. The control unit 30 includes a CPU 31, a ROM 32, a RAM 33, and a timer (not shown). The control program executed by the CPU 31 is stored in the ROM 32. Various values such as various threshold values are also stored in the ROM 32. The RAM 33 provides a work area when the CPU 31 executes the control program.

[0039] The display unit 34 is composed of a liquid crystal display or the like and displays various information. The display unit 34 may have two or more screens or may have a function of displaying two or more screens by screen splitting. The operation input unit 35 receives the input of various instructions from the user and sends the input information to the CPU 31. Further, the operation input unit 35 may have a function of giving a warning to the user by voice, a lamp, or the like based on an instruction from the CPU 31. The storage unit 36 is composed of, for example, a non-volatile memory. The storage unit 36 may be an external memory. The communication I / F 37 performs wired or wireless communication between the control unit 30 and the robot 302.

[0040] Next, each function of the control unit 30 will be described with reference to FIG. 2. The control unit 30 includes a sensing unit 201, a protection area setting unit 202, an area information acquisition unit 203, a dead angle determination unit 204, a three-dimensional space coordinate display unit 205, and a warning unit 206. Each of these functions is realized software-wise by a program stored in the ROM 32. That is, the CPU 31 expands and executes the necessary program in the RAM 33 to perform various calculations and control of various hardware resources, thereby providing each function. In other words, the function of the sensing unit 201 is mainly realized by the cooperation of the CPU 31, ROM 32, RAM 33, and sensor I / F 44. The functions of the protection area setting unit 202 and the area information acquisition unit 203 are mainly realized by the cooperation of the CPU 31, ROM 32, RAM 33, display unit 34, and operation input unit 35. The function of the dead angle determination unit 204 is mainly realized by the cooperation of the CPU 31, ROM 32, and RAM 33. The function of the warning unit 206 is mainly realized by the cooperation of the CPU 31, ROM 32, RAM 33, operation input unit 35, and communication I / F 37.

[0041] As shown in FIG. 3, the sensing unit 201 acquires distance information from a sensor 10 installed so that at least a part of the robot 302 enters the measurable area 301 via the sensor I / F 44. The distance information may be, for example, a distance image in which information on the depth distance from the sensor 10 is associated with each pixel, or point cloud data. Before the actual operation of the robot 302, the sensing unit 201 acquires the distance information (1) measured by the sensor 10 when the robot 302 is stopped and the distance information (2) measured by the sensor 10 at a plurality of timings while moving the robot 302. These acquired distance information (1) and (2) are temporarily stored in the RAM 33.

[0042] The distance information (1) is used to recognize objects other than the robot 302 (also referred to as peripheral objects) in the measurable area 301, such as the workbench 303 and the shelf 304 shown in FIG. 3, as well as the floor, wall, safety fence, etc., and to calculate the dead angle areas 303a and 304a formed by these objects.

[0043] The distance information (2) is used to recognize the movement path 306 of the robot 302, and the blind spot area 306a is calculated based on the movement path 306. Therefore, in order to recognize the same blind spot area as that generated by the robot 302 during actual operation, it is preferable that the distance information (2) is acquired while moving the robot 302 with the same operation as during actual operation.

[0044] The protection area setting unit 202 sets a virtual three-dimensional protection area 305 within the measurable area 301 in consideration of the installation position and orientation of the sensor 10, the installation position of the robot 302, and the safety distance. Specifically, the protection area setting unit 202 generates, as data of the protection area 305, for example, data defined by the XYZ coordinates of each vertex of a pentagonal prism as shown in FIG. 3, and stores it in the storage unit 36.

[0045] Here, the safety distance is a distance that can guarantee that the robot 302 can complete deceleration and stop before an intruder into the protection area 305 reaches the robot 302, and is determined according to safety standards in consideration of the response speed of the sensor 10, the operating speed and braking performance of the robot 302, etc.

[0046] Therefore, the protection area setting unit 202 reads out the used robot information (information such as the operating speed and braking performance of the robot 302) and the used safety standard information from the storage unit 36, and calculates the safety distance based on these.

[0047] Also, the storage unit 36 pre-retains information on the installation position and orientation of the sensor 10 and the installation position of the robot 302. When setting the protection area 305, the protection area setting unit 202 also reads out this information from the storage unit 36.

[0048] The area information acquisition unit 203 acquires the information of the protection area set by the protection area setting unit 202. Alternatively, the area information acquisition unit 203 may acquire the information of the set protection area from the storage unit 36. Further, the area information acquisition unit 203 acquires the data of the 3D model of the object (for example, the robot 301, the workbench 303, the shelf 304, etc.) existing in the measurable area 301 from the storage unit 36. The 3D model is data that defines the three-dimensional shape of an object, such as CAD data. Note that the information of the protection area and the 3D model of the object may be acquired from an external computer, an external storage, etc. instead of the storage unit 36.

[0049] The dead angle determination unit 204 (recognition means) first recognizes the movement path 306 of the robot 302. This is because the dead angle area 306a viewed from the sensor 10 generated by the operation of the robot 302 is calculated based on the range of the movement path 306.

[0050] Specifically, the dead angle determination unit 204 reads the distance information (2) acquired by the sensing unit 201 from the RAM 33 and converts it into point cloud information (object range information) in the orthogonal XYZ coordinate system in the global coordinate system. Similarly, the dead angle determination unit 204 reads the distance information (1) acquired by the sensing unit 201 from the RAM 33 and converts it into point cloud information (surrounding environment information) in the orthogonal XYZ coordinate system in the global coordinate system. Then, the dead angle determination unit 204 excludes the surrounding environment information from the object range information. Thereby, point cloud information (actual operation information) indicating the robot 302 at each timing when the distance information (2) is measured is generated. After that, the dead angle determination unit 204 sequentially plots the point cloud information of the robot 302 at each timing when the distance information (2) is measured, and recognizes a three-dimensional shape 401 that surrounds all the plotted point cloud information as the movement path 306 (FIG. 4) of the robot 302. Thereby, since the operation range of the robot 302 is recognized as the movement path 306, an accurate dead angle area 306a can be calculated.

[0051] Next, the blind spot determination unit 204 calculates blind spot regions 303a, 304a, and 306a, which are regions of blind spots as viewed from the sensor 10 caused by surrounding objects (workbench 303 and shelf 304) existing within the measurable region and the movement path 306 of the robot 302. Specifically, the blind spot determination unit 204 reads out information on the installation position and orientation of the sensor 10 from the storage unit 36, and based on this information, the surrounding environment information obtained from the distance information (1), and the three-dimensional shape of the movement path 306 shown in FIG. 4, calculates the blind spot regions 303a, 304a, and 306a. Specifically, the region on the back side of the upper surface of the workbench 303 as viewed from the sensor 10 becomes the blind spot region 303a, the region on the back side of the upper surface of the shelf 304 as viewed from the sensor 10 becomes the blind spot region 304a, and the region on the back side of the movement path 306 of the robot 302 as viewed from the sensor 10 becomes the blind spot region 306a.

[0052] Also, the blind spot determination unit 204 determines whether at least a part of the blind spot regions 303a, 304a, and 306a is within the protection area 305, and if so, sends a blind spot detection notification to the warning unit 206.

[0053] The three-dimensional space coordinate display unit 205 displays the three-dimensional diagram shown in FIG. 3 in a virtual three-dimensional space on the display unit 34. Although the information processing device 50 and the measurable region 301 are shown in FIG. 3, these may be non-displayed in the virtual three-dimensional space on the display unit 34.

[0054] Specifically, the three-dimensional space coordinate display unit 205 displays the three-dimensional diagram shown in FIG. 3 in the three-dimensional space on the display unit 34 as follows.

[0055] First, based on the data of the protection area 305 acquired by the area information acquisition unit 203, the three-dimensional space coordinate display unit 205 generates a CG representing the protection area 305 and displays it in the three-dimensional space on the display unit 34. In the example of FIG. 3, the outer shape of the protection area 305 defined by the XYZ coordinates of each vertex of the pentagonal prism is drawn in wireframe.

[0056] In addition, the three-dimensional space coordinate display unit 205 generates a CG (e.g., wireframe) representing the blind spot regions 303a, 304a, 306a calculated by the blind spot discrimination unit 204, and superimposes and displays it on the virtual three-dimensional space.

[0057] Furthermore, the three-dimensional space coordinate display unit 205 may superimpose and display a point cloud generated from the distance information (1) acquired by the sensing unit 201 on the virtual three-dimensional space. Each point constituting the point cloud corresponds to a point (measurement point) on the surface of an object whose distance is measured by the sensor 10. Therefore, by displaying the point cloud in a virtual three-dimensional space, the object (outer shape) existing within the measurable region 301 can be represented. Thereby, it is also possible to grasp the positional and size relationships with the objects 303, 304 existing inside or near the protection area 305, and what kind of blind spot regions 303a, 304b are generated by the objects 303, 304.

[0058] Furthermore, the three-dimensional space coordinate display unit 205 may superimpose and display the 3D model of the object acquired by the area information acquisition unit 203 on the virtual three-dimensional space. By performing the display using such a 3D model, the object existing within the measurable region 301 can be represented more precisely and accurately. Note that both the point cloud and the 3D model may be superimposed and displayed, or only one of them may be displayed or switched.

[0059] In addition, the range of the protection area 305 displayed on the display unit 34 can be changed by a method such as moving the vertices of the wireframe by a user operation using the operation input unit 35. This facilitates the appropriate setting of the protection area 305. For example, when there is no danger in the area under the shelf 304 because the arm of the robot 302 does not enter, the user can exclude such an area from the protection area 305.

[0060] In addition, in this embodiment, the case where the CGs of the protection area 305 and the blind spot areas 303a, 304a, and 306a are displayed as wireframes has been exemplified. However, it is not limited to this as long as the user can recognize the shapes and ranges of these areas. For example, it may be a box frame, a spherical frame, or a polygon frame instead of a wireframe.

[0061] Further, the blind spot discrimination unit 204 (second blind spot calculation means) calculates a three-dimensional blind spot area 306a' (not shown: second blind spot area) that can be a blind spot of the three-dimensional distance sensor from the information on the three-dimensional maximum movable range of the robot 302 read from the storage unit 36 and the surrounding environment information stored in the storage unit 36. The three-dimensional space coordinate display unit 205 superimposes and displays the blind spot areas 306a and 306a' in a distinguishable state, such as by color coding, on the virtual three-dimensional space, or switches the blind spot area 306a to the blind spot area 306a' in response to a mode switch by the user and superimposes and displays it on the virtual three-dimensional space. Thereby, the user can grasp the critical ones among the blind spot areas 306a' within the maximum movable range of the robot 302.

[0062] When the warning unit 206 receives a blind spot detection notification from the blind spot discrimination unit 204, it warns the user through the operation input unit 35. Thereby, the user can more easily recognize the critical blind spot area.

[0063] In addition, the warning unit 206 may issue a danger warning when the size of the blind spot area in the protection area 305 is equal to or greater than a threshold value. Thereby, when the blind spot area within the protection area is, for example, large enough for a person to enter, it is possible to inform the user that the sensor arrangement has a high degree of danger.

[0064] Further, the warning unit 206 (instruction transmission means) transmits an instruction to change the movement path 306 so that the blind spot area 306a decreases to the robot 302 via the communication I / F 37 when the blind spot area is within the protection area 305. Thereby, it is possible to give feedback on the measurement result by the sensor 10 to the robot 302.

[0065] Figure 5 is a flowchart showing the blind spot area display process.

[0066] This process is realized by the CPU 31 expanding and executing the program stored in the ROM 32 in the RAM 33. This process is started by the user's instruction after the user installs the sensor 10 so that at least a part of the robot 302 enters its measurable area 301.

[0067] First, in step S500, the protection area setting unit 202 reads out the used robot information, the safety standard information to be used, and the information on the installation position / orientation of the sensor 10 and the installation position of the robot 302 from the storage unit 36, and sets a virtual three-dimensional protection area 305 in the measurable area 301 based on these pieces of information. Note that if the protection area 305 has already been set, the process of step S500 may be omitted.

[0068] In step S501, while the user stops the robot 302, the user inputs an acquisition instruction for the distance information (1) using the operation input unit 35. When there is an acquisition instruction for the distance information (1), the sensing unit 201 acquires the distance information (1) measured by the sensor 10.

[0069] In step S502, after the user starts the robot 302 to move in the same motion as during actual operation, the user inputs an acquisition instruction for the distance information (2) using the operation input unit 35. When there is an acquisition instruction for the distance information (2), the sensing unit 201 acquires the distance information (2) measured by the sensor 10.

[0070] In step S503, the area information acquisition unit 203 acquires data such as the information on the protection area and the 3D model of the object from the storage unit 36.

[0071] In step S504, the blind spot discrimination unit 204 recognizes the movement path 306 of the robot based on the distance information (1) and the distance information (2).

[0072] In step S505, the dead angle determination unit 204 calculates dead angle regions 303a, 304a, and 306a caused by surrounding objects (workbench 303 and shelf 304) and the movement path 306 of the robot 302.

[0073] In step S506, the three-dimensional space coordinate display unit 205 displays the protection area 305 acquired in step S503 and the dead angle regions 303a, 304a, and 306a calculated in step S505 in a virtual three-dimensional space on the display unit 34. Further, a point cloud generated from the distance information (1), the 3D model acquired in step S503, etc. may be superimposed and displayed in the virtual three-dimensional space.

[0074] In step S507, the dead angle determination unit 204 determines whether at least a part of the dead angle regions 303a, 304a, and 306a is within the protection area 305. If at least a part of the dead angle regions 303a, 304a, and 306a is within the protection area 305 (YES in step S507), after sending a dead angle detection notification to the warning unit 206, the process proceeds to step S508. On the other hand, if there is no dead angle region within the protection area 305 (NO in step S507), this process is terminated as it is.

[0075] In step S508, when the warning unit 206 receives the dead angle detection notification from the dead angle determination unit 204, it warns the user through the operation input unit 35 and transmits an instruction to change the movement path 306 so that the dead angle region 306a decreases in the robot 302 via the communication I / F 37, and this process is terminated.

[0076] According to this embodiment, when the sensor 10 is installed such that at least a part of the robot 302 enters its measurable region 301, the control unit 30 sets a virtual three-dimensional protection area 305 within the measurable region 301, and calculates the dead angle regions generated by each object based on the distance information measured by the sensor 10. Thereafter, the control unit 30 displays the protection area 305 and the dead angle regions in a virtual three-dimensional space on the display unit 34. Thereby, the user can easily recognize the critical dead angle regions.

[0077] Furthermore, as the three-dimensional distance sensor used as the sensor 10, other types of sensors may be adopted as long as they are sensors that measure and output distance information (three-dimensional information). When adopting a TOF sensor, either a direct type or an indirect type may be used. In addition, sensors that use something other than light, such as radio waves, are also applicable.

[0078] Furthermore, the information processing apparatus 50 can be configured by, for example, a computer including a processor, a memory, a storage, and the like. In that case, the configuration shown in FIG. 2 is realized by loading a program stored in the storage into the memory and the processor executing the program. Such a computer may be a general-purpose computer such as a personal computer, a server computer, a tablet terminal, or a smartphone, or may be an embedded computer such as an on-board computer. Alternatively, all or part of the configuration shown in FIG. 2 may be configured by an ASIC, an FPGA, or the like. Alternatively, all or part of the configuration shown in FIG. 2 may be realized by cloud computing or distributed computing.

[0079] <Supplementary Note> 〔1〕A distance information acquisition means (201) for acquiring distance information measured by the sensor (10); An area information acquisition means (202) for acquiring information on a virtual three-dimensional protection area set within the measurable area of the sensor; A dead angle calculation means (204) for calculating a three-dimensional dead angle area that is a dead angle of the sensor based on the distance information; A display means (205) for displaying the protection area and the dead angle area in a virtual three-dimensional space; An information processing apparatus (50), characterized by comprising the above.

[0080] 〔2〕Steps (S501, S503) of acquiring distance information measured by the sensor (10); A step (S500) of acquiring information on a virtual three-dimensional protection area set within the measurable range of the sensor; A step (S505) of calculating a three-dimensional dead zone area that is a dead zone of the three-dimensional distance sensor based on the distance information; A step (S506) of displaying the protection area and the dead zone area in a virtual three-dimensional space; An information processing method, characterized by comprising the above.

Explanation of Signs

[0081] 10: Sensor 30: Control unit 50: Information processing device 201: Sensing unit 202: Protection area setting unit 203: Area information acquisition unit 204: Dead zone discrimination unit 205: Three-dimensional space coordinate display unit 206: Warning unit 301: Measurable range 302: Robot 303: Workbench 304: Shelf 305: Protection area 306: Traffic line 303a, 304a, 306a: Dead zone area

Claims

1. distance information acquisition means for acquiring distance information measured by a sensor; area information acquisition means for acquiring information on a virtual three-dimensional protection area set within the measurable area of the sensor; dead angle calculation means for calculating a three-dimensional dead angle area that is a dead angle of the sensor based on the distance information; display means for displaying the protection area and the dead angle area in a virtual three-dimensional space; recognition means for recognizing the operating range of the robot from distance information measured by the sensor at a plurality of timings while moving the robot installed within the measurable area; comprising; The dead angle calculation means calculates a dead angle area generated by the robot based on the operating range of the robot recognized by the recognition means. An information processing apparatus characterized by this.

2. The information processing apparatus according to claim 1, wherein the display means further displays a point cloud based on the distance information in the virtual three-dimensional space.

3. The area information acquisition means acquires a 3D model of an object existing within the measurable area, The information processing apparatus according to claim 1 or 2, wherein the display means further displays the 3D model in the virtual three-dimensional space.

4. The information processing apparatus according to any one of claims 1 to 3, further comprising warning means for giving a warning when at least a part of the dead angle area is within the protection area.

5. The information processing apparatus according to claim 4, wherein the warning means gives a danger warning when the size of the dead angle area within the protection area is equal to or greater than a threshold value.

6. The information processing apparatus according to any one of claims 1 to 5, wherein the range of the protection area displayed in the virtual three-dimensional space can be changed by a user operation.

7. The display means superimposes and displays the operating range of the robot in the virtual three-dimensional space. The information processing apparatus according to any one of claims 1 to 6, characterized by this.

8. The information processing apparatus according to claim 7, further comprising instruction transmission means for transmitting an instruction to change the operating range so that the dead angle area decreases when the dead angle area generated by the robot is within the protection area to the robot.

9. The recognition means, Convert each of the distance information measured by the sensor at the plurality of timings into object range information indicating the range occupied by each object in the measurable region. Generate actual operation information indicating the range occupied by the robot in the measurable region, excluding the peripheral environment information indicating the range occupied by objects other than the robot in the measurable region, from the object range information. The information processing apparatus according to claim 8, characterized in that the actual operation information at the plurality of timings is superimposed to calculate the operation range of the robot.

10. The information processing apparatus according to claim 9, further comprising setting means for setting the peripheral environment information based on the distance information measured by the sensor with the robot stopped.

11. The distance information measured by the sensor at the plurality of timings is the distance information measured by the sensor while moving the robot with the same operation as during actual operation. The information processing apparatus according to claim 9 or 10.

12. The dead angle calculating means further calculates the dead angle region based on the peripheral environment information. The information processing apparatus according to any one of claims 9 to 11.

13. The apparatus further comprises second dead angle calculating means for calculating a second dead angle region that can be a dead angle of the sensor from the maximum movable range of the robot. The display means further superimposes and displays the second dead angle region in a distinguishable state from the dead angle region calculated from the operation range of the robot in the virtual three-dimensional space. The information processing apparatus according to any one of claims 9 to 12.

14. The apparatus further comprises second dead angle calculating means for calculating a second dead angle region that can be a dead angle of the sensor from the maximum movable range of the robot. The display means alternately displays the dead angle region calculated from the operation range of the robot and the second dead angle region, and superimposes and displays them in the virtual three-dimensional space. The information processing apparatus according to any one of claims 9 to 12.

15. A step of acquiring distance information measured by a sensor; A step of acquiring information on a virtual three-dimensional protection area set within the measurable region of the sensor; A step of calculating a three-dimensional dead angle region that is a dead angle of the sensor based on the distance information; A step of displaying the protection area and the dead angle region in a virtual three-dimensional space; While moving the robot installed within the measurable area, recognizing the operating range of the robot from distance information measured by the sensor at a plurality of timings; comprising; In the step of calculating the blind spot area, based on the recognized operating range of the robot, calculating a blind spot area generated by the robot, the information processing method being characterized thereby.

16. A program for causing a computer to execute each step of the information processing method according to Claim 15.

Images