Sensor, control method for sensor, program, and safety monitoring system

Abstract

To enhance productivity at a production site where a robot and a worker work together.SOLUTION: A sensor includes: an image generation unit configured to generate a distance image and a luminance image of a predetermined region; a first point cloud data creation unit configured to create, based on the distance image, first point cloud data that is point cloud data of the predetermined region; a mask data creation unit configured to create, when a tag is detected from the luminance image, mask data for masking a predetermined range in the predetermined region based on an analysis result obtained by analyzing the tag; and a second point cloud data creation unit configured to create second point cloud data by excluding point cloud data of a mobile body provided with the tag from the first point cloud data based on the mask data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a sensor, a sensor control method, a program, and a safety monitoring system. [Background technology]

[0002] In production sites, workers and robots work together. In such production sites, efforts are made to ensure the safety of workers and to increase productivity. For example, Patent Document 1 discloses a technology for increasing both safety and productivity in production sites where robots and workers work together. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7036078 Summary of the Invention [Problem to be solved by the invention]

[0004] In production sites, when a sensor detects that a person, such as a worker, is approaching a hazard such as a robot, the robot is slowed down or stopped. In production sites, there are cases where mobile devices (moving objects) other than people are in operation. When an object approaching the robot is detected, regardless of whether it is a person or a mobile device, there is a problem that productivity is reduced by slowing down or stopping the robot.

[0005] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a technique for increasing productivity in a production site where a robot and a worker work together. [Means for solving the problem]

[0006] A sensor according to one aspect of the present invention includes an image generation unit that generates a distance image and a brightness image of a predetermined area, a first point cloud data creation unit that creates first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image, a mask data creation unit that, when a tag is detected in the brightness image, creates mask data for masking a predetermined range in the predetermined area based on an analysis result of analyzing the tag, and a second point cloud data creation unit that creates second point cloud data by excluding point cloud data of mobile objects with tags from the first point cloud data, based on the mask data. Safety monitoring of the predetermined area using the second point cloud data by excluding point cloud data of mobile objects with tags from the first point cloud data makes it possible to increase productivity in production sites where robots and workers work together.

[0007] The sensor includes an analysis unit that analyzes the tag, and the analysis unit acquires identification information of the tag based on the luminance image, acquires size data related to the moving object based on the identification information of the tag, calculates position coordinates and angle information of the tag based on the distance image and the luminance image or the luminance image, and generates the mask data based on the size data, the position coordinates of the tag, the angle information of the tag, and relationship information indicating a relative positional relationship between the tag and the moving object. The tag has geometric characteristics.

[0008] The sensor includes a determination unit that determines the positional relationship between the worker and a hazard within the specified area based on the second point cloud data. The sensor also includes a determination unit that determines the positional relationship between the worker and a hazard within the specified area based on the first point cloud data when the tag is not detected from the luminance image.

[0009] The present invention can also be understood as a control method including at least a part of the above-described processing, a program for causing a computer to execute at least a part of the above-described processing, or a computer-readable recording medium on which such a program is non-temporarily recorded. It can also be understood as a safety monitoring system including at least a part of the above-described processing. The above configurations and processing can be combined with each other to constitute the present invention as long as no technical contradiction occurs. [Effects of the Invention]

[0010] According to the present invention, it is possible to increase productivity at a production site where a robot and a worker work together. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram of the control system. [Figure 2] FIG. 2 is a block diagram showing the configuration of the sensor. [Figure 3] FIG. 3 is a flowchart illustrating the processing flow of the control system. [Figure 4] FIG. 4 is a diagram showing an example of a traveling device having a tag. [Figure 5] FIG. 5 is a diagram illustrating an example of a transport device having a tag. [Figure 6] FIG. 6 is a functional block diagram showing the configuration of the control unit. [Figure 7] FIG. 7 is a flowchart illustrating the processing flow of the control system. [Figure 8] FIG. 8 is a schematic configuration diagram of a control system according to the first modification. [Figure 9] FIG. 9 is a functional block diagram showing the configuration of the control unit according to the second modification. [Figure 10] FIG. 10 is a functional block diagram showing the configuration of a control device according to the second modification. DETAILED DESCRIPTION OF THE INVENTION

[0012] Application examples and embodiments will be described below with reference to the drawings. The application examples and embodiments described below are aspects of the present application and do not limit the scope of the rights of the present application. <Application example> FIG. 1 is a schematic diagram of a control system (safety monitoring system). In the control system of FIG. 1, in an environment where a hazard source such as a fixed robot 1, a worker (person) 100, and a mobile robot 101 cooperate with each other, for example, in a production site such as a factory, the movement of the worker 100 is grasped and the fixed robot 1 is controlled. The fixed robot 1 is, for example, a fixed installation robot such as a vertical articulated robot, a horizontal articulated robot, or a parallel link robot. The mobile robot 101 is, for example, an autonomous mobile robot (AMR), an automatic guided vehicle (AGV), or the like. The mobile robot 101 is a traveling device such as a GV (Automated Guided Vehicle). The mobile robot 101 may also be a transport device with a manipulator attached to the traveling device. In FIG. 1, the fixed robot 1 is installed on a workbench 200. The fixed robot 1 is controlled by a robot controller (control device) 2. A protection area (danger area) 10 is set in the vicinity of or around the fixed robot 1. The protection area 10 is a virtual three-dimensional area for detecting an object approaching the fixed robot 1, and is set in at least a part of the vicinity of or around the fixed robot 1. In FIG. 1, the protection area 10 is set around the fixed robot 1, but the protection area 10 may be set in at least one of the front, rear, and side of the fixed robot 1. It is also possible to set multiple protection areas 10. The protection area 10 is determined in accordance with safety standards and taking into account the operating range of the fixed robot 1.

[0013] A three-dimensional safety sensor 3 (hereinafter referred to as sensor 3) is installed at any position in the production site. The protection area 10 and monitoring area can be set arbitrarily. The monitoring area is an area monitored by the sensor 3. The user may set the protection area 10 and monitoring area for the sensor 3 by operating the information processing device 4. The information processing device 4 is, for example, a personal computer, a tablet terminal, etc. For example, the entire periphery of the protection area 10 may be monitored. The monitoring area may be an area, or a part of the periphery of the protection region 10. The monitoring area may be set as a part of the measurement area of ​​the sensor 3, or may be set as the entire measurement area of ​​the sensor 3. The measurement area of ​​the sensor 3 is the range that the sensor 3 can measure.

[0014] The sensor 3 generates point cloud data of the monitoring area and uses the point cloud data of the monitoring area to detect whether an object has entered the protection area 10. The point cloud data of the monitoring area is data that indicates the three-dimensional coordinates of each point of an object present in the monitoring area. If the distance between a predetermined position coordinate (e.g., center position coordinate) in the protection area 10 and the position coordinate of an object such as a worker is equal to or less than a predetermined distance (safety distance), the sensor 3 may determine that an object has entered the protection area 10. If the sensor 3 detects the intrusion of an object into the protection area 10, a predetermined control (safety control) is performed. For example, a control signal (detection signal) is input from the sensor 3 to the robot controller 2, and the robot controller 2 performs control to slow down or stop the operation of the fixed robot 1 based on the control signal. The control signal may be input from the sensor 3 to an alarm device, which then outputs an alert.

[0015] In a production site, moving objects such as traveling devices and conveying devices may be operating. If the sensor 3 detects the intrusion of a moving object into the protection area 10 and safety control such as slowing down or stopping the operation of the fixed robot 1 is performed, productivity will decrease. In the control system of FIG. 1, even if a moving object intrudes into the protection area 10, safety control such as slowing down or stopping the operation of the fixed robot 1 is not performed. By using such a control system, it is possible to increase productivity in a production site where the fixed robot 1 and the worker 100 work together.

[0016] <Embodiment> FIG. 2 is a block diagram showing the configuration of the sensor 3. As shown in FIG. 2, the sensor 3 includes a light-emitting unit 11, a light-receiving unit 12, a control unit 13, a storage unit 14, and a communication unit 15. The light-emitting unit 11 emits light (for example, infrared light), and the light-receiving unit 12 receives the reflected light. The light-emitting unit 11 is, for example, an LED (Light Emitting Diode). The light-receiving unit 12 is, for example, a photodiode. As an example of the sensor 3, a distance image is acquired from the time of flight (TOF) of light. For example, the sensor 3 may be an indirect TOF sensor that estimates the time difference from the phase difference between emitted light and reflected light.

[0017] The control unit 13 is a control device (controller) that controls the overall operation of the sensor 3. The control unit 13 may be configured with a dedicated device or a general-purpose computer. The control unit 13 includes hardware resources such as a processor (CPU), memory, and storage. The memory may be RAM. The storage may be a non-volatile storage device (e.g., ROM, flash memory, etc.). The functions of each processing unit (functional unit) of the control unit 13 are realized by loading a program stored in the storage into the memory and executing it with the processor. Note that the configuration of the control unit 13 is not limited to these. For example, all or part of the functions of the control unit 13 may be configured with circuits such as ASIC or FPGA, or all or part of the functions of the control unit 13 may be executed by a cloud server or other device.

[0018] The control unit 13 periodically generates distance images and luminance images. The control unit 13 measures the distance to an object (subject) using the TOF method, calculates the distance to the surface of the measured object for each pixel, and generates a distance image. The TOF method is a method for measuring distance from the time of flight of light. The control unit 13 generates point cloud data of the monitoring area based on the distance image. The control unit 13 measures the intensity of reflected light, and generates a luminance image based on the intensity of reflected light. The control unit 13 accesses a database (DB) 21 of the external device 20 to obtain information and data from the database 21. The memory unit 14 stores information and data. The memory unit 14 may have at least one of a memory and a storage. The communication unit 15 communicates via wired or wireless It is an interface for communication via wire.

[0019] FIG. 3 is a flowchart illustrating the processing flow of the control system. The processing flow of the control system will be described with reference to the flowchart of FIG. 3. In step S101, the control unit 13 determines whether or not a tag (marker) has been detected from the luminance image. The tag has geometric characteristics. A geometric pattern made up of two colors with different brightness (for example, white and black) may be formed on the tag. The tag is, for example, an April Tag, but is not limited to this. , or other geometric tags.

[0020] If a tag can be detected from the luminance image (S101; YES), the process proceeds to step S102. If a tag cannot be detected from the luminance image (S101; NO), the process proceeds to step S108. If the control unit 13 cannot reliably detect the tag, it determines that the tag cannot be detected from the luminance image. For example, if the outline of the tag can be recognized in the luminance image but the content of the tag cannot be analyzed, the control unit 13 may determine that the tag cannot be detected from the luminance image.

[0021] The tag is provided on the traveling device or the conveying device. FIG. 4 is a diagram showing an example of a traveling device 32 having a tag 31. In the example shown in FIG. 4, the tag 31 is attached to the front side of the housing of the traveling device 32. The example is not limited to that shown in FIG. 4, and the tag 31 may be attached to the side or back side of the housing of the traveling device 32. Furthermore, multiple tags 31 may be attached to multiple locations on the housing of the traveling device 32. If the outer shape of the tag 31 of the traveling device 32 can be recognized in the brightness image but the content of the tag 31 cannot be analyzed, the control unit 13 may communicate with the traveling device 32 via the communication unit 15 and output an instruction signal to the traveling device 32. An instruction signal may be output to instruct the traveling device 32 to move so that the tag 31 faces the front of the sensor 3.

[0022] FIG. 5 is a diagram illustrating an example of a transport device 33 having a tag 31. The transport device 33 includes a traveling device 34 and a manipulator 35. In the example illustrated in FIG. 5, the tag 31 is provided on the front side of the housing of the traveling device 34. The example is not limited to the example illustrated in FIG. 5, and the tag 31 may be attached to the side or rear side of the housing of the traveling device 34. Furthermore, multiple tags 31 may be attached to the housing of the traveling device 34. The tag 31 may be attached to the manipulator 35, or multiple tags 31 may be attached to the manipulator 35. If the outer shape of the tag 31 of the transport device 33 can be recognized in the brightness image but the content of the tag 31 cannot be analyzed, the control unit 13 may communicate with the transport device 33 via the communication unit 15 and output an instruction signal to the transport device 33. An instruction signal may be output to instruct the transport device 33 to move so that the tag 31 faces the front of the sensor 3. An instruction signal may be output to instruct the manipulator 35 to move so that the tag 31 faces the front of the sensor 3.

[0023] In step S102, the control unit 13 acquires ID information by reading the geometric pattern of the tag in the luminance image. The ID information is unique identification information that does not overlap and is different for each moving body. The moving body is an example of a traveling device and a transport device.

[0024] In step S103, the control unit 13 calculates the position coordinates (x, y, z) of the tag and angle information of the tag based on the brightness image and the distance image. The position coordinates of the tag are, for example, the coordinates (x, y, z) of the tag in the camera coordinate system. The position coordinates of the tag may be the coordinates of the center of the tag. The angle information of the tag includes the angle with respect to a reference plane (horizontal plane, ground) and the angle with respect to a plane perpendicular to the reference plane. In step S103, the control unit 13 acquires the reference coordinates stored in the memory unit 14. The reference coordinates are relationship information that indicates the relative positional relationship between the tag and the moving body. The reference coordinates indicate, for example, the installation position of the tag with respect to the moving body. The coordinates are (x, y, z), and the position of a predetermined part (for example, a corner) of the moving object may be set as the origin. In step S103, the control unit 13 may calculate the position coordinates of the tag and angle information of the tag based on the luminance image.

[0025] In step S104, the control unit 13 acquires size data related to the moving object from the database 21 based on the ID information. The ID information and size data are associated and stored in the database 21. The size data related to the moving object may include data related to the dimensions (length, width, and height) of the moving object, or may include data related to values ​​obtained by adding a margin to the dimensions of the moving object.

[0026] In step S105, the control unit 13 generates point cloud range data for the moving object based on size data for the moving object, position coordinates of the tag, angle information of the tag, and reference coordinates. The tag is attached to the moving object. The control unit 13 calculates the position coordinates (x, y, z) of the moving object based on the position coordinates of the tag, the reference coordinates, and size data for the moving object. The control unit 13 may calculate the coordinates at the center of the moving object as the position coordinates of the moving object. Since the tag attached to the moving object tilts depending on angle information of the moving object (information regarding the direction the moving object is facing), the angle information of the tag differs for each angle information of the moving object. The control unit 13 calculates angle information for the moving object based on the angle information of the tag. The control unit 13 generates point cloud range data for the moving object based on the position coordinates of the moving object, angle information of the moving object, and size data for the moving object. The point cloud range data for the moving object is mask data for masking a predetermined range within a monitoring area, and is data indicating the range in which the moving object exists within the monitoring area. That is, the point cloud range data of the moving body is data indicating the range occupied by the point cloud data of the moving body out of the point cloud data of the monitoring area.

[0027] In step S106, the control unit 13 excludes (deletes) the point cloud data of the moving body from the point cloud data of the monitoring area based on the point cloud range data of the moving body, thereby invalidating (masking) the point cloud data of the moving body in the point cloud data of the monitoring area. Hereinafter, the point cloud data after excluding the point cloud data of the moving body from the point cloud data of the monitoring area will be referred to as point cloud data after masking.

[0028] In step S107, the control unit 13 performs safety monitoring based on the masked point cloud data. For example, the control unit 13 performs safety monitoring by determining whether an object such as a worker 100 has entered the protection area 10.

[0029] In step S108, the control unit 13 performs safety monitoring based on the point cloud data of the monitoring area. Since no tag is detected from the luminance image, the control unit 13 treats objects in the monitoring area as people.

[0030] According to the embodiment, the control unit 13 performs safety monitoring based on the masked point cloud data, and therefore does not perform safety control such as slowing down or stopping the operation of the fixed robot 1 even if a moving object enters the protection area 10. In this way, by not performing safety control when a moving object enters the protection area 10, it is possible to increase productivity at a production site where the fixed robot 1 and the worker 100 work together.

[0031] It is preferable to attach a tag to the moving object so that the tag can be easily detected from the brightness image. For example, if the sensor 3 is attached above the monitoring area, the tag may be attached to the moving object so that the tag faces upward. For example, if the sensor 3 is installed so that the tag faces diagonally downward, the tag may be attached to the moving object so that the tag faces diagonally upward.

[0032] FIG. 6 is a functional block diagram showing the configuration of the control unit 13. The control unit 13 includes an image generation unit 110, a first point cloud data creation unit 120, a detection unit 130, an analysis unit 140, a mask data creation unit 150, a second point cloud data creation unit 160, a determination unit 170, and an output unit 180. The image generation unit 110 generates a distance image and a brightness image of a predetermined area. The predetermined area may be, for example, a monitoring area. The first point cloud data creation unit 120 creates first point cloud data, which is point cloud data of the predetermined area. The detection unit 130 detects tags from the brightness image. If a tag is detected from the brightness image, the analysis unit 140 analyzes the detected tag. The mask data creation unit 150 creates mask data for masking a predetermined range within the predetermined area based on the analysis result of the tag analysis by the analysis unit 140. The second point cloud data creation unit 160 creates second point cloud data by excluding point cloud data of moving objects from the first point cloud data based on the mask data. The determination unit 170 determines the positional relationship between the worker 100 within a predetermined area and a hazard such as the fixed robot 1. The output unit 180 outputs a predetermined signal.

[0033] An example of processing by the analysis unit 140 when the tag has geometric characteristics will be described. The analysis unit 140 acquires ID information (identification information) of the tag based on the luminance image. The geometric characteristic of the tag is, for example, a geometric pattern provided on the tag. The storage unit 14 stores the tag's geometric characteristics and the tag's ID information in association with each other. The analysis unit 140 analyzes the luminance image and extracts the tag's geometric characteristics in the luminance image. The analysis unit 140 acquires the tag's ID information from the tag's geometric characteristics in the luminance image. The analysis unit 140 acquires size data related to the moving object from the database 21 based on the tag's ID information. The ID information and the size data may be associated and stored in the storage unit 14. The analysis unit 140 may acquire the size data related to the moving object from the storage unit 14. The analysis unit 140 calculates the tag's position coordinates, the tag's angle information, and the reference coordinates based on the distance image and the luminance image. The mask data generating unit 150 generates point cloud range data (mask data) of the moving object based on size data relating to the moving object, position coordinates of the tag, angle information of the tag, and reference coordinates.

[0034] If a tag is detected from the luminance image, the determination unit 170 determines the positional relationship between the worker 100 and a hazard, such as the fixed robot 1, within the predetermined area based on the second point cloud data. The determination unit 170 may determine whether or not the distance between the position coordinates of the worker 100 and a predetermined position coordinate (the center position coordinate of the hazard, such as the fixed robot 1) within the predetermined area is equal to or less than a predetermined distance based on the second point cloud data. If a tag is not detected from the luminance image, the determination unit 170 determines the positional relationship between the worker 100 and a hazard, such as the fixed robot 1, within the predetermined area based on the first point cloud data. The determination unit 170 may determine whether or not the distance between the position coordinates of the worker 100 and a predetermined position coordinate within the predetermined area is equal to or less than a predetermined distance based on the first point cloud data. If the distance between the position coordinates of the worker 100 and the predetermined position coordinate within the predetermined area is equal to or less than the predetermined distance, the output unit 180 outputs a control signal (detection signal) to the robot controller 2. The robot controller 2 performs control to slow down or stop the operation of the fixed robot 1 based on the control signal.

[0035] In the above, tags having geometric features are used, but this is not limiting, and tags having character strings, numeric strings, or combinations of letters and numbers indicating ID information (identification information) may also be used. Fig. 7 is a flowchart explaining the processing flow of the control system when tags having ID information are used. The processing flow of the control system when tags having ID information are used will be explained with reference to the flowchart of Fig. 7.

[0036] In step S201, the control unit 13 determines whether or not a tag has been detected from the luminance image. The tag has a character string, a number string, or a combination of letters and numbers that indicates the ID information of the tag. For example, a character string, a number string, or a combination of letters and numbers that indicates the ID information of the tag may be displayed on the surface of the tag. If the tag has been extracted from the luminance image (S201; YES), the process proceeds to step S202. If the tag has not been extracted from the luminance image (S 201; NO), the process proceeds to step S208. If the control unit 13 cannot reliably detect the tag, it determines that the tag cannot be detected from the luminance image. For example, if the outline of the tag can be recognized in the luminance image but the content of the tag cannot be analyzed, the control unit 13 may determine that the tag cannot be detected from the luminance image.

[0037] In step S202, the control unit 13 acquires ID information by reading the ID information of the tag in the luminance image. The ID information is unique identification information that does not overlap and is different for each moving object.

[0038] In step S203, the control unit 13 calculates the position coordinates (x, y, z) of the tag based on the luminance image and the distance image. In step S203, the control unit 13 acquires angle information of the moving object from the moving object via the communication unit 15. The angle information of the moving object may include the angle of the moving object in a planar direction or the direction in which the moving object is facing. For example, the moving object may calculate the angle information of the moving object by performing a self-position estimation process based on map data of the production site and landmarks placed at the production site. Alternatively, in step S203, the control unit 13 may calculate the angle information of the moving object by recognizing the shape of the moving object based on the distance image and the luminance image.

[0039] In step S204, the control unit 13 acquires size data related to the moving object from the database 21 based on the ID information. The database 21 stores ID information and size data in association with each other. The size data related to the moving object may include data related to the dimensions (length, width, and height) of the moving object, or may include data related to values ​​obtained by adding a margin to the dimensions of the moving object. The control unit 13 may acquire the size data related to the moving object using the results of recognizing the shape of the moving object based on the distance image and the brightness image.

[0040] In step S205, the control unit 13 generates point cloud range data for the moving object based on size data for the moving object, the position coordinates of the tag, and angle information for the moving object. The tag is provided on the moving object. The control unit 13 may calculate the position coordinates (x, y, z) of the moving object based on the position coordinates of the tag. The control unit 13 may calculate the position coordinates of the tag as the position coordinates of the moving object. The control unit 13 may calculate the position coordinates of the moving object based on the position coordinates of the tag and size data for the moving object. The control unit 13 generates point cloud range data for the moving object based on the position coordinates of the moving object, angle information for the moving object, and size data for the moving object. The point cloud range data for the moving object is mask data for masking a predetermined range within the monitoring area, and is data indicating the range within the monitoring area in which the moving object exists. In other words, the point cloud range data for the moving object is data indicating the range occupied by the point cloud data for the moving object among the point cloud data for the monitoring area.

[0041] In step S206, the control unit 13 excludes (deletes) the point cloud data of the moving body from the point cloud data of the monitoring area based on the point cloud range data of the moving body, thereby invalidating (masking) the point cloud data of the moving body in the point cloud data of the monitoring area. Hereinafter, the point cloud data after excluding the point cloud data of the moving body from the point cloud data of the monitoring area will be referred to as the point cloud data after masking.

[0042] In step S207, the control unit 13 performs safety monitoring based on the masked point cloud data. For example, the control unit 13 performs safety monitoring by determining whether an object such as a worker 100 has entered the protection area 10.

[0043] In step S208, the control unit 13 performs safety monitoring based on the point cloud data of the monitoring area. Since no tag is detected from the luminance image, the control unit 13 determines that there is no tag in the monitoring area. Treat objects in the world as people.

[0044] An example of processing by the analysis unit 140 when a tag has a character string, a number string, or a combination of letters and numbers indicating ID information (identification information) of the tag will be described. The analysis unit 140 analyzes the luminance image. The analysis unit 140 acquires the ID information (identification information) of the tag based on the luminance image. The analysis unit 140 acquires size data related to the moving object from the database 21 based on the ID information of the tag. The ID information and the size data may be associated and stored in the storage unit 14. The analysis unit 140 may acquire the size data related to the moving object from the storage unit 14. The analysis unit 140 calculates the position coordinates of the tag based on the distance image and the luminance image. The analysis unit 140 acquires angle information of the moving object from the moving object via the communication unit 15. The mask data creation unit 150 generates point cloud range data (mask data) of the moving object based on the size data related to the moving object, the position coordinates of the tag, and the angle information of the moving object.

[0045] This embodiment can also be applied to a two-dimensional laser scanner. In addition to generating a range image using a two-dimensional laser scanner, a separate camera may be provided to generate a luminance image and send it to the two-dimensional laser scanner.

[0046] <Variation 1> Modifications will be described. FIG. 8 is a schematic configuration diagram of a control system (safety monitoring system) according to Modification 1. In the control system shown in FIG. 8, an interference prevention area 40 is set in addition to the control system shown in FIG. 1. The interference prevention area 40 is an area for preventing interference between the fixed robot 1 and the mobile robot 101. The interference prevention area 40 is a virtual three-dimensional area for detecting when the mobile robot 101 approaches the fixed robot 1, and is set in at least a part of the vicinity or periphery of the fixed robot 1. In the example shown in FIG. 8, the interference prevention area 40 is set to a range narrower than the protection area 10, but this is not limited to this example. The interference prevention area 40 may be set to the same range as the protection area 10 or a range wider than the protection area 10. Furthermore, in FIG. 8, the interference prevention area 40 is set around the fixed robot 1, but the interference prevention area 40 may be set to at least one of the front, rear, and side of the fixed robot 1. It is also possible to set multiple interference prevention areas 40. The user may set the interference prevention area 40 for the sensor 3 by operating the information processing device 4.

[0047] The sensor 3 generates point cloud data of the monitoring area and uses the point cloud data of the monitoring area to detect whether the mobile robot 101 has entered the interference prevention area 40. If the distance between a predetermined position coordinate (for example, a center position coordinate) within the interference prevention area 40 and the position coordinate of the mobile robot 101 is equal to or less than a predetermined distance (interference distance), the sensor 3 may determine that the mobile robot 101 has entered the interference prevention area 40. If the sensor 3 detects that the mobile robot 101 has entered the interference prevention area 40, a predetermined control (safety control) is performed. For example, a control signal (interference flag) is input from the sensor 3 to the robot controller 2, and the robot controller 2 performs control to slow down or stop the operation of the fixed robot 1 based on the control signal.

[0048] <Variation 2> Modification 2 will now be described. Fig. 9 is a functional block diagram showing the configuration of control unit 13 according to Modification 2. Control unit 13 includes determination unit 170, output unit 180, and image acquisition unit 190. Image acquisition unit 190 acquires a distance image and a luminance image of a predetermined area. Image acquisition unit 190 may generate a distance image and a luminance image of the predetermined area. Image acquisition unit 190 may generate a distance image of the predetermined area, or may acquire a luminance image from another sensor (camera).

[0049] The control system according to the second modification may include a control device 300. 1 is a functional block diagram showing the configuration of a control device 300 according to the present invention. The control device 300 may be, for example, an information processing device such as a personal computer or a tablet terminal. The control device 300 may be configured as a dedicated device or a general-purpose computer. The control device 300 includes hardware resources such as a processor (CPU), memory, and storage. The functions of each processing unit (functional unit) of the control device 300 are realized by loading a program stored in the storage into the memory and executing it with the processor. Note that the configuration of the control device 300 is not limited to this. For example, all or part of the functions of the control device 300 may be configured using circuits such as ASICs or FPGAs, or all or part of the functions of the control device 300 may be executed by a cloud server or other device. The control device 300 includes a communication unit, which is an interface for wired or wireless communication. Communication is performed between the control unit 13 and the control device 300.

[0050] The control device 300 includes a first point cloud data creation unit 310, a detection unit 320, an analysis unit 330, a mask data creation unit 340, and a second point cloud data creation unit 350. The first point cloud data creation unit 310 creates first point cloud data, which is point cloud data of a predetermined area. The detection unit 320 detects tags from the luminance image. If a tag is detected from the luminance image, the analysis unit 330 analyzes the detected tag. The mask data creation unit 340 creates mask data for masking a predetermined range within the predetermined area based on the analysis result of the tag analysis by the analysis unit 330. The second point cloud data creation unit 350 creates second point cloud data by excluding point cloud data of moving objects from the first point cloud data based on the mask data. The processes executed by the first point cloud data creation unit 310, the detection unit 320, the analysis unit 330, the mask data creation unit 340, and the second point cloud data creation unit 350 are the same as the processes executed by the first point cloud data creation unit 120, the detection unit 130, the analysis unit 140, the mask data creation unit 150, and the second point cloud data creation unit 160.

[0051] In Modification 2, the control device 300 may execute the processes of steps S101 to S106 in FIG. 3, and the control unit 13 may execute the processes of steps S107 and S108 in FIG. 3. In Modification 2, the control device 300 may execute the processes of steps S201 to S206 in FIG. 7, and the control unit 13 may execute the processes of steps S207 and S208 in FIG. 7. Furthermore, the control unit 13 according to Modification 2 may include at least one of the first point cloud data creation unit 120, the detection unit 130, the analysis unit 140, the mask data creation unit 150, and the second point cloud data creation unit 160. In Modification 2, the control unit 13 and the control device 300 may share and execute the processes of steps S101 to S106 in FIG. 3. Furthermore, in Modification 2, the control unit 13 and the control device 300 may share and execute the processes of steps S201 to S206 in FIG. 7.

[0052] <Variation 3> A third modification will be described. The sensor 3 may acquire the movement range of the mobile robot 101 and determine whether a portion of the movement range of the mobile robot 101 overlaps with a portion of the protection area 10. The sensor 3 may acquire the movement range of the mobile robot 101 from the mobile robot 101. The sensor 3 may acquire the movement range of the mobile robot 101 from a host device that manages the movement range of the mobile robot 101. If a portion of the movement range of the mobile robot 101 overlaps with a portion of the protection area 10, the sensor 3 may input a control signal (interference flag) to the robot controller 2. The robot controller 2 may perform control to slow down or stop the operation of the fixed robot 1 based on the control signal. If a portion of the movement range of the mobile robot 101 overlaps with a portion of the protection area 10, the sensor 3 may issue an instruction to change the movement range of the mobile robot 101 (a movement range change instruction). A control signal for the movement range change instruction may be input from the sensor 3 to the mobile robot 101. A control signal for the movement range change instruction may be input from a host device to the mobile robot 101. When a control signal for instructing the mobile robot 101 to change the movement range is input to the mobile robot 101, the mobile robot 101 is instructed to change the movement range. For example, by narrowing the movement range of the mobile robot 101, the movement range of the mobile robot 101 is changed so that it does not overlap with the protection area 10. This prevents interference between the operation of the fixed robot 1 and the movement of the mobile robot 101. Therefore, frequent safety control is avoided, and productivity can be improved.

[0053] In Modification 3, the sensor 3 may determine whether a portion of the movement range of the mobile robot 101 overlaps with a portion of the interference prevention area 40 in Modification 1. If a portion of the movement range of the mobile robot 101 overlaps with a portion of the interference prevention area 40, the sensor 3 may input a control signal (interference flag) to the robot controller 2. The robot controller 2 may perform control to slow down or stop the movement of the fixed robot 1 based on the control signal. If a portion of the movement range of the mobile robot 101 overlaps with a portion of the interference prevention area 40, the sensor 3 may issue a movement range change instruction. Changing the movement range of the mobile robot 101 prevents interference between the movement of the fixed robot 1 and the movement of the mobile robot 101. This avoids frequent safety control, making it possible to increase productivity.

[0054] <<Computer-readable recording medium>> A program that causes an information processing device or other machine or device (hereinafter referred to as a computer, etc.) to realize any of the above functions can be recorded on a computer-readable recording medium. Then, by having the computer, etc. read and execute the program from this recording medium, the function can be provided.

[0055] Here, a computer-readable recording medium refers to a recording medium that stores information such as data and programs electrically, magnetically, optically, mechanically, or chemically and that can be read by a computer. Among such recording media, those that can be removed from a computer include, for example, flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, Blu-ray disks, flash memories, etc. Recording media that are fixed to a computer include hard disks and ROMs.

[0056] <Appendix 1> an image generation unit (110) that generates a distance image and a brightness image of a predetermined area; a first point cloud data creation unit (120) that creates first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a mask data creation unit (150) that, when a tag is detected from the luminance image, creates mask data for masking a predetermined range in the predetermined region based on an analysis result of the tag; a second point cloud data creation unit (160) that creates second point cloud data by excluding point cloud data of the moving body provided with the tag from the first point cloud data based on the mask data; A sensor (3) comprising: <Appendix 2> An analysis unit (140) that analyzes the tag, The analysis unit (140) acquiring identification information of the tag based on the luminance image; obtaining size data about the moving object based on the identification information of the tag; calculating position coordinates of the tag and angle information of the tag based on the distance image and the luminance image, or based on the luminance image; generating the mask data based on the size data, the position coordinates of the tag, the angle information of the tag, and relationship information indicating a relative positional relationship between the tag and the moving object; Attachment 1, the sensor (3). <Appendix 3> The tag has a geometric feature. The sensor (3) described in Appendix 2. <Appendix 4> a determination unit that determines a positional relationship between a worker and a hazard within the predetermined area based on the second point cloud data, Attachments 1 to 3. A sensor (3) according to any one of claims 1 to 3. <Appendix 5> a determination unit that determines a positional relationship between the worker and a hazard within the predetermined area based on the first point cloud data when the tag is not detected from the luminance image; Attachments 1 to 3. A sensor (3) according to any one of claims 1 to 3. <Appendix 6> generating a distance image and a brightness image of a predetermined area; creating first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a step of generating mask data for masking a predetermined range in the predetermined region based on an analysis result of the tag when the tag is detected from the luminance image; creating second point cloud data by excluding point cloud data of the moving object provided with the tag from the first point cloud data based on the mask data; A method for controlling a sensor (3) having the above structure. <Appendix 7> On the computer, generating a distance image and a brightness image of a predetermined area; creating first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a step of generating mask data for masking a predetermined range in the predetermined region based on an analysis result of the tag when the tag is detected from the luminance image; creating second point cloud data by excluding point cloud data of the moving object provided with the tag from the first point cloud data based on the mask data; A program to execute. <Appendix 8> an image acquisition unit (190) that acquires a distance image and a brightness image of a predetermined area; a first point cloud data creation unit (310) that creates first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a mask data creation unit (340) that, when a tag is detected from the luminance image, creates mask data for masking a predetermined range in the predetermined region based on an analysis result of the tag; a second point cloud data creation unit (350) that creates second point cloud data by excluding point cloud data of the moving body provided with the tag from the first point cloud data based on the mask data; A safety monitoring system comprising: [Explanation of symbols]

[0057] 1: Fixed robot 2: Robot controller 3: 3D safety sensor 4: Information processing device 10:Protection area 11: Light emitting part 12: Light receiving part 13: Control unit 14: Storage part 15: Communications Department 20: External device 21: Database 31: Tag 32, 34: Running gear 33:Transportation equipment 34: Running gear 35: Manipulator 40: Interference prevention area 100: Worker 101: Mobile robot 110: Image generation unit 120, 310: First point cloud data creation section 130, 320: Detector 140, 330: Analysis Department 150, 340: Mask data creation section 160, 350: Second point cloud data creation section 170: Judgment section 180: Output section 300: Control device

Claims

1. an image generation unit that generates a distance image and a luminance image of a predetermined area; a first point cloud data creation unit that creates first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a mask data creation unit that creates mask data for masking a predetermined range in the predetermined region based on an analysis result of analyzing the tag when the tag is detected from the luminance image; a second point cloud data creation unit that creates second point cloud data by excluding point cloud data of the moving object provided with the tag from the first point cloud data based on the mask data; A sensor comprising:

2. an analysis unit that analyzes the tag, The analysis unit acquiring identification information of the tag based on the luminance image; obtaining size data about the moving object based on the identification information of the tag; calculating position coordinates of the tag and angle information of the tag based on the distance image and the luminance image, or based on the luminance image; generating the mask data based on the size data, the position coordinates of the tag, the angle information of the tag, and relationship information indicating a relative positional relationship between the tag and the moving object; The sensor of claim 1 .

3. The tag has a geometric feature. The sensor of claim 2 .

4. a determination unit that determines a positional relationship between a worker and a hazard within the predetermined area based on the second point cloud data, A sensor according to any one of claims 1 to 3.

5. a determination unit that determines a positional relationship between the worker and a hazard within the predetermined area based on the first point cloud data when the tag is not detected from the luminance image; A sensor according to any one of claims 1 to 3.

6. generating a distance image and a brightness image of a predetermined area; generating first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a step of generating mask data for masking a predetermined range in the predetermined region based on an analysis result of the tag when the tag is detected from the luminance image; generating second point cloud data by excluding point cloud data of the moving object provided with the tag from the first point cloud data based on the mask data; A method for controlling a sensor having the above structure.

7. On the computer, generating a distance image and a brightness image of a predetermined area; generating first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a step of generating mask data for masking a predetermined range in the predetermined region based on an analysis result of the tag when the tag is detected from the luminance image; Based on the mask data, the first point cloud data is used to identify the mobile object on which the tag is provided. creating second point cloud data excluding the point cloud data; A program to execute.

8. an image acquisition unit that acquires a distance image and a brightness image of a predetermined area; a first point cloud data creation unit that creates first point cloud data, which is three-dimensional point cloud data of the predetermined area, based on the distance image; a mask data creation unit that creates mask data for masking a predetermined range in the predetermined region based on an analysis result of analyzing the tag when the tag is detected from the luminance image; a second point cloud data creation unit that creates second point cloud data by excluding point cloud data of the moving object provided with the tag from the first point cloud data based on the mask data; A safety monitoring system comprising:

Images