Industrial device safety management system and method

Abstract

The present invention relates to an industrial device safety management system, and more specifically to an industrial device safety management system and method, enabling safety management to be carried out by displaying a safety area for a region configured through object recognition and object analysis inside a workplace including multiple devices, or by outputting a warning sound.

Description

Safety management system and safety management method for industrial devices

[0001] The present invention relates to a safety management system for industrial devices, and more particularly, to a safety management system and safety management method for industrial devices that enable safety management by displaying a safety zone for a set area or outputting a warning sound through object recognition and object analysis inside a workplace including a plurality of devices.

[0002] Recently, due to various reasons such as factory automation, unmanned operation, and labor shortage due to the development of the 4th industrial revolution, various types of industrial devices such as robots and conveyors are being used in manufacturing sites, and industrial accidents are occurring among workers due to these industrial devices.

[0003] In this regard, entrapment accidents account for the largest proportion of fatalities in the manufacturing industry, with an average of 49 workers dying each year. Of these 49 entrapment deaths, a significant number are caused by robots.

[0004] According to Article 13 of the Industrial Safety and Health Standards, Industrial Robot Standards, an 18m long safety fence without openings must be installed as a basic requirement, and additional features such as light curtains, safety mats, and door locks must be added.

[0005] However, despite the installation of such protective fences, industrial accidents involving workers are occurring frequently, and it is necessary to develop a workplace safety system that can provide real-time worker safety through a real-time monitoring system for worker safety in the environment.

[0006]

[0007] [Prior Art Literature]

[0008] (Patent Document 0001) Korean Intellectual Property Office Registered Patent Publication No. 10-2147065 (Publication 20200824)

[0009] (Patent Document 0002) Korean Intellectual Property Office Registered Patent Publication No. 10-2101584 (Publication 20200416)

[0010]

[0011] The purpose of the present invention is to provide a safety management system for industrial devices that provides real-time safety to workers by displaying a safety zone for a set area or outputting a warning sound through object recognition and object analysis inside a workplace including a plurality of devices.

[0012] The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

[0013] In order to achieve the above-described object, the safety management system of the industrial device of the present invention is characterized by including: an image acquisition unit (100) for capturing an image of the inside of a workplace including a plurality of devices (D) and detecting an object in the captured image to generate object data; an image setting unit (200) for setting a surveillance area for the workplace image captured by the image acquisition unit (100); an image judgment unit (300) for analyzing the object data generated by the image acquisition unit (100) to determine the object and generating a safety signal according to the determination; a display device (400) for receiving a safety signal transmitted from the image judgment unit (300) and displaying a safety area or outputting a warning sound according to the safety signal; a monitoring unit (500) for monitoring the inside of the workplace detected through the image acquisition unit (100); and a control unit (600) for receiving a safety signal transmitted from the image judgment unit (300) to control a device.

[0014] In addition, the image acquisition unit (100) is characterized by including a plurality of cameras that photograph the inside of a workplace including a device; a motion sensor that recognizes the movement of an object existing inside a workplace including a device; and an object data generation unit that generates object data.

[0015] In addition, the image judgment unit (300) is characterized by including an object judgment unit (310) that analyzes object data generated through the image acquisition unit (100) using an object detection algorithm; a behavior judgment unit (320) that recognizes object movement and generates skeleton data using a joint detection algorithm; a positive error judgment unit (320) that collects information generated by the object judgment unit (310) and the behavior judgment unit (320) to determine whether a worker is stationary, whether an object is within a safe zone distance, and whether the object's movement is legitimate, and determines a positive error; and a safety signal generation unit (340) that generates a safety signal according to the judgment of the image judgment unit (300).

[0016] In addition, the safety signal generation unit (340) is characterized in that, when the image judgment unit (300) detects that an object in the surveillance area is a worker, it generates a safety signal that notifies the level of caution, warning, and emergency corresponding to each surveillance area in stages, and outputs the safety signal to the display device (400) or monitoring unit (500).

[0017] In addition, the joint detection algorithm is characterized by utilizing a Kinetics data set or an NTU-RGB-D data set.

[0018] In addition, the object detection algorithm is characterized by being one of the SIFT, LBP, and YOLO algorithms.

[0019] In addition, in a safety management method using a safety management system, it is characterized by including an image acquisition step (S100) in which an image processing unit acquires an image of a workplace including a device; a surveillance area setting step (S200) in which an image setting unit sets a surveillance area using a setting tool for the acquired workplace image after the image acquisition step (S100); an object determination step (S300) in which an object determination unit analyzes object data generated by the image acquisition unit using an object detection algorithm to determine whether the object is a person or a device after the surveillance area setting step (S200); and a device control step (S400) in which, when a control unit detects a worker in a risk area (R) corresponding to an emergency level, the operating speed of the device is forcibly reduced or stopped.

[0020] In addition, the object judgment step (S300) is characterized in that it further includes a skeleton data generation step in which the action judgment unit recognizes the object movement and generates skeleton data; and a positive error judgment step in which the positive error judgment unit collects information generated by the object judgment unit and the action judgment unit and determines whether the worker is stationary, whether the object is within the safe zone distance, and whether the object's movement is justified to determine a positive error.

[0021] According to the safety management system and safety management method of the industrial device of the present invention, it is possible to display a safety zone for a set area through object recognition and object analysis within a workplace including a plurality of devices, or to output a warning sound to provide real-time safety to workers.

[0022] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention belongs from the description below.

[0023] Figure 1 is a drawing showing the configuration of the safety management system of the present invention;

[0024] Figure 2 is a drawing showing an embodiment of the safety management system of the present invention;

[0025] Figure 3 is a drawing showing the configuration of the image judgment unit of the present invention.

[0026] Figure 4 is a diagram showing the operation of the YOLO algorithm of the present invention.

[0027] Figure 5 is a diagram showing an embodiment of a positive error judgment of the present invention;

[0028] Figure 6 is a drawing showing an embodiment of a monitoring unit of the present invention;

[0029] Figure 7 is a flowchart showing the sequence of the safety management method of the present invention.

[0030]

[0031] When describing embodiments of the present invention, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined based on their functions in the embodiments of the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0032] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Like reference numerals designate like elements throughout the specification.

[0033] At this time, it will be understood that each block of the processing flowchart drawings and combinations of the flowchart drawings can be performed by computer program instructions. These computer program instructions can be installed in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create a means for performing the functions described in the flowchart block(s). These computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-available or computer-readable memory can also produce a manufactured item that includes an instruction means for performing the functions described in the flowchart block(s). Since the computer program instructions may be installed on a computer or other programmable data processing device, a series of operational steps may be performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform the steps for performing the functions described in the flowchart block(s) may also provide steps for performing the functions described in the flowchart block(s).

[0034] Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative implementation examples, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on their respective functions.

[0035] Here, the term '~ part' used in the present embodiment means software or hardware components such as FPGA (field-programmable gate array) or ASIC (application specific integrated circuit), and the '~ part' performs certain roles. However, the '~ part' is not limited to software or hardware. The '~ part' may be configured to be in an addressable storage medium and may be configured to play one or more processors. Therefore, as an example, the '~ part' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and '~ parts' may be combined into a smaller number of components and '~ parts' or further separated into additional components and '~ parts'. Additionally, components and '~parts' may be implemented to regenerate one or more CPUs within a device or secure multimedia card.

[0036] In specifically describing embodiments of the present invention, examples of specific systems will be primarily used, but the main points claimed in this specification can be applied to other communication systems and services having similar technical backgrounds without significantly departing from the scope disclosed in this specification, and this can be done at the discretion of a person skilled in the relevant technical field.

[0037] Hereinafter, a safety management system for an industrial device (hereinafter referred to as “safety management system”) according to one embodiment of the present invention will be described in detail with reference to the attached drawings.

[0038] Figure 1 is a drawing showing the configuration of the safety management system of the present invention, and Figure 2 is a drawing showing an embodiment of the safety management system of the present invention.

[0039] Referring to FIG. 1, a safety management system (10) according to one embodiment of the present invention may include an image acquisition unit (100), an image setting unit (200), an image judgment unit (300), a display device (400), a monitoring unit (500), and a control unit (600), and each component may be connected to each other through a network.

[0040] Referring to FIG. 2, an industrial device (D) according to one embodiment of the present invention may include a conveyor, and may include, but is not limited to, an automatic or semi-automatic device such as a robot, a conveyor, a hydraulic machine, an arm, a press, a tool, a drill, etc.

[0041] The image acquisition unit (100) can capture images of the interior of a workplace including multiple devices (D) and detect objects within the captured images to generate object data. To this end, the image acquisition unit (100) can include multiple cameras that capture images of the interior of the workplace, a motion sensor that recognizes the movement of objects present within the workplace, and an object data generation unit that generates object data.

[0042] Cameras can be USB cameras, IP cameras, Gig-E cameras, CCTV cameras, etc., and these cameras can perform various functions such as object recognition, tracking surveillance, and abnormal situation detection, and can include functions such as direction and angle switching, and zooming.

[0043] The object data generation unit can detect dynamic objects, including objects, in the workplace using images acquired from each camera to generate object data, and transmit the generated object data to an image setting unit, etc., using a network.

[0044] The image setting unit (200) can set a surveillance area for the workplace image acquired from the image acquisition unit. To this end, the image setting unit may include an area setting tool for setting the surveillance area.

[0045] The surveillance area may include an operation radius that is a certain distance away from multiple devices (D), and may be set as an alarm area corresponding to a caution level by the area setting tool, a boundary area corresponding to a warning level within the alarm area, and a danger area corresponding to an emergency level within the boundary area, and each area may be displayed in a different color, for example, an alarm area in yellow, a boundary area in orange, a danger area in red, etc., but is not limited thereto and may be expressed in various colors or levels.

[0046] Fig. 3 is a diagram showing the configuration of the image judgment unit of the present invention, and Fig. 4 is a diagram showing the operation of the YOLO algorithm of the present invention.

[0047] The image judgment unit (300) can determine an object by analyzing the object data generated by the image acquisition unit (100). To this end, the image judgment unit (300) may include an object judgment unit (310), an action judgment unit (320), a positive error judgment unit (330), and a safety signal generation unit (340).

[0048] The object determination unit (310) can analyze object data generated by the image acquisition unit (100) using an object detection algorithm to determine whether the object is a person or a device.

[0049] Object detection algorithms may include Scale Invariant Feature Transform (SIFT), Histogram of Gradient (HOG), Local Binary Pattern (LBP), etc., and preferably, the You Only Look Once (YOLO) algorithm may be used.

[0050] The YOLO algorithm has a pipeline structure for redundant processing of classification information for object candidates, and since each individual element is learned separately, it is an algorithm that processes computationally intensive problems in a single network. Referring to Fig. 4, the YOLO algorithm can divide the input image into NxN grid cells and learn through a CNN network. The YOLO algorithm can pre-segment multiple grid cells using a bounding box prediction method, and when a person is detected in the input image, it can mark the person as detected if the confidence value is high by matching it with the grid cell. Each grid cell predicts multiple bounding boxes and the confidence value for each bounding box, and the grid cell can predict the class probability that each bounding box overlaps with other bounding boxes and select the final bounding box through NMS (Non-Max Suppresision).

[0051] The object determination unit (310) acquires data including images from the image acquisition unit and can determine objects through the YOLO algorithm. That is, among the objects detected by the YOLO algorithm, only detections related to workers are listed, and detections related to devices are not listed. In addition, the object determination unit can digitize objects that cannot be determined and transmit them to the action determination unit.

[0052] The action determination unit (320) can recognize object movement and generate skeleton data using a joint detection algorithm. To this end, the action determination unit may include a skeleton data generation unit that generates skeleton data and may be linked to a motion sensor that recognizes object movement.

[0053] At this time, skeleton data can be utilized with joint detection algorithms such as the Kinetics dataset or the NTU-RGB-D (Nanyang Technological University's Red Blue Green and Depth information) dataset.

[0054] The Kinetics dataset can represent human joints with 17 key points, and NTU-RGB-D can represent human joints with 25 key points.

[0055] Skeleton data generated using the above data set can be expressed as XY coordinate values ​​on the image of the part where the joints of the object are located.

[0056] By using this type of action judgment unit (320), analysis can be performed on objects that are not judged by the object judgment unit (310), thereby enabling more precise object judgment.

[0057] The false positive judgment unit (330) collects information generated by the object judgment unit (310) and the action judgment unit (320) to determine whether the worker is stationary, whether the object is within the safe zone, and whether the object's movement is legitimate, thereby determining a false positive. To this end, the false positive judgment unit may be linked to a motion sensor and a processor.

[0058] FIG. 5 is a diagram showing an embodiment of a false positive judgment of the present invention. Referring to FIG. 5, a false positive judgment unit (330) may collect data from a motion sensor (not shown), an object judgment unit (310), an action judgment unit (320), etc., and perform a step (S10) of confirming whether a worker exists. When confirming whether a worker exists, if it is determined that the worker does not exist, the false positive judgment unit may ignore the data or proceed to a step of receiving an instruction to move the object. If it is determined that a worker exists, the unit may notify the safety management system and perform a step (S20) of determining whether the object is stationary.

[0059] In the step (S20) of determining whether an object is static, if the object is determined to be static, the positive error determination unit can ignore it or proceed to the step of receiving an instruction to move the object, and if the object is determined not to be static, the safety management system can be notified to perform the step (S30) of determining whether the object is within the safety zone.

[0060] In the step (S30) of determining whether an object is within a safe area, if it is determined that the object is within a safe area, the positive error determination unit may proceed to a step of ignoring the object or receiving an instruction to move the object, and if it is determined that the object is not within a safe area, the system may be notified and a step (S40) of determining whether the object's movement is legitimate may be performed.

[0061] In the step (S40) of determining whether the movement of the object is legitimate, if the movement of the object is determined to be legitimate, the system is notified, and if the movement of the object is not determined to be legitimate, a step (S50) of determining this as a positive error and transmitting a notification signal can be performed.

[0062] In this way, by determining a positive error through a positive error judgment unit, the accuracy of object judgment, along with object judgment and object action judgment, can be increased, and the device can be prevented from stopping due to an object judgment being determined to be an error, thereby reducing the efficiency of the work.

[0063] The safety signal generation unit (340) can generate a safety signal based on the judgment of the image judgment unit (300). That is, if the image judgment unit (300) detects that an object in the surveillance area is a worker, it can generate a safety signal that gradually notifies the level of caution, warning, and emergency corresponding to each surveillance area. Such a safety signal can be output to the display device (400) or monitoring unit (500) described below.

[0064] When a safety signal is output to a display device (400), an actual safety zone within the workplace may be displayed or a warning sound may be output. In addition, when a safety signal is output to a monitoring unit (500), it may be displayed on the monitor in stages according to a preset monitoring zone.

[0065] The display device (400) can receive a safety signal transmitted from a safety signal generation unit and display a safety area or output a warning sound according to the safety signal.

[0066] The display device (400) may be displayed or expressed in various forms, including generating a warning sound, generating a warning light signal, and setting a safety zone. To this end, the display device may include a voice output device including a speaker for generating a warning sound, a warning signal generating device (not shown) including a warning lamp for generating a warning light signal, a safety zone display device (not shown) for indicating a safety zone, etc.

[0067] Safety zone marking devices can display actual safety zones in different colors so that workers can visually check the safety zone within the workplace.

[0068] The actual safety zone can be divided into multiple stages according to preset levels, and can be set as an alarm zone, a boundary zone, and a danger zone, and each zone can be displayed with a unique color.

[0069] For example, alarm zones may be marked yellow, speed limit zones orange, stop work zones red, etc., and unmarked areas may be safety zones.

[0070] The monitoring unit (500) can monitor the interior of the workplace detected through the image acquisition unit (100) and receive a safety signal generated from the safety signal generation unit to display a virtual safety zone on the monitor.

[0071] FIG. 6 is a drawing showing an embodiment of a monitoring unit of the present invention. Referring to FIG. 6, a virtual safety zone can be configured in multiple stages according to preset levels, similar to an actual safety zone, and can be set as an alarm zone (Y), a boundary zone (O), and a danger zone (R), and each zone can be displayed in a unique color.

[0072] For example, the alarm zone (Y) may be displayed in yellow, the alert zone (O) in orange, and the danger zone (R) in red, but is not limited thereto and may be displayed in various colors.

[0073] The control unit (600) can control the device by receiving a safety signal generated by the safety signal generation unit.

[0074] For example, if a worker is detected in a risk area (R) corresponding to an emergency level, the control unit (600) can forcibly reduce or stop the operating speed of the device. At this time, the control unit (600) can receive data from the positive error determination unit (330) before stopping the device to check whether there is a positive error, and if there is no positive error, stop the device, and if there is a positive error, prevent the device from stopping.

[0075] Figure 7 is a flowchart of the safety management method of the present invention.

[0076] Referring to FIG. 7, the safety management method of the present invention may include an image acquisition step (S100), a surveillance area setting step (S200), an object determination step (S300), and a device control step (S400).

[0077] The image acquisition step (S100) is a step in which the image processing unit acquires an image of the workplace including the device.

[0078] In the image acquisition step, the image acquisition unit can capture an image of the inside of a workplace including a plurality of devices, detect an object in the captured image, and generate object data.

[0079] The surveillance area setting step (S200) is a step in which, after the image acquisition step, the image setting unit sets the surveillance area using a setting tool for the acquired workplace image.

[0080] The surveillance area may include an operating radius that is a certain distance away from multiple devices, and may be set as an alarm area corresponding to a caution level, a boundary area corresponding to a warning level within the alarm area, and a danger area corresponding to an emergency level within the boundary area by the area setting tool.

[0081] The object judgment step (S300) is a step in which, after the surveillance area setting step, the object judgment unit analyzes object data generated through the image acquisition unit using an object detection algorithm to determine whether the object is a person or a device.

[0082] The object determination step (S300) is a step in which the determination unit analyzes the data recognized by the object recognition unit using an object detection algorithm to determine the object. The object detection algorithm is preferably the YOLO algorithm, but is not limited thereto, and other algorithms such as SIFT (Scale Invariant Feature Transform), HOG (Histogram of Gradient), and LBP (Local Binary Pattern) can be applied.

[0083] In an object judgment step (S300) according to one embodiment of the present invention, the action judgment unit may include a skeleton data generation step in which the action judgment unit recognizes object movement and generates skeleton data; and a positive error judgment step in which the positive error judgment unit collects information generated by the object judgment unit and the action judgment unit to determine whether the worker is stationary, whether the object is within a safe zone distance, and whether the object's movement is legitimate, thereby determining a positive error.

[0084] By including a skeleton data generation step and a positive error judgment step in the object judgment step, the accuracy of object judgment can be increased.

[0085] The device control step (S400) is a step in which, if the control unit detects an operator in a risk area (R) corresponding to an emergency level, the device's operating speed is forcibly reduced or stopped. In the device control step (S400), before stopping the device, the data of the positive error determination unit (330) is received to determine whether there is a positive error. If there is no positive error, the device is stopped, and if there is a positive error, the device is not stopped.

[0086] According to the safety management system and safety management method of the industrial device of the present invention, it is possible to display a safety zone for a set area through object recognition and object analysis within a workplace including a plurality of devices, or to output a warning sound to provide real-time safety to workers.

[0087] The embodiments of the present invention disclosed in this specification and drawings are intended to provide a simple explanation of the technical content of the present invention and provide specific examples to aid understanding of the present invention, and are not intended to limit the scope of the present invention. Furthermore, the embodiments of the present invention described above are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments are possible. Therefore, the true technical protection scope of the present invention should be defined by the following claims.

[0088] The present invention relates to a safety management system and a safety management method for industrial devices that enable safety management by displaying a safety zone for a set area or outputting a warning sound through object recognition and object analysis within a workplace including a plurality of devices.

Claims

1. An image acquisition unit (100) that captures an image of the inside of a workplace including multiple devices (D) and detects an object in the captured image to generate object data; An image setting unit (200) that sets a surveillance area for the workplace image acquired from the image acquisition unit (100); An image judgment unit (300) that analyzes the object data generated by the image acquisition unit (100) to determine the object and generates a safety signal based on the determination; A display device (400) that receives a safety signal transmitted from the above image judgment unit (300) and displays a safety area or outputs a warning sound according to the safety signal; A monitoring unit (500) that monitors the interior of the workplace detected through the above image acquisition unit (100); A safety management system for an industrial device, including a control unit (600) that controls the device by receiving a safety signal transmitted from the image judgment unit (300).

2. In claim 1, The above image acquisition unit (100) is Multiple cameras filming the interior of the workplace, including the device; A motion sensor that recognizes the movement of objects inside the workplace, including devices; and A safety management system for an industrial device, comprising an object data generation unit that generates object data; 3. In claim 1, The above image judgment unit (300) An object determination unit (310) that analyzes object data generated through an image acquisition unit (100) using an object detection algorithm; An action judgment unit (320) that recognizes object movement and generates skeleton data using a joint detection algorithm; A positive error judgment unit (320) that collects information generated from the object judgment unit (310) and the action judgment unit (320) to determine whether the worker is stationary, whether the object exists within the safe zone distance, and whether the object's movement is justified, and then judges a positive error; A safety management system for an industrial device, comprising a safety signal generation unit (340) that generates a safety signal according to the judgment of an image judgment unit (300).

4. In claim 1, The above safety signal generating unit (340) A safety management system for an industrial device that, when an object in a surveillance area is detected as a worker in the image judgment unit (300), generates a safety signal that gradually notifies the level of caution, warning, and emergency corresponding to each surveillance area, and outputs the safety signal to a display device (400) or monitoring unit (500).

5. In claim 3, The above joint detection algorithm is, Safety management system for industrial devices utilizing Kinetics data sets or NTU-RGB-D data sets 6. In claim 3, The above object detection algorithm, Safety management system for industrial devices, using one of the SIFT, LBP, or YOLO algorithms 7. In the safety management method using the safety management system, An image acquisition step (S100) in which an image processing unit acquires an image of a work site including a device; After the above image acquisition step (S100), the image setting unit sets the surveillance area using a setting tool for the acquired workplace image (S200); After the above surveillance area setting step (S200), the object determination step (S300) in which the object determination unit analyzes the object data generated by the image acquisition unit through an object detection algorithm to determine whether the object is a person or a device; A safety management method for an industrial device, comprising a device control step (S400) for forcibly reducing or stopping the operating speed of the device when a worker is detected in a risk area (R) corresponding to an emergency level by the control unit; 8. In claim 7, The above object judgment step (S300) is A skeleton data generation step in which the action judgment unit recognizes object movement and generates skeleton data; and A safety management method for an industrial device, further comprising a positive error judgment step in which a positive error judgment unit collects information generated by an object judgment unit and an action judgment unit, determines whether a worker is stationary, whether an object is within a safe zone distance, and whether the object's movement is justified, and then determines a positive error;

Images