Safety management system and safety management method for industrial device

Abstract

To provide a safety management system for an industrial device.SOLUTION: A safety management system for an industrial device includes: an image acquisition unit configured to capture an image of an inside of a workplace including a plurality of devices and detect an object in the captured image to generate object data; an image setting unit configured to set a monitoring region for a workplace image acquired by the image acquisition unit; an image determination unit configured to analyze the object data generated by the image acquisition unit to determine the object and generate a safety signal based on the determination; a display device configured to receive the safety signal transmitted from the image determination unit and display a safety region or output a warning sound based on the safety signal; a monitoring unit configured to monitor the inside of the workplace sensed through the image acquisition unit; and a control unit configured to receive the safety signal transmitted from the image determination unit and control the device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a safety management system for industrial devices, and more particularly to a safety management system and method for industrial devices that performs safety management by displaying a safety area for a set area or outputting an alarm sound through object recognition and object analysis within a workplace including a large number of devices. [Background technology]

[0002] Recently, with the development of the fourth industrial revolution, various types of industrial devices such as robots and conveyors are being used in manufacturing sites due to various reasons such as factory automation, unmanned operation, and a shortage of workers, but these industrial devices are causing industrial accidents among workers.

[0003] In relation to this, the largest proportion of fatal accidents in the manufacturing industry are entrapment accidents, with an average of 49 workers dying from entrapment accidents each year, with a particularly large number of fatalities caused by robots.

[0004] According to the Industrial Robot Standards in Section 13 of the Regulations on Industrial Safety and Health Standards, an 18m protective fence without openings must be installed as a basic requirement, and light curtains, safety mats, door locking devices, etc. must also be added.

[0005] However, despite the installation of such protective fences, industrial accidents involving workers frequently occur, and therefore there is a need for the development of a workplace safety system that can provide real-time worker safety through a real-time monitoring system for worker safety in the environment. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Korean Patent No. 10-2147065 (published on August 24, 2020) [Patent Document 2] Korean Patent No. 10-2101584 (published on April 16, 2020) Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to provide a safety management system for industrial devices that provides worker safety in real time by displaying a safety area for a set area or outputting an alarm sound through object recognition and object analysis within a workplace including a large number of devices.

[0008] The technical problems to be solved by the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned above will be clearly understood by those skilled in the art to which the present invention pertains from the following description. [Means for solving the problem]

[0009] In order to achieve the above object, the safety management system for industrial devices of the present invention is characterized by including an image acquisition unit that takes an image of the inside of a workplace including a large number of devices, detects objects in the taken image, and generates object data; an image setting unit that sets a monitoring area for the workplace image taken by the image acquisition unit; an image judgment unit that analyzes the object data generated by the image acquisition unit to judge the object and generate a safety signal based on the judgment; a display device that receives the safety signal transmitted from the image judgment unit and displays a safety area or outputs a warning sound based on the safety signal; a monitoring unit that monitors the inside of the workplace sensed through the image acquisition unit; and a control unit that receives the safety signal transmitted from the image judgment unit and controls devices.

[0010] The image acquisition unit is characterized by including a number of cameras that capture images of the inside of the workplace including the device, a motion sensor that recognizes the movement of objects present inside the workplace including the device, and an object data generation unit that generates object data.

[0011] The image judgment unit is characterized by including an object judgment unit that analyzes object data generated through the image acquisition unit using an object detection algorithm; a behavior judgment unit that recognizes object movements and generates skeleton data using a joint detection algorithm; a positive error judgment unit that collects information generated by the object judgment unit and the behavior judgment unit, and determines whether the worker is stationary, whether the object is within a safe area distance, and whether the object movements are legitimate, and judges a positive error; and a safety signal generation unit that generates a safety signal based on the judgment of the image judgment unit.

[0012] In addition, the safety signal generating unit generates safety signals indicating levels of caution, warning, and emergency corresponding to each monitoring area when the image determining unit detects that the object in the monitoring area is a worker, and outputs the safety signals through a display device or a monitoring unit.

[0013] Furthermore, the joint detection algorithm utilizes the Kinetics dataset or the NTU-RGB-D dataset. The object detection algorithm is one of SIFT, LBP, and YOLO algorithms.

[0014] Furthermore, the safety management method using the safety management system includes an image acquisition step in which an image processing unit acquires an image of a workplace including a device; a monitoring area setting step in which, after the image acquisition step, an image setting unit sets a monitoring area for the acquired workplace image using a setting tool; an object determination step in which, after the monitoring area setting step, an object determination unit analyzes object data generated via the image acquisition unit using an object detection algorithm to determine whether the object is a human or a device; and a device control step in which, if a worker is detected in a danger area corresponding to an emergency level, the control unit forcibly reduces or stops the operating speed of the device.

[0015] The object judgment step further includes a skeleton data generation step in which the behavior judgment unit recognizes the movement of the object and generates skeleton data, and a positive error judgment step in which the positive error judgment unit collects the information generated by the object judgment unit and the behavior judgment unit, and determines whether the worker is stationary, whether the object is within a safe area distance, and whether the movement of the object is legitimate, thereby judging a positive error. [Effects of the Invention]

[0016] According to the safety management system and safety management method for industrial devices of the present invention, it is possible to provide real-time worker safety by displaying a safety area for a set area or outputting an alarm sound through object recognition and object analysis within a workplace including a large number of devices.

[0017] The effects obtained by the present invention are not limited to those described above, and other effects not described above will be clearly understood by those having ordinary skill in the art to which the present invention pertains from the following description. [Brief explanation of the drawings]

[0018] [Figure 1]1 is a diagram showing the configuration of a safety management system of the present invention. [Figure 2] 1 is a diagram illustrating an embodiment of a safety management system of the present invention. [Figure 3] FIG. 2 is a diagram showing a configuration of an image determination unit of the present invention. [Figure 4] FIG. 1 illustrates the operation of the YOLO algorithm of the present invention. [Figure 5] FIG. 1 illustrates an embodiment of a positive error judgment of the present invention. [Figure 6] 1 shows an embodiment of a monitoring unit of the present invention. [Figure 7] 1 is a flowchart showing the sequence of a safety management method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] In describing embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. Furthermore, the terms described below are defined in consideration of the functions in the embodiments of the present invention, and may vary depending on the intentions or practices of users or operators. Therefore, the definitions should be based on the overall content of this specification.

[0020] The advantages and features of the present invention, as well as methods for achieving them, will become clearer with reference to the following detailed description of the embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and can be realized in various different forms. However, the present embodiments are provided to complete the disclosure of the present invention and fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the claims. The same reference numerals refer to the same elements throughout the specification.

[0021] It will be understood that each block of the process flowchart and combinations of flowcharts can be implemented by computer program instructions. These computer program instructions can be loaded onto a processor in a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the instructions, executed by the processor of the computer or other programmable data processing device, create means for performing the functions described in the flowchart blocks. These computer program instructions can also be stored in computer-usable or computer-readable memory that can direct the computer or other programmable data processing device to implement functions in a particular manner, such that the instructions stored in the computer-usable or computer-readable memory can produce an article of manufacture containing instruction means for performing the functions described in the flowchart blocks. Computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable data processing device to create a computer-implemented process, causing the computer or other programmable data processing device to provide instructions for performing the functions described in the flowchart blocks.

[0022] Furthermore, each block may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of order. For example, two blocks shown in succession may actually be performed substantially simultaneously, or the blocks may sometimes be performed in reverse order according to their respective functions.

[0023] In this embodiment, the term "module" refers to software or hardware components such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), and the "module" performs a certain function. However, the term "module" is not limited to software or hardware. The "module" may be configured to reside on an addressable storage medium or to implement one or more processors. Thus, as an example, the "module" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The components and functions provided in the "modules" may be combined into fewer components and "modules" or further separated into additional components and "modules." Furthermore, the components and "modules" may be implemented to implement one or more CPUs within a device or a security multimedia card.

[0024] In describing the embodiments of the present invention in detail, the present invention will be primarily directed to examples of specific systems. However, the main gist of the present invention is applicable to other communication systems and services having similar technical backgrounds without significantly departing from the scope disclosed in the present specification, and this would be within the judgment of a person skilled in the art.

[0025] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A safety management system for industrial devices (hereinafter referred to as a safety management system) according to an embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] FIG. 1 is a diagram showing the configuration of a safety management system of the present invention, and FIG. 2 is a diagram showing an embodiment of the safety management system of the present invention.

[0027] First, in one embodiment of the present invention, the industrial device D may include, but is not limited to, a conveyor, and may include automatic or semi-automatic devices such as a robot, a hydraulic machine, an arm, a press, a tool, a drill, etc.

[0028] Referring to FIG. 1, a safety management system 10 according to one embodiment of the present invention may include an image acquisition unit 100, an image setting unit 200, an image judgment unit 300, a display device 400, a monitoring unit 500, and a control unit 600, and each component may be connected to each other via a network.

[0029] The image acquisition unit 100 according to an embodiment of the present invention may capture an image of a workplace including a plurality of devices D, detect objects in the captured image, and generate object data. To this end, the image acquisition unit 100 may include a plurality of cameras, a motion sensor, and an object data generation unit.

[0030] The cameras capture the inside of the workplace and can be USB cameras, IP cameras, Gig-E cameras, CCTV cameras, etc. These cameras can perform various functions such as object recognition, tracking monitoring, and abnormal situation detection, and can include functions such as switching direction and angle and zooming.

[0031] The motion sensor is capable of recognizing the movement of objects present within the workspace.

[0032] The object data generation unit detects dynamic objects, including objects, within the workplace from images acquired by each camera, generates object data, and can transmit the generated object data to an image setting unit or the like via a network.

[0033] According to an embodiment of the present invention, the image setting unit 200 can set a monitoring area for a workplace image acquired by the image acquisition unit 200. To this end, the image setting unit 200 can include an area setting tool (not shown) for setting the monitoring area.

[0034] The monitoring area may include a working radius that is more than a certain distance away from multiple devices D, and may be set using an area setting tool as an alarm area corresponding to a caution level, a boundary area within the alarm area corresponding to a warning level, and a danger area within the boundary area corresponding to an emergency level, and each area may be displayed in a different color. For example, the alarm area may be displayed in yellow, the boundary area in orange, and the danger area in red, but is not limited thereto, and may be displayed in various colors or stages.

[0035] FIG. 3 is a diagram showing the configuration of the image judgment unit of the present invention, and FIG. 4 is a diagram showing the operation of the YOLO algorithm of the present invention.

[0036] The image determination unit 300 according to an embodiment of the present invention may determine an object by analyzing object data generated by the image acquisition unit 100. To this end, the image determination unit 300 may include an object determination unit 310, an action determination unit 320, a positive error determination unit 330, and a safety signal generation unit 340.

[0037] The object determination unit 310 analyzes the object data generated by the image acquisition unit 100 using an object detection algorithm to determine whether the object is a human or a device.

[0038] The object detection algorithm may include SIFT (Scale Invariant Feature Transform), HOG (Histogram of Gradient), LBP (Local Binary Pattern), etc., and preferably, YOLO (You Only Look Once) algorithm can be used.

[0039] The YOLO algorithm has a pipeline structure for overlapping processing of classification information for object candidates, and each individual element is trained separately, allowing computationally intensive problems to be processed in a single network. Referring to Figure 4, the YOLO algorithm divides an input image into NxN grid cells and performs training through a CNN network. The YOLO algorithm pre-divides multiple grid cells using a bounding box prediction method. When detecting a human in an input image, a high confidence value obtained by matching with the grid cell indicates that a human has been detected. Each grid cell predicts multiple bounding boxes and their confidence values. The grid cell predicts the class probability taking into account the overlap of each bounding box with other bounding boxes, and the final bounding box can be selected through non-max suppression (NMS).

[0040] The object determination unit 310 can acquire data including images from the image acquisition unit and determine objects using the YOLO algorithm. That is, among the objects detected by the YOLO algorithm, only those detected as workers are listed, and those detected as devices are not listed. Furthermore, the object determination unit 310 can convert undetermined objects into data and transmit it to the behavior determination unit.

[0041] The behavior determination unit 320 can recognize the movement of the object and generate skeleton data using a joint detection algorithm. To this end, the behavior determination unit can include a skeleton data generation unit that generates skeleton data and can be linked to a motion sensor that recognizes the movement of the object.

[0042] Skeleton data can be used to detect joints using algorithms such as the Kinetics dataset and the NTU-RGB-D (Nanyang Technological University's Red Blue Green and Depth information) dataset.

[0043] The Kinetics dataset can represent human joints with 17 key points, while NTU-RGB-D can represent human joints with 25 key points.

[0044] The skeleton data generated using the above data set can be expressed as XY coordinate values on an image of the parts where the joints of the object are located.

[0045] By analyzing the objects not determined by the object determination unit 310 through the behavior determination unit 320, it is possible to perform more precise object determination.

[0046] The positive error determination unit 330 may determine a positive error by collecting information generated by the object determination unit 310 and the behavior determination unit 320 and determining whether the worker is stationary, whether the object is within a safe area distance, and whether the object's movement is valid. To this end, the positive error determination unit may be connected to a motion sensor and a processor.

[0047] 5 is a diagram illustrating an embodiment of the positive error judgment of the present invention. Referring to FIG. 5, the positive error judgment unit 330 can collect data from the motion sensor, the object judgment unit 310, the behavior judgment unit 320, etc., and perform a step (S10) of confirming the presence or absence of a worker. When it is determined that a worker is not present during the confirmation of the presence or absence of a worker, the positive error judgment unit can ignore the data or proceed to a step of receiving an instruction to move the object. When it is determined that a worker is present, the positive error judgment unit can notify the safety management system 10 and perform a step (S20) of determining whether the object is static.

[0048] In the step (S20) of determining whether the object is static, if it is determined that the object is static, the positive error determination unit can ignore it or proceed to a step of receiving an instruction to move the object, and if it is determined that the object is not static, it can notify the safety management system 10 and perform a step (S30) of determining whether the object is within the safety area.

[0049] In the step (S30) of determining whether the object is within the safety area, if it is determined that the object is within the safety area, the positive error determination unit 330 can ignore it or proceed to a step of receiving an instruction to move the object, and if it is determined that the object is not within the safety area, it can notify the safety management system 10 and perform a step (S40) of determining whether the movement of the object is valid.

[0050] In the step (S40) of determining whether the object movement is valid, if it is determined that the object movement is valid, the safety management system 10 is notified, and if the object movement is not determined to be valid, a step (S50) can be performed in which this is determined to be a positive error and a notification signal is transmitted.

[0051] In this way, by having the positive error judgment unit 330 judge positive errors, the accuracy of object judgment can be improved along with object judgment and object behavior judgment, and it is possible to prevent the device from stopping due to an object judgment being judged to be an error, which would reduce work efficiency.

[0052] The safety signal generator 340 may generate a safety signal based on the determination of the image determination unit 300. That is, when the image determination unit 300 detects that an object in a monitoring area is a worker, the safety signal generator 340 may generate a safety signal indicating a level of caution, warning, or emergency corresponding to each monitoring area. Such a safety signal may be output via the display device 400 or the monitoring unit 500, which will be described later.

[0053] When the safety signal is output through the display device 400, an actual safety area may be displayed in the workplace or a warning sound may be output. When the safety signal is output through the monitoring unit 500, it may be displayed on the monitor in stages according to a preset monitoring area.

[0054] The display device 400 receives the safety signal transmitted from the safety signal generator and can display a safety area or output a warning sound based on the safety signal.

[0055] The display device 400 can display or express in various forms, including generating a warning sound, generating a warning light signal, setting a safety area, etc. To this end, the display device can include an audio output device including a speaker for generating a warning sound, a warning light signal generating device (not shown) including a warning lamp for generating a warning light signal, a safety area display device (not shown) for displaying the safety area, etc.

[0056] The safety area display device can display the actual safety area in colors differentiated from each other so that a worker can visually confirm the safety area in the workplace.

[0057] The actual safety area can be configured in multiple steps according to preset levels, and can be set into alarm area, boundary area and danger area, and each area can be displayed with a unique color.

[0058] For example, alarm areas can be displayed in yellow, speed limit areas in orange, work stop areas in red, etc., and undisplayed areas can be safety areas.

[0059] The monitoring unit 500 monitors the inside of the workplace sensed through the image acquisition unit 100, receives the safety signal generated by the safety signal generation unit, and can display a virtual safety area on the monitor.

[0060] Fig. 6 is a diagram showing an embodiment of the monitoring unit of the present invention. Referring to Fig. 6, the virtual safety area can be configured in multiple steps according to preset levels, just like the actual safety area, and can be set as an alarm area Y, a boundary area O, and a danger area R, and each area can be displayed in a unique color.

[0061] For example, the alarm area Y can be displayed in yellow, the boundary area O in orange, and the danger area R in red, but the display is not limited to these and can be displayed in various colors.

[0062] The control unit 600 can control the device by receiving the safety signal generated by the safety signal generation unit. For example, the control unit 600 can forcibly slow down or stop the operation speed of the device when a worker is detected in the danger area R corresponding to an emergency level. In this case, the control unit 600 can receive data from the positive error determination unit 330 before stopping the device and check whether it is a positive error. If it is not a positive error, the control unit 600 can stop the device, and if it is a positive error, the control unit 600 can not stop the device.

[0063] FIG. 7 is a flowchart of the safety management method of the present invention. Referring to FIG. 7, the safety management method of the present invention may include an image capturing step (S100), a monitoring area setting step (S200), an object determining step (S300), and a device control step (S400).

[0064] The image acquisition step (S100) is a step in which the image processing unit acquires an image of the workplace including the device.

[0065] In the image capturing step (S100), the image capturing unit captures an image of the inside of a workplace including a large number of devices, and can detect objects in the captured image to generate object data.

[0066] The monitoring area setting step (S200) is a step in which, after the image acquisition step, the image setting unit sets a monitoring area for the acquired work place image using a setting tool.

[0067] The monitoring area can include a working radius that is more than a certain distance away from multiple devices, and can be set using the area setting tool as an alarm area corresponding to a caution level, a boundary area within the alarm area corresponding to a warning level, or a danger area within the boundary area corresponding to an emergency level.

[0068] The object determination step (S300) is a step that occurs after the monitoring area setting step, in which the object determination unit analyzes the object data generated via the image acquisition unit using an object detection algorithm to determine whether the object is a human or a device.

[0069] The object determination step (S300) is a step in which the determination unit analyzes the data recognized by the object recognition unit using an object detection algorithm to determine an object. The object detection algorithm is preferably, but not limited to, the YOLO algorithm, and may be SIFT (Scale Invariant Feature Transform), HOG (Histogram of Gradient), LBP (Local Binary Pattern), etc.

[0070] The object judgment step (S300) according to one embodiment of the present invention may include a skeleton data generation step in which the behavior judgment unit recognizes the movement of the object and generates skeleton data, and a positive error judgment step in which the positive error judgment unit collects the information generated by the object judgment unit and the behavior judgment unit, and determines whether the worker is stationary, whether the object is within a safe area distance, and whether the movement of the object is valid, and judges a positive error.

[0071] In this way, by further including the skeleton data generation step and the positive error judgment step in the object judgment step, the accuracy of the object judgment can be improved.

[0072] The device control step (S400) is a step in which the control unit forcibly slows down or stops the operation speed of the device when a worker is detected in the danger area R corresponding to the emergency level. In the device control step (S400), before stopping the device, the control unit receives data from the positive error determination unit 330 and checks whether it is a positive error, and if it is not a positive error, the control unit stops the device, and if it is a positive error, the control unit does not stop the device.

[0073] According to the safety management system and safety management method for industrial devices of the present invention, it is possible to provide worker safety in real time by displaying a safety area for a set area or outputting an alarm sound through object recognition and object analysis within a workplace including a large number of devices.

[0074] The embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to easily explain the technical content of the present invention and facilitate understanding of the present invention, and are not intended to limit the scope of the present invention. Furthermore, the above-described embodiments of the present invention are merely illustrative, and a person skilled in the art will understand that various modifications and equivalent embodiments are possible. Therefore, the true technical scope of protection of the present invention should be determined by the following claims. [Explanation of symbols]

[0075] 10 Safety Management System 100 Image acquisition unit 200 Image Settings 300 Image Judgment Department 400 display device 500 Monitoring Department 600 control section

Claims

1. an image acquisition unit that captures an image of the inside of a workplace including a large number of devices, detects objects in the captured image, and generates object data; an image setting unit that sets a monitoring area for the work place image acquired by the image acquisition unit; an image determination unit that analyzes the object data generated by the image acquisition unit to determine the object and generates a safety signal based on the determination; a display device that receives a safety signal transmitted from the image determination unit and displays a safety area or outputs a warning sound based on the safety signal; a monitoring unit that monitors the inside of the workplace sensed through the image acquisition unit; a control unit that receives a safety signal transmitted from the image determination unit and controls the device.

2. The image acquisition unit A number of cameras capture the inside of the workplace, including the device, a motion sensor that recognizes the movement of objects present within a workspace including the device; 2. The safety management system for industrial devices according to claim 1, further comprising: an object data generation unit that generates object data.

3. The image determination unit an object determination unit that analyzes the object data generated by the image acquisition unit through an object detection algorithm; a behavior determination unit that recognizes the movement of an object and generates skeleton data using a joint detection algorithm; a positive error determination unit that collects the information generated by the object determination unit and the behavior determination unit, and determines whether the worker is stationary, whether the object is within a safe area distance, and whether the object's movement is legitimate, thereby determining a positive error; 2. The safety management system for industrial devices according to claim 1, further comprising: a safety signal generating unit that generates a safety signal based on the determination of the image determining unit.

4. The safety signal generating unit 2. The safety management system for industrial devices according to claim 1, wherein when the image determination unit detects that an object in a monitoring area is a worker, a safety signal is generated to notify a worker of a level corresponding to each monitoring area, such as caution, warning, or emergency, and the safety signal is output via a display device or a monitoring unit.

5. The safety management system for industrial devices according to claim 3 , wherein the joint detection algorithm utilizes a Kinetics dataset or an NTU-RGB-D dataset.

6. The safety management system for industrial devices according to claim 3 , wherein the object detection algorithm is one of SIFT, LBP, and YOLO algorithms.

7. In a safety management method using a safety management system, an image acquisition step in which an image processing unit acquires an image of a workplace including the device; a monitoring area setting step in which, after the image acquisition step, an image setting unit sets a monitoring area for the acquired workplace image using a setting tool; an object determination step in which, after the monitoring area setting step, an object determination unit analyzes the object data generated by the image acquisition unit using an object detection algorithm to determine whether the object is a human or a device; A device control step in which the control unit forcibly reduces or stops the operating speed of the device when a worker is detected in a danger area corresponding to an emergency level.

8. The object determination step includes: a skeleton data generation step in which the behavior determination unit recognizes the movement of the object and generates skeleton data; 8. The industrial device safety management method according to claim 7, further comprising a positive error judgment step in which the positive error judgment unit collects information generated by the object judgment unit and the behavior judgment unit, and determines whether the worker is stationary, whether the object is within a safety area distance, and whether the movement of the object is legitimate, thereby judging a positive error.

Images