Method and System for Implementation of Multi-level On-site Safety Control of Worker Safety

Abstract

A multi-stage workplace monitoring method and system that introduces division of labor through multi-stage monitoring and analysis in monitoring worker safety, and enhances the efficiency of on-site analysis in the analysis of sensing data and images. The workplace monitoring method of the present invention may be performed in a workplace monitoring system comprising: a plurality of cameras; a plurality of control devices that are hierarchically connected and include a lower-level control device and an upper-level control device, which detect the dangerous state of a worker at the workplace by dividing roles based on images acquired by the plurality of cameras. First, the lower-level control device determines whether there is an image among the images acquired by the plurality of cameras that does not conform to a predetermined rule, analyzes whether a risk factor exists, and transmits the analysis result to the upper-level control device as safety-related information. Subsequently, the upper-level control device predicts the associated risk level by linking and synthesizing the plurality of safety-related information. Then, the upper-level control device generates a risk alert when the associated risk level is high.

Description

Technology Field

[0001] The present invention relates to a safety control method and system utilizing closed-circuit cameras and sensors, and in particular, to a method and system for safety control of on-site safety utilizing multiple cameras in a worker's workspace. Background Technology

[0002] A safety control system is being utilized in which surveillance cameras and various types of sensors are installed at the workplace, and employees at a central control center connected to these cameras and sensors monitor the camera footage. According to this control system, since on-site footage is viewed by the human eye, it relies on the experience of control center staff rather than precisely analyzing various accident cases; furthermore, because employees must monitor multiple videos simultaneously, it is easy to miss minor safety threats.

[0003] To compensate for these shortcomings, there are attempts to analyze video by introducing Artificial Intelligence (AI) technology. However, while AI video analysis is effective in cases where there are clearly distinguishable features in the video, such as fire detection, it has limitations in determining the level of danger from workers' movements based solely on video. Furthermore, given the limited volume of video data collected at the control center, it is difficult to assess danger based on such a restricted amount of footage. In addition, because communication from the site to the central control center takes time, as does the process of capturing video, analyzing it at the central center, and notifying the site of the danger, AI video analysis is difficult to apply in emergency situations.

[0004] Considering these issues, on-site response safety control systems are sometimes installed at every workplace to enable immediate response. While such systems are useful for immediate response in emergencies, they still require the analysis of a large amount of video footage and make immediate response difficult when a single site is large or when multiple workers are performing various tasks simultaneously.

[0005] Published Patent Application 10-2018-0095261 describes a location tracking and video safety control system that utilizes a worker terminal equipped by a worker and a video surveillance module. According to this system, the worker's location is confirmed by utilizing beacon signals received by the worker terminal, and the worker's safety is verified based on signals transmitted by the worker terminal; furthermore, the video acquired by the video surveillance module, i.e., the surveillance camera, is refined through certain processing within the camera. However, it is very difficult to verify the worker's safety status in real time by primarily utilizing the signals from the worker terminal. Moreover, processing and refining data at the surveillance camera level carries a high risk of potentially discarding important information. Prior art literature

[0006] Published Patent Application 10-2018-0095261, Aug. 27, 2018, FIG. 1 and FIG. 8, identification numbers

[31] ~

[34] ,

[40] ~

[43] The problem to be solved

[0007] The present invention aims to solve such problems by providing a multi-stage workplace safety control method and system that introduces division of labor through multi-stage safety control and analysis in worker safety control, and enhances the efficiency of on-site analysis in the analysis of sensing data and images. means of solving the problem

[0008] According to one aspect of the present invention, a workplace safety control device operates as a higher-level control device that determines the possibility of risk by synthesizing safety-related information generated by a lower-level control device among a plurality of hierarchically connected control devices based on workplace images acquired by a plurality of cameras. The control device includes a processor and a memory that stores program instructions executed by the processor. When the program instructions are executed by the processor, they perform: an operation of predicting an associated risk level by linking and synthesizing a plurality of safety-related information based on relationship information between each process; and an operation of generating a risk alert when the associated risk level is high.

[0009] The control device may further be equipped with a storage device that stores work process information for each process and the relationships between work processes. In this case, the operation of predicting the associated risk can be performed based on the work process information for each process and the work process relationship information.

[0010] The above program instructions may further include instructions that, when executed by the processor, receive a still image acquired by each camera from the lower-level control device, classify the process based on the still image, generate process type information, store it in the storage device, and provide it to the lower-level control device.

[0011] Program commands that receive the above still image and perform the operation of distinguishing processes based on the above still image can be executed during the device registration process when a new camera is connected to the above lower-level control device.

[0012] The above program instructions may further include instructions that, when executed by the processor, perform learning about the risk level through images for each work process and determine neural network parameters for image analysis based on the learning results.

[0013] The above program instructions may further include instructions that perform an operation of learning the ability to distinguish processes from input still images when executed by the processor.

[0014] The above program instructions may further include instructions that, when executed by the processor: accept camera attribute information for at least one of the plurality of cameras and one or more still images acquired by the at least one camera; and determine process information for a process captured by the at least one camera based on the camera attribute information and the one or more still images, and transmit the neural network parameters for the process to the lower-level control device, thereby enabling the work site images to be utilized for refinement.

[0015] The above safety-related information may include information on hazardous situations at the work site, object information on objects found in the image to be analyzed, and still images extracted from the image to be analyzed.

[0016] The above-mentioned work site safety control device may be connected to a plurality of lower-level control devices, each of which is connected to one or more cameras.

[0017] According to another aspect of the present invention, a workplace safety monitoring device operates as a lower-level monitoring device that determines the dangerous state of a worker based on workplace images acquired by a plurality of cameras, generates safety-related information for the worker, and transmits it to a higher-level monitoring device among a plurality of monitoring devices connected hierarchically. The monitoring device includes a processor and a memory that stores program instructions executed by the processor. The program instructions include instructions that, when executed by the processor, perform: an operation of analyzing whether a dangerous situation exists at the workplace by determining whether there is an image among the workplace images that does not conform to a predetermined rule based on process information and process-specific risk factor information; and an operation of transmitting the analysis result as safety-related information to the higher-level monitoring device.

[0018] The control device may further be equipped with a storage device that receives and stores the process information, process-specific risk factor information, and neural network parameters from the upper-level control device. In this case, a determination regarding whether there is an image that does not conform to the rule may be made based on the process information and the process-specific risk factor information.

[0019] The above program instructions may further include instructions that perform an operation to refine the images acquired by the plurality of cameras according to the work process and risk level when executed by the processor. In this case, the determination of whether there are images that do not conform to the rule can be made using the refined images.

[0020] Program commands that perform the operation of analyzing whether a dangerous situation exists at the above work site are provided separately for each process, and each can operate specialized for a specific process.

[0021] The above safety-related information may include information on the above-mentioned hazardous situation at the above-mentioned work site, object information on an object found in the image to be analyzed, and a still image extracted from the image to be analyzed.

[0022] The workplace safety control method of the present invention can be performed in a workplace safety control system comprising: a plurality of cameras, each acquiring images of a workplace; and a plurality of control devices, which are hierarchically connected and include a lower-level control device and an upper-level control device, which detect the dangerous state of a worker at the workplace by dividing roles based on images acquired by the plurality of cameras. (a) First, the lower-level control device determines whether there is an image among the images acquired by the plurality of cameras that does not conform to a predetermined rule, analyzes whether a dangerous situation exists, and transmits the analysis result to the upper-level control device as safety-related information. (b) Subsequently, the upper-level control device predicts the associated risk level by linking and synthesizing the plurality of safety-related information. (c) Then, the upper-level control device generates a risk alert when the associated risk level is high.

[0023] The above step (a) may include a step of refining the images acquired by the plurality of cameras according to the work process and risk level. In this case, when analyzing whether the risk situation exists, it is determined whether there is an image among the refined images that does not conform to the rule.

[0024] The method for controlling safety at a work site may further include a step performed no later than before the above step (b) is performed, wherein the above-level control device learns about the risk level through video for each work process and determines neural network parameters for video analysis based on the learning results.

[0025] The method for controlling safety at a work site may further include a step in which the upper-level control device learns the ability to distinguish processes from input still images.

[0026] A method for controlling safety at a work site may further include: a step in which the upper-level control device receives camera attribute information and one or more still images from at least one of the plurality of cameras through the lower-level control device; and a step of determining process information for a process captured by the at least one camera and the neural network parameters for the corresponding process based on the camera attribute information and the one or more still images, and transmitting them to the lower-level control device so as to enable them to be utilized for refinement. Effects of the invention

[0027] According to the present invention, safety monitoring of workers at a work site is performed in multiple stages by introducing a division of labor, thereby increasing the efficiency of on-site analysis of sensing data and images. Accordingly, information collected at the site through cameras, etc., can be effectively analyzed step-by-step, and necessary information can be transmitted to a higher level.

[0028] As the proportion of on-site analysis increases in the multi-stage safety control and analysis process, optimal video analysis becomes possible for various work processes, effective video analysis becomes possible even at lower stages, and many camera images can be analyzed simultaneously.

[0029] In addition, as safety monitoring and analysis are performed in multiple stages, there is the advantage of being able to reduce the amount of video data transmitted to the central control center and decrease the channel bandwidth required for transmission. Furthermore, by installing cameras at various angles at the work site, there is an effect of significantly reducing industrial accidents.

[0030] As such, the present invention enables effective implementation of industrial site safety control and reduces accidents at workers' workplaces by using low communication bandwidth while analyzing a large amount of image information quickly and efficiently. Brief explanation of the drawing

[0031] Hereinafter, preferred embodiments of the present invention will be described with reference to the attached drawings. In the drawings, FIG. 1 is a block diagram of a work site safety control system according to one embodiment of the present invention. FIG. 2 is a block diagram of a work site safety control system according to another embodiment of the present invention. FIG. 3 is a block diagram of a work site safety control system according to another embodiment of the present invention. FIG. 4 is a block diagram of a work site safety control system according to another embodiment of the present invention. Figure 5 is a flowchart showing the overall operation process of the work site safety control system illustrated in Figure 4. Figure 6 is a flowchart specifically showing the initial setup process illustrated in Figure 5. Figure 7 is a diagram specifically showing the device registration process. Figure 8 is a diagram showing the safety control process in a lower-level control device in more detail. Figure 9 is a diagram showing the safety control process in a higher-level control device in more detail. Figure 10 is a block diagram of lower-level control devices and upper-level control devices. Specific details for implementing the invention

[0032] The present invention is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0033] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0034] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0035] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0036] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.

[0037] In order to facilitate an overall understanding of the present invention, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0038] FIG. 1 is a block diagram of a work site safety control system according to an embodiment of the present invention. The illustrated work site safety control system can be installed in an industrial site, such as a construction site, and comprises a plurality of cameras (10a to 10c) for acquiring site images to visually check the situation at the work site, one or more sensors (20), and a control device (100) connected to the plurality of cameras (10a to 10c) and one or more sensors (20) via a network.

[0039] Each of the multiple cameras (10a to 10c) is installed to be scattered throughout the work site to acquire images of the site. At least some of the multiple cameras (10a to 10c) may be capable of pan, tilt, and zoom operation. The multiple cameras (10a to 10c) may be connected to a control device (100), transmit image data regarding the acquired images to the control device (100), and receive control signals from the control device (100). In one embodiment, the multiple cameras (10a to 10c) are IP cameras that are assigned IP addresses and can be connected to the control device (100) via a public network such as the Internet, but the present invention is not limited thereto.

[0040] The sensor (20) may include various sensors for detecting dangerous situations at a work site, such as fire detection sensors, and may include or be connected to IoT (Internet of Things) devices equipped with network interface functions. Accordingly, the sensor (20) can transmit detection data to the control device (100) via the Internet.

[0041] The control device (100) receives field video data and sensor data from the cameras (10a~10c) and the sensor (20), respectively, through a network, and displays them so that a safety manager can understand the situation at the work site.

[0042] In one embodiment, the control device (100) comprises a device connection service unit (110), a core service unit (120), and a management service unit (130).

[0043] The device connection service unit (110) provides device connection services using various communication methods. The communication methods supported by the device connection service unit (110) include WiFi, 5G / LTE, Ethernet, etc., but the present invention is not limited thereto and may include other communication methods.

[0044] The core service unit (120) may be equipped with a risk analysis module (122) that analyzes whether there is a dangerous situation harmful to a worker based on field video data and sensor data, and a field collected data database (DB) (124) that stores field video data and sensor data. In this embodiment, the risk analysis module (122) enables the setting of notifications for conditions of specific sensors, such as 'fire detection', and can analyze whether there is a dangerous situation based on pre-determined rules.

[0045] The management service unit (130) may be equipped with a system management module (132) that performs overall system management, a user interface (134) that enables the safety manager's monitor (140) to connect so that the safety manager can know about dangerous situations through the monitor (140), and an application programming interface (API: Application Programming Interface, 136) that enables data to be linked with an external data processing device.

[0046] According to this work site safety control system, a safety manager can identify risk factors that threaten the safety of workers while viewing real-time video. In addition, the safety manager can additionally recognize dangerous situations based on sensor data detected by various sensors (20) installed at the site. For example, the risk analysis module (122) can recognize a fire situation at the work site early based on a rule that 'recognizes a fire situation if the temperature from the temperature sensor is 50 degrees or higher.'

[0047] In an embodiment modified from the embodiment of FIG. 1, artificial intelligence (AI) technology may be added to the risk analysis module (122). FIG. 2 shows such an embodiment.

[0048] In the embodiment of FIG. 2, the risk analysis module of the control device (100) is implemented in the form of an AI video analysis module (122a) to which AI technology is applied. The AI ​​video analysis module (122a) receives field video data acquired by a plurality of cameras (10a to 10c) and performs video analysis on the field video. The video analysis of the AI ​​video analysis module (122a) includes object detection, face recognition, and recognition of special situations such as fire monitoring. The object detection includes the detection of safety equipment such as safety helmets, and through the detection of safety equipment, it is possible to verify whether the worker is wearing safety equipment. The AI ​​video analysis module (122a) can check the worker's safety status or safety hazards through primary video analysis and sensor data analysis. Therefore, there is no need for a safety manager to be stationed at the control center where the control device (100) is installed and to directly monitor real-time field video with their own eyes.

[0049] Although AI video analysis technology is currently developing rapidly, while it may be helpful for safety monitoring at the level of simply checking whether safety equipment is being worn, it is not easy to expect, at the time of this application, to determine whether the work environment is safe, or if it is in a dangerous situation or could develop into a dangerous situation, solely through video analysis. In other words, while AI video analysis by the AI ​​video analysis module (122a) can detect danger in special situations such as fire flame detection or situations where special objects like flames occur, it may be limited in detecting a wide range of dangerous situations or risk factors.

[0050] Meanwhile, due to the characteristics of video data, which has a large amount of data, wide bandwidth is required for video transmission. For example, compressing and transmitting one channel of high-definition (HD) video corresponding to the resolution of broadcast TV video using the MPEG-2 (Moving Picture Expert Group-2) standard requires a bandwidth of about 20 Mbps (Mega-bit per second). Considering that dozens of cameras can be installed even at small and medium-sized construction sites, it is nearly impossible to transmit video from all cameras at the site to a control device (100) to perform AI video analysis.

[0051] Taking these difficulties into account, another embodiment of the present invention introduces division of labor through multi-stage safety control and analysis, and improves the efficiency of on-site analysis in the analysis of sensing data and images.

[0052] FIG. 3 shows a work site safety control system according to another embodiment of the present invention. The work site safety control system according to the present embodiment comprises a plurality of cameras (10a to 10c), one or more sensors (20), local analysis devices (200a to 200c) each provided corresponding to each of the plurality of cameras (10a to 10c), and a central control device (300) connected to the local analysis devices (200a to 200c) via a network.

[0053] The local analysis device (200a) is equipped with a device connection service unit (210a), a core service unit (220a), and a management service unit (230a). The device connection service unit (210a) provides a connection service with the corresponding camera (10a). The core service unit (220a) analyzes whether there is a dangerous situation harmful to the worker based on video data and sensor data acquired by the camera (10a). The management service unit (230a) performs overall management of the local analysis device (200a), enables an interface with the central control device (300) at the work site, and transmits and receives data with the central control device (300) at the work site. Such a local analysis device (200a) can be implemented based on a commercial processing board capable of AI processing and can be installed on a gateway or access point (AP) equipment of a safety control network installed at the work site. Other local analysis devices (200b, 200c) shown in FIG. 3 have a configuration similar to the local analysis device (200a), so a detailed description thereof is omitted.

[0054] In the central control device (300) for the work site, the device connection service unit (310) provides device connection services using various communication methods, and in particular, provides an interface with local analysis devices (200a to 200c). The core service unit (320) may be equipped with a safety rule comprehensive module (322), a video comprehensive analysis module (324), and a field collected data DB (124). The safety rule comprehensive module (322) manages rules regarding risk factor analysis by the video comprehensive analysis module (324), determines whether to maintain the rules, and manages analysis rules in each local analysis device (200a to 200c). The video comprehensive analysis module (324) analyzes whether there are dangerous situations harmful to workers at the work site based on field video data and sensor data from the local analysis devices (200a to 200c) and analysis data from the local analysis devices (200a to 200c). The other configurations of the central control device (300) at the work site are similar to the configuration of the control device (100) shown in FIGS. 1 and 2, so a detailed description thereof is omitted.

[0055] In the work site safety control system according to the present embodiment, local analysis devices (200a to 300c) primarily analyze images acquired by corresponding cameras (10a to 10c) to detect objects, recognize faces, and analyze postures. The local analysis devices (200a to 200c) transmit the analysis data along with location and time information of the images to the work site central control device (300). Since only the information analyzed by AI from the images is transmitted to the upper level, rather than the entire image data, the amount of data transmitted can be significantly reduced. Accordingly, the work site central control device (300) can simultaneously receive and analyze image information acquired by dozens of cameras, and can perform on-site safety control more accurately.

[0056] FIG. 4 shows a work site safety control system according to another embodiment of the present invention. The work site safety control system according to this embodiment comprises a plurality of lower-level control devices (400a to 400n) and an upper-level control device (500) connected to the plurality of lower-level control devices (400a to 400n). A plurality of cameras (10a to 10c) and one or more sensors (20) may be connected to each lower-level control device (400a to 400n). Meanwhile, although the control devices (400a to 400n, 500) are hierarchically arranged in two stages in FIG. 4, the control devices may be hierarchically connected in more stages depending on the scale of the system, particularly the number of cameras and / or sensors.

[0057] Each lower-level control device (400a to 400n) receives and analyzes field video data and sensor data from cameras (10a to 10c) and sensors (20), respectively, and can immediately provide notification to the worker if a dangerous situation is found as a result of the analysis. Additionally, the lower-level control devices (400a to 400n) transmit the analysis results to the upper-level control device (500) as safety-related information. At this time, sensor data may also be transmitted. The safety-related information transmitted by the lower-level control devices (400a to 400n) to the upper-level control device (500) may include current safety status, object information detected in the video, and still pictures for reference in the comprehensive analysis by the upper-level control device (500). The object information may include human body detection information, information on the presence of protective equipment, dangerous work tools, etc.

[0058] The lower-level control device (400a) is equipped with a device connection service unit (410), a core service unit (420), and a management service unit (430). The device connection service unit (410) provides connection services with cameras (10a to 10c). The core service unit (420) analyzes whether there are dangerous situations harmful to the worker based on video data and sensor data acquired by the cameras (10a to 10c). The management service unit (430) performs overall management of the lower-level control device (400a), enables an interface with the upper-level control device (500), and transmits and receives data with the upper-level control device (500). Additionally, the management service unit (430) can provide an immediate notification to the worker if a dangerous situation is detected as a result of analyzing the video data and sensor data.

[0059] The above core service unit (420) may include a safety rule processing module (422), a process-specific image analysis module (424), a work process DB (426), a metadata DB (428), and a field-collected data DB (429).

[0060] The above safety rule processing module (422) manages rules regarding the analysis of dangerous situations by the process-specific image analysis module (424) and determines whether the current situation conforms to the rules. Specifically, the safety rule processing module (422) determines whether a dangerous situation exists within an image by determining whether the image conforms to a predetermined rule based on process information of each process stored in the work process DB (426), camera attribute information and process-specific risk factor information stored in the metadata DB (428). The safety rule processing module (422) operates in a rule-based manner, but in modified embodiments, it may operate in a learning-based manner.

[0061] The process-specific image analysis module (424) may be provided separately for each process or each camera (10a to 10c). Accordingly, the process-specific image analysis module (424) operates specifically for a particular process or a particular camera (10a to 10c). The process-specific image analysis module (424) refines the images received from the cameras (10a to 10c) according to the work process and risk level based on camera attribute information and neural network parameters stored in the metadata DB (428), thereby reducing the processing burden of the safety rule processing module (522) and enabling the safety rule processing module (422) to perform accurate image analysis.

[0062] The work process DB (426) stores process information for each process at the work site.

[0063] The metadata DB (428) stores attribute information for each camera (10a~10c) and sensor (20), neural network parameters for image analysis received from the upper-level control device (500), and risk factor information for each process.

[0064] The field collection data DB (429) stores image data and sensor data received from cameras (10a~10c) and sensors (20), respectively.

[0065] The upper-level control device (500) receives safety-related information and sensor data from multiple lower-level control devices (400a to 400n) and analyzes whether the associated risk is high. At this time, the upper-level control device (500) performs a comprehensive analysis by linking the safety-related information received from multiple lower-level control devices (400a to 400n). In particular, the upper-level control device (500) comprehensively analyzes still images, etc. included in the safety-related information and can analyze whether work with a risk-related relationship is being performed simultaneously in adjacent spaces by utilizing the relationships between work processes stored in the work process relationship DB (526) and the work process information stored in the metadata DB (528). If it is determined as a result of the analysis that work with a risk-related relationship is being performed simultaneously, the upper-level control device (500) notifies the safety manager that it is a dangerous situation.

[0066] The upper-level control device (500) is equipped with a device connection service unit (510), a core service unit (520), and a management service unit (530). The device connection service unit (510) provides device connection services using various communication methods, and in particular, provides an interface with lower-level control devices (400a to 400n). The core service unit (520) performs comprehensive analysis by linking safety-related information received from lower-level control devices (400a to 400n). The management service unit (530) performs overall management of the upper-level control device (500) and enables the safety manager's monitor (140) to connect, allowing the safety manager to know the situation at the work site. In addition, if a dangerous situation is detected at the work site as a result of the analysis, the management service unit (530) notifies the worker of the danger.

[0067] The above core service unit (520) may include a safety rule processing module (522), a comprehensive image analysis module (524), a work process relationship DB (526), ​​a metadata DB (528), and a field collected data DB (529).

[0068] The above safety rule processing module (522) manages rules regarding risk situation analysis by the video comprehensive analysis module (524), determines whether the current situation conforms to the rules, and manages analysis rules in lower-level control devices (400a~400n). Specifically, the safety rule processing module (522) analyzes the relationships between safety-related information based on the relationships between safety-related information stored in the work process relationship DB (526) and the neural network parameters stored in the metadata DB (528) to predict the associated risk level. When the associated risk level between safety-related information is high, that is, when work with a risk association is performed simultaneously, the safety rule processing module (522) transmits the risk information to the risk notification module (536).

[0069] The comprehensive image analysis module (524) is equipped with a process classifier capable of classifying processes from photos through learning. The process classifier undergoes learning to distinguish processes when a specific still image is given during the initial system setup process, thereby acquiring the ability to classify or refine characteristic still image information for each process. By refining safety-related information generated from lower-level control devices (400a to 400n), the comprehensive image analysis module (524) can reduce the burden of related risk analysis by the safety rule processing module (522).

[0070] The work process relationship DB (526) stores the relationships between all work processes at the work site and the relationships between safety-related information generated by lower-level control devices (400a to 400n).

[0071] The metadata DB (528) stores image analysis neural network parameter information determined based on the learning results after the system learns about the risk level based on information such as dangerous postures, tool placement, and surrounding environment through images for each work process.

[0072] The field collection data DB (529) stores safety-related information received from lower-level control devices (400a to 400n).

[0073] The other configurations of the lower-level control devices (400a~400n) and the upper-level control device (500) are similar to the configurations of the control devices in the other embodiments described above, so a detailed description thereof is omitted.

[0074] In this embodiment, when the lower-level control devices (400a to 400n) transmit video data, they transmit only the analyzed safety-related information to the upper-level control device (500) instead of the entire video data, so the amount of data transmitted can be significantly reduced and the bandwidth used by the network is also greatly reduced. Accordingly, it is possible to simultaneously analyze video information acquired by multiple cameras while reducing the network load of the field network.

[0075] Hereinafter, the operation of the work site safety control system illustrated in FIG. 4 will be explained with reference to FIG. 5 to FIG. 9.

[0076] Figure 5 is a flowchart showing the overall operation process of the work site safety control system illustrated in Figure 4.

[0077] First, initial setup is performed on the work site safety control system (Step 600). Figure 6 is a flowchart showing the initial setup process in more detail.

[0078] In industrial settings, a wide variety of tasks are performed. When defining a posture that differs from the standard work position as a dangerous one, it is advisable to differentiate between work processes that frequently involve squatting, such as welding, and tasks like standing bricklaying, as their basic postures are completely different; thus, it is desirable to assess whether a posture constitutes a dangerous situation differently. In other words, it is difficult and practically impossible to define dangerous postures using the same standards for all work processes.

[0079] Accordingly, in order to set different criteria for determining the presence of risk for each work process, the work site safety control system according to one embodiment of the present invention first subdivides the work process (Step 602). Subsequently, for each work process, learning regarding the risk level can be performed based on information such as dangerous postures, tool placement, and surrounding environment through video (Step 604). Then, based on the learning results, video analysis neural network parameter information is determined, and the parameter information is stored in the metadata DB (528) of the upper-level control device (500) (Step 606). At this time, learning for the process classifier of the video comprehensive analysis module (524) can also be performed. Accordingly, the process classifier of the video comprehensive analysis module (524) acquires the ability to distinguish the process when a specific still image is given (Step 608).

[0080] Referring again to FIG. 5, after the initial setup of step 600 is executed, device registration for cameras (10a to 10c) and sensors (20) is performed (step 610). Device registration is performed sequentially for all devices such as cameras (10a to 10c) and sensors (20), and the registration for cameras (10a to 10c) includes the process of receiving the first image of the corresponding camera.

[0081] The device registration process will be explained in detail with reference to Fig. 7.

[0082] For example, registration for the first camera (10a) can be performed as follows. First, the first camera (10a) transmits camera attribute information and several still images to lower-level control devices (400a to 400n), and this information is stored in a metadata DB (428) (step 612). In addition, the camera attribute information and still images are transmitted to the image comprehensive analysis module (524) of the upper-level control device (500) (step 614). Based on the received still images, the image comprehensive analysis module (524) identifies or classifies which work process the first camera (10a) is a device for capturing (step 616), queries the metadata DB (528) using the classification information (step 618), and transmits process information for the corresponding work process and neural network parameters for image analysis of the corresponding process to the lower-level control device (400a) (steps 620 and 622). Process information and neural network parameters received from the lower-level control device (400a) are stored in the work process DB (426) and metadata DB (428), respectively, so that the process-specific image analysis module (424) can utilize them. Accordingly, each process-specific image analysis module (424), which can be separately provided for each process or each camera (10a~10c) in the lower-level control device (400a), is specialized for a specific process or specific camera (10a~10c) and can effectively perform image analysis regarding safety status and dangerous situations through accurate image analysis.

[0083] Subsequently, through steps 632 to 642, the registration of the second camera (10b) can be carried out in the same manner as the registration of the first camera (10a). At this time, the neural network parameters transmitted from the upper-level control device (500) are utilized by a process-specific image analysis module (424) separate from the process-specific image analysis module (424) responsible for the first camera (10a). In this way, the process-specific image analysis module (424) can be provided separately for each process or each camera (10a to 10c), and optimal image analysis becomes possible for each process or camera (10a to 10c).

[0084] A registration process can also be performed for the sensor (20). When the registration of the sensor (20) via IoT communication is completed, attribute information of the sensor (20) is stored in the metadata DB (428) (step 644). In this state, the sensor (20) can collect field data and transmit it to the field collected data DB (429).

[0085] In FIG. 7, it is shown that each piece of information passes through the device connection service (510) and the external connection API (434) to improve readability, but in reality, each piece of information is transmitted through the connection service (510) according to communication protocols such as WiFi, 5G / LTE, and Ethernet, and the connection to the upper-level control device (500) is transmitted through the field network or public network via the external connection API (434).

[0086] Meanwhile, although it has been described above that device registration (Step 610) is executed after the completion of initial setup (Step 600), which includes work process subdivision (Step 602), risk learning by work process (Step 604), determination of image analysis neural network parameters based on learning results (Step 606), and learning of the process classifier (Step 608), the present invention is not limited thereto. That is, device registration may be executed first followed by initial setup, device registration may be executed in parallel with the initial setup process, or device registration may be additionally executed after device registration following the initial setup.

[0087] Referring again to FIG. 5, after the device registration of step 610 is executed, safety control is performed based on camera images and sensing data from the IoT sensor (20) that is measured periodically, and monitoring is performed to see if an event that may be related to worker safety occurs (step 650). If an event that may be related to worker safety occurs (step 660), the lower-level control device (400a~400n) analyzes the event to determine if a dangerous situation exists (step 670). If it is determined in step 680 that there is no dangerous situation, the process returns to step 650 to monitor for the occurrence of events; however, if it is determined in step 680 that there is a dangerous situation, a danger alert is issued to the worker (step 690). Then, safety-related information is transmitted to the upper-level control device (500) (step 700). Subsequently, the upper-level control device (500) synthesizes multiple safety-related information to analyze the associated risk level and, if necessary, issues a danger alert to the worker and / or safety manager (step 710).

[0088] FIG. 8 shows the safety control process of steps 650 to 700 executed in lower-level control devices (400a to 400n) in more detail.

[0089] During the execution of safety control, the video output from the camera (10a) passes through the field-collected data DB (429), undergoes analysis by the process-specific video analysis module (424), and is then transmitted to the safety rule processing module (422) (steps 750–754). The safety rule processing module (422) queries the work process DB (426) and the metadata DB (428) to determine whether a dangerous situation exists. That is, the safety rule processing module (422) checks process information regarding the process where the camera (10a) is installed in the work process DB (426) (step 756), checks information regarding process risk factors in the process in the metadata DB (428) (step 758), and then determines whether a dangerous situation exists in the video by determining whether the video conforms to a predetermined rule based on the process information and the risk factor information within the process (step 760). In one embodiment, the safety rule processing module (422) operates in a rule-based manner. However, the present invention is not limited thereto, and the safety rule processing module (422) may also be learned, and through such learning, the risk may be determined.

[0090] When it is determined that a dangerous situation exists, that is, that a dangerous situation has occurred, the safety rule processing module (422) transmits the danger information to the danger notification module (436) (step 762). Upon receiving the danger information, the danger notification module (436) immediately notifies the field worker of the danger (step 764) and also transmits safety-related information to the upper-level control device (500) to report it (step 766). At this time, the danger notification can also be transmitted to the safety manager at the same time.

[0091] FIG. 9 shows the safety control process of step 710, executed in the upper-level control device (500), in more detail.

[0092] Safety-related information transmitted from lower-level control devices (400a~400n) is stored in the field-collected data DB (529). The safety-related information stored in the field-collected data DB (529) is read and refined by the video comprehensive analysis module (524) (steps 780 and 782), and then transmitted to the safety rule comprehensive module (522) (step 784). The safety rule comprehensive module (522)

[0093] The exact work process related to each safety-related information is identified by querying the metadata DB (528) (Step 786). Then, the safety rule comprehensive module (522) identifies the relationships between the processes by querying the work process relationship DB (526) (Step 788). Subsequently, the safety rule comprehensive module (522) analyzes the associated risk level by utilizing the relationships between the work processes and the work process information for each process to determine whether work with risk-related relationships is being performed simultaneously in adjacent spaces (Step 790).

[0094] If, through such comprehensive analysis, it is determined that the associated risk among safety-related information is high, the safety rule comprehensive module (522) transmits the risk information to the risk notification module (536) (step 792). Upon receiving the risk information, the risk notification module (536) immediately notifies the on-site worker of the risk (step 794) and also notifies the safety manager of the situation (step 796). For example, if a process using flammable materials such as thinner and welding work that may cause sparks to fly are being carried out simultaneously within a specific distance, the safety rule comprehensive module (522) determines the risk level to be the highest and immediately stops the work through a notification.

[0095] FIG. 10 is a block diagram of lower-level control devices (400a to 400n) and upper-level control device (500).

[0096] Referring to FIG. 10, control devices (400a to 400n, 500) according to one embodiment of the present invention may include at least one processor (1020), memory (1040), and storage device (1060).

[0097] The processor (1020) can execute program instructions stored in memory (1040) and / or storage device (1060). The processor (1020) may be implemented by at least one central processing unit (CPU) or graphics processing unit (GPU), and may also be implemented by other processors capable of performing the workplace safety control method according to the present invention.

[0098] The memory (1040) may include volatile memory such as ROM (Read Only Memory) and non-volatile memory such as RAM (Random Access Memory). The memory (1040) may load program instructions stored in the storage device (1060) and provide them to the processor (1020).

[0099] The storage device (1060) is a recording medium suitable for storing program instructions and data, and may include, for example, magnetic media such as a hard disk, floppy disk and magnetic tape, optical media such as a CD-ROM (Compact Disk Read Only Memory) and DVD (Digital Video Disk), magneto-optical media such as a floptical disk, flash memory or EPROM (Erasable Programmable ROM), or semiconductor memory such as SSE made based thereon.

[0100] The storage device (1060) stores the program command. For example, in the case of the lower-level control devices (400a to 400n), the program command includes program commands necessary to implement the device connection service unit (410), the core service unit (420), and the management service unit (430) shown in FIG. 4. Meanwhile, for example, in the case of the upper-level control device (500), the program command includes program commands necessary to implement the device connection service unit (510), the core service unit (520), and the management service unit (530) shown in FIG. 4. Such program commands can be executed by the processor (1020) while loaded into memory (1040) under the control of the processor (1020) to implement the work site safety control method according to the present invention.

[0101] Each lower-level control device (400a to 400n) may be implemented by a processor with specifications identical or similar to those of the upper-level control device (500). However, considering that the data processing load of each lower-level control device (400a to 400n) is lower than that of the upper-level control device (500), each lower-level control device (400a to 400n) may be implemented by a processor with specifications lower than those of the upper-level control device (500).

[0102] Meanwhile, the functional configuration of the lower-level control devices (400a to 400n) and the upper-level control device (500) illustrated in FIG. 4 enumerates the functions of these devices, and the modules, service units, and DBs do not necessarily have to be implemented in the units of the enumerated names. That is, multiple modules or services may be integrated or grouped differently from each other. The same applies to the DBs.

[0103] As mentioned above, the operation according to the embodiment of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. Additionally, the computer-readable recording medium may be distributed across networked computer systems, allowing computer-readable programs or code to be stored and executed in a distributed manner.

[0104] The above computer-readable recording medium may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Program instructions may include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0105] Some aspects of the invention have been described in the context of a device, but may also be described according to a corresponding method, wherein a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method may also be described according to a corresponding block or item or a feature of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware device, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such a device.

[0106] In the embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In the embodiments, the field-programmable gate array may operate with a microprocessor to perform one of the methods described herein. Generally, it is preferable that the methods be performed by some hardware device.

[0107] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims.

Claims

Claim 1 A workplace safety control device that determines the possibility of risk by synthesizing safety-related information generated by a lower-level control device among a plurality of hierarchically connected control devices based on images of the workplace acquired by a plurality of cameras, comprising a processor and a memory that stores program instructions executed by the processor, wherein the program instructions include instructions that, when executed by the processor: predict an associated risk level by linking a plurality of safety-related information based on relationship information between each process and synthesizing and analyzing whether work with a risk-related relationship is being performed simultaneously in an adjacent space; and generate a risk alert when the associated risk level is high. Claim 2 A workplace safety control device according to claim 1, further comprising a storage device for storing work process information for each process and information regarding the relationship between work processes; and performing an operation to predict the associated risk based on the work process information for each process and the information regarding the relationship between work processes. Claim 3 A workplace safety control device according to claim 2, wherein the program instructions, when executed by the processor, further include instructions for performing the operation of: receiving a still image acquired by each camera from the lower-level control device, classifying a process based on the still image, generating process type information, storing it in the storage device, and providing it to the lower-level control device. Claim 4 A workplace safety control device according to claim 3, wherein program instructions that receive the still image and perform the operation of distinguishing processes based on the still image are executed during the device registration process when a new camera is connected to the lower-level control device. Claim 5 A workplace safety control device according to claim 4, further comprising instructions that, when the program instructions are executed by the processor: perform the operation of learning about the risk level through images for each work process and determining neural network parameters for image analysis based on the learning results. Claim 6 A workplace safety control device according to claim 5, wherein the program instructions further include instructions that perform the operation of learning the ability to distinguish a process from an input still image when executed by the processor. Claim 7 A workplace safety control device according to claim 6, further comprising instructions that, when the program instructions are executed by the processor: an operation of receiving camera attribute information for at least one camera among the plurality of cameras and one or more still images acquired by the at least one camera; and an operation of determining process information for a process captured by the at least one camera and the neural network parameters for the said process based on the camera attribute information and the one or more still images and transmitting them to the lower-level control device so as to enable the work site images to be utilized for refinement. Claim 8 A workplace safety control device according to claim 1, wherein the safety-related information comprises information on a dangerous situation at a workplace, object information on an object found in an image to be analyzed, and a still image extracted from the image to be analyzed. Claim 9 A work site safety control device according to claim 1, wherein the work site safety control device is connected to a plurality of lower-level control devices, each of which is connected to one or more cameras. Claim 10 A workplace safety control device according to claim 1, wherein the program instructions, when executed by the processor: an operation of analyzing whether a dangerous situation exists at a workplace by determining whether there is an image among the workplace images that does not conform to a predetermined rule based on process information and process-specific risk factor information; and an operation of transmitting the analysis result as safety-related information to a higher-level control device. Claim 11 In claim 10, the work site safety control device further comprises a storage device that receives and stores the process information, the process-specific risk factor information, and neural network parameters from the upper-level control device, and the determination of whether there is an image that does not conform to the rule is performed based on the process information and the process-specific risk factor information. Claim 12 In claim 10, the above program instructions further include instructions that perform an operation of refining the images acquired by the plurality of cameras according to the work process and risk level when executed by the processor, and the judgment on whether there are images that do not conform to the rule is performed using the refined images, a work site safety control device. Claim 13 In Clause 10, the program commands for performing the operation of analyzing whether a dangerous situation exists at the above-mentioned work site are provided separately for each process, and each is a work site safety control device that operates specialized for a specific process. Claim 14 A workplace safety control device according to claim 10, wherein the safety-related information comprises information on the dangerous situation of the workplace, object information on an object found in an image to be analyzed, and a still image extracted from the image to be analyzed. Claim 15 A workplace safety control system comprising: a plurality of cameras, each acquiring images of a workplace; and a plurality of control devices, each connected hierarchically and including a lower-level control device and an upper-level control device, which detect the dangerous state of a worker at the workplace by dividing roles based on images acquired by the plurality of cameras; wherein the system comprises: (a) a step in which the lower-level control device determines whether there is an image among the images acquired by the plurality of cameras that does not conform to a predetermined rule, analyzes whether a dangerous situation exists, and transmits the analysis result as safety-related information to the upper-level control device; (b) a step in which the upper-level control device predicts the associated risk level by linking a plurality of safety-related information and comprehensively analyzing whether work with a risk-related relationship is performed simultaneously in an adjacent space; and (c) a step in which the upper-level control device generates a risk alert when the associated risk level is high. Claim 16 A method for workplace safety control according to claim 15, wherein step (a) comprises the step of refining the images acquired by the plurality of cameras according to the work process and risk level; and determining whether there is an image among the refined images that does not conform to the rule when analyzing whether the dangerous situation exists. Claim 17 A method for workplace safety control according to claim 15, further comprising the step of being performed no later than before step (b) is performed, wherein the upper-level control device performs learning on risk levels through images for each work process and determines neural network parameters for image analysis based on the learning results. Claim 18 A workplace safety control method according to claim 17, further comprising the step of the upper-level control device learning the ability to distinguish processes from input still images. Claim 19 A method for workplace safety control according to claim 18, further comprising: a step in which the upper-level control device receives camera attribute information and one or more still images from at least one of the plurality of cameras through the lower-level control device; and a step of determining process information for a process captured by the at least one camera and neural network parameters for the said process based on the camera attribute information and the one or more still images and transmitting them to the lower-level control device so as to enable them to be utilized for refinement.

Images