System and method for supporting security management service by using AI vision and Internet of Things

Abstract

The present invention provides a system for supporting a security management service using AI vision and Internet of Things, comprising: a plurality of vision cameras for capturing an image of a security target area to which an ID is assigned; a plurality of Internet of Things devices located within the safe target area and sensing or detecting any of the following items: dynamic objects, static objects, temperature / humidity, gas, cracks, heat, vibration, dust, and electric waves; and the AI safety diagnosis management server is used for analyzing the risk degree of the safety target area based on the images shot by the plurality of visual cameras and the sensing information sensed by the plurality of Internet of Things devices, and providing a dangerous situation coping scheme conforming to the risk degree for a worker terminal located in or close to the safety target area.

Description

Technical Field

[0001] The present invention relates to a system and method for supporting security management services by utilizing AI vision and the Internet of Things. Background Art

[0002] Generally speaking, there are many dangerous factors in industries, factories, sites, workplaces, etc., and many disasters, incidents, accidents, etc. occur in manufacturing, production, construction, construction, and work. In particular, the manufacturing industry, the building industry, and the construction industry are labor-intensive industries, with many workers involved and multiple tasks performed at the same time. Therefore, it is crucial to carry out systematic and effective safety management of personnel and equipment. At industrial sites, accidents and disasters frequently occur due to the inherent dangers of various environmental factors inherent in the site itself and the negligence of workers.

[0003] South Korea has implemented the Industrial Safety and Health Act since 1981 to reduce industrial disasters, and the Major Disaster Penalty Act since 2022 to prevent major disasters. In particular, with the implementation of the Major Disaster Act, industrial systems, construction industries, etc. must take more rigorous measures to prevent accidents. In particular, in the case of the construction industry, the accident rate is higher than other industries, so it is necessary to further strengthen measures against accidents and disasters. Generally speaking, there are fewer administrators in charge of management work compared to on-site workers, which leads to poor control and management by administrators or the existence of unmanaged areas.

[0004] Therefore, there is a problem that workers who perform the work approach the site for a dangerous factor or perform the work carelessly.

[0005] On some construction sites, incidents, accidents, disasters, gases, pollution, falls, etc. may occur due to various facilities, structures, noise, vibration, dust, radio interference, or difficulties in location tracking, wired / wireless communication, GPS, etc. In addition, there are many hazards caused by accidents or situations that occur to workers and cannot be quickly transmitted to supervisors.

[0006] In addition, due to safety issues, accidents, disasters, industrial development, mechanical development, population decline, worker reduction, manpower shortage, craftsman shortage, expert shortage, shirking 3D work, shirking hard work, reducing construction costs, poor working environment, five-day work week, shortened working hours and other reasons, in work, business, affairs, life, operations, manufacturing, production, construction, and construction, it is necessary to achieve unmanned, mechanized, robotized, safe, systematized, automatic control, remote control, and period animation.

[0007] Prior art literature

[0008] Patent Literature

[0009] Patent Document 1: Korean Patent Publication No. 10-2023-0129146 Summary of the invention

[0010] Issues to be solved

[0011] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a system and method for supporting security management services using AI vision and the Internet of Things.

[0012] Problem Solution

[0013] A system for supporting safety management services using AI vision and the Internet of Things according to an embodiment of the present invention for solving the above-mentioned problem comprises: a plurality of visual cameras for photographing a safety target area assigned an ID; a plurality of Internet of Things devices located within the safety target area and sensing or detecting any of the following: dynamic objects, static objects, temperature / humidity, gas, cracks, heat, vibration, dust, and radio waves; an AI safety diagnosis management server for analyzing the degree of danger of the safety target area based on the images photographed by the plurality of visual cameras and the perception information sensed by the plurality of Internet of Things devices, and providing a dangerous situation response plan that meets the degree of danger to a worker terminal located in or near the safety target area; wherein the AI ​​safety diagnosis management server determines the type of the safety target area through the above-mentioned ID, selects the type of visual cameras and Internet of Things devices located in the safety target area to judge the dangerous situation of the safety target area, and then receives data from the selected visual cameras and Internet of Things devices.

[0014] A method for supporting safety management services using AI vision and the Internet of Things according to an embodiment of the present invention for solving the above-mentioned problem comprises: in an AI safety diagnosis management server, collecting visual images from a plurality of visual cameras that capture a safety target area assigned an ID; the AI ​​safety diagnosis management server collecting measurement data from a plurality of Internet of Things devices, wherein the plurality of Internet of Things devices are located in the safety target area and sense or detect any of the following: dynamic objects, static objects, temperature / humidity, gas, cracks, heat, vibration, dust, and radio waves; in the AI ​​safety diagnosis management server, analyzing or predicting the degree of danger and the occurrence of dangerous situations in the safety target area based on the images captured by the plurality of visual cameras and the sensed information sensed by the plurality of Internet of Things devices; and in the AI ​​safety diagnosis management server, providing a dangerous situation response plan that meets the degree of danger to a worker (enterprise) terminal located in or near the safety target area; wherein the AI ​​safety diagnosis management server determines the type of the safety target area through the above-mentioned ID, selects the type of visual cameras and Internet of Things devices located in the safety target area to determine the dangerous situation of the safety target area, and then receives data from the selected visual cameras and Internet of Things devices.

[0015] Effects of the Invention

[0016] Therefore, a safety management system and method using artificial intelligence technology according to an embodiment of the present invention can be installed in construction sites, logistics warehouses, factories, airports, subway stations, railway stations, tunnels, buildings, bridges, railways and other areas that require safety management to predict workers' work hazards in advance and respond quickly to dangerous situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A network structure diagram of a system using AI vision and the Internet of Things to support security management services according to an embodiment of the present invention;

[0018] Figure 2 for Figure 1 A detailed structural diagram of the visual camera shown;

[0019] Figure 3 for Figure 1 The detailed structure diagram of the AI ​​security diagnosis management server shown;

[0020] Figure 4 is a flowchart of a method for supporting security management services using AI vision and the Internet of Things according to an embodiment of the present invention;

[0021] Figure 5 The present invention is a schematic diagram of an exemplary computing environment in which one or more embodiments disclosed in this specification may be implemented.

[0022] * Reference Numbers *

[0023] 100: A system that uses AI vision and IoT to support security management services

[0024] 110: Visual Camera Department

[0025] 120: IoT devices

[0026] 130: AI security diagnosis management server DETAILED DESCRIPTION

[0027] Below, various embodiments and / or aspects will be described in conjunction with the accompanying drawings. In the following description, for ease of explanation, some specific details are disclosed to provide an overall understanding of one or more aspects. However, it will be appreciated by those skilled in the art that various aspects of the present invention may be implemented without these specific details. The following description and the accompanying drawings describe in detail certain exemplary aspects of one or more aspects. However, these aspects are exemplary, and some different methods may be used to practice the principles of various aspects, and the purpose of the description is to include all of these aspects and their equivalents.

[0028] Furthermore, various aspects and functionalities will be presented by systems that may include multiple devices, components and / or modules, etc. It should also be understood and appreciated that the various systems may include additional devices, components and / or modules, and / or may not include all of the devices, components, modules, etc. discussed in connection with the figures.

[0029] The terms "embodiment", "example", "aspect", "example", etc. used in this specification shall not be interpreted as any aspect or design described is better or more advantageous than any other aspect or design. The terms "component", "module", "system", "interface", etc. used below generally refer to computer-related entities, such as hardware, hardware and software combinations, and software.

[0030] At the same time, the term "or" means an implied "or" rather than an exclusive "or". That is, unless otherwise specified or the context clearly indicates, "X uses A or B" means one of the natural implicit substitutions. That is, when X uses A; or X uses B; or X uses both A and B, "X uses A or B" may apply to any of the cases. In addition, the term "and / or" used in this specification should be understood to refer to and include all possible combinations of one or more of the listed related items.

[0031] In addition, the terms "include" and / or "including..." should be understood as meaning that although the feature and / or component is present, it does not exclude the presence or addition of one or more other features, components and / or their combinations. In addition, unless otherwise specified or it is obvious from the context that it is in the singular form, the singular in this specification and claims should be understood as "one or more". In addition, the terms "client", "user" and "user" used in this specification are generally used interchangeably. In addition, the terms "component" and "element" used in this specification are generally used interchangeably.

[0032] Below, a security management system and method using artificial intelligence technology according to an embodiment of the present invention will be described in detail based on the accompanying drawings.

[0033] Figure 1 This is a network structure diagram of a system using AI vision and the Internet of Things to support security management services according to an embodiment of the present invention. Figure 2 for Figure 1 The detailed structure diagram of the visual camera is shown in Figure 3 for Figure 1 The detailed structure diagram of the AI ​​security diagnosis management server is shown.

[0034] First, if Figure 1As shown, a safety management system 100 using visual artificial intelligence technology according to an embodiment of the present invention is installed in construction sites, logistics warehouses, factories, airports, subway stations, railway stations, tunnels, buildings, bridges, railways and other areas that require safety management. It is used to detect whether on-site workers are wearing safety equipment, predict dangers and respond to accidents through behavioral analysis of workers, detect and manage abnormal phenomena, cracks and defects in facilities such as bridges, railways, buildings, tunnels, etc., determine whether workers enter dangerous areas, detect intruders outside working hours, monitor on-site fires and gas leaks, and determine whether there are dangerous situations caused by proximity to mobile equipment such as work vehicles and forklifts, thereby monitoring the safety of workers.

[0035] Specifically, the security management system 100 using visual artificial intelligence technology includes an IP network camera unit 110 , an IoT sensor unit 120 , and an AI security diagnosis management server 130 .

[0036] Each configuration communicates through a network, wherein the network refers to a connection structure that can exchange information between nodes such as multiple terminals and servers. Examples of the above-mentioned network include RF, 3GPP (3rd Generation Partnership Project) network, LTE (Long Term Evolution) network, 5GPP (5th Generation Partnership Project) network, WIMAX (World Interoperability for Microwave Access) network, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth network, NFC network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc., but not limited to.

[0037] The IP network camera unit 110 may be configured to detect and photograph dynamic objects moving in a preset indoor space or outdoor space using a radar.

[0038] The IP network camera unit 110 includes a CCD camera module 111 , a radar sensor module 112 , a data processing module 113 , a CAN communication module 114 and an environment control unit 115 .

[0039] The CCD camera module 111 may be configured to capture an indoor space or an outdoor space.

[0040] The radar sensor module 112 may be configured to detect a dynamic object moving within the indoor space or the outdoor space or a change of a static object using a radar.

[0041] The data processing module 113 can extract dynamic objects or static objects from the image captured by the CCD camera module 111, match the dynamic objects or static objects detected by the radar sensor module 220 with the dynamic objects or static objects, and assign object IDs to the dynamic objects or static objects.

[0042] The CAN communication module 113 can transmit the output information of the CCD camera module 111 , the radar sensor module 112 and the data processing module 113 , and receive the configuration of dynamic environmental parameters from the AI ​​safety diagnosis management server 130 .

[0043] The environment control unit 114 may be configured to control the operation of the camera module and the radar sensor module based on the dynamic environment parameters.

[0044] Next, the AI ​​safety diagnosis management server 130 may be configured to monitor the safety of the target object based on the image captured by the IP network camera unit 200 and the information measured by the IoT sensor unit 300.

[0045] Specifically, the AI ​​safety diagnosis management server 130 includes a safety management object registration unit 131, a visual camera management unit 132, an IoT device management unit 133, a data classification / analysis / learning unit 134, a dangerous situation prediction and analysis unit 135 and a safety solution providing unit 136.

[0046] The security management object registration unit 131 may configure a database for the type, area, and location information of the security management object.

[0047] The security management object registration unit 131 may be configured to register the IDs of the visual cameras and IoT devices used to monitor the security management objects when registering the information of the security management objects.

[0048] The safety management object registration part 131 may be a configuration for setting and registering a dangerous situation factor according to a type of a safety management object.

[0049] Among them, safety management objects can be all objects where safety management or dangerous situations may occur, such as buildings, workers, railway tracks, fire sensors, bridges, tunnels, and crossings.

[0050] Therefore, the safety management object registration unit 210 is designed to be required to register an IoT device that provides data and / or images that can identify or determine the dangerous status of the safety management object.

[0051] If the safety management object is the first construction area (location) or a worker stumbles / falls / collapses (type), the ID information of the visual camera that captures the construction area will be registered as the safety management object during registration. This will allow the work image captured by the visual camera to read and predict whether the worker is wearing a safety helmet, whether a stumble / falls / collapses, etc., so as to provide information that can respond to dangerous situations.

[0052] That is, the security management object registration unit 131 sets and registers the required IoT device type according to the type of the security management object.

[0053] The visual camera management unit 132 may manage the remote operation status and financial resource information configuration of the visual cameras registered in the security management object registration unit 210 .

[0054] The IoT management unit 133 may be configured to manage the remote operation status and financial resource information of the IoT devices registered in the security management object registration unit 131 .

[0055] Among them, the IoT device is an electronic device related to the actions to be described later, performing sensing, measuring, measuring, observing, observing, detecting, identifying, monitoring, remote sensing, wireless sensing, unmanned sensing, automatic sensing, remote sensing, etc. In addition, the sensor can be an instrument for measuring, sensing, measuring, observing, observing, detecting, detecting, and identifying the following factors: a) position, altitude, height, latitude, longitude, depth, deep, coordinates, GPS, air pressure, tilt, activity, action, phenomenon, movement, movement, action, movement, fall, fall, trend, fall, body, biology, health, hygiene, gas, pollution, oil, oils, environment, cleaning, purification, sterilization, disinfection, aeration, reaction, agglomeration, precipitation, filtration, concentration, pollution concentration, gas concentration, pH value, ORP (oxidation-reduction potential), redox potential, pollution, light, heat, light, wave, short wave, long wave, fluctuation, magnetism, electricity, power supply, voltage, current, magnetic field, sound, signal, sound, sound wave, image, shooting, recording, transmission, broadcasting, broadcasting, receiving, storage, recording, scanning, input, output, unmanned, manned, luminous, light source, UV, ultraviolet, infrared, far infrared, component, taste, smell, taste, odor, thermal imaging, ultrasound, water quality, air, air quality, harmfulness, dangerousness, meteorology, climate, atmosphere, wind speed, wind direction, water, flow, water volume, water leakage, oil leakage, liquid leakage, waterproof, waterproof, pipe, pipeline, pipeline, water supply pipe, pipe leakage, membrane leakage, pipeline leakage, water-proof membrane leakage, water pipe leakage, temperature, humidity, body temperature, water temperature, water surface, underwater, water level, conductivity, electrical resistivity, pollution concentration, shock, vibration, earthquake, noise, fall, force, weight, weight, load, stress, pressure, compression, strength, rigidity, stretch, pull out, extraction, heat, volume, length, area, deformation, displacement, tilt, inclination, slope, facility inclination, building inclination, function, performance, substance, material, materials, object, thing, everything, living thing, approach, human approach, animal approach, fatigue, destruction, breakage, damage, destruction, damage, destruction, non-destruction, fire, flame, disaster, power outage, air conditioning, cooling, heating, speed, acceleration, three-axis acceleration, gyroscope, brain wave, EEG (electroencephalogram), body temperature, breathing, pulse, blood pressure, heart rate, heartbeat, heart disease, electrocardiogram (electrocardiogram,ECG), tracking, equipment, device, mechanism, person, human, robot, animal, operator, worker, entrant, work, business, operation, manufacturing, production, construction, building, construction, blasting, drilling, digging, excavation, excavation, storage, mixing, pressurization, suction, injection, pouring, injection, laying, spreading, compaction, solidification, demolition, screening, separation, crushing, pulverization, separation, sensing, measurement, observation, determination, measurement, testing, analysis, ground, underground, underground, upper, soil land, topography, facilities, equipment, buildings, tunnels, roads, bridges, retaining walls, subways, structures, anchors, bolts, rock bolts, nails, steel bars, concrete, reinforcing materials, quantity, number, capacity, business volume, workload, construction volume, earthwork volume, global navigation satellite system, gas leakage, lack of oxygen, worker collapse, helmet, vest, wearable tool, whether a hard hat is worn or not, whether a wearable tool is worn or not; b) the quantity or degree of one of the above positions or whether a wearable tool is worn or not; c) a combination of two or more of the above positions or quantities.

[0056] In addition, the IoT device may include various sensors, which may be sensors, elements, lenses, beacons, kits, wires, measuring instruments, measuring devices, sensor phones, mobile phone sensors, cables, position sensors, altitude sensors, latitude sensors, longitude sensors, air pressure sensors, motion sensors, biometric sensors, health sensors, human sensors, gas sensors, environmental sensors, kit sensors, air quality sensors, electrical sensors, electronic sensors, communication sensors, wireless sensors, wired sensors, electrical sensors, measuring sensors, metering sensors, optical sensors, wire sensors, cable sensors, portable sensors, wireless measurement sensors, wired measurement sensors, chips, sensor chips, sensor chips, position chips, motion chips, biometric chips, gas chips, environmental chips, air quality chips, sensor chips, position sensor chips, ID chips, GPS chips, identification chips, motion sensor chips, biometric identification chips, gas sensor chips, environmental sensor chips, air quality sensor chips, codes, barcodes, ID codes, tags, tags, ID tags, identification tags, trackers, tracker sensors, mobile phones, mobile phone sensors, sensors, multiple sensor, gauge, strain gauge, strain gauge, ID sensor, RFID sensor, NFC sensor, MEMS sensor, IR sensor, FOD sensor, PIR sensor, UWB sensor, IMU, CMOS sensor, IoT sensor, ICT sensor, GPS (global positioning system), INS (inertial navigation system), biosensor, multi-sensor, multi-sensor, light sensor, thermal sensor, optical sensor, wave sensor, magnetic sensor, fever sensor, detection sensor, leak detection sensor, vibration sensor, tactile sensor, concentration sensor, luminescence sensor, light source sensor, component sensor, temperature sensor, humidity sensor, pressure sensor, stress sensor, deformation sensor, displacement sensor, sound sensor, signal sensor, flow sensor, magnetic sensor, acoustic sensor, taste sensor, olfactory sensor, device sensor, equipment sensor, sound wave sensor, fiber sensor, optical fiber sensor, infrared sensor, far infrared sensor, thermal imaging sensor, ultrasonic sensor, camera, infrared sensor, near infrared sensor, ultraviolet sensor, digital camera, various sensors, CDS sensor, compass sensor, altimeter sensor, tilt sensor, triaxial tilt sensor, crack sensor, wire sensor, vibrating wire sensor, stress sensor, load sensor, millivoltsensor, displacement sensor, optical displacement sensor and a composite sensor combining two or more of the above.

[0057] The data classification / analysis / learning unit 134 may classify and learn the measurement data of the IoT devices managed by the visual camera management unit 220 and the IoT management unit 230 according to the security management objects (types) registered in the security management object registration unit 210 .

[0058] In addition, the data classification / analysis / learning unit 134 extracts object objects of factors that may cause dangerous situations related to the safety management object (type) from the images captured by the visual camera, and classifies and learns them.

[0059] In addition, when the factor that may cause a dangerous situation is a dynamic object, the data classification / analysis / learning unit 134 may include a configuration for analyzing and learning the behavior pattern and / or posture of the dynamic object.

[0060] In addition, when the factor that may cause a dangerous situation is a static object, the data classification / analysis / learning unit 134 may include a configuration for analyzing and learning the size, shape, color, etc. of the static object.

[0061] For reference, dynamic objects can be industrial equipment, workers, robots, cars, ships, cranes, hoists, etc., while static objects can be configurations that change shape and / or color, such as bolts, nuts, connecting plates, welds, joints, cracks, fissures, colors, etc. on railway lines.

[0062] Furthermore, the data classification and learning section 240 may be configured to track dynamic objects and static objects and classify them as time series information.

[0063] On the other hand, the data classification / analysis / learning unit 240 may include a configuration for detecting the position coordinates (x, y) and size (a, h: height and width) of an object (bounding box) in a visual image using a deep learning algorithm.

[0064] At this time, when detecting moving objects or static objects, after merging and expanding the object detection areas of two different deep learning object detection algorithms to generate a total object detection area, objects located within the comprehensive object detection area can be detected according to preset detection condition values.

[0065] The two different deep learning object detection algorithms may be a first deep learning object detection algorithm (Gaussian YOLO v3) and a second deep learning object detection algorithm (MASK R-CNN), and overlap each object detection area (bounding box) to generate a preset overlap value, preferably a comprehensive object detection area greater than 70%.

[0066] For reference, the first deep learning object detection algorithm (Gaussian YOLO v3) is characterized by fast object detection speed but low recognition accuracy, while the second deep learning object detection algorithm (MASK R-CNN) is characterized by slow object detection speed but high recognition accuracy.

[0067] Therefore, in the present invention, the detection bounding boxes of two deep learning object detection algorithms are integrated and expanded to form an integrated object detection area to highlight the advantages of the two deep learning object detection algorithms.

[0068] On the other hand, if the comprehensive object detection area is smaller than the preset overlap value (60%), the second deep learning object detection algorithm (MASK R-CNN) may be used to detect the object (bounding box).

[0069] Furthermore, the data classification / analysis / learning section 134 may be a configuration that provides a comparison value that compares a detected object (static object) and learning data of the object.

[0070] In addition, the data classification / analysis / learning unit 134 may include a configuration for estimating the movement, posture, and condition of a detected object (dynamic object) using a deep learning algorithm.

[0071] In addition, the data classification / analysis / learning unit 134 may include a configuration for analyzing the measurement values ​​of the IoT devices to select and analyze the dangerous situation factors of the safety management object.

[0072] Next, the dangerous situation prediction and analysis unit 135 can be configured to judge or predict the existence and severity of dangerous situations of the safety management object based on the dangerous situation factors or comparison values ​​extracted and analyzed by the data classification / analysis / learning unit 240.

[0073] The risk prediction and analysis unit 135 may be a configuration for predicting the risk level of the safety management object by comparing the risk manual of the safety management object with the risk factor or comparison value.

[0074] The security solution providing unit 136 may be configured to provide the risk level of the security management object and a security inspection guide according to the risk level to a terminal of a company that manages the security management object.

[0075] In addition, according to an embodiment of the present invention, the system 100 using AI vision and the Internet of Things to support security management services may also include an information providing unit 137, and the information providing unit 270 may provide an insurance company that joins the company that manages the security management object with information on the degree of danger of the security management object and the possibility of the occurrence of dangerous situations.

[0076] At this point, the insurance company can change and / or (re)set the insurance premium rates based on the client company's data.

[0077] Figure 4 Flow chart of a method for supporting security management services using AI vision and the Internet of Things according to an embodiment of the present invention.

[0078] See also Figure 4 According to an embodiment of the present invention, a method (S700) for supporting safety management services using AI vision and the Internet of Things includes the following series of processes: collecting visual images from multiple visual cameras that capture a safety target area with an ID (S710), collecting measurement data from multiple Internet of Things devices, wherein the multiple Internet of Things devices are located in the safety target area and sense or detect any of the following: dynamic objects, static objects, temperature / humidity, gas, cracks, heat, vibration, dust, and radio waves (S720), and then in the AI ​​safety diagnosis management server, based on the images captured by the multiple visual cameras and the sensed information sensed by the multiple Internet of Things devices, analyzing or predicting the danger level and occurrence of dangerous situations in the safety target area (S730), and then providing a dangerous situation response plan that meets the danger level to a worker (enterprise) terminal located in or near the safety target area (S740).

[0079] Here, the AI ​​safety diagnosis management server 400 may be configured to monitor the safety of a target object based on the image captured by the IP network camera unit 200 and the information measured by the IoT sensor unit 300 .

[0080] Specifically, the AI ​​safety diagnosis management server 400 includes a safety management object registration unit 210, a visual camera management unit 220, an IoT management unit 230, a data classification and learning unit 240, a dangerous situation prediction and analysis unit 250 and a safety solution providing unit 260.

[0081] The security management object registration unit 210 may configure a database for the type, region, and location information of the security management object.

[0082] The security management object registration unit 210 may be configured to register the IDs of the visual cameras and IoT devices used to monitor the security management objects when registering the information of the security management objects.

[0083] The safety management object registration part 210 may be a configuration for setting and registering a dangerous situation factor according to a type of a safety management object.

[0084] Among them, safety management objects can be all objects where safety management or dangerous situations may occur, such as buildings, workers, railway tracks, fire sensors, bridges, tunnels, and crossings.

[0085] Therefore, the safety management object registration unit 210 is designed to be required to register an IoT device that provides data and / or images that can identify or determine the dangerous status of the safety management object.

[0086] If the safety management object is the first construction area (location) or a worker stumbles / falls / collapses (type), the ID information of the visual camera that captures the construction area will be registered as the safety management object during registration. This will allow the work image captured by the visual camera to read and predict whether the worker is wearing a safety helmet, whether a stumble / falls / collapses, etc., so as to provide information that can respond to dangerous situations.

[0087] That is, the security management object registration unit 210 sets and registers the required IoT device type according to the type of the security management object.

[0088] The visual camera management unit 220 may be configured to manage the remote operation status and financial information of the visual cameras registered in the security management object registration unit 210 .

[0089] The IoT management unit 230 may be configured to manage the remote operation status and financial information of the IoT devices registered in the security management object registration unit 210 .

[0090] Among them, the IoT device is an electronic device related to the actions to be described later, performing sensing, measuring, measuring, observing, observing, detecting, identifying, monitoring, remote sensing, wireless sensing, unmanned sensing, automatic sensing, remote sensing, etc. In addition, the sensor can be an instrument for measuring, sensing, measuring, observing, observing, detecting, detecting, and identifying the following factors: a) position, altitude, height, latitude, longitude, depth, deep, coordinates, GPS, air pressure, tilt, activity, action, phenomenon, movement, movement, action, movement, fall, fall, trend, fall, body, biology, health, hygiene, gas, pollution, oil, oils, environment, cleaning, purification, sterilization, disinfection, aeration, reaction, agglomeration, precipitation, filtration, concentration, pollution concentration, gas concentration, pH value, ORP (oxidation-reduction potential), redox potential, pollution, light, heat, light, wave, short wave, long wave, fluctuation, magnetism, electricity, power supply, voltage, current, magnetic field, sound, signal, sound, sound wave, image, shooting, recording, transmission, broadcasting, broadcasting, receiving, storage, recording, scanning, input, output, unmanned, manned, luminous, light source, UV, ultraviolet, infrared, far infrared, component, taste, smell, taste, odor, thermal imaging, ultrasound, water quality, air, air quality, harmfulness, dangerousness, meteorology, climate, atmosphere, wind speed, wind direction, water, flow, water volume, water leakage, oil leakage, liquid leakage, waterproof, waterproof, pipe, pipeline, pipeline, water supply pipe, pipe leakage, membrane leakage, pipeline leakage, water-proof membrane leakage, water pipe leakage, temperature, humidity, body temperature, water temperature, water surface, underwater, water level, conductivity, electrical resistivity, pollution concentration, shock, vibration, earthquake, noise, fall, force, weight, weight, load, stress, pressure, compression, strength, rigidity, stretch, pull out, extraction, heat, volume, length, area, deformation, displacement, tilt, inclination, slope, facility inclination, building inclination, function, performance, substance, material, materials, object, thing, everything, living thing, approach, human approach, animal approach, fatigue, destruction, breakage, damage, destruction, damage, destruction, non-destruction, fire, flame, disaster, power outage, air conditioning, cooling, heating, speed, acceleration, three-axis acceleration, gyroscope, brain wave, EEG (electroencephalogram), body temperature, breathing, pulse, blood pressure, heart rate, heartbeat, heart disease, electrocardiogram (electrocardiogram,ECG), tracking, equipment, device, mechanism, person, human, robot, animal, operator, worker, entrant, work, business, operation, manufacturing, production, construction, building, construction, blasting, drilling, digging, excavation, excavation, storage, mixing, pressurization, suction, injection, pouring, injection, laying, spreading, compaction, solidification, demolition, screening, separation, crushing, pulverization, separation, sensing, measurement, observation, determination, measurement, testing, analysis, ground, underground, underground, upper, soil land, topography, facilities, equipment, buildings, tunnels, roads, bridges, retaining walls, subways, structures, anchors, bolts, rock bolts, nails, steel bars, concrete, reinforcing materials, quantity, number, capacity, business volume, workload, construction volume, earthwork volume, global navigation satellite system, gas leakage, lack of oxygen, worker collapse, helmet, vest, wearable tool, whether a hard hat is worn or not, whether a wearable tool is worn or not; b) the quantity or degree of one of the above positions or whether a wearable tool is worn or not; c) a combination of two or more of the above positions or quantities.

[0091] In addition, the IoT device may include various sensors, which may be sensors, elements, lenses, beacons, kits, wires, measuring instruments, measuring devices, sensor phones, mobile phone sensors, cables, position sensors, altitude sensors, latitude sensors, longitude sensors, air pressure sensors, motion sensors, biometric sensors, health sensors, human sensors, gas sensors, environmental sensors, kit sensors, air quality sensors, electrical sensors, electronic sensors, communication sensors, wireless sensors, wired sensors, electrical sensors, measuring sensors, metering sensors, optical sensors, wire sensors, cable sensors, portable sensors, wireless measurement sensors, wired measurement sensors, chips, sensor chips, sensor chips, position chips, motion chips, biometric chips, gas chips, environmental chips, air quality chips, sensor chips, position sensor chips, ID chips, GPS chips, identification chips, motion sensor chips, biometric identification chips, gas sensor chips, environmental sensor chips, air quality sensor chips, codes, barcodes, ID codes, tags, tags, ID tags, identification tags, trackers, tracker sensors, mobile phones, mobile phone sensors, sensors, multiple sensor, gauge, strain gauge, strain gauge, ID sensor, RFID sensor, NFC sensor, MEMS sensor, IR sensor, FOD sensor, PIR sensor, UWB sensor, IMU, CMOS sensor, IoT sensor, ICT sensor, GPS (global positioning system), INS (inertial navigation system), biosensor, multi-sensor, multi-sensor, light sensor, thermal sensor, optical sensor, wave sensor, magnetic sensor, fever sensor, detection sensor, leak detection sensor, vibration sensor, tactile sensor, concentration sensor, luminescence sensor, light source sensor, component sensor, temperature sensor, humidity sensor, pressure sensor, stress sensor, deformation sensor, displacement sensor, sound sensor, signal sensor, flow sensor, magnetic sensor, acoustic sensor, taste sensor, olfactory sensor, device sensor, equipment sensor, sound wave sensor, fiber sensor, optical fiber sensor, infrared sensor, far infrared sensor, thermal imaging sensor, ultrasonic sensor, camera, infrared sensor, near infrared sensor, ultraviolet sensor, digital camera, various sensors, CDS sensor, compass sensor, altimeter sensor, tilt sensor, triaxial tilt sensor, crack sensor, wire sensor, vibrating wire sensor, stress sensor, load sensor, millivoltsensor, displacement sensor, optical displacement sensor and a composite sensor combining two or more of the above.

[0092] The data classification / analysis / learning unit 240 may classify and learn the measurement data of the IoT devices managed by the visual camera management unit 220 and the IoT management unit 230 according to the security management objects (types) registered in the security management object registration unit 210 .

[0093] In addition, the data classification / analysis / learning unit 240 extracts and classifies and learns object objects of factors that may cause dangerous situations related to the safety management object (type) from the images captured by the visual camera.

[0094] In addition, when the factor that may cause a dangerous situation is a dynamic object, the data classification / analysis / learning unit 240 may include a configuration for analyzing and learning the behavior pattern and / or posture of the dynamic object.

[0095] In addition, when the factor that may cause a dangerous situation is a static object, the data classification / analysis / learning unit 240 may include a configuration for analyzing and learning the size, shape, color, etc. of the static object.

[0096] For reference, dynamic objects can be industrial equipment, workers, robots, cars, ships, cranes, hoists, etc., while static objects can be configurations that change shape and / or color, such as bolts, nuts, connecting plates, welds, joints, cracks, fissures, colors, etc. on railway lines.

[0097] Furthermore, the data classification and learning section 240 may be configured to track dynamic objects and static objects and classify them as time series information.

[0098] On the other hand, the data classification / analysis / learning unit 240 may include a configuration for detecting the position coordinates (x, y) and size (a, h: height and width) of an object (bounding box) in a visual image using a deep learning algorithm.

[0099] At this time, when detecting moving objects or static objects, after merging and expanding the object detection areas of two different deep learning object detection algorithms to generate a total object detection area, objects located within the comprehensive object detection area can be detected according to preset detection condition values.

[0100] The two different deep learning object detection algorithms may be a first deep learning object detection algorithm (Gaussian YOLO v3) and a second deep learning object detection algorithm (MASK R-CNN), and overlap each object detection area (bounding box) to generate a preset overlap value, preferably a comprehensive object detection area greater than 70%.

[0101] For reference, the first deep learning object detection algorithm (Gaussian YOLO v3) is characterized by fast object detection speed but low recognition accuracy, while the second deep learning object detection algorithm (MASK R-CNN) is characterized by slow object detection speed but high recognition accuracy.

[0102] Therefore, in the present invention, the detection bounding boxes of two deep learning object detection algorithms are integrated and expanded to form an integrated object detection area to highlight the advantages of the two deep learning object detection algorithms.

[0103] On the other hand, if the comprehensive object detection area is smaller than the preset overlap value (60%), the second deep learning object detection algorithm (MASK R-CNN) may be used to detect the object (bounding box).

[0104] Furthermore, the data classification / analysis / learning section 240 may be a configuration that provides a comparison value that compares a detected object (static object) and learning data of the object.

[0105] In addition, the data classification / analysis / learning unit 240 may include a configuration for estimating the movement, posture, and condition of a detected object (dynamic object) using a deep learning algorithm.

[0106] In addition, the data classification / analysis / learning unit 240 may include a configuration for analyzing the measurement values ​​of the IoT devices to select and analyze the dangerous situation factors of the safety management object.

[0107] Next, the dangerous situation prediction and analysis unit 250 can be configured to judge or predict the existence and severity of dangerous situations of the safety management object based on the dangerous situation factors or comparison values ​​extracted and analyzed by the data classification / analysis / learning unit 240.

[0108] The risk prediction and analysis unit 250 may be a configuration for predicting the risk level of the safety management object by comparing the risk manual of the safety management object with the risk factor or comparison value.

[0109] The security solution providing unit 260 may be configured to provide the risk level of the security management object and the security inspection guide according to the risk level to the terminal of the company that manages the security management object.

[0110] In addition, according to an embodiment of the present invention, the system 100 using AI vision and the Internet of Things to support security management services may also include an information providing unit 270, which can provide information on the degree of danger of the security management object and the possibility of the occurrence of dangerous situations to the insurance company that joins the company that manages the security management object.

[0111] At this point, the insurance company can change and / or (re)set the insurance premium rates based on the client company's data.

[0112] Therefore, a safety management system and method using artificial intelligence technology according to an embodiment of the present invention can be installed in construction sites, logistics warehouses, factories, airports, subway stations, railway stations, tunnels, buildings, bridges, railways and other areas that require safety management to predict workers' work hazards in advance and respond quickly to dangerous situations.

[0113] Figure 5 The schematic diagram of an exemplary computing environment in which one or more embodiments disclosed in this specification may be implemented shows an example of a system 1000 including a computing device 1100 configured to implement the one or more embodiments. For example, the computing device 1100 may include, but is not limited to, a personal computer, a server computer, a handheld device or a notebook device, a mobile device (a mobile phone, a PDA, a media player, etc.), a multiprocessor system, a consumer electronic device, a microcomputer, a mainframe computer, a distributed computing environment including any of the aforementioned systems or devices, etc.

[0114] The computing device 1100 may include at least one processing unit 1110 and a memory 1120. The processing unit 1110 may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., and may have multiple cores.

[0115] The memory 1120 may be a volatile memory (such as RAM, etc.), a non-volatile memory (such as ROM, flash memory, etc.), or a combination thereof. In addition, the computing device 1100 may further include an additional storage 1130. The storage 1130 may include, but is not limited to, magnetic storage, optical storage, etc. The storage 1130 may store computer-readable instructions for implementing the embodiments disclosed in this specification or multiple embodiments, and may also store other computer-readable instructions for implementing an operating system, an application program, etc. The computer-readable instructions stored in the memory 1130 may be loaded into the memory 1120 for execution by the processing unit 1110. In addition, the computing device 1100 may further include an input device 1140 and an output device 1150.

[0116] The input device 1140 may include, for example, a keyboard, a mouse, a pen, a voice input device, a touch input device, an infrared camera, a video input device, or any other input device. In addition, the output device 1150 may include, for example, one or more displays, speakers, a printer, or any other output device. In addition, the computing device 1100 may use an input device or an output device provided on another computing device as the input device 1140 or the output device 1150. In addition, the computing device 1100 may include a communication connection 1160 that allows the computing device 1100 to communicate with another device (e.g., the computing device 1300).

[0117] Among them, the communication connection 1160 may include a modem, a network interface card (NIC), an integrated network interface, a radio frequency transmitter / receiver, an infrared port, a USB connection, or other interfaces for connecting the computing device 1100 to another computing device. In addition, the communication connection 1160 may include a wired connection or a wireless connection. The components of the computing device 1100 can be connected by various interconnection methods, such as a bus (such as a peripheral component interconnect (PCI), USB, firmware (IEEE1394), an optical bus structure, etc.). Terms such as "component" and "system" used in this specification generally refer to hardware, a combination of hardware and software, software, or a computer-related entity that executes software.

[0118] The present invention is not limited to the embodiments and drawings. It is obvious to those skilled in the art that the components according to the present invention can be replaced, modified and changed without departing from the technical concept of the present invention.

Claims

1. A system using AI vision and the Internet of Things to support security management services, characterized in that: include: Multiple visual cameras to capture the secure target areas that are assigned IDs; A plurality of IoT devices are located in the security target area and sense or detect any of the following: dynamic objects, static objects, temperature / humidity, gas, cracks, heat, vibration, dust, radio waves; The AI ​​safety diagnosis management server analyzes the danger level of the safety target area based on the images captured by the multiple visual cameras and the perception information sensed by the multiple IoT devices, and provides dangerous situation response plans that meet the danger level to worker terminals located in or near the safety target area.

2. The system for supporting security management services using AI vision and the Internet of Things according to claim 1, characterized in that: The AI ​​safety diagnosis management server determines the type of the safety target area through the above-mentioned ID, selects the type of visual cameras and IoT devices located in the safety target area to judge the dangerous situation of the safety target area, and then receives data from the selected visual cameras and IoT devices.

3. The system for supporting security management services using AI vision and the Internet of Things according to claim 1, characterized in that: The AI ​​safety diagnosis management server uses multiple safety risk analysis system programs that conform to multiple dangerous situations to predict and analyze the safety status of workers or facilities working in the safety target area.

4. The system for supporting security management services using AI vision and the Internet of Things according to claim 3, characterized in that: The AI ​​safety diagnosis management server provides the worker terminal with preliminary measures information for the current dangerous situation predicted in the safety target area.

5. The system for supporting security management services using AI vision and the Internet of Things according to claim 4, characterized in that: The AI ​​safety diagnosis management server extracts objects of factors that may cause dangerous situations related to safety management objects from the images captured by the visual camera for classification and learning; When the factor that may cause a dangerous situation is a dynamic object, analyzing and learning the behavior pattern and / or posture of the dynamic object; When the dangerous situation factor is a static object, analyze and learn the size, shape, and color of the static object.

6. A method for supporting security management services using AI vision and the Internet of Things, characterized in that: include: In the AI ​​safety diagnosis management server, visual images are collected from multiple visual cameras that capture the safety target area assigned with an ID; The AI ​​security diagnosis management server collects measurement data from a plurality of IoT devices, wherein the plurality of IoT devices are located in the security target area and sense or detect any of the following: dynamic objects, static objects, temperature / humidity, gas, cracks, heat, vibration, dust, radio waves; In the AI ​​safety diagnosis management server, based on the images captured by the multiple visual cameras and the perception information perceived by the multiple IoT devices, the danger level of the safety target area and the occurrence of dangerous situations are analyzed or predicted; and In the AI ​​safety diagnosis management server, a dangerous situation response plan that meets the dangerousness level is provided to a worker terminal located in or near the safety target area; Among them, the AI ​​safety diagnosis management server determines the type of the safety target area through the above-mentioned ID, selects the type of visual cameras and Internet of Things devices located in the safety target area to judge the dangerous situation of the safety target area, and then receives data from the selected visual cameras and Internet of Things devices.

Images