Embedded video recognition safety integrated control platform including artificial intelligence-based video recognition system

Abstract

The present invention relates to an embedded video recognition safety integrated control platform including an artificial intelligence-based video recognition system, and more particularly, to a video recognition system including: a video data receiving unit that receives video data of the vicinity of a construction or industrial site from a camera module; a CNN-based object recognition unit that recognizes a human object in the video data received from the video data receiving unit and detects the recognized human object by boxing it according to the recognized human object data; and an event information generating unit that generates a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit that receives event information detected from the CNN-based object recognition unit and classifies it into positive detection, false detection, or undetected, and a deep learning-based object recognition learning algorithm, YOLO (You Only Look Back), which classifies the false detection or undetected classification received from the false detection / undetected classification unit. the re-learning unit re-learns the data set generated by the re-learning unit in real time through a CNN-based object recognition unit (CNN-based object recognition unit), a learning model update unit uploads the re-learned data set generated by the re-learning unit to the CNN-based object recognition unit in online real time, and an AI video recognition platform including an LTE communication unit for communication between the video recognition system and an external device including an external server or an administrator terminal.

Description

[Technical field]

[0001] The present invention relates to an embedded video recognition safety integrated control platform including an artificial intelligence-based video recognition system, and more particularly, to a video recognition system including: a video data receiving unit that receives video data of a construction or industrial site from a camera module; a CNN (Convolutional Neural Network)-based object recognition unit that recognizes a human object in the video data received from the video data receiving unit and detects the recognized human object by boxing it in accordance with the recognized human object data; and an event information generating unit that generates a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit that receives event information detected from the CNN-based object recognition unit and classifies it into positive detection, false detection, or undetected, and a deep learning-based object recognition learning algorithm, YOLO (You Only Look Back), which classifies the false detection or undetected classification received from the false detection / undetected classification unit. the re-learning unit re-learns the data set generated by the re-learning unit in real time through a CNN-based object recognition unit (CNN-based object recognition unit), a learning model update unit uploads the re-learned data set generated by the re-learning unit to the CNN-based object recognition unit in online real time, and an AI video recognition platform including an LTE communication unit for communication between the video recognition system and an external device including an external server or an administrator terminal. [Background technology]

[0002] According to the OECD National Industrial Accident Fatality Comparison and Analysis Report published by the Korea Construction Industry Institute in September 2020, it was confirmed that in 2017, there were approximately 3.61 industrial accident fatalities for every 100,000 workers in Korea.

[0003] In particular, the average industrial death rate among OECD member countries is 2.43 per 100,000 people, with South Korea (3.61) the fifth highest after Canada (5.84), Turkey (5.17, as of 2016), Chile (4.04), and Luxembourg (3.69). Not only that, but among countries with a population of over 50 million and a per capita national income of over $30,000, the number of industrial deaths among these "3050 club" countries is 2.43 per 100,000 people, with South Korea ranking first behind South Korea (3.61), Japan (1.50), the US (3.36), the UK (0.88), France (2.18), Germany (1.03), and Italy (2.10). While the industrial death rate is generally on a downward trend, South Korea still has a higher industrial death rate than other countries.

[0004] Additionally, according to the current industrial accident occurrence status released by the Ministry of Employment and Labor in 2020, the total number of accidents was confirmed to be 92,383. In addition, when examining accident victims by type of industry, excluding other businesses, the construction industry had the highest number of accidents at 24,617 (26.6% of the total), followed by the manufacturing industry at 23,127 (25.0% of the total), accounting for 51.6% of all industrial sites.

[0005] The majority of deaths occurred in the construction industry (567, 27.5%) and manufacturing industry (469, 22.7%), with the highest number of deaths occurring in manufacturing (249) and construction sites (225), both of which are in the 5-49 range.

[0006] In addition, it was confirmed that more than 51% of all industrial accidents such as those mentioned above were conventional accidents (falls / trips / being pinched) and occurred due to non-compliance with basic safety rules and safe work methods.

[0007] For reference, among the current fatal accident occurrence status by disaster type, examples of collision and entrapment accidents involving construction machinery at industrial sites are representative of collision and entrapment with heavy equipment such as excavators, dump trucks, forklifts, and containers.

[0008] In addition, the direct losses (industrial accident compensation payments) due to the above-mentioned industrial accidents amounted to 5,996,819 million won in 2021, up 8.45% from the previous year. In addition, the estimated economic losses, including direct and indirect losses, were 29,984,095 million won, up 8.45% from the previous year, which is higher than Gyeonggi Province's entire budget for 2021 (28,792.5 billion won) and reaches 1.5% of the national GDP.

[0009] As mentioned above, the government is also implementing various policies to prevent the continued occurrence of industrial disasters and safety accidents. The revised Occupational Safety and Health Act, which significantly strengthened safety regulations at industrial sites, including preventing the "outsourcing of risks," came into effect on January 27, 2022. The Act also provides for management to be sentenced to prison when a worker is killed or involved in an accident, introduces a punitive damages system, and makes employers and corporations liable for compensation of up to five times the amount of damages if they intentionally or with gross negligence violate their safety and health obligations and cause a serious disaster or damage.

[0010] However, at construction and industrial sites, various types of construction machinery are used in large quantities, so there are potential disaster-causing elements everywhere, and at construction sites, numerous processes are carried out in conjunction with each other, so if the previous process is not performed properly, it will have an immediate impact on the next process, and the combination of various potential dangerous elements can cause large-scale disasters all at once. As a result, it is extremely difficult to reduce industrial accidents at construction sites.

[0011] In particular, in industrial sites, various construction machines, such as forklifts, excavators, dump trucks, etc., are used at various industrial and construction sites to improve the ease and productivity of work such as moving materials and excavating. However, since there are many cooperative tasks in which such construction equipment is operated together with workers, there are frequent fatal accidents due to inexperienced equipment operation, carelessness, or failure to recognize blind spots. Therefore, there is a need to develop technology to prevent accidents by developing a safety system for construction machinery equipment.

[0012] As a result, current Korean industrial sites are making efforts to prevent collision and entrapment accidents by equipping construction equipment with rear cameras and collision prevention bars and by stationing traffic lights, as shown in [Figure 1]. However, workers and traffic lights are constantly exposed to danger as they work at the same time around heavy equipment, and in most cases, traffic lights and surveyors still end up being involved in collisions and entrapment accidents.

[0013] Therefore, in recent years, various proximity warning devices have been developed and used to prevent the risk of collision safety accidents involving workers due to heavy equipment at industrial sites. These can be broadly divided into tag-based and non-tag-based technologies that use various sensors, and products incorporating various technologies are being developed, commercialized, and sold.

[0014] The tag-based technology is a method in which vehicles and workers carry sensor tags capable of direct radio wave transmission and reception to measure the distance between heavy equipment and workers, and is classified into unidirectional and bidirectional tag types according to the radio wave transmission and reception method of the tag, and RF, UWB, etc. are mainly used as sensors in the bidirectional tag type, and a method is used in which the radio wave strength indicator (RSSI) and radio wave time of arrival (TOA) received by the sensor are measured and converted into distance. RF-type proximity alarm products are mainly mainstream, but UWB-type products, which have better performance in terms of accuracy and uniformity of distance measurement, have been developed and commercialized.

[0015] In addition, the unidirectional tag system includes a passive type product that uses RFID tags, but in this system an RFID reader is installed on the vehicle and pedestrians carry directional passive RFID tags as tags. This type of product has a larger distance recognition error than the bidirectional tag system, which reduces the reliability of the product.

[0016] In comparison, non-tag-based approach warnings use simple sensors such as non-tag-based cameras and ultrasound, and due to the characteristics of the non-tag-based sensors themselves and their limited functionality in terms of operation, radar products using radio wave signals and radar products using optical technology have been developed, and recently, with the development of AI technology, object image recognition products using cameras have also been developed.

[0017] In particular, when considering the conventional technology for managing and controlling dangerous situations of construction equipment and workers based on the AI ​​image recognition system, Korean Patent No. 10-1808587 (registration date: December 7, 2017) describes a video input unit including a PTZ (Pan / Tilt / Zoom) camera or a fixed camera that can rotate 360 ​​degrees and has up / down / left / right and zoom functions, an abnormal situation detection unit that detects whether the video captured from the video input unit corresponds to one abnormal situation selected from intrusion, crowding, loitering, abandonment, emergence, entry / exit, climbing a wall, people counting, falling, running in the wrong direction, and number recognition according to a preset abnormal situation algorithm, and when an abnormal situation is detected by the abnormal situation detection unit, object recognition is performed through an image pre-processing process, an object extraction image generation process, and an object analysis process, and the object analysis process is performed by extracting an object extracted from the object extraction image generation process. An intelligent integrated surveillance and control system using object recognition, tracking and abnormal situation detection technologies has been developed, which includes an object recognition unit that extracts edge patterns from a detected object using a Haar algorithm, a HOG algorithm or a SURF algorithm depending on an abnormal situation, and recognizes and discriminates objects through pattern matching with learned data accumulated and stored through a deep learning algorithm; an object tracking unit that analyzes changes in coordinates of an object recognized by the object recognition unit, predicts a moving path or moving direction of the object in the captured image, and tracks the object so that the object is located at the center of the captured image; and an integrated control unit that displays and monitors images captured by the image input unit, and sets and controls the image input unit, the abnormal situation detection unit, the object recognition unit and the object tracking unit.

[0018] In addition, Korean Patent 10-2185859 (registration date: November 26, 2020) relates to an object tracking device through deep learning that recognizes human objects from video data and tracks the corresponding human objects on a frame-by-frame basis, the device comprising: a video data receiving unit that receives the video data from a camera module; a pre-processing unit that resizes the received video data and reduces the effects of light; an object recognition unit that recognizes human objects in video data that has been pre-processed through deep learning-based object recognition learning and boxes the recognized human objects according to the recognized human object data; a calculation unit that calculates whether or not the boxed human object in a first frame portion of the video data matches the data of the human object in a second frame portion following the first frame, and recognizes boxes that show a degree of match equal to or higher than a set degree of match as the same human object; and a box that is recognized as the same human object by the calculation unit. a moving direction measuring unit that measures a moving direction of a human object of a corresponding box based on a position of the human object in the first frame and a position of the human object in the second frame; a coordinate information generating unit that accumulates and collects the moving directions of the human objects measured by the moving direction measuring unit, extracts a moving path of the human object, and stores coordinate information for a shape of a road through the extracted moving path; a coordinate information receiving unit that receives coordinate information for measuring a moving population from an administrator terminal; and a moving population calculating unit that calculates a moving population as 0 if the coordinate information received from the coordinate information receiving unit is not included in the coordinate information for the road, and calculates a moving population based on each box that passes through the corresponding coordinate and sets a straight line perpendicular to an extension direction of the road at the corresponding coordinate when the coordinate information received from the coordinate information receiving unit is included in the coordinate information for the road, and calculates a moving population based on each box that passes through the set straight line;

[0019] In addition, Korean Patent No. 10-2206662 (registered date: January 18, 2021) is a patent for a port gate, yard / block, block entrance, etc. at a port container terminal. and an FPGA-based embedded vision system (TLEM) that receives the video data from the cameras, detects each object in the camera video according to the learning data through a deep learning module, recognizes characters, and performs lane recognition, vehicle number recognition, character recognition of container number (ISO code), container damage recognition, recognition of vehicles entering the block entrance, detection of vehicles and workers entering dangerous areas, vehicles traveling in the wrong direction, detection of whether yard workers are wearing safety protective gear / safety vests, and detection of the location of loading and unloading equipment, and transmits the learned object extraction event (text) and object extraction video data marked with a square box to the middleware. The FPGA-based embedded vision system (TLEM) is equipped with a deep learning module that detects each object in a non-GPU-based video and recognizes characters of vehicle number and container number (ISO code), and a vision camera system that performs vehicle entrance and exit management and object recognition at a port container terminal has been developed.

[0020] In addition, Korean Patent Registered No. 10-2263512 (Registration Date: June 4, 2021) describes an IoT integrated intelligent video analysis platform system that integrates and analyzes video data and non-video data, comprising: a video data acquisition unit that acquires at least one video data; a non-video data acquisition unit that acquires at least one non-video data; a video data processing unit that analyzes the video data; a non-video data processing unit that analyzes the non-video data; and an integrated data judgment unit that finally judges the abnormal situation when the video data processing unit or the non-video data processing unit judges that an abnormal situation exists from the video data or the non-video data; and the video data processing unit recognizes an object from the acquired video data, estimates the state of the object, estimates the authenticity of the object, or determines the action of the object. the non-video data processing unit, in analyzing the non-video data, defines a case where a measurement value of the non-video data deviates from a data range of a normal situation as an abnormal event, and determines an abnormal situation by considering whether the abnormal event has occurred, the time of occurrence, and a count number of occurrences per a predefined unit time; the video data processing unit further includes an object processing unit that processes a function of recognizing an object from the acquired video data; and a user learning setting unit that provides a user with a function related to machine learning of video data; the object processing unit further includes an object authenticity identification unit that extracts an object from the video data and determines whether it is forged; an object state recognition unit that estimates a state of an object from the video data; and an object action recognition unit that estimates an action event of an object from the video data;The object authenticity identification unit extracts an image from the video data, analyzes the hue of each pixel constituting the extracted image, extracts a desired hue from each analyzed hue, and derives a probability that the object is an authentic product through an authenticity determination algorithm. In order to extract a hue ratio through a K-mean Clustering algorithm, a variance between clusters is minimized using a clustering algorithm based on similarity between data, and a hue ratio within an item is identified using the clustered hues. The hue ratio can be extracted through OpenCV, and a hue magnification factor for an authentic product can be learned, and a difference between an authentic product and a counterfeit product image can be distinguished according to the extracted hue ratio. The object authenticity identification unit can also use a DCGAN (Deep Convolution Generative Adversarial An IoT integrated intelligent image analysis platform system capable of smart object recognition has been developed, which generates a copy image using a 3D Network algorithm, and derives the probability that an object is genuine by applying an illegal copy detection algorithm that corresponds to a learning model for genuine detection through feedback and learning between genuine and imitation models using the difference in surface material.

[0021] However, while the above-mentioned conventional technologies are positive in terms of vehicle entrance / exit management and worker safety management through object recognition and tracking from video data, they have fatal problems such as insufficient object detection accuracy due to misrecognition issues caused by various objects (pillars) characteristic of industrial sites, non-detection or misdetection of objects due to harsh external environmental factors characteristic of construction sites, and difficulty in collecting and updating automatic learning datasets in various environments.

[0022] As a result, the inventors have overcome the limitations of existing video recognition proximity warning systems, improved the visibility of construction equipment operators, ensured durability suitable for industrial sites so that blind spots around the equipment can be accurately monitored in real time, and developed an AI-based video recognition platform that can detect objects in all directions while ensuring real-timeness and accuracy, communicates to manage dangerous situations and equipment fleets, and can build an AI learning data set, thereby completing the present invention. [Prior art documents] [Patent documents]

[0023] [Patent Document 1] Republic of Korea Patent Registered 10-1808587 (Registration Date: December 7, 2017) [Patent Document 2] Republic of Korea Registered Patent No. 10-2185859 (Registration Date: November 26, 2020) [Patent Document 3] Republic of Korea Registered Patent No. 10-2206662 (Registration Date: January 18, 2021) [Patent Document 4] Republic of Korea Registered Patent No. 10-2263512 (Registration Date: June 4, 2021) Summary of the Invention [Problem to be solved by the invention]

[0024] The present invention is to solve the above-mentioned problems, and provides an image recognition system including: a video data receiving unit that receives video data of the vicinity of a construction or industrial site from a camera module; a CNN-based object recognition unit that recognizes a human object in the video data received from the video data receiving unit and detects the recognized human object by boxing it according to the recognized human object data; and an event information generating unit that generates a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit that receives the event information detected from the CNN-based object recognition unit and classifies it into positive detection, false detection, or undetected, and a deep learning-based object recognition learning algorithm, YOLO (You Only Look Back), which classifies the false detection or undetected classification received from the false detection / undetected classification unit. a re-learning unit that re-learns in real time via a CNN-based object recognition unit (CNN-based object recognition unit), a learning model update unit that uploads a re-learned dataset generated by the re-learning unit to the CNN-based object recognition unit in online real time, and an AI video recognition platform including an LTE communication unit for communication between the video recognition system and an external device including an external server or an administrator terminal. [Means for solving the problem]

[0025] In order to solve the above technical problem, the present invention provides a video recognition system including: a video data receiving unit that receives video data of the vicinity of a construction or industrial site from a camera module; a CNN-based object recognition unit that recognizes a human object in the video data received from the video data receiving unit and detects the recognized human object by boxing it according to the data of the recognized human object; and an event information generating unit that generates a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit that receives the event information detected from the CNN-based object recognition unit and classifies it into positive detection, false detection, or undetected, and a deep learning-based object recognition learning algorithm, YOLO (You Only Look Back), which classifies the false detection or undetected classification received from the false detection / undetected classification unit. a re-learning unit that re-learns in real time through a CNN-based object recognition unit (CNN-based object recognition unit), a learning model update unit that uploads the re-learned dataset generated by the re-learning unit to the CNN-based object recognition unit in online real time, and an AI video recognition platform including an LTE communication unit for communication between the video recognition system and an external device including an external server or an administrator terminal.

[0026] The preset danger radius and situation including the periphery of the heavy equipment in the CNN-based object recognition unit includes setting a virtual boundary or virtual area in the video shooting data received from the video data receiving unit, and appearing, entering, or falling within the virtual boundary or virtual area.

[0027] The re-learning data set is stored together with the video shooting data after labeling the video shooting data according to the misdetection or undetection classification using an Auto-Labeling Tool, and is uploaded online in real time.

[0028] The external device communicating with the LTE communication unit further includes an equipment fleet management system including equipment operation time, movement location tracking, and equipment downtime, and the equipment fleet management system is linked to the CNN-based object recognition unit to update and reflect a preset danger radius and situation including the periphery of the heavy equipment.

[0029] The external device in communication with the LTE communication unit further includes a site danger map display unit that collects event information detected from the CNN-based object recognition unit and displays danger zones on a map.

[0030] The external device communicating with the LTE communication unit further includes an equipment-specific and worker-specific dangerous event monitoring display unit for tracking equipment work paths and worker movement lines based on event information detected from the CNN-based object recognition unit.

[0031] The LTE communication unit is equipped with a GPS for determining and tracking the location of equipment or workers.

[0032] The external server or the administrator terminal receives and uploads event information including an event image or clip video in real time when a worker approaches within a danger radius detected by the CNN-based object recognition unit.

[0033] The embedded video recognition safety integrated control platform, which includes the AI-based video recognition system, has an object detection accuracy of over 88% (human, F1 Score standard), a maximum recognition distance of over 7m, a range of 360 degrees, high temperature reliability of 60℃, and a risk factor recognition speed of less than 0.5s. Effect of the Invention

[0034] The embedded video recognition safety integrated control platform including the artificial intelligence-based video recognition system of the present invention includes a video recognition system including: a video data receiving unit that receives video data of the vicinity of a construction or industrial site from a camera module; a CNN-based object recognition unit that recognizes a human object in the video data received from the video data receiving unit and detects the recognized human object by boxing it according to the recognized human object data; and an event information generating unit that generates a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit that receives the event information detected from the CNN-based object recognition unit and classifies it into positive detection, false detection, or undetected, and a deep learning-based object recognition learning algorithm, YOLO (You Only Look Back), which classifies the false detection or undetected classification received from the false detection / undetected classification unit. The present invention is configured to provide an AI video recognition platform including an AI image recognition system that includes an AI image recognition unit that performs real-time retrieval through a learning model update unit that uploads the retrieval dataset generated by the retrieval unit to the CNN-based object recognition unit in online real time, and an LTE communication unit for communication between the image recognition system and an external device including an external server or an administrator terminal. The present invention has a groundbreaking effect in ensuring object recognition accuracy in various industrial or construction site environments, improving the reliability of dangerous situation events, and preventing collision and entrapment accidents through sustainable learning data collection and retrieval. [Brief description of the drawings]

[0035] [Figure 1] This is an example of preventing collision and entrapment accidents involving construction equipment. [Diagram 2] FIG. 1 is an overall schematic diagram of an embedded video recognition safety integrated control platform including the artificial intelligence-based video recognition system of the present invention. [Diagram 3] FIG. 1 is a detailed schematic diagram of an embedded video recognition safety integrated control platform including an artificial intelligence-based video recognition system of the present invention; [Figure 4]1 is an overall flowchart of an embedded video recognition safety integrated control platform including an artificial intelligence-based video recognition system of the present invention; [Diagram 5] FIG. 2 is a process diagram of the re-learning part of the embedded video recognition safety integrated control platform including the artificial intelligence-based video recognition system of the present invention. [Figure 6] FIG. 2 is a configuration diagram of the re-learning part of the embedded image recognition safety integrated control platform including the artificial intelligence-based image recognition system of the present invention. [Figure 7] FIG. 2 is a diagram showing an example of positive detection, false detection, or undetected classification according to the present invention. [Figure 8] FIG. 13 is an example diagram of a re-learning dataset using the video shooting data Auto-Labeling Tool. [Figure 9] This shows the structure of YOLO. [Figure 10] FIG. 2 is an example diagram of an equipment fleet management system added to the present invention. [Figure 11] This is an example diagram of dangerous event monitoring by equipment and by worker. [Figure 12] FIG. 2 is a test and performance specification diagram of the video recognition safety integrated control platform of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the following examples and / or drawings so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments and / or drawings set forth herein.

[0037] First, as shown in FIG. 2 and FIG. 3, an embedded video recognition safety integrated control platform including an AI-based video recognition system of the present invention includes a video data receiving unit 101 that receives video data of the vicinity of a construction or industrial site from a camera module; a CNN-based object recognition unit that recognizes a human object from the video data received from the video data receiving unit and detects the recognized human object by boxing it according to the recognized human object data, and an event information generating unit that generates a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit that receives the event information detected from the CNN-based object recognition unit and classifies it into positive detection, false detection, or undetected, and a deep learning-based object recognition learning algorithm, YOLO (You Only Look Back), which classifies the false detection or undetected classification received from the false detection / undetected classification unit. a learning model update unit that uploads the re-learned dataset generated by the re-learning unit to the CNN-based object recognition unit in online real time; and an AI video recognition platform 103 including an LTE communication unit for communication between the video recognition system and an external device including an external server or an administrator terminal.

[0038] In this case, the preset danger radius and situation including the periphery of the heavy equipment in the CNN-based object recognition unit may include setting a virtual boundary or virtual area in the video shooting data received from the video data receiving unit 101, and appearing, entering, and falling within the virtual boundary or virtual area, and the setting of the virtual boundary or virtual area may be changed or set at any time by the CNN-based object recognition unit.

[0039] More specifically, looking at the operation flowchart of the embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system of the present invention, as shown in [FIG. 2] to [FIG. 4], in addition to the video recognition system 102 including an event information generating unit that 1) recognizes a human object by a CNN-based object recognition unit in the video shooting data received from the video data receiving unit 101, boxes and detects the recognized human object according to the recognized human object data, and 3) generates a warning light or a warning alarm when the detected human object is exposed to a preset danger radius and situation including around heavy equipment, the event information detected from the CNN-based object recognition unit is received and classified into positive detection, false detection, or unknown detection before generating a warning light or a warning alarm, and the false detection / undetected classification is re-learned in real time through YOLO (You Only Look Once), which is a deep learning based object recognition learning algorithm, and then transmitted to the event information generating unit, thereby enabling more accurate event information detection through video recognition.

[0040] In particular, the process of re-learning in real time through YOLO (You Only Look Once) is as shown in [FIGS. 5] to [FIG. 6], and includes the steps of: image resizing and pre-processing (S11) the video shooting data received from the video data receiving unit 101; object detection step (S12) of performing individual recognition on the image resized in the pre-processing step (S11) using YOLO4 and TensorRT and generating and tracking a Bbox; primary object recognition step (S13) of inferring whether the object detected in the object detection step (S12) corresponds to a dangerous situation; if the object inferred in the primary object recognition step (S13) corresponds to a dangerous situation, a step (S14) of generating event information from an event information generating unit; and if the object inferred in the primary object recognition step (S13) does not correspond to a dangerous situation, a secondary object recognition step (S15) of inferring that it is not event information.

[0041] Meanwhile, the re-learning data set is stored together with the corresponding video data after labeling the video data according to the misdetection or undetection classification using an Auto-Labeling Tool, and is uploaded online in real time.

[0042] That is, as shown in [FIGS. 7] to [FIGS. 8], there is a false positive / missing classification unit that receives detected event information from the CNN-based object recognition unit and classifies it into true positive, false positive, or missing, a re-learning unit that re-learns the false positive or missing classification received from the false positive / missing classification unit in real time through YOLO (You Only Look Once), which is a deep learning-based object recognition learning algorithm, and a re-learning dataset generated from the re-learning unit is labeled on video shooting data according to the false positive or missing classification using an Auto-Labeling Tool, and then stored together with the corresponding video shooting data and uploaded online in real time.

[0043] Here, YOLO is one of the deep learning-based map learning algorithms for object detection. It is an abbreviation of You Only Look Once, which means looking at it just once, and determines the classification and location of an object by a single regression of the image. YOLO is based on a CNN structure, and the network architecture is based on the GoogleNet model and is composed of 24 convolutional layers and 2 fully connected layers.

[0044] [Figure 9] shows the structure of YOLO. The way YOLO processes images is to scale the input image, run a convolutional network on the image, and threshold the result detection according to the model's confidence. A bounding box consists of five elements: x, y, w, h, and a confidence score. (x,y) are the center coordinates of the bounding box based on the grid cell boundary. (w,h) are the width and height of the bounding box. The confidence score indicates the IOU between the predicted bounding box and all ground truth bounding boxes. Each grid cell also predicts a conditional class probability, Pr(Classi|Object), C. These probabilities are conditioned on the grid cell containing the object. Only one set of class probabilities is predicted per grid cell, regardless of the number of bounding boxes, B. At test time, the conditional class probabilities are multiplied to provide a class-specific confidence score for each box.

[0045] In Figure 9, the score encodes the probability that the class appears in the box and how well the predicted box fits the individual. YOLO's system models detection as a regression problem. It divides the image into an S×S grid and predicts for each grid cell B bounding boxes, a confidence for the box, and C class probability. These predictions are encoded as S×S×(B≦5+C) tensors.

[0046] In the present invention, we use YOLOv4, the fourth version of YOLO, which has the advantages of being fast, capable of real-time detection, and having greatly improved accuracy and excellent performance.

[0047] In addition, the external device communicating with the LTE communication unit further includes an equipment fleet management system including equipment operation time, movement location tracking, and equipment downtime, as shown in FIG. 10, and the equipment fleet management system can be linked to the CNN-based object recognition unit to update and reflect a preset danger radius and situation including the periphery of the heavy equipment.

[0048] In addition, the external device communicating with the LTE communication unit may further include a dangerous event monitoring display unit for each equipment and each worker for tracking the equipment work path and the worker's movement line based on the event information detected by the CNN-based object recognition unit, as shown in FIG. 11.

[0049] Furthermore, the LTE communication unit is equipped with a GPS for determining and tracking the location of equipment or workers.

[0050] In addition, the external server or the administrator terminal can be configured to receive and upload event information including an event image or clip video in real time when a worker approaches within a danger radius detected by the CNN-based object recognition unit.

[0051] In particular, the embedded image recognition safety integrated control platform including the artificial intelligence-based image recognition system according to the present invention has an object detection accuracy of 88% or more (human, F1 Score standard), a maximum recognition distance of 7m or more, a range of 360 degrees, high temperature reliability of 60℃, and a risk factor recognition speed of 0.5s or less, as shown in FIG. 12. [Industrial Applicability]

[0052] The embedded video recognition safety integrated control platform including the artificial intelligence-based video recognition system of the present invention includes a video recognition system including: a video data receiving unit that receives video data of the vicinity of a construction or industrial site from a camera module; a CNN-based object recognition unit that recognizes a human object in the video data received from the video data receiving unit and detects the recognized human object by boxing it according to the recognized human object data; and an event information generating unit that generates a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit that receives the event information detected from the CNN-based object recognition unit and classifies it into positive detection, false detection, or undetected, and a deep learning-based object recognition learning algorithm, YOLO (You Only Look Back), which classifies the false detection or undetected classification received from the false detection / undetected classification unit. the CNN-based object recognition system includes an AI video recognition platform including a re-learning unit that re-learns in real time via a learning model update unit that uploads the re-learned dataset generated by the re-learning unit to the CNN-based object recognition unit in online real time, and an LTE communication unit for communication between the video recognition system and an external device including an external server or an administrator terminal; and the AI ​​video recognition platform ... has an industrial applicability because it has a groundbreaking effect in ensuring object recognition accuracy in various industrial or construction environments, improving reliability of dangerous situation events, and preventing collision and entrapment accidents through sustainable collection and re-learning of learning data.

Claims

1. a video data receiving unit for receiving video data of a construction or industrial site from a camera module; a CNN (convolutional neural network)-based object recognition unit for recognizing a human object in the video data received from the video data receiving unit and detecting the recognized human object by boxing it in response to the recognized human object data; and an event information generating unit for generating a warning light or a warning alarm when the human object detected by the CNN-based object recognition unit is exposed to a preset danger radius and situation including the vicinity of heavy equipment; a false detection / undetected classification unit for receiving the event information detected from the CNN-based object recognition unit and classifying it into a positive detection, a false detection, or an undetected event; and a YOLO (You Only Look) classification unit for classifying the false detection or undetected event classification received from the false detection / undetected classification unit into a positive detection, a false detection, or an undetected event classification based on a deep learning object recognition learning algorithm. an AI video recognition platform including an AI based video recognition system, the AI ​​video recognition platform including: a re-learning unit that re-learns in real time via a CNN-based object recognition unit (CNN) that re-uploads a re-learned dataset generated from the re-learning unit to the CNN-based object recognition unit in online real time; and an LTE communication unit for communication between the video recognition system and an external device including an external server or an administrator terminal.

2. 2. The embedded video recognition safety integrated control platform including an artificial intelligence based video recognition system according to claim 1, wherein the preset danger radius and situation including the periphery of the heavy equipment in the CNN-based object recognition unit includes setting a virtual boundary or a virtual area in the video shooting data received from the video data receiving unit and detecting an appearance, an entry, or a fall within the virtual boundary or the virtual area.

3. The embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system of claim 1, wherein the re-learning data set is stored together with the corresponding video shooting data after labeling the video shooting data according to the misdetection or undetection classification using an auto-labeling tool, and is uploaded online in real time.

4. The embedded video recognition safety integrated control platform including an artificial intelligence based video recognition system according to claim 1, wherein the external device communicating with the LTE communication unit further includes an equipment fleet management system including equipment operation time, movement location tracking, and equipment downtime, and the equipment fleet management system is linked to the CNN-based object recognition unit to update and reflect a preset danger radius and situation including the periphery of the heavy equipment.

5. The embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system of claim 1, wherein the external device communicating with the LTE communication unit further includes a site danger map display unit that collects event information detected from the CNN-based object recognition unit and displays danger zones on a map.

6. 2. The embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system according to claim 1, wherein the external device communicating with the LTE communication unit further includes a dangerous event monitoring display unit for each equipment and each worker for tracking an equipment work path and a worker's movement line based on event information detected from the CNN-based object recognition unit.

7. 2. The embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system of claim 1, wherein the LTE communication unit is equipped with a GPS for locating and tracking equipment or workers.

8. 2. The embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system of claim 1, wherein the external server or the manager terminal receives and uploads event information including an event image or a clip video in real time when a worker approaches within a danger radius detected by the CNN-based object recognition unit.

9. The embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system according to any one of claims 1 to 8, characterized in that the embedded video recognition safety integrated control platform including the artificial intelligence based video recognition system has an object detection accuracy of 88% or more (human, F1 Score standard), a maximum recognition distance of 7m or more, a range of 360 degrees, a high temperature reliability of 60°C, and a hazard recognition speed of 0.5s or less.

Images