System for predicting safety accident at railway work site

Abstract

The present invention relates to a system for predicting a safety accident at a railway work site, and to a railway work site safety accident prediction system comprising intelligent CCTV, a network, and a safety accident prediction server in order to monitor and control railway safety information in real time by integrating data of a conventional safety detection device and data of an intelligent safety monitoring device, wherein the safety accident prediction server controls a transmission and reception unit such that video information acquired through the intelligent CCTV is received through the network, and thus collects the video information in a database (DB), thereby verifying whether a worker is working in a designated work area and whether the worker is wearing protective gear, requests verification from a big data server corresponding to a railway safety information DB server if the worker is not in the work area or is not wearing protective gear, and stores the history thereof in the DB.

Description

Railway worksite safety accident prediction system

[0001] The present invention relates to a railway work site safety accident prediction system, and more specifically, to a railway work site safety accident prediction system capable of monitoring and controlling railway safety information in real time by integrating data from existing safety monitoring devices and data from intelligent safety monitoring devices.

[0002] As railways have grown as a means of transportation, the number of railway-related accidents has also steadily increased. A recent example is a freight train connection incident in which a worker was struck and killed by a freight train. An investigation revealed that the area involved had no CCTV installed and that the confined space lacked adequate evacuation routes.

[0003] The current operational Railway Safety Information Management System (RSIS) posts information on railway accidents and operational disruptions. However, this RSIS calculates and forecasts accident risks by reflecting the number of fatalities, injuries, and property damage resulting from railway accidents over the past decade. Therefore, it focuses more on post-accident management than on accident prevention.

[0004] Accordingly, in the railway-related technology field, there is a continuous demand for technology development to monitor and control railway safety information in real time by integrating data from existing safety monitoring devices and data from intelligent safety monitoring devices through research on the development of an integrated railway safety monitoring and control system.

[0005] (Prior Document 001) Patent Publication No. 10-2018-0055956

[0006] The present invention aims to address the problems of the prior art described above. To reduce railway-related safety accidents, the present invention provides a railway worksite safety accident prediction system capable of monitoring and controlling railway safety information in real time by integrating data from existing safety monitoring devices with data from intelligent safety monitoring devices. However, the purpose of the present invention is not limited thereto, and other objectives not mentioned will be clearly understood by those skilled in the art from the description below.

[0007] In order to solve the above problem, the present invention provides a railway work site safety accident prediction system including an intelligent CCTV, a network, and a safety accident prediction server, wherein the safety accident prediction server controls a transmission and reception unit to receive image information acquired through the intelligent CCTV through a network, collects the image information in a database, confirms whether a worker is working in a work area and wearing equipment, and requests confirmation from a big data server corresponding to a railway safety information DB server when the worker is not located in the work area or is not wearing equipment, and stores the history in the DB.

[0008] In addition, in the present invention, it is preferable that the safety accident prediction server detects the status of the area where the worker is working using a signal in the work area and separates it into a safe area and a dangerous area.

[0009] In addition, in the present invention, it is preferable that the safety accident prediction server issue a warning when detecting a mismatch between a track switch and a shunting signal, and transmit CCTV images of a preset area to a control officer terminal via a network.

[0010] In addition, in the present invention, it is preferable that the safety accident prediction server provides CCTV placement information located near a traffic light to a worker terminal via a network, and the worker terminal receives information about the color of the traffic light based on an image captured from the CCTV.

[0011] In addition, in the present invention, it is preferable that the safety accident prediction server accumulates behavior evaluation metadata including accidents and failures occurring in the field in a database through accident risk behavior and object metadata collection and management.

[0012] The railway worksite safety accident prediction system according to the present invention integrates data from existing safety monitoring devices with data from intelligent safety monitoring devices to monitor and control railway safety information in real time, thereby significantly reducing safety accidents. However, the effects of the present invention are not limited to the literal description, but encompass all those that a person skilled in the art might infer from the present invention.

[0013] FIG. 1 is a drawing showing a railway work site safety accident prediction system according to one embodiment of the present invention.

[0014] FIGS. 2 to 4 are diagrams showing event detection processing of a railway work site safety accident prediction system according to one embodiment of the present invention.

[0015] FIG. 5 is a diagram showing the concept of monitoring each facility of a safety accident prediction server according to one embodiment of the present invention.

[0016] FIG. 6 is a diagram showing the implementation of an object recognition model and an anomaly detection model by a safety accident prediction server according to one embodiment of the present invention.

[0017] Figure 7 is a diagram showing the main work forms of supervisors and workers by the information analysis module according to one embodiment of the present invention.

[0018] FIG. 8 is a diagram showing an object recognition model training process using an image set by an information analysis module according to one embodiment of the present invention.

[0019] FIG. 9 is a diagram showing the results of algorithm training for each image set by an information analysis module according to one embodiment of the present invention.

[0020] FIG. 10 is a diagram showing the weight names and output data capacity of an algorithm by an information analysis module according to one embodiment of the present invention.

[0021] FIG. 11 is a diagram showing an example of utilizing area division data through signal color status recognition by an information analysis module according to one embodiment of the present invention.

[0022] FIGS. 12 and 13 are drawings showing a set of images of major railway facilities for facility detection used in image recognition according to one embodiment of the present invention.

[0023] Figure 14 is a diagram showing a learning feedback process by an information analysis module according to one embodiment of the present invention.

[0024] The present invention will be described with reference to the attached drawings. In describing the present invention, descriptions of previously known functions will be omitted. Furthermore, when one component transmits data or the like to another component, this means that the data may be transmitted directly to the other component or through at least one other component.

[0025] Figure 1 is a drawing showing a railway work site safety accident prediction system according to an embodiment of the present invention.

[0026] A railway work site safety accident prediction system (1) may include multiple intelligent CCTVs (100), a network (200), a safety accident prediction server (300), a big data server (400), a control officer terminal (500), and a worker terminal (600).

[0027] The safety accident prediction server (300) includes a transmission / reception unit (310), a control unit (320), and a database (330). The control unit (320) may include an information collection module (321), an information analysis module (322), an alert provision module (323), and a detection provision module (324).

[0028] Through this, the railway work site safety accident prediction system of the present invention uses images captured by multiple intelligent CCTVs (100) to detect abnormal behavior, confirm the time and image thereof, and provide a specialized alarm function.

[0029] That is, by using multiple intelligent CCTVs (100) operated by a railway management agency, it is possible to check whether a worker is working in a designated work area and wearing the correct safety equipment (e.g., safety equipment such as a hard hat, X-band, etc.), and if the worker leaves the work area or is not wearing the equipment, necessary measures, including an alarm, can be taken.

[0030] Next, the present invention has a function for identifying the point in time when abnormal behavior, such as a worker leaving the work area or not wearing equipment, is detected in the captured video. Furthermore, in addition to detecting abnormal behavior based on the captured video, a history of such behavior can be accumulated and recorded.

[0031] And as a function specialized for railway safety, it can detect the current status of signals and track switches, detect and provide guidance on mismatch situations between track switches and shunting signals, and transmit CCTV (100) video and images of a specific area to a control officer terminal (500) as a predetermined specific mobile terminal.

[0032] To this end, the present invention can utilize artificial intelligence algorithms, various facilities, and work site photographing datasets. The present invention will examine a railway work site safety accident prediction system (1) focusing on the configuration of a control unit (320) forming a safety accident prediction server (300).

[0033] The information collection module (321) controls the transmission / reception unit (310) to receive image information acquired through multiple intelligent CCTVs (100) through a network (200) and collects and stores the information in a database (330).

[0034] The information analysis module (322) detects whether a worker is working in a designated work area through the video provided from CCTV (100), and checks matters related to on-site work or safety, including whether the worker is wearing the correct equipment.

[0035] If a worker is outside the work area or is not wearing equipment, an alarm can be generated through the alarm provision module (323).

[0036] In addition, the information analysis module (322) can confirm the time when an abnormal behavior is detected in the video. When an abnormal behavior is detected in the video captured using video analysis technology, it requests confirmation from the big data server (400), which corresponds to the railway safety information DB server, as to whether the abnormal behavior is within the permissible range, and performs history accumulation in the DB, thereby enabling real-time monitoring and increasing efficiency related to safety.

[0037] The alarm provision module (323) provides an alarm function specialized for railway site safety.

[0038] For the area where workers are working, the current status is detected by a signal in the work area, and a warning is issued by separating the safe area and the dangerous area, and a warning is issued when a mismatch is detected between the track switch and the shunting signal.

[0039] In addition, the transmission / reception unit (310) can be controlled to transmit CCTV (100) video and images of a preset area to the control manager's terminal (500) via the network (200).

[0040] FIG. 2 is a drawing for explaining event detection processing of a railway work site safety accident prediction system (1) according to an embodiment of the present invention.

[0041] The information collection module (321) of the safety accident prediction server (300) collects and manages accident risk behavior and object metadata to accumulate behavior evaluation metadata such as various accidents and failures occurring in the field in a database (330).

[0042] The information collection module (321) can accumulate risk metadata of objects detected by multiple fixed and mobile intelligent CCTVs (100) in a database (330) and provide it to a big data server (400) so that a suitable post-evaluation can be performed by classifying the form and behavior of an artificial intelligence algorithm.

[0043] The information collection module (321) performs event management, object metadata management, etc. for analysis and alerts performed by the information analysis module (322) and alert provision module (323). The information collection module (321) performs data collection, storage, and processing, and the information analysis module (322) can perform data analysis.

[0044] In addition, the service provision method through the network infrastructure by the railway work site safety accident prediction system (1) described above may be the same as the system illustrated in Fig. 3.

[0045] Meanwhile, the intelligent CCTV (100) according to the present invention is a camera for observing worker movements and railway conditions, and utilizes a horizontal field of view and focal length selected according to the height of the work environment facility, and an appropriate lens. Since most railway facilities are located outdoors, it may be desirable to apply an image sensor of an optimal size for digital image processing to enable identification of the acquired images.

[0046] In addition, a camera module equipped with an algorithm capable of basic processing, such as object recognition of the camera itself, is installed, and a rail-type camera or an object recognition camera can be used. Such a CCTV (100) camera module can provide an auto-tracking function, an intelligent video analysis function (including tampering, intrusion, loitering, and line crossing), a video analysis event area setting function, an impact detection function, and a shake correction function.

[0047] The railway work site safety accident prediction system (1) according to the present invention can be applied to various railway-related environments, such as bridge sections, tunnel sections, raised sections, and cutting sections. In addition, the information analysis module (322) can receive information by accessing the big data server (400) via the network (200) for each railway work category, and control the transceiver (310) to transmit the information to the worker terminal (600) via the network (200).

[0048] In addition, the information analysis module (322) can provide, through a preset algorithm, the determination of the wiring method and connection for power supply of the mobile camera and rail drive unit corresponding to the CCTV (100) according to the shape and classification of the image according to the CCTV (100) installation environment.

[0049] To this end, the information analysis module (322) can analyze the collected data distributed and stored in the DCS DB classified by the analysis / control program on the big data server (400) through a machine learning algorithm to determine the wiring method for power supply to the mobile camera and rail drive unit corresponding to the CCTV (100) according to the shape and classification of the image according to the CCTV (100) installation environment, and make a wiring and cabling decision.

[0050] Specifically, the machine learning algorithm used in the analysis / control program may be one of a decision tree classification algorithm, a random forest classification algorithm, and a support vector machine (SVM) classification algorithm.

[0051] The information analysis module (322) analyzes the shape and classification of the image according to the installation environment, which is the collected data distributed and stored in the DCS DB by the distributed file program, and extracts at least one piece of feature information (including a connection point and a wiring point) as a result of the analysis, and compares the extracted feature information with the feature information and the information on which the shape and classification of the image according to each installation environment are updated using at least one or more of a plurality of machine learning algorithms, and analyzes the decision on the connection and wiring method as a result of the learning.

[0052] In addition, the information analysis module (322) can apply an ensemble structure composed of multiple complementary machine learning algorithms to improve the accuracy of the results.

[0053] Figure 4 is a drawing showing the concept of monitoring each facility of the safety accident prediction server (300) among the railway work site safety accident prediction systems (1) according to the present invention.

[0054] The detection provision module (324) of the safety accident prediction server (300) collects operation data for each work facility, such as railway vehicles, track signal equipment, power equipment, facilities, and safety monitoring devices, for predicting safety accidents at railway work sites, through each safety detection device I / F and stores the data in a database (330).

[0055] And, in order to perform surveillance on a target by detecting safety accidents and predicting data and generating object and event metadata for the stored data, the information analysis module (322) and the alarm provision module (323) can be controlled and driven.

[0056] Figure 5 is a drawing showing that a safety accident prediction server (300) can be constructed as a server capable of multiple streaming inputs.

[0057] The safety accident prediction server (300) can provide a stream (RTSP) input function transmitted from a CCTV (surveillance camera), a screen (frame buffer) capture function of an installed surveillance system, and an artificial intelligence image recognition and continuous monitoring function of received streaming video.

[0058] FIG. 6 is a drawing for explaining the implementation of an object recognition model and an anomaly detection model by a safety accident prediction server (300) according to an embodiment of the present invention, and shows an example of event detection due to not wearing a safety helmet and falling.

[0059] To implement an object recognition model and anomaly detection model, the information analysis module (322) performs image learning using an image labeling tool.

[0060] Learning data is stored in at least one of a database (330) and a big data server (400), and learning is performed by behavior pattern based on the coordinate information of the bounding box for each object and the abnormal behavior category information, and the learning data is stored in at least one of the database (330) and a big data server (400).

[0061] The information analysis module (322) can perform object recognition and abnormality detection by comparing detection pattern information and similar pattern information for each object stored in at least one of a database (330) and a big data server (400) with detection pattern information and similar pattern information for each object corresponding to abnormal behavior, including sudden abnormalities, falling down due to hitting an abnormal object, abnormal movement, and standing still for a certain period of time in an abnormal posture.

[0062] Here, the similar pattern information may be the reverse of each detection pattern, the tilt at a preset angle, and the changed information on saturation, brightness, and color of the image.

[0063] In addition, the information analysis module (322) can perform error correction for object overlapping by expanding a two-dimensional video image frame into a three-dimensional space and correcting human body posture estimation that is distorted by planar interpretation, which is a limitation of two-dimensional images.

[0064] In addition, the posture estimation model can detect abnormal signs by estimating the duration of abnormal postures such as lying down and prone in addition to individual movements by distinguishing between the upper and lower body.

[0065] The information analysis module (322) can perform customization of an object through an object recognition model algorithm, and can perform training of a deep learning model by utilizing primary training data corresponding to an open dataset {e.g., Classification (ImageNET, CIFAR-10), Object Detection (Pascal VOC, COCO), Action Recognition (NTU RGB+D, UCF101), etc.} according to a learning model for each of the preset learning models for the customized object, i.e., Classification (ResNET, DenseNET), Object Detection (YOLO, Faster RCNN), and Action Recognition (ST-GCN, LSTM).

[0066] Afterwards, in order to refine the computer vision object recognition model learning data, the information analysis module (322) secures a certain level of reliability through learning, verification, and testing, and the secondary learning data produced through the verification and correction procedures can be reused as learning data for the deep learning model, thereby improving the reliability of the authoring tool.

[0067] In addition, the information analysis module (322) can perform automatic labeling on user datasets by providing checkpoints in which deep learning networks capable of object detection, such as YOLO and Mask-RCNN, are pre-trained using open datasets such as COCO and Pascal VOC.

[0068] The information analysis module (322) can increase the accuracy of the location of the bounding box through verification of the labeling results of the deep learning network, or can build a user dataset in at least one of the database (330) and the big data server (400) by additionally labeling a new class that was not defined in the open dataset.

[0069] Additionally, the information analysis module (322) can customize the action recognition network to fit the user dataset by repeating the process of retraining the action recognition network using the constructed dataset.

[0070] The detection provision module (324) can control the execution of a railway object recognition model by the information analysis module (322) and then provide an event notification function by the alarm provision module (323).

[0071] To this end, the information analysis module (322) can standardize worker classification, work type, and pattern using images provided from CCTV (100) or data on a big data server (400) and store the data in at least one of a database (330) and a big data server (400).

[0072] In this case, the information analysis module (322) can standardize the work area and work pattern by considering the interaction between workers, train monitors, and controllers on the data through object recognition, thereby increasing the reliability of the data during learning.

[0073] Figure 7 is a drawing showing an example of storing the main work types of supervisors and workers in at least one of a database (330) and a big data server (400) by an information analysis module (322).

[0074] According to the standardized work area and work pattern, the work personnel (workers) and the supervisors (supervisors) are distinguished, and the transmission / reception unit (310) can be controlled to request the worker terminal (600) through the network (200) to perform line and facility work in the work area within the designated area and time limit.

[0075] The information analysis module (322) can perform railway facility image information dataset and event labeling using data stored by the information collection module (321). To this end, the information analysis module (322) can request the information collection module (321) to secure datasets from railway-related organizations and to secure its own dataset, and can also perform object labeling tasks for work area labeling and worker behavior pattern prediction according to the facility status of the requested data.

[0076] Figure 8 is a diagram showing the process of training an object recognition model using an image set by an information analysis module (322), and is a diagram showing various railway facility image training sets, situations to be recognized, and shape labeling.

[0077] Fig. 9 is a diagram showing the results of algorithm training for each image set, Fig. 10 is a diagram showing the weight names and output data capacity of the YOLOv5 algorithm by the information analysis module for result production, and Fig. 11 is a diagram showing an example of utilizing area division data through signal color status recognition by the information analysis module.

[0078] Referring to Fig. 11, since the direction of travel of a train changes depending on the color or shape of the signal, the information analysis module (322) provides CCTV (100) arrangement information located near the signal to the worker terminal (600) via the network (200), and image sets for subsequent worker search and warning transmission and object detection are provided according to the color of the signal to perform status detection.

[0079] In addition, the worker terminal (600) can perform status detection by receiving information about the color of the signal based on the image captured from the CCTV (100).

[0080] In addition, the information analysis module (322) can perform a worker detection function around a railway using a worker-recognizable dataset to perform detection using a worker protective gear dataset when applying an object recognition and multi-object recognition model, and can implement a separation model of multiple objects within a class using a Mask R-CNN model that takes into account clustering between workers and between workers and train monitors.

[0081] Figure 11 is a drawing showing an example of a worker image set, and Figure 12 is a drawing showing a main railway facility image set for facility detection used in image recognition.

[0082] The information analysis module (322) provides a video and streaming recognition function for railway facilities, and can use the DB secured by the information collection module (321) for a dataset of 8 TB of 4K video with a preset amount of time and a dataset of 60,000 railway facilities to enhance the function of identifying the status and location of railway facilities from videos and images using a dataset of facilities around the railway.

[0083] Accordingly, the information analysis module (322) can perform the function of recognizing railway facilities and storing location data as shown in Fig. 12.

[0084] In addition, the information analysis module (322) performs test bed construction and performance test analysis, and can perform parameter tuning to increase the accuracy of the prediction model by performing pre-learning using already secured data and data on abnormal behavior for each situation, thereby converting it into a database. In the present invention, converting to a database may mean stacking data on at least one of the database (330) and the big data server (400).

[0085] The information analysis module (322) learns data based on the GPU Server based on the data labeling results, and can improve accuracy through transfer learning. It can also analyze and learn parameters that minimize overfitting based on the loss results during data learning. Fig. 14 illustrates the additional image learning feedback process for the object recognition model by the information analysis module (322).

[0086] The present invention can also be implemented as computer-readable code on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system.

[0087] The present invention is not limited to the disclosed embodiments and the accompanying drawings, and various modifications may be made by those skilled in the art without departing from the technical spirit of the present invention. Furthermore, the technical concepts described in the embodiments of the present invention may be implemented independently, or two or more may be combined.

[0088] The present invention relates to a safety accident prediction system at a railway work site and has industrial applicability.

Claims

1. In a railway work site safety accident prediction system (1) including an intelligent CCTV (100), a network (200), and a safety accident prediction server (300), The above safety accident prediction server (300) is The transmission / reception unit (310) is controlled to receive image information obtained through an intelligent CCTV (100) through a network (200) and collect the image information on the database (330), thereby checking whether the worker is working in the work area and whether the worker is wearing equipment. A railway work site safety accident prediction system characterized in that, when a worker is not located in a work area or is not wearing equipment, a confirmation request is made to a big data server (400) corresponding to a railway safety information DB server, and the history is stored in the DB.

2. In claim 1, The above safety accident prediction server (300) is A railway work site safety accident prediction system characterized by detecting the status of the area where workers are working using a signal in the work area and dividing it into a safe area and a dangerous area.

3. In claim 2, The above safety accident prediction server (300) is If a mismatch between the track switch and the shunting signal is detected, a warning is issued. A railway work site safety accident prediction system characterized by transmitting CCTV (100) images of a preset area to a control officer terminal (500) via a network (200).

4. In claim 3, The above safety accident prediction server (300) is CCTV (100) placement information located near the signal is provided to the worker terminal (600) through the network (200), A railway work site safety accident prediction system characterized in that the worker terminal (600) receives information about the color of the signal based on the image captured from the CCTV (100).

5. In claim 3, The above safety accident prediction server (300) is A railway work site safety accident prediction system characterized by accumulating behavior evaluation metadata including accidents and failures occurring on site in a database (330) through the collection and management of accident risk behavior and object metadata.

Images