A safety identification and assessment method and system for operation behaviors in coal mine scenarios

Abstract

The present invention relates to a method for safety identification and assessment of operation behaviors in a coal mine scenario. Based on behavior recognition devices provided at preset designated positions in the coal mine scenario for detecting preset designated types of non-safe behaviors, and image capture devices for capturing images of each designated type of non-safe behavior, through the triggering of non-safe behavior recognition, based on the corresponding captured images, perform person recognition of the operators of non-safe behaviors, and then combine with a question bank composed of questions preset for each mine operation label, select corresponding questions to construct an assessment paper, and push it to the corresponding operators for assessment, so as to improve the safety operation skills of coal mine operators; and design a corresponding system, based on the designed method, construct a network architecture with corresponding functions, which can efficiently realize the safety identification and assessment training of operation behaviors in the coal mine scenario.

Description

Technical Field

[0001] The present invention relates to a method and system for safety identification and assessment of operation behaviors in coal mine scenarios, belonging to the technical field of safety management of coal mine enterprises. Background Art

[0002] Traditional coal mine video monitoring systems usually only have a recording function. The system design is simple and the functions are single, only meeting the needs of monitoring and storage, and not achieving the purpose of early warning analysis. When an abnormal situation occurs, the monitoring personnel can only view it through video recordings, and can only manually issue various penalty forms to punish the unsafe behaviors that have occurred. However, due to the lack of corresponding safety knowledge training, the staff do not know where they are wrong, and it is very easy to cause unsafe behaviors again.

[0003] With the rapid development of technologies such as the Internet, artificial intelligence, big data, and image recognition, intelligent video recognition technology has gradually received attention and emphasis, bringing a subversive change to traditional coal mine safety management, and also making the management of various unsafe behaviors in coal mine scenarios more user-friendly. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for safety identification and assessment of operation behaviors in coal mine scenarios. Based on automated intelligent monitoring, it identifies non-safe behaviors in coal mine scenarios, and combines with a question bank to push assessment papers to relevant operators to achieve safety assessment.

[0005] The present invention adopts the following technical solutions to solve the above technical problems: The present invention designs a method for safety identification and assessment of operation behaviors in coal mine scenarios, based on behavior recognition devices respectively arranged at preset designated positions in the coal mine scenario for detecting preset designated types of non-safe behaviors, and image capture devices for capturing images of each designated type of non-safe behavior. Each designated position respectively executes the following steps:

[0006] Step A. The behavior recognition device detects whether a designated type of non-safe behavior appears at its designated position. If so, enter Step B; otherwise, the behavior recognition device continues to detect.

[0007] Step B. The image capture device obtains a facial image of the operator performing the designated type of non-safe behavior, and then enters Step C.

[0008] Step C. According to the facial image of the operator of the designated type of non-safe behavior, apply a trained facial recognition model that takes the facial image as the input and the identity information of the operator corresponding to the facial image as the output to obtain the identity information of the operator of the designated type of non-safe behavior, and then enter Step D.

[0009] Step D. According to the specified type of non-safe behavior, combined with the corresponding relationship between each preset specified type of non-safe behavior and each mine operation label, randomly select from the question bank composed of each question preset to correspond to each mine operation label, a preset test quantity of questions corresponding to the specified type of non-safe behavior to form an assessment paper, and then enter Step E;

[0010] Step E. According to the identity information of the operator of the specified type of non-safe behavior, push the assessment paper to the operator of the specified type of non-safe behavior to perform the assessment.

[0011] As a preferred technical solution of the present invention: during the process of pushing the test paper to the operator of the specified type of non-safe behavior to perform the assessment in Step E, if the assessment is passed, it means that the operator of the specified type of non-safe behavior has passed the assessment; if the assessment fails, return to Step D.

[0012] As a preferred technical solution of the present invention: the face recognition model in Step C is obtained according to the following Steps C1 to C2;

[0013] Step C1. For each preset specified operator, collect the face images of each operator corresponding to each preset face posture, and construct the corresponding relationship between each face image and the corresponding operator identity information, and then enter Step C2;

[0014] Step C2. According to each face image and the operator identity information corresponding to each face image, using the face image as the input and the operator identity information corresponding to the face image as the output, train the preset classification network to obtain the face recognition model.

[0015] As a preferred technical solution of the present invention: the preset face postures in Step C1 include preset face angles, as well as covering the face, cleaning the face, wearing a safety helmet, and not wearing a safety helmet.

[0016] As a preferred technical solution of the present invention: each specified type of non-safe behavior includes prohibiting entry into a preset specified area. Then, the behavior recognition device for detecting the prohibition of entry into the preset specified area is an image capture device whose image capture area covers the preset specified area, and whether this type of non-safe behavior occurs is detected by comparing whether there is an image of an operator in the image captured by the image capture device.

[0017] As a preferred technical solution of the present invention: each of the specified types of non-safe behaviors includes non-safe behaviors based on preset alarm for each preset sensing data corresponding to each specified object to be operated. Then, the behavior recognition device for this specified type of non-safe behavior is each of the sensors provided on each of the specified objects to be operated, which is used to detect each of the sensing data corresponding to each of the specified objects to be operated. By comparing whether each of the obtained sensing data exceeds the preset corresponding threshold range respectively, it is detected whether this type of non-safe behavior occurs.

[0018] As a preferred technical solution of the present invention: each of the mine operation labels corresponding to each of the questions in the question bank includes coal mining operation type labels, tunneling operation type labels, blasting operation type labels, hoisting, transportation and electromechanical equipment operation type labels, ventilation and prevention operation type labels, auxiliary transportation operation type labels, geodetic survey operation type labels, monitoring and control operation type labels, comprehensive operation type labels, general operation type labels.

[0019] Correspondingly, the technical problem further solved by the present invention is to provide a system for a coal mine scenario operation behavior safety recognition and assessment method. Based on the designed method, a network architecture with corresponding functions is constructed, which can efficiently realize the safety recognition and assessment of coal mine scenario operation behaviors.

[0020] The present invention adopts the following technical solutions to solve the above technical problems: The present invention designs a system for a coal mine scenario operation behavior safety recognition and assessment method, including an access layer, a network layer, a data layer, an application layer, a presentation layer and a user layer. Among them, the access layer is used to realize the video monitoring access of each image capture device in the coal mine scenario; the network layer is used to provide a channel for the transmission of the data received by the access layer; the data layer is used to parse and identify the data transmitted through the network layer, that is, to execute the steps A to C; the application layer is used to realize the construction of the question bank and execute the step D; the presentation layer is used for the display of the system operation; the user layer is used for the question assessment of coal mine operators, that is, to execute the step E.

[0021] For the coal mine scenario operation behavior safety recognition and assessment method and system of the present invention, compared with the prior art by adopting the above technical solutions, it has the following technical effects:

[0022] (1) The safety identification and assessment method for coal mine scene operation behaviors designed by the present invention is based on behavior recognition devices respectively set at preset designated positions in the coal mine scene for detecting preset designated types of non-safe behaviors, and image capture devices for capturing images of each designated type of non-safe behavior. Through the triggering of non-safe behavior recognition, based on the corresponding captured images, person recognition of non-safe behavior operators is performed, and then combined with a question bank composed of questions preset for each mine operation label respectively, corresponding questions are selected to construct an assessment paper and pushed to the corresponding operators for assessment, so as to improve the safe operation skills of coal mine operators; and a corresponding system is designed. Based on the designed method, a network architecture with corresponding functions is constructed, which can efficiently realize the safety identification and assessment of coal mine scene operation behaviors.

[0023] (2) The safety identification and assessment method and system for coal mine scene operation behaviors designed by the present invention associate video recognition of personnel's unsafe behaviors with training. By applying video recognition technology, cameras replace people's eyes and computers replace people's brains to achieve 24-hour real-time detection, identify personnel's unsafe behaviors, and solve the problems of insufficient personnel monitoring and safety management blind spots; automatically generate a training question bank according to the severity and type classification of personnel's unsafe behaviors for users to study, simulate, and take exams, realizing precise and effective training for users, effectively improving the efficiency and accuracy of personnel training, and thus ensuring the smooth progress of coal mine safety production. Brief Description of the Drawings

[0024] Figure 1 It is a system technical architecture diagram of the safety identification and assessment method for coal mine scene operation behaviors designed by the present invention;

[0025] Figure 2 It is a structure diagram designed by the present invention;

[0026] Figure 3 It is a schematic diagram of question bank entry classification and marking in the design of the present invention;

[0027] Figure 4 It is a schematic diagram of the system process in the design of the present invention;

[0028] Figure 5 It is a flowchart for identifying operators' unsafe behaviors in the design of the present invention. Detailed Embodiment

[0029] The following further elaborates on the detailed embodiment of the present invention in conjunction with the attached drawings of the specification.

[0030] The present invention designs a method for safety identification and assessment of operation behaviors in coal mine scenarios. Based on behavior recognition devices respectively installed at preset designated positions in coal mine scenarios for detecting preset various types of non-safe behaviors, and image capture devices for capturing images of various types of non-safe behaviors, each designated position executes according to the following steps.

[0031] Step A. The behavior recognition device detects whether a specified type of non-safe behavior appears at its designated position. If so, go to Step B; otherwise, the behavior recognition device continues to detect.

[0032] Step B. The image capture device obtains a facial image of the operator performing the specified type of non-safe behavior, and then goes to Step C.

[0033] Step C. According to the facial image of the operator of the specified type of non-safe behavior, apply a trained facial recognition model that takes the facial image as input and outputs the identity information of the operator corresponding to the facial image to obtain the identity information of the operator of the specified type of non-safe behavior, and then go to Step D.

[0034] In actual application, the facial recognition model in the above Step C is obtained according to the following Steps C1 to C2.

[0035] Step C1. For each preset designated operator, collect facial images of each operator corresponding to preset various facial postures, and establish the corresponding relationship between each facial image and the identity information of the corresponding operator, and then go to Step C2; where the preset various facial postures include preset various facial angles, as well as covering the face, clean face, wearing a safety helmet, and not wearing a safety helmet.

[0036] Step C2. According to each facial image and the identity information of the operator corresponding to each facial image, use the facial image as input and the identity information of the operator corresponding to the facial image as output to train a preset classification network to obtain a facial recognition model.

[0037] Such deep learning-based detection algorithms, computer vision object detection algorithms, and intelligent recognition algorithm packages are used to perform real-time moving object detection and recognition. The computer analyzes and extracts specific behaviors of personnel within the field of view, and through technologies such as computer vision, artificial intelligence, and deep learning, conducts a series of personnel behavior detections such as safety helmet detection, work uniform detection, phone call detection, seat belt detection, personnel leaving their posts, sleeping on the job, entering restricted areas, and chasing the monkey car. In the case of insufficient underground lighting, interference from miner's lamps and safety helmets, and unstable facial features, illumination normalization of face images under strong conditions is adopted. On this basis, personnel features are extracted and then recognition is carried out; specifically, convolutional neural networks and video compensation technologies are used to optimize the algorithm, so as to achieve the best effect of the design purpose of the present invention.

[0038] Step D. According to this specified type of non-safe behavior, combined with the corresponding relationship between each preset specified type of non-safe behavior and each mine operation label, randomly select from the question bank composed of each question corresponding to each mine operation label preset respectively, a preset test quantity of questions corresponding to this specified type of non-safe behavior to form an assessment paper, and then enter Step E.

[0039] In the application, such as Figure 3 shown, the mine operation labels corresponding to each question in the question bank include coal mining operation type labels, tunneling operation type labels, blasting operation type labels, hoisting and transportation electromechanical equipment operation type labels, ventilation and prevention operation type labels, auxiliary transportation operation type labels, surveying operation type labels, monitoring and control operation type labels, comprehensive operation type labels, and general operation type labels.

[0040] Step E. According to the identity information of the operator of this specified type of non-safe behavior, push the assessment paper to the operator of this specified type of non-safe behavior to perform the assessment. If the assessment is passed, it means that the operator of this specified type of non-safe behavior has passed the assessment; if the assessment fails, return to Step D.

[0041] In practical applications, regarding the behavior recognition devices respectively set at each specified position for detecting each preset specified type of non-safe behavior, for example, if each specified type of non-safe behavior includes prohibiting entry into a preset specified area, the behavior recognition device for detecting the prohibition of entry into the preset specified area is an image capture device whose image capture area covers the preset specified area, and by comparing whether there is an image of an operator in the image captured by this image capture device, it is detected whether this type of non-safe behavior occurs.

[0042] Alternatively, if each specified type of unsafe behavior includes unsafe behaviors that alarm based on the preset sensing data corresponding to each preset specified object to be operated, the behavior recognition device for this specified type of unsafe behavior is the respective sensors provided on each of the specified objects to be operated, which are used to detect the respective sensing data corresponding to each of the specified objects to be operated, and by comparing whether each of the obtained sensing data exceeds the preset corresponding threshold range, it is detected whether this type of unsafe behavior occurs.

[0043] Based on the above-designed safety recognition and assessment method for coal mine scenario operation behaviors, the present invention further designs a corresponding system, as Figure 1 shown, including an access layer, a network layer, a data layer, an application layer, a presentation layer, and a user layer. Among them, the access layer is used to realize the video monitoring access of each image capture device in the coal mine scenario, collect the coal mine video data required for system construction, and provide data support; the network layer is used to provide a channel for the transmission of the data received by the access layer, mainly involving the Internet, video private networks, etc.; the system accesses the ring network switch nearby through an Ethernet interface or RS485 / 232, etc.; based on security and confidentiality requirements, data transmission security is ensured between each network through a firewall and a security boundary; the data layer is used to parse and identify the data transmitted through the network layer, that is, to execute the above steps A to C, perform intelligent parsing on the video image data based on the big data platform, comprehensively analyze various real-time personnel image data, identify and classify the types and severities based on personnel's unsafe behaviors, and provide high-value data through statistical analysis; the application layer is used to realize the construction of the above test question bank and execute the above step D, specifically realizing functions such as real-time monitoring of personnel behaviors, input, query, editing, and automatic generation of training question banks; the presentation layer is used for the display of system operation; the user layer is used for the test question assessment of coal mine operators, that is, to execute the above step E.

[0044] In the application, the decoupling of the algorithm package and the video stream is realized, an algorithm resource scheduling layer is added, a large database of computing resources for each unsafe behavior event is formed according to the user's configuration options, and then the data is classified and learned, so as to meet the requirements of classifying the severity and type of each unsafe behavior. Then, online distributed computing is carried out through networking, and computing resources (GPU, RAM) are dynamically allocated for online video detection according to the severity and type of events formed by pre-detection, so as to optimize parallel computing, prevent the occurrence of computing congestion and resource exhaustion, and achieve the goal of fast real-time response.

[0045] Specifically based on the above network architecture, the designed system includes video recognition management, training plan management, training material management, simulation question bank management, training examination system, and training record management.

[0046] The system adopts the B / S mode and consists of two major parts: the front-end user training part and the back-end system management part. There is no need to install a client, and authorized users can log in to the system through a browser.

[0047] Front-end user training part: The user training application end, including a data learning module, a simulation practice module, a training examination module, and an examination result query module.

[0048] Back-end management part: The administrator application end, including many functions such as video recognition management, training plan management, training material management, simulation question bank management, training test paper management, examination status management, user information management, and administrator permission management.

[0049] As Figure 4 shown, first, the system administrator uniformly enters the question bank and registers each question with labels according to the categories of unsafe behaviors and risk classifications. Personnel information is collected in advance to establish a personnel image database including states such as masked faces, unmasked faces, wearing safety helmets, and not wearing safety helmets. Based on AI algorithms, deep learning detection algorithms, computer vision target detection algorithms, etc., video recognition of personnel's unsafe behaviors is realized. Secondly, the system automatically generates corresponding training examination questions according to the severity and type of the unsafe behaviors recognized by the video and pushes the examination information. Finally, personnel take the examination through the received examination push messages. If the examination is passed, the training ends; if the examination is not passed, the examination information is pushed again, thus realizing closed-loop and precise training management.

[0050] As Figure 5 shown, for the identification of personnel's unsafe behaviors in the coal mine intelligent video recognition and precise training system, first, personnel information is collected to establish a personnel image database including states such as masked faces, unmasked faces, wearing safety helmets, and not wearing safety helmets. Secondly, based on AI algorithms, deep learning detection algorithms, and computer vision target detection algorithms, the comparison between the portrait and personnel behaviors is carried out to identify the unsafe behaviors of personnel. Finally, the comparison results are displayed / queried for the severity and type of personnel's unsafe behaviors.

[0051] The method and system for identifying and assessing the safety of coal mine scene operation behaviors designed by the present invention are based on the analysis of the current coal mine safety management status and apply intelligent video recognition technology. In actual implementation, it is specifically constructed based on the Windows operating system, the SQL Server database platform, and the J2EE platform, associating video recognition of personnel's unsafe behaviors with training.

[0052] The intelligent video recognition involved in the present invention can automatically perform real-time moving target detection, recognition, and tracking on video images. Through technologies such as artificial intelligence and deep learning, it conducts real-time structured analysis on videos, understands the image scenes, and realizes real-time detection of personnel's unsafe behaviors. The system can form corresponding question banks for pushing according to the severity and type of the detected unsafe behaviors to achieve precise training. The system adopts the B / S mode and consists of two major parts: the front-end user training part and the back-end system management part. It does not require installing a client, and authorized users can log in to the system through a browser. The system adopts an open structure and has good compatibility. As long as a database interface is provided, it can be docked with various application software. Combining with user requirements, the system can implement the management of multiple question banks for learning, simulation, and examination; users can consult historical training and examination scores and test papers; score record management supports archiving the scores of examinees and managing historical data to meet the learning, training, examination, and record management in a network environment. This system realizes the paperless, networked, and automated computer online learning, training, and examination in enterprises.

[0053] The entire design is based on behavior recognition devices respectively set at preset specified positions in the coal mine scene for detecting preset specified types of unsafe behaviors, and image capture devices for capturing images of each specified type of unsafe behavior. Through the triggering of unsafe behavior recognition, based on the corresponding captured images, it performs person recognition on the operators of unsafe behaviors, and then combines with the question bank composed of each question preset corresponding to each mine operation label, selects corresponding questions to construct an assessment test paper, and pushes it to the corresponding operator for assessment, improving the safe operation skills of coal mine operators; and designs a corresponding system, based on the designed method, constructs a network architecture with corresponding functions, and can efficiently realize the safe recognition and assessment of coal mine scene operation behaviors.

[0054] In the application, the video recognition of personnel's unsafe behaviors is associated with training. Applying video recognition technology, the camera replaces people's eyes, and the computer replaces people's brains to achieve 24-hour real-time detection, recognize personnel's unsafe behaviors, and solve the problems of insufficient personnel monitoring and safety management blind spots; automatically generate training question banks according to the severity and type classification of personnel's unsafe behaviors for users to learn, simulate, and take examinations, realizing precise and effective training for users, effectively improving the efficiency and accuracy of personnel training, and thus ensuring the smooth progress of coal mine safety production.

[0055] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art, various changes can be made without departing from the gist of the present invention.

Claims

1. A system for a safety identification and assessment method of operation behaviors in a coal mine scenario, characterized in that: It includes an access layer, a network layer, a data layer, an application layer, a presentation layer and a user layer. The method is based on behavior recognition devices respectively set at preset specified positions in the coal mine scenario for detecting preset specified types of non-safe behaviors, and image capture devices for capturing images of each specified type of non-safe behavior. Each specified position performs the following steps: Step A. The behavior recognition device detects whether a specified type of non-safe behavior occurs at its specified position. If yes, go to Step B; otherwise, the behavior recognition device continues to detect. Step B. The image capture device obtains a facial image of the operator performing the specified type of non-safe behavior, and then goes to Step C. Step C. Based on the facial image of the operator of the specified type of non-safe behavior, apply a facial recognition model that has been trained with the facial image as the input and the identity information of the operator corresponding to the facial image as the output to obtain the identity information of the operator of the specified type of non-safe behavior, and then go to Step D. Step D. According to the specified type of non-safe behavior, combined with the corresponding relationship between the preset specified types of non-safe behaviors and each mine operation label, randomly select a preset test quantity of questions corresponding to the specified type of non-safe behavior from the question bank composed of questions respectively corresponding to each mine operation label to form an assessment paper, and then go to Step E. Step E. According to the identity information of the operator of the specified type of non-safe behavior, push the assessment paper to the operator of the specified type of non-safe behavior to perform the assessment. Each specified type of non-safe behavior includes non-safe behaviors based on preset alarm of each sensing data corresponding to preset specified objects to be operated. Then, the behavior recognition device for the specified type of non-safe behavior is each of these sensors provided on each of the specified objects to be operated, which is used to detect each of the sensing data corresponding to each of the specified objects to be operated, and detect whether this type of non-safe behavior occurs by comparing whether each of the obtained sensing data exceeds the preset corresponding threshold range. The access layer is used to realize the video monitoring access of each image capture device in the coal mine scenario; the network layer is used to provide a channel for the transmission of the data received by the access layer. The data layer is used to parse and identify the data transmitted through the network layer, that is, to execute Steps A to C; the application layer is used to realize the construction of the question bank and execute Step D; the presentation layer is used for the display of the system operation; the user layer is used for the question assessment of coal mine operators, that is, to execute Step E.

2. The system for a safety identification and assessment method of operation behaviors in a coal mine scenario according to Claim 1, characterized in that: During the process of pushing the test paper to the operator of the specified type of non-safe behavior to perform the assessment in Step E, if the assessment is passed, it means that the operator of the specified type of non-safe behavior passes the assessment; if the assessment fails, return to Step D.

3. The system for a safety identification and assessment method of operation behaviors in a coal mine scenario according to Claim 1, characterized in that: The face recognition model in step C is obtained according to the following steps C1 to C2; Step C1. For each preset specified operator, collect face images of each operator corresponding to each preset face posture, and establish the corresponding relationship between each face image and the corresponding operator identity information, and then enter step C2; Step C2. According to each face image and the operator identity information corresponding to each face image, use the face image as the input and the operator identity information corresponding to the face image as the output to train the preset classification network to obtain the face recognition model.

4. The system of a coal mine scene operation behavior safety recognition and assessment method according to claim 3, characterized in that: Each preset face posture in step C1 includes each preset face angle, as well as covering the face, clean face, wearing a safety helmet, and not wearing a safety helmet.

5. The system of a coal mine scene operation behavior safety recognition and assessment method according to claim 1, characterized in that: Each mine operation label corresponding to each question in the question bank includes a coal mining operation label, a tunneling operation label, a blasting operation label, a hoisting, transportation and electromechanical equipment operation label, a ventilation and prevention operation label, an auxiliary transportation operation label, a geological survey operation label, a monitoring and control operation label, a comprehensive operation label, and a general operation label.

Images