Coal mine underground safety detection method and system based on face image recognition

Abstract

The invention discloses a coal mine underground safety detection method and system based on face image recognition, and relates to the technical field of coal mine safety detection. The method comprises the following specific implementation steps: through a video target recognition technology and a self-adaptive face recognition method, recognizing the identity of a miner and the wearing condition of safety equipment in each working area, and obtaining a miner number and a first danger index; secondly, according to the motion trail matching and behavior analysis technology, analyzing the danger degree of various behaviors of the miner to obtain a second danger index; then, according to the environmental parameters, evaluating the danger level of the miner working environment, and calculating a third danger index; finally, in combination with the danger index of the miner and the danger degree of the working area, the working safety condition of the miner is comprehensively evaluated, the comprehensive danger index is obtained, danger grading is carried out, early warning information can be sent out in time, the safety risk of underground operation of the coal mine can be effectively reduced, the safety of operators is ensured, and safety accidents are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mine safety detection, and in particular to a coal mine underground safety detection method and system based on face image recognition. Background Art

[0002] The working space in coal mines is narrow, and safety accidents such as gas explosions, coal dust explosions, mine water damage and roof falls are prone to occur, which pose a serious threat to the life safety of workers and mine production. With the continuous increase in the depth and scale of coal mining, such safety problems are becoming more frequent. Therefore, underground safety detection technology in coal mines is particularly important. This technology can timely detect potential safety hazards and provide early warning information by real-time monitoring of underground environmental parameters such as gas concentration, carbon monoxide content, temperature, humidity and wind speed. It not only provides important technical support for mine safety management, but also effectively improves emergency response capabilities and accident handling efficiency, and significantly reduces the probability of safety accidents.

[0003] At present, the existing technology for safety inspection of miners is limited to the inspection of the miners' own physical fitness and dangerous behaviors. It ignores the interference of safety equipment (such as safety helmets and masks) on the identity recognition of miners, and also does not consider the impact of work fatigue and the danger level of the working area on the life safety of miners. As a result, the safety inspection results are difficult to fully reflect the safety risks of mining operations, and thus it is impossible to track the safety status of each miner in a timely manner, increasing the risk of operational errors and accidents.

[0004] Therefore, a coal mine underground safety detection method and system based on face image recognition is proposed. Summary of the invention

[0005] The present invention provides a coal mine underground safety detection method and system based on face image recognition, which is used for comprehensive detection of the safety of personnel in coal mines. In order to solve the problems existing in the prior art, the present invention uses video target recognition technology and adaptive face recognition method to identify the identity of miners in each working area and the wearing of safety equipment, and obtains the miner number and the first danger index; secondly, according to the motion trajectory matching and behavior analysis technology, the danger level of various behaviors of miners is analyzed to obtain the second danger index; then, according to the environmental parameters, the danger level of the miner's working environment is evaluated, and the third danger index is calculated; finally, the miner's work safety situation is comprehensively evaluated by combining the miner's danger index and the danger level of the working area, and a comprehensive danger index is obtained and the danger level is divided, so as to issue early warning information in time, which can effectively reduce the safety risk of underground coal mine operations, ensure the safety of operators, and reduce the occurrence of safety accidents.

[0006] A coal mine underground safety detection method based on face image recognition, the specific implementation steps include:

[0007] According to the monitoring cameras, inspection robots and sensors in each working area of ​​the target coal mine, real-time monitoring video data, inspection video data and environmental parameters are collected; according to the face recognition equipment at the entrance of the target coal mine, the first facial image and identity information data of the underground miners are obtained;

[0008] Cleaning the monitoring video data and the inspection video data of each working area to obtain first video data and second video data;

[0009] By means of video target recognition technology, all personnel video data in the second video data are obtained; by means of a target recognition method, the wearing conditions of safety equipment in the personnel video data are classified; by means of an adaptive face recognition method, the classified video data of miners are identified to obtain a miner number and a first danger index;

[0010] Matching the identity of the person in the first video data by a motion trajectory matching method; performing behavior analysis on the matched first video data and the second video data to obtain a second danger index;

[0011] According to the environmental parameters, an environmental assessment is performed on different working areas to obtain a third hazard index;

[0012] The first hazard index, the second hazard index and the third hazard index are combined to comprehensively evaluate the working safety of the miner to obtain a comprehensive hazard index; and the miners are divided into hazard levels according to the comprehensive hazard indexes of different miners.

[0013] Preferably, the environmental parameters include gas concentration index, carbon monoxide concentration, dust index, temperature index, humidity index, noise index and vibration index; the first facial image represents the facial image of the underground miner of the target coal mine before work; the identity information data includes the miner number and facial image set.

[0014] Preferably, the specific implementation process of the adaptive face recognition method includes:

[0015] The classified video data of the miners is obtained, including first category videos, second category videos, third category videos and fourth category videos; facial recognition is performed on the first category videos, second category videos and third category videos in combination with the multi-region face recognition unit and the identity information data to obtain the miner number; the miner number of the person in the fourth category video is obtained according to the first facial image and eyebrow parameter estimation unit; different danger weights are set for different categories of videos, and the first danger index of different miners is obtained according to the safety equipment worn by the miners.

[0016] Preferably, the process of determining the miner number by the multi-region face recognition unit includes:

[0017] In the training phase, a facial data set of miners is obtained according to the identity information data; each image in the facial data set is divided into m×n image blocks of equal size through a data enhancement layer, and g types of occlusion are performed on k of the image blocks to obtain a facial expansion data set; the facial expansion data set is input into a feature extraction layer to obtain facial features; texture feature weights of the facial features are obtained through a feature enhancement layer, and facial enhancement features are obtained through feature reconstruction; the facial enhancement features are input into a classifier to obtain a facial recognition result; the facial recognition result is supervised according to an angle boundary loss function and a cross entropy loss function to obtain the trained multi-region face recognition unit;

[0018] In the inference stage, the first category video, the second category video and the third category video are input into the trained multi-region face recognition unit to determine the identity of the miner; and the miner number is obtained by combining the identity information data.

[0019] Preferably, the implementation process of obtaining the miner number according to the first facial image and the eyebrow parameter estimation unit includes:

[0020] Through the facial key point detection algorithm, the frontal eyebrow and eye key points of the first facial image are obtained; according to the coordinates of the frontal eyebrow and eye key points, the first eyebrow and eye parameters are obtained, including eye distance, eye aspect ratio, eye opening and closing angle, eyebrow aspect ratio, eyebrow and eye distance and eyebrow curvature angle; according to the facial key point detection algorithm, the multi-view eyebrow and eye key points of the fourth category video are obtained; through the head posture estimation algorithm, the Euler angles of each video frame in the fourth category video are obtained; the second eyebrow and eye parameters are obtained by combining the multi-view eyebrow and eye key points and the Euler angles of the corresponding video frames; according to the cosine similarity and the average absolute distance, the matching degree of the second eyebrow and eye parameters and the first eyebrow and eye parameters is obtained; if the matching degree is greater than the predetermined matching degree, the miner number is determined by the first eyebrow and eye parameter corresponding to the highest matching degree value; if the matching degree is less than or equal to the predetermined matching degree, it indicates that the person corresponding to the second eyebrow and eye parameter is not a miner.

[0021] Preferably, the process of performing behavior analysis on the matched first video data and the second video data to obtain the second risk index includes:

[0022] The second video data is input into the eye closing behavior detection unit to obtain the eye closing duration and eye closing frequency corresponding to the miner number; the second video data is input into the head posture estimation unit to obtain the miner's Euler angle; the change amplitude of the miner's head is obtained by the change of the Euler angle in the video; if the head change amplitude is greater than the predetermined amplitude threshold, it indicates that the miner has head shaking behavior; according to the head shaking behavior of the miner in the second video data, the head shaking frequency corresponding to the miner number is obtained; the first video data is input into the body shaking detection unit to obtain the body shaking frequency and body shaking amplitude through the center of gravity offset algorithm; according to the dangerous action recognition unit, the miner's dangerous actions are identified through the spatiotemporal convolutional neural network to obtain the number of occurrences and danger levels of the dangerous actions; the second danger index corresponding to the miner number is obtained by combining the eye closing risk weight, the head shaking risk weight, the body shaking risk weight and the dangerous action weight.

[0023] Preferably, the implementation process of performing environmental assessment on different working areas includes:

[0024] Compare each of the environmental parameters of each of the working areas with the corresponding predetermined safety threshold, and obtain the degree of excess and the number of excesses of the environmental parameter over the corresponding predetermined safety threshold; if the degree of excess is greater than the predetermined boundary threshold or the number of excesses is greater than the predetermined number threshold, the corresponding environmental parameter is in the first environmental hazard level; if the degree of excess is less than or equal to the predetermined boundary threshold or the number of excesses is less than or equal to the predetermined number threshold, the corresponding environmental parameter is in the second environmental hazard level; based on the environmental parameters of all the working areas and the corresponding environmental hazard levels, obtain the third hazard index of each of the working areas.

[0025] Preferably, the specific implementation process of comprehensively evaluating the working safety of the miners includes:

[0026] Obtain the work hazard weights of all the work areas, and combine them with the third hazard index corresponding to the work area to obtain the work hazard index; obtain the work hazard index corresponding to the miner number according to the work area where each miner is located; and obtain the comprehensive hazard index corresponding to all the miner numbers by combining the first hazard index, the second hazard index and the work hazard index corresponding to the miner number.

[0027] Preferably, the process of classifying the risk levels of miners according to the comprehensive risk index of different miners includes:

[0028] The miner number, the first danger index, the second danger index and the third danger index are sent to the management personnel; if the comprehensive danger index is greater than or equal to the predetermined first danger threshold, the miner is at the first danger level and a red warning is issued; if the comprehensive danger index is less than the predetermined first danger threshold and greater than the predetermined second danger threshold, the miner is at the second danger level, the corresponding miner number and the work area are sent to the management personnel, and a yellow warning is issued; if the comprehensive danger index is less than or equal to the predetermined second danger threshold, the miner is at the third danger level and no warning is issued.

[0029] A coal mine underground safety detection system based on face image recognition, comprising:

[0030] The data acquisition module collects monitoring video data, inspection video data and environmental parameters in real time based on the monitoring cameras, inspection robots and sensors in each working area of ​​the target coal mine; obtains the first facial image and identity information data of the underground miners based on the face recognition equipment at the entrance of the target coal mine;

[0031] A data cleaning module performs data cleaning on the monitoring video data and the inspection video data of each working area to obtain first video data and second video data;

[0032] The miner identification module obtains all personnel video data in the second video data through video target recognition technology; classifies the wearing conditions of safety equipment in the personnel video data through target recognition method; and identifies the miner classification video data through adaptive face recognition method to obtain the miner number and the first danger index;

[0033] The miner behavior analysis module matches the identity of the person in the first video data by a motion trajectory matching method; performs behavior analysis on the matched first video data and the second video data to obtain a second danger index;

[0034] An environmental assessment module, performing an environmental assessment on the work area according to the environmental parameters to obtain a third hazard index;

[0035] A miner safety comprehensive assessment module, combining the first hazard index, the second hazard index and the third hazard index, comprehensively assesses the working safety of the miner to obtain a comprehensive hazard index; and classifies the miners into hazard levels according to the comprehensive hazard indexes of different miners;

[0036] The supervision and early warning module sends the numbers of miners who are not wearing safety equipment and the danger levels of all miners to the management personnel in real time; if the danger level of the miners reaches the first danger level or the second danger level, an early warning is issued to the management personnel.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The adaptive face recognition method proposed in the present invention significantly improves the recognition accuracy in the complex environment of coal mines by combining a multi-region face recognition unit and an eyebrow and eye parameter estimation unit. When miners wear safety helmets or masks, the multi-region face recognition unit introduces face occlusion interference to the data set through a data enhancement strategy, thereby enhancing the ability to extract facial features, which can effectively deal with the partial occlusion of the face by the safety equipment of underground miners and improve the accuracy of face recognition; while wearing a safety helmet and a mask at the same time makes it impossible to extract sufficient facial features, the eyebrow and eye parameter estimation unit uses the matching of the front eyebrow and eye key points and the multi-view eyebrow and eye key points to identify the miner's identity while only capturing the eyebrow and eye features, further improving the accuracy of face recognition, thereby timely tracking the safety status of the miners.

[0039] 2. The behavior analysis method proposed in the present invention uses a variety of technical means such as eye closure behavior detection, head posture estimation, body shaking detection and dangerous action recognition to comprehensively monitor and analyze the behavior of miners when working underground. This method can not only capture the fatigue state of miners in real time, but also identify potential dangerous actions, thereby accurately assessing the behavioral risks of miners, helping to promptly discover and correct unsafe behaviors and prevent accidents.

[0040] 3. The present invention comprehensively evaluates the safety of miners' work, combines the results of face recognition, behavior analysis and environmental assessment, and forms a comprehensive assessment of miners' work safety. At the same time, combined with the hazard weight of the work area, the comprehensive hazard index of the miners is calculated, which not only provides managers with an intuitive basis for safety management, but also makes safety management decisions more scientific and reasonable. Managers can take targeted management measures according to the hazard levels of different miners, such as strengthening training, adjusting positions or conducting key monitoring, thereby effectively improving the safety management efficiency of underground coal mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flow chart of a coal mine underground safety detection method based on face image recognition provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of different image masking methods in a multi-region face recognition unit provided by an embodiment of the present invention;

[0043] Figure 3 A flow chart of miner behavior analysis provided by an embodiment of the present invention;

[0044] Figure 4A flow chart of a coal mine underground safety detection system based on face image recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The working environment of coal mines is complex and changeable, often accompanied by major safety hazards such as gas explosions, landslides and water seepage, which pose a great threat to the life and safety of underground workers. In addition, due to the high intensity of work and long working hours, miners are prone to fatigue, which reduces their reaction ability and judgment and increases the risk of accidents. To this end, the introduction of coal mine underground personnel safety detection methods can monitor the location, behavior and important parameters of the surrounding environment of underground workers in real time. When an abnormal situation is detected, an early warning signal can be issued in time to help managers quickly take emergency measures to reduce the risk of accidents and effectively improve the safety of underground workers.

[0047] The present invention proposes a coal mine underground safety detection method and system based on face image recognition to detect the safety of coal mine personnel underground to ensure the life safety of miners. In order to illustrate the effectiveness of the method and system of the present invention, it will be specifically described in combination with the accompanying drawings of this embodiment and the following two embodiments.

[0048] Embodiment 1

[0049] The embodiment of the present application discloses a coal mine underground safety detection method based on face image recognition to detect the safety of personnel in A coal mine underground. Figure 1 The specific steps of the method proposed in the present invention include: S1. Collecting video data and environmental data of each working area to obtain the first facial image and identity information data of the miner; S2. Cleaning the video data of each working area; S3. Using an adaptive face recognition method, identifying the video data of miners wearing different safety equipment, obtaining the miner number and the first danger index; S4. Analyzing the miner's behavior on the video data to obtain the second danger index; S5. Performing environmental assessment on different working areas to obtain the third danger index; S6. Comprehensively assessing the work safety of the miners to obtain a comprehensive danger index, and classifying the miners into danger levels.

[0050] Furthermore, the working area underground in coal mine A is divided into k working areas, including ordinary working areas and dangerous working areas; each working area is provided with at least one monitoring camera, at least one inspection robot and a number of sensors; monitoring video data, inspection video data and environmental parameters are collected in real time according to the monitoring cameras, inspection robots and sensors in each working area; based on the face recognition equipment at the entrance of the target coal mine, the first facial image and identity information data of the underground miners are obtained, corresponding to the above-mentioned step S1; wherein the environmental parameters include gas concentration index, carbon monoxide concentration, dust index, temperature index, humidity index, noise index and vibration index; the first facial image represents the facial image of the underground miners of the target coal mine before work; the identity information data includes the miner number and the facial image set.

[0051] The embodiments of the present application provide a comprehensive information basis for subsequent safety inspections through real-time collected monitoring videos and environmental parameters; it not only helps to timely discover potential safety hazards and reduce the probability of accidents, but also can quickly confirm the identity of miners through face recognition technology and track the safety of each miner in real time.

[0052] Furthermore, the monitoring video data and the inspection video data of each working area are cleaned to obtain first video data and second video data, corresponding to the above-mentioned step S2; wherein, the data cleaning includes operations such as denoising, defogging, low-light enhancement, deglare and grayscale.

[0053] Specifically, in the underground working environment of coal mine A, monitoring video data is often affected by various factors, such as coal dust, humidity and low light, resulting in reduced image quality. De-noising can eliminate environmental noise, improve image clarity, and help identify miners and safety equipment more accurately; defogging technology can eliminate blur caused by moisture or coal dust, making the monitoring image more thorough; since low light conditions often exist in the working environment, especially in areas far away from light sources or equipment failures, monitoring video images may be blurred and details may be lost, so low light enhancement can improve visibility under dark conditions and ensure that personnel status can still be effectively monitored in dim environments; in addition, light reflections from mining lamps may cause glare in monitoring videos, thereby covering up important safety information, such as the wearing of safety equipment by miners and potential dangerous behaviors, so de-glare operations can reduce interference caused by light reflections and ensure the accuracy of video information; grayscale processing simplifies image data and reduces computational complexity, thereby speeding up subsequent analysis and recognition processes.

[0054] Further, through the video target recognition technology, the video data of all personnel in the second video data are obtained; through the target recognition method, the wearing conditions of the safety equipment in the personnel video data are classified to obtain the classified video data of the miners, including the first category video, the second category video, the third category video and the fourth category video; through the adaptive face recognition method, the classified video data of the miners are recognized to obtain the miner number and the first danger index, corresponding to the above step S3; the specific implementation process of the adaptive face recognition method includes:

[0055] In combination with the multi-region face recognition unit and the identity information data, face recognition is performed on the first category video, the second category video and the third category video to obtain the miner number; the miner number of the person in the fourth category video is obtained according to the first facial image and eyebrow parameter estimation unit; different danger weights are set for the first category video, the second category video, the third category video and the fourth category video, and the first danger index of different miners is obtained according to the safety equipment wearing condition of the miner. The specific calculation formula is:

[0056]

[0057] in, represents the first risk index of the miner numbered i; They respectively represent the probability that the miner numbered i is in the first category of videos, the second category of videos, the third category of videos, and the fourth category of videos, And each miner has only one is 1; α1, α2, α3, α4 represent the danger weights of the first category video, the second category video, the third category video and the fourth category video respectively, and 1>α1>α2≥α3>α4>0.

[0058] Specifically, the first category of videos is the video data of the personnel not wearing safety helmets and masks; the second category of videos is the video data of the personnel only wearing masks; the third category of videos is the video data of the personnel only wearing safety helmets; and the fourth category of videos is the video data of the personnel wearing safety helmets and masks. In the underground working environment of coal mine A, the wearing of safety equipment directly affects the safety of miners. Classifying the wearing of safety equipment helps to identify which workers do not wear necessary protective equipment as required, so as to take timely measures to reduce safety risks; among them, the miners in the first category of videos are at the highest risk because they lack basic head and face protection and are vulnerable to direct damage from collapse, flying objects or harmful gases; although the masks of the miners in the second category of videos provide respiratory protection, the lack of helmets means that the heads are still exposed to the risk of physical damage, and the degree of danger is relatively high; the heads of the miners in the third category of videos are protected, but the lack of masks may expose them to harmful gases or dust, and there are still health risks; the miners in the fourth category of videos have the highest safety level, have comprehensive protective measures, and are at the lowest risk.

[0059] The embodiment of the present application adopts a method combining video target recognition and adaptive face recognition technology to conduct detailed analysis of mining operation videos, significantly improving the safety management level of mines, effectively reducing the occurrence of safety accidents, and greatly protecting the lives of miners. This method can accurately identify all personnel in the video, automatically classify and evaluate the wearing of their safety equipment, quickly identify the situation of not wearing or wearing irregular safety equipment, and obtain the immediate danger index of each miner by setting different categories of danger weights. It not only greatly improves the efficiency of safety supervision, but also encourages miners to consciously abide by safety regulations and reduce potential accidents caused by human negligence.

[0060] Furthermore, the process of determining the miner number through the multi-region face recognition unit includes:

[0061] During the training phase, the miner's facial data set is obtained based on the identity information data; see Figure 2 , through the data enhancement layer, each image in the facial data set is divided into m×n image blocks of equal size, and k of the image blocks are occluded in g ways to obtain a facial expansion data set; the facial expansion data set is input into the feature extraction layer to obtain facial features; through the feature enhancement layer, the texture feature weights of the facial features are obtained, and facial enhancement features are obtained through feature reconstruction; the facial enhancement features are input into the classifier to obtain a facial recognition result; according to the angle boundary loss function and the cross entropy loss function, the facial recognition result is supervised to obtain the trained multi-region face recognition unit; the specific calculation formula of the loss function is:

[0062] L=β1·L arc +β2·L cro ;

[0063]

[0064] Among them, L represents the total loss function; L arc ,L cro They represent the angle boundary loss function and the cross entropy loss function respectively; N represents the number of samples; θ u ,θ v They represent the angles between the facial enhancement features of the uth and vth samples and the texture feature weights of the corresponding real categories respectively; s represents the magnification factor, which is used to enhance the distinction between different categories and is obtained by fitting a large amount of experimental data; m represents the angle boundary, which is used to enhance the similarity between the same categories and is obtained through multiple experimental tests; C represents the number of categories, which refers to the total number of miners here; y u represents the true category of the u-th sample; ρ u represents the probability that the uth sample is predicted to be the true category; log() represents the logarithmic function; cos() represents the cosine function; e () Represents a power function with a natural constant as base;

[0065] In the inference stage, the first category video, the second category video and the third category video are input into the trained multi-region face recognition unit to determine the identity of the miner; and the miner number is obtained by combining the identity information data.

[0066] Specifically, in the underground of coal mine A, due to the complex and dangerous working environment, it is crucial to ensure the accurate identification of miners. By enhancing and extracting features from the miners' facial data, the accuracy of face recognition can be improved, ensuring that miners can be accurately identified in different wearing conditions of safety equipment; for example, wearing only a mask or only a safety helmet may cause occlusion and changes in some facial features, making it difficult for traditional face recognition algorithms to effectively identify; therefore, occluding the training images during the training phase can increase the robustness of the model under different wearing conditions, ensuring that miners can be accurately identified under various working conditions, thereby improving the effectiveness of safety management and risk assessment.

[0067] The embodiment of the present application significantly enhances the accuracy and robustness of miner identification through a multi-region face recognition unit, providing strong technical support for reducing safety accidents and protecting the lives of miners. The unit uses data enhancement technology to generate a rich facial expansion data set, combined with feature enhancement and angle boundary loss functions, effectively improving the model's ability to recognize miners' faces in complex environments, ensuring that miners can be accurately identified even under adverse conditions such as occlusion and lighting changes, buying valuable time for timely rescue, and thus greatly reducing the risk of safety accidents.

[0068] Furthermore, according to the first facial image and the eyebrow parameter estimation unit, the implementation process of obtaining the miner number includes:

[0069] The frontal eyebrow and eye key points of the first facial image are obtained by using a facial key point detection algorithm; the first eyebrow and eye parameters are obtained according to the coordinates of the frontal eyebrow and eye key points, including eye distance, eye aspect ratio, eye opening angle, eyebrow aspect ratio, eyebrow and eye distance, and eyebrow curvature angle; the multi-view eyebrow and eye key points of the fourth category video are obtained according to the facial key point detection algorithm; the Euler angles of each video frame in the fourth category video are obtained by using a head posture estimation algorithm; the corresponding rotation matrix is ​​obtained according to the Euler angles of the corresponding video frame, and the multi-view eyebrow and eye key points are mapped to the frontal perspective to obtain the corresponding pseudo-orthogonal perspective key point coordinates; the corresponding eyebrow and eye parameters are obtained according to the pseudo-orthogonal perspective key point coordinates of different persons in the fourth category video, and the means of different eyebrow and eye parameters are calculated to obtain the second eyebrow and eye parameters; the matching degree of the second eyebrow and eye parameters and the first eyebrow and eye parameters is obtained according to the cosine similarity and the average absolute distance, and the specific calculation formula is:

[0070]

[0071] Among them, MD oi A represents the matching degree between the second eyebrow and eye parameter of the oth person in the fourth category video and the first eyebrow and eye parameter of the miner numbered i; o A vector representing the second eyebrow and eye parameters of the oth person in the fourth category video; B i represents the vector of the first eyebrow parameter of the miner numbered i; |||| represents a vector modulus operation; γ1, γ2 represent adjustment parameters and 0<γ2<γ1<1; χ represents the number of eyebrow parameters and χ=6;

[0072] If the matching degree is greater than the predetermined matching degree, the miner number is determined by the first eyebrow parameter corresponding to the highest matching degree value; if the matching degree is less than or equal to the predetermined matching degree, it indicates that the person corresponding to the second eyebrow parameter is not a miner; the working area and video data of the non-miner are transmitted to the management personnel, and a blue warning is issued.

[0073] In underground operations at coal mine A, most facial features are blocked because miners often wear safety helmets and masks. By extracting the frontal eyebrow and eye key points of miners before work through the facial key point detection algorithm, and combining the multi-view eyebrow and eye key points of people in different working areas, the occlusion problem caused by wearing equipment can be effectively overcome; by matching the eyebrow and eye parameters, the miners' identities can be accurately identified, ensuring real-time monitoring of operators in high-risk environments.

[0074] The embodiment of the present application combines the frontal eyebrow and eye parameters of the first facial image with the multi-view eyebrow and eye parameters in the fourth category video to improve the flexibility and accuracy of miner identification. The method can capture the eyebrow and eye features of miners at different viewing angles, and combine the head posture information to accurately analyze the eyebrow and eye parameters at multiple viewing angles, thereby quickly determining the miner's identity, which not only improves the recognition accuracy, but also enhances the system's adaptability to complex working environments, ensuring that the miner's identity can be accurately identified even when the miner is at different angles or postures.

[0075] Furthermore, the identity of the person in the first video data is matched by a motion trajectory matching method; the behavior analysis is performed on the matched first video data and the second video data to obtain a second danger index corresponding to Figure 1 S4 step of Figure 3 The specific implementation process of the behavior analysis includes:

[0076] The second video data is input into the eye closing behavior detection unit to obtain the eye closing duration and eye closing frequency corresponding to the miner number; the second video data is input into the head posture estimation unit to obtain the miner's Euler angle; the change amplitude of the miner's head is obtained by the change of the Euler angle in the video; if the head change amplitude is greater than the predetermined amplitude threshold, it indicates that the miner has a head shaking behavior; according to the head shaking behavior of the miner in the second video data, the head shaking frequency corresponding to the miner number is obtained; the first video data is input into the body shaking detection unit, and the body shaking frequency and body shaking amplitude are obtained by the center of gravity offset algorithm; according to the dangerous action recognition unit, the miner's dangerous action is identified by the spatiotemporal convolutional neural network to obtain the number of occurrences and the danger level of the dangerous action; the second danger index corresponding to the miner number is obtained by combining the eye closing risk weight, the head shaking risk weight, the body shaking risk weight and the dangerous action weight. The specific calculation formula is:

[0077]

[0078]

[0079] in, represents the second risk index of the miner numbered i; w1, w2, w3, w4 represent weight coefficients of different actions and 0 <w1≤w2≤w3<w4<1; respectively represent the risk index of the miner numbered i when the miner closes his eyes, shakes his head, shakes his body, and performs dangerous behaviors;

[0080] represents the closing time of the miner numbered i in the second video; T represents the total duration of the second video; represents the eye closing frequency of the miner numbered i; f eye represents the normal eye closing frequency; δ1, δ2 represent the corresponding weight coefficients and 0<δ2≤δ1<1; represents the average head shaking amplitude of the miner numbered i; ψ head Indicates the normal head shaking amplitude; represents the head shaking frequency of the miner numbered i; f head represents the normal head shaking frequency; ε1, ε2 represent the corresponding weight coefficients and 0<ε2≤ε1<1; represents the average body shaking amplitude of the miner numbered i; ψ body Indicates the normal body shaking amplitude; represents the body shaking frequency of the miner numbered b; f body represents the normal body shaking frequency; ∈1,∈2 represents the corresponding weight coefficient and 0<∈2≤∈1<1; represents the number of times the miner numbered i performs the u-th dangerous behavior; λ u represents the risk level weight of the u-th dangerous behavior and 0<λ u <1; m represents the total number of dangerous behaviors of miners.

[0081] During underground operations in coal mine A, miners may experience fatigue and dizziness after working for a long time. The eye closure behavior detection unit, head posture estimation unit and body shaking detection unit can evaluate the miners' fatigue and dizziness states in real time, thereby avoiding dangers caused by fatigue and unconsciousness.

[0082] The embodiment of the present application can comprehensively monitor the safety status of miners during operation by comprehensively using motion trajectory matching and analysis of multiple behaviors, thereby effectively reducing the occurrence of safety accidents. This method can not only monitor the duration and frequency of miners' eye closure and the shaking of their heads and bodies in real time, but also accurately identify dangerous actions through advanced spatiotemporal convolutional neural networks, and achieve a comprehensive and in-depth analysis of miners' behaviors, thereby ensuring that miners can work safely in a complex and changing working environment.

[0083] Further, according to the environmental parameters, an environmental assessment is performed on different working areas to obtain a third hazard index corresponding to Figure 1 Step S5; the implementation process of environmental assessment of different working areas includes:

[0084] Each of the environmental parameters of each of the working areas is compared with the corresponding predetermined safety threshold, and the degree and number of times the environmental parameter exceeds the corresponding predetermined safety threshold are obtained; if the degree of excess is greater than the predetermined boundary threshold or the number of excesses is greater than the predetermined number threshold, the corresponding environmental parameter is in the first environmental hazard level; if the degree of excess is less than or equal to the predetermined boundary threshold or the number of excesses is less than or equal to the predetermined number threshold, the corresponding environmental parameter is in the second environmental hazard level; according to the environmental parameters of all the working areas and the corresponding environmental hazard levels, the third hazard index of each of the working areas is obtained, and the specific calculation formula is:

[0085]

[0086] in, The third hazard index of the work area numbered x; They represent the probability of the working area numbered x in different environmental hazard levels, And each of the working areas has only one is 1; represents the weight coefficient of the u-th environmental parameter in different environmental hazard levels and It represents the total number of the environmental parameters, which include gas concentration index, carbon monoxide concentration, dust index, temperature index, humidity index, noise index and vibration index.

[0087] The embodiment of the present application can timely discover and quantify potential environmental risks by comprehensively monitoring and evaluating the environmental parameters of each working area, providing strong data support for mine safety management and effectively reducing safety accidents caused by environmental factors. The method compares environmental parameters with predetermined safety thresholds, determines the danger level of environmental parameters, and comprehensively considers the degree and number of times of exceeding, thereby ensuring the comprehensiveness and accuracy of the evaluation results, thereby reducing the occurrence of safety accidents and protecting the lives of miners.

[0088] Furthermore, the first hazard index, the second hazard index and the third hazard index are combined to comprehensively evaluate the working safety of the miner to obtain a comprehensive hazard index corresponding to Figure 1 The specific implementation process of step S5 includes:

[0089] The work hazard weights of all the working areas are obtained, and the work hazard index is obtained by combining the third hazard index corresponding to the working area; the work hazard index corresponding to the miner number is obtained according to the working area where each miner is located; the comprehensive hazard index corresponding to all the miner numbers is obtained by combining the first hazard index, the second hazard index and the work hazard index corresponding to the miner number. The specific calculation formula is:

[0090]

[0091]

[0092] Among them, DI i represents the comprehensive risk factor of the miner numbered i; WD i They respectively represent the first hazard index, the second hazard index and the work hazard index of the miner numbered i; ω1, ω2, ω3 respectively represent weight coefficients and 0<ω1≤ω2≤ω3<1; Indicates the total working time of the miner numbered i; Indicates the length of time worked in the work area numbered x; Indicates the total number of the working areas; represents the third danger index of the working area numbered x; ξ x represents the danger weight of the work area numbered x and 0<ξ x ≤1.

[0093] In order to further illustrate the role of the comprehensive risk assessment proposed by the present invention, the comprehensive risk index of the miners in the A coal mine in one day is given as an example for comparison. The comprehensive risk indexes of different miners are listed in Table 1; wherein the weight coefficients ω1=0.3, ω2=0.3, ω3=0.4. According to the data in Table 1, it can be seen that the comprehensive risk index of different miners is affected by the first risk index, the second risk index and the work risk index. For example, for the miner numbered KG01, the first hazard index, the second hazard index and the work hazard index are 0.17, 0.54 and 0.66 respectively; while for the miner KG02, the first hazard index, the second hazard index and the work hazard index are 0.47, 0.78 and 0.61 respectively; when the work hazard indexes of the two miners are similar, since the first hazard index and the second hazard index of KG01 are much lower than those of KG02, the comprehensive hazard index of miner KG02 is higher; this shows that the degree of danger in the miner's working area cannot be equated with the safety risks faced by the miner himself, and the miner's own protection situation and behavioral status should be comprehensively considered to comprehensively assess the miner's physical safety status.

[0094] Table 1. Comprehensive risk index of different miners in A coal mine The embodiment of the present application combines the first, second and third hazard indexes and the hazard weights of different work areas to comprehensively and objectively evaluate the work safety of miners. The method not only covers the personal safety behaviors and environmental factors of miners, but also fully considers the hazard level of the work area itself, ensuring the comprehensiveness and accuracy of the evaluation results, thereby effectively preventing the occurrence of safety accidents.

[0095] Furthermore, according to the comprehensive risk index of different miners, the process of classifying the risk levels of miners includes:

[0096] The miner number, the first danger index, the second danger index and the third danger index are sent to the management personnel; if the comprehensive danger index is greater than or equal to the predetermined first danger threshold, the miner is at the first danger level and a red warning is issued; if the comprehensive danger index is less than the predetermined first danger threshold and greater than the predetermined second danger threshold, the miner is at the second danger level, the corresponding miner number and the work area are sent to the management personnel, and a yellow warning is issued; if the comprehensive danger index is less than or equal to the predetermined second danger threshold, the miner is at the third danger level and no warning is issued.

[0097] The embodiment of the present application divides the miners into risk levels according to the comprehensive risk index and takes corresponding early warning measures, thereby achieving accurate identification and timely response to the safety risks of miners and effectively reducing the risk of safety accidents. For miners at the first risk level, a red warning is immediately issued and various risk indexes are reported in detail to ensure that management personnel can intervene quickly and take emergency measures to avoid potential safety accidents; for miners at the second risk level, the system issues a yellow warning and indicates their work area, so that management personnel can pay targeted attention and supervision and prevent safety hazards in a timely manner.

[0098] During underground operations at coal mine A, miners may be threatened by insufficient protection, work fatigue, dangerous behaviors, and work environment risks. By accurately identifying the danger level of each miner, it can ensure that managers can take appropriate countermeasures in a timely manner; among them, the issuance of a red warning indicates that the miner is in an extremely high-risk state. At this time, the miner can no longer continue to work and must be forced to evacuate or rest immediately to prevent major accidents; the yellow warning reminds managers to pay attention to potential risks and may need to monitor the corresponding miners in real time to prevent safety accidents; in addition, the real-time transmission of the miner's number and the corresponding different danger indexes is conducive to managers analyzing the causes of safety risks, so as to facilitate the subsequent adoption of corresponding measures, such as safety training and updating management systems.

[0099] The embodiment of the present application realizes a comprehensive detection of the safety status of miners in coal mines through an adaptive face recognition method, a behavior analysis method and a comprehensive evaluation mechanism. The specific implementation process mainly includes the following steps: (1) Identify the miner's identity according to the different wearing conditions of the personnel's safety equipment; (2) Comprehensively detect and analyze the miner's behavior; (3) Comprehensively evaluate the miner's work safety. The present invention proposes an adaptive face recognition method, behavior analysis and comprehensive risk assessment for the above three processes respectively; the adaptive face recognition method realizes the confirmation of the miner's identity when the safety equipment is blocked, and improves the reliability of safety detection; the behavior analysis improves the real-time detection capability of the miner's fatigue state and dangerous actions; the comprehensive risk assessment further improves the scientificity and rationality of safety management decisions, and helps to effectively reduce the risk of accidents.

[0100] Embodiment 2

[0101] In the first embodiment, the method of the present invention realizes the safety detection of personnel in the A coal mine. In the embodiment of the present application, a coal mine underground safety detection system based on face image recognition proposed by the present invention is described. Figure 4 , the coal mine underground safety detection system comprises:

[0102] The data acquisition module collects monitoring video data, inspection video data and environmental parameters in real time based on the monitoring cameras, inspection robots and sensors in each working area of ​​the target coal mine; obtains the first facial image and identity information data of the underground miners based on the face recognition equipment at the entrance of the target coal mine;

[0103] A data cleaning module performs data cleaning on the monitoring video data and the inspection video data of each working area to obtain first video data and second video data;

[0104] The miner identification module obtains all personnel video data in the second video data through video target recognition technology; classifies the wearing conditions of safety equipment in the personnel video data through target recognition method; and identifies the miner classification video data through adaptive face recognition method to obtain the miner number and the first danger index;

[0105] The miner behavior analysis module matches the identity of the person in the first video data by a motion trajectory matching method; performs behavior analysis on the matched first video data and the second video data to obtain a second danger index;

[0106] An environmental assessment module, performing an environmental assessment on the work area according to the environmental parameters to obtain a third hazard index;

[0107] A miner safety comprehensive assessment module, combining the first hazard index, the second hazard index and the third hazard index, comprehensively assesses the working safety of the miner to obtain a comprehensive hazard index; and classifies the miners into hazard levels according to the comprehensive hazard indexes of different miners;

[0108] The supervision and early warning module sends the numbers of miners who are not wearing safety equipment and the danger levels of all miners to the management personnel in real time; if the danger level of the miners reaches the first danger level or the second danger level, an early warning is issued to the management personnel.

[0109] Specifically, in the supervision and early warning module, the management personnel's processing process depends on the miners' danger level and the wearing of safety equipment; when a miner who is not wearing safety equipment is identified, his miner number and danger level will be sent to the management personnel in real time, providing data basis for subsequent targeted safety training; if the miner's danger level reaches the first or second level, the management personnel will receive a red or yellow warning, prompting immediate action to reduce safety risks; for example, a red warning means that the miner is at extremely high risk, and the management personnel need to intervene quickly, such as stopping operations, forced rest and emergency evacuation; a yellow warning requires management personnel to monitor the corresponding miners in real time to prevent the occurrence of safety accidents.

[0110] The embodiment of the present application has built a comprehensive and efficient miner safety detection system, which has significantly improved the safety management level of coal mines and effectively reduced the occurrence of safety accidents. The system can collect monitoring videos, inspection videos and environmental parameters in real time to ensure the comprehensiveness and timeliness of the data; through adaptive face recognition methods, it can accurately identify the identity of miners and the wearing of safety equipment, and track the safety of different miners in real time; at the same time, it conducts multi-dimensional analysis of miners' behaviors and environmental factors, comprehensively evaluates the work safety of miners, and realizes the accurate division of danger levels; in addition, the real-time feedback mechanism of the supervision and early warning module ensures that managers can quickly respond to potential safety risks and take necessary intervention measures, thereby effectively preventing the occurrence of safety accidents and protecting the lives of miners.

[0111] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A coal mine underground safety detection method based on face image recognition, characterized in that: include: According to the monitoring cameras, inspection robots and sensors in each working area of ​​the target coal mine, monitoring video data, inspection video data and environmental parameters are collected in real time; Acquire a first facial image and identity information data of an underground miner according to a face recognition device at the entrance of the target coal mine; Cleaning the monitoring video data and the inspection video data of each working area to obtain first video data and second video data; Obtaining video data of all persons in the second video data by using video target recognition technology; The wearing conditions of the safety equipment in the personnel video data are classified by the target recognition method; the classified video data of the miners are recognized by the adaptive face recognition method to obtain the miner number and the first danger index; Matching the identity of the person in the first video data by a motion trajectory matching method; performing behavior analysis on the matched first video data and the second video data to obtain a second danger index; According to the environmental parameters, an environmental assessment is performed on different working areas to obtain a third hazard index; The first hazard index, the second hazard index and the third hazard index are combined to comprehensively evaluate the working safety of the miner to obtain a comprehensive hazard index; and the miners are divided into hazard levels according to the comprehensive hazard indexes of different miners.

2. A coal mine underground safety detection method based on face image recognition according to claim 1, characterized in that: The first facial image represents the facial image of the target coal mine underground miner before work; the identity information data includes the miner number and the facial image set.

3. The method for underground coal mine safety detection based on face image recognition according to claim 1, characterized in that: The specific implementation process of the adaptive face recognition method includes: The classified video data of the miners is obtained, including first category videos, second category videos, third category videos and fourth category videos; facial recognition is performed on the first category videos, second category videos and third category videos in combination with the multi-region face recognition unit and the identity information data to obtain the miner number; the miner number of the person in the fourth category video is obtained according to the first facial image and eyebrow parameter estimation unit; different danger weights are set for different categories of videos, and the first danger index of different miners is obtained according to the safety equipment worn by the miners.

4. A coal mine underground safety detection method based on face image recognition according to claim 3, characterized in that: The process of determining the miner number by the multi-region face recognition unit includes: In the training phase, a facial data set of miners is obtained according to the identity information data; each image in the facial data set is divided into m×n image blocks of equal size through a data enhancement layer, and g types of occlusion are performed on k of the image blocks to obtain a facial expansion data set; the facial expansion data set is input into a feature extraction layer to obtain facial features; texture feature weights of the facial features are obtained through a feature enhancement layer, and facial enhancement features are obtained through feature reconstruction; the facial enhancement features are input into a classifier to obtain a facial recognition result; the facial recognition result is supervised according to an angle boundary loss function and a cross entropy loss function to obtain the trained multi-region face recognition unit; In the inference stage, the first category video, the second category video and the third category video are input into the trained multi-region face recognition unit to determine the identity of the miner; and the miner number is obtained by combining the identity information data.

5. The method for underground coal mine safety detection based on face image recognition according to claim 3 is characterized in that: The implementation process of obtaining the miner number according to the first facial image and the eyebrow parameter estimation unit includes: Through the facial key point detection algorithm, the frontal eyebrow and eye key points of the first facial image are obtained; according to the coordinates of the frontal eyebrow and eye key points, the first eyebrow and eye parameters are obtained, including eye distance, eye aspect ratio, eye opening and closing angle, eyebrow aspect ratio, eyebrow and eye distance and eyebrow curvature angle; according to the facial key point detection algorithm, the multi-view eyebrow and eye key points of the fourth category video are obtained; through the head posture estimation algorithm, the Euler angles of each video frame in the fourth category video are obtained; the second eyebrow and eye parameters are obtained by combining the multi-view eyebrow and eye key points and the Euler angles of the corresponding video frames; according to the cosine similarity and the average absolute distance, the matching degree of the second eyebrow and eye parameters and the first eyebrow and eye parameters is obtained; if the matching degree is greater than the predetermined matching degree, the miner number is determined by the first eyebrow and eye parameter corresponding to the highest matching degree value; if the matching degree is less than or equal to the predetermined matching degree, it indicates that the person corresponding to the second eyebrow and eye parameter is not a miner.

6. A coal mine underground safety detection method based on face image recognition according to claim 1, characterized in that: The process of performing behavior analysis on the matched first video data and the second video data to obtain the second danger index includes: The second video data is input into the eye closing behavior detection unit to obtain the eye closing duration and eye closing frequency corresponding to the miner number; the second video data is input into the head posture estimation unit to obtain the miner's Euler angle; the change amplitude of the miner's head is obtained by the change of the Euler angle in the video; if the head change amplitude is greater than the predetermined amplitude threshold, it indicates that the miner has head shaking behavior; according to the head shaking behavior of the miner in the second video data, the head shaking frequency corresponding to the miner number is obtained; the first video data is input into the body shaking detection unit to obtain the body shaking frequency and body shaking amplitude through the center of gravity offset algorithm; according to the dangerous action recognition unit, the miner's dangerous actions are identified through the spatiotemporal convolutional neural network to obtain the number of occurrences and danger levels of the dangerous actions; the second danger index corresponding to the miner number is obtained by combining the eye closing risk weight, the head shaking risk weight, the body shaking risk weight and the dangerous action weight.

7. The method for underground coal mine safety detection based on face image recognition according to claim 1, characterized in that: The implementation process of environmental assessment for the different work areas includes: Compare each of the environmental parameters of each of the working areas with the corresponding predetermined safety threshold, and obtain the degree of excess and the number of excesses of the environmental parameter over the corresponding predetermined safety threshold; if the degree of excess is greater than the predetermined boundary threshold or the number of excesses is greater than the predetermined number threshold, the corresponding environmental parameter is in the first environmental hazard level; if the degree of excess is less than or equal to the predetermined boundary threshold or the number of excesses is less than or equal to the predetermined number threshold, the corresponding environmental parameter is in the second environmental hazard level; based on the environmental parameters of all the working areas and the corresponding environmental hazard levels, obtain the third hazard index of each of the working areas.

8. The method for underground coal mine safety detection based on face image recognition according to claim 1, characterized in that: The specific implementation process of the comprehensive assessment of the working safety of the miners includes: Obtain the work hazard weights of all the work areas, and combine them with the third hazard index corresponding to the work area to obtain the work hazard index; obtain the work hazard index corresponding to the miner number according to the work area where each miner is located; and obtain the comprehensive hazard index corresponding to all the miner numbers by combining the first hazard index, the second hazard index and the work hazard index corresponding to the miner number.

9. The method for underground coal mine safety detection based on face image recognition according to claim 1, characterized in that: According to the comprehensive risk index of different miners, the process of classifying the risk levels of miners includes: The miner number, the first danger index, the second danger index and the third danger index are sent to the management personnel; if the comprehensive danger index is greater than or equal to the predetermined first danger threshold, the miner is at the first danger level and a red warning is issued; if the comprehensive danger index is less than the predetermined first danger threshold and greater than the predetermined second danger threshold, the miner is at the second danger level, the corresponding miner number and the work area are sent to the management personnel, and a yellow warning is issued; if the comprehensive danger index is less than or equal to the predetermined second danger threshold, the miner is at the third danger level and no warning is issued.

10. A coal mine underground safety detection system based on face image recognition, characterized in that: include: The data acquisition module collects monitoring video data, inspection video data and environmental parameters in real time based on the monitoring cameras, inspection robots and sensors in each working area of ​​the target coal mine; obtains the first facial image and identity information data of the underground miners based on the face recognition equipment at the entrance of the target coal mine; A data cleaning module performs data cleaning on the monitoring video data and the inspection video data of each working area to obtain first video data and second video data; A miner identification module, which obtains video data of all personnel in the second video data through video target recognition technology; The wearing conditions of the safety equipment in the personnel video data are classified by the target recognition method; the classified video data of the miners are recognized by the adaptive face recognition method to obtain the miner number and the first danger index; The miner behavior analysis module matches the identity of the person in the first video data by a motion trajectory matching method; performs behavior analysis on the matched first video data and the second video data to obtain a second danger index; An environmental assessment module, performing an environmental assessment on the work area according to the environmental parameters to obtain a third hazard index; A miner safety comprehensive assessment module, combining the first hazard index, the second hazard index and the third hazard index, comprehensively assesses the working safety of the miner to obtain a comprehensive hazard index; and classifies the miners into hazard levels according to the comprehensive hazard indexes of different miners; The supervision and early warning module sends the numbers of miners who are not wearing safety equipment and the danger levels of all miners to the management personnel in real time; if the danger level of the miners reaches the first danger level or the second danger level, an early warning is issued to the management personnel.

Images