AI-based open pit coal mine monitoring video analysis method

Abstract

The invention relates to the technical field of artificial intelligence, and particularly discloses an open pit coal mine monitoring video analysis method based on AI, and the method comprises the following steps: S1, building a smoke region recognition model, and carrying out the training and verification of the smoke region recognition model through a database; s2, acquiring a monitoring video frame, and determining a target area in the monitoring video frame based on the smoke area identification model; obtaining a gray frame, and calculating a theoretical concentration based on the gray values of the pixel points in the gray frame; s3, obtaining a target proportion, and calculating an influence value based on the theoretical concentration and the target proportion; determining an abnormal area based on the influence value, and determining a warning area according to the abnormal area. According to the invention, the staff is prompted when entering the warning area, so that the influence of smoke dust on the staff is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly relates to an AI-based analysis method for open-pit coal mine monitoring videos. Background Art

[0002] AI, that is, artificial intelligence, is an important driving force for the new round of scientific and technological revolution and industrial transformation. It is a new technical science that studies, develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence.

[0003] With the continuous development of artificial intelligence, its applications in various fields have become more extensive and mature. In the field of coal mine monitoring, an artificial intelligence model after training can identify abnormalities in monitoring videos, such as employees not wearing safety helmets, etc., and issue corresponding alarms, thereby reducing the labor and time costs brought by staff viewing monitoring videos.

[0004] However, most of the current technological applications mainly focus on detecting whether employees have violated regulations, such as whether they wear safety helmets, etc., and rarely involve paying attention to the health risks of employees in a high-dust environment for a long time. The dust pollution in the open-pit coal mine working environment poses a long-term threat to the health of workers. In particular, the damage of coal dust and stone dust to the respiratory system may lead to serious diseases such as pneumoconiosis. Although the existing monitoring systems can help managers supervise operation specifications, there is still a lack of effective solutions for analyzing the impact of dust on employees. Summary of the Invention

[0005] The purpose of the present invention is to provide an AI-based analysis method for open-pit coal mine monitoring videos to solve the following technical problems:

[0006] Most of the current technological applications mainly focus on detecting whether employees have violated regulations, such as whether they wear safety helmets, etc., and rarely involve paying attention to the health risks of employees in a high-dust environment for a long time. The dust pollution in the open-pit coal mine working environment poses a long-term threat to the health of workers. In particular, the damage of coal dust and stone dust to the respiratory system may lead to serious diseases such as pneumoconiosis. Although the existing monitoring systems can help managers supervise operation specifications, there is still a lack of effective solutions for analyzing the impact of dust on employees.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] An AI-based analysis method for open-pit coal mine monitoring videos includes the following steps:

[0009] S1: Establish a database which stores the monitored images of open-pit coal mines with marked soot areas. Based on a deep learning model, establish a soot area recognition model, and train and verify the soot area recognition model through the database;

[0010] S2: Obtain the monitored video of the open-pit coal mine within a preset monitoring period, perform frame division on the monitored video to obtain monitored video frames, input the monitored video frames into the verified soot area recognition model, and obtain the soot area in the monitored video frames, denoted as the target area;

[0011] Perform grayscale processing on the monitored video frames to obtain grayscale frames, determine the grayscale values of the pixel points belonging to the target area, and calculate the theoretical concentration Dys represents a preset standard grayscale value, c represents the dust concentration corresponding to the standard grayscale value, Di represents the grayscale value of the i-th pixel point belonging to the target area, and n represents the total number of pixel points belonging to the target area;

[0012] S3: Obtain the ratio of the area of the target area to the monitored video frame, denoted as the target ratio b, then the influence value P = η * b * C, where η represents a preset correction coefficient;

[0013] When the influence value is greater than or equal to a preset influence value threshold, regard it as an outlier, regard the target area corresponding to the outlier as an abnormal area, and regard the intersection of the abnormal areas corresponding to the same monitoring device as the warning area. When it is recognized that an employee enters the warning area, send a warning message for prompt.

[0014] As a further solution of the present invention: In step S3, the process of identifying that a user enters the warning area specifically includes:

[0015] Obtain the real-time monitored video of the open-pit coal mine, perform frame division on the real-time monitored video to obtain real-time frames, input the real-time frames into a pre-trained person recognition model, and identify the people in the real-time frames. When a person is in the warning area, regard him / her as a pending person;

[0016] When the pending person has no protective measures, mark him / her as an abnormal person and send a warning message for prompt to the abnormal person.

[0017] As a further solution of the present invention: The process of pre-training the person recognition model specifically includes:

[0018] Establish a second database which stores the monitored images of open-pit coal mines with marked people;

[0019] A person recognition model is established based on a deep learning model, and the person recognition model is trained and verified based on the second database to obtain a pre-trained person recognition model.

[0020] As a further solution of the present invention: the open-pit coal mine monitoring images with marked soot areas stored in the database are manually marked, and the open-pit coal mine monitoring images with marked persons stored in the second database are manually marked.

[0021] As a further solution of the present invention: the soot area recognition model and the person recognition model are trained based on the backpropagation algorithm, and the soot area recognition model and the person recognition model are verified based on five-fold cross-validation.

[0022] As a further solution of the present invention: in step S3, the following steps are further included:

[0023] Generate coordinate points (a, Pa), where Pa represents the influence value corresponding to the a-th monitoring video frame, and fit the coordinate points to obtain a fitting curve f(t), where t represents time;

[0024] Obtain the monotonicity of the fitting curve. When the fitting curve is monotonically decreasing and the average influence value is less than the influence value threshold, the corresponding target area is not used as an abnormal area.

[0025] As a further solution of the present invention: in step S3, when the number of abnormal persons exceeds the preset abnormal person number threshold, a prompt message is sent to the preset management personnel.

[0026] As a further solution of the present invention: in step S3, when the number of employees entering the warning area exceeds the preset employee number threshold, a prompt message is sent to the preset management personnel.

[0027] Advantages of the present invention: In this solution, by establishing a dataset and using a deep learning model to identify the soot area, the soot area and concentration in the monitoring video can be effectively identified; through the accurate identification of soot, the dust pollution situation in the coal mine environment can be monitored in real time, so as to better evaluate the safety of the working environment. By accurately identifying the soot area and evaluating its concentration, timely prompts are issued when employees enter the warning area to prevent them from being exposed to a high-concentration dust environment and avoid long-term harm to their health. Secondly, by calculating the soot concentration, a quantitative index can be provided for the soot concentration in each monitoring area; it can be understood that the digitalization of the soot concentration facilitates the management of dangerous areas. By calculating and predicting the concentration, the soot concentration in different areas can be accurately grasped, and then the warning area can be set more reasonably. After that, by calculating the influence degree of the target area and determining the abnormal area, it should be noted that the abnormal area may be constantly changing, and the reasons include but are not limited to mining behaviors, mining locations, etc. Therefore, the intersection of the abnormal areas corresponding to the same monitoring device is used as the warning area for subsequent monitoring. The present invention can effectively prevent employees from being exposed to harmful environments for a long time and reduce the threat to health. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention will be further described below with reference to the accompanying drawings.

[0029] Figure 1 is a schematic flow chart of a method for analyzing open-pit coal mine monitoring video based on AI of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0031] Please refer to Figure 1 as shown, the present invention is a method for analyzing open-pit coal mine monitoring video based on AI, including the following steps:

[0032] S1: Establish a database, in which open-pit coal mine monitoring images with marked soot areas are stored, establish a soot area recognition model based on a deep learning model, and train and verify the soot area recognition model through the database;

[0033] S2: Obtain the monitoring video of the open-pit coal mine within a preset monitoring period, perform frame segmentation on the monitoring video to obtain monitoring video frames, input the monitoring video frames into the verified soot area recognition model, and obtain the soot area in the monitoring video frames, denoted as the target area;

[0034] Perform gray-scale processing on the monitoring video frames to obtain gray-scale frames, determine the gray-scale values of the pixel points belonging to the target area, and calculate the theoretical concentration Dys represents a preset standard gray-scale value, c represents the dust concentration corresponding to the standard gray-scale value, Di represents the gray-scale value of the i-th pixel point belonging to the target area, and n represents the total number of pixel points belonging to the target area;

[0035] S3: Obtain the ratio of the area of the target area to the monitoring video frame, denoted as the target ratio b, then the influence value P = η * b * C, where η represents a preset correction coefficient;

[0036] When the influence value is greater than or equal to a preset influence value threshold, use it as an outlier, use the target area corresponding to the outlier as an abnormal area, and use the intersection of the abnormal areas corresponding to the same monitoring device as a warning area. When it is recognized that an employee enters the warning area, send a warning message for prompt.

[0037] It should be noted that by establishing a data set and using a deep learning model to identify the soot area, the soot area and concentration in the monitoring video can be effectively identified; through the accurate identification of soot, the dust pollution situation in the coal mine environment can be monitored in real time, so as to better evaluate the safety of the working environment. By accurately identifying the soot area and evaluating its concentration, timely prompts are issued when employees enter the warning area to prevent them from being exposed to high-concentration dust environments and avoid long-term harm to health; secondly, by calculating the soot concentration, a quantitative index can be provided for the soot concentration in each monitoring area; it can be understood that digitizing the soot concentration facilitates the management of dangerous areas. By calculating the predicted concentration, the soot concentration in different areas can be accurately grasped, and then the warning area can be set more reasonably; afterwards, by calculating the influence degree of the target area and determining the abnormal area, it should be noted that the abnormal area may be constantly changing, and the reasons include but are not limited to mining behaviors, mining locations, etc. Therefore, the intersection of the abnormal areas corresponding to the same monitoring device is used as the warning area for subsequent monitoring.

[0038] In another preferred embodiment of the present invention, in step S3, the process of identifying that a user enters the warning area specifically includes:

[0039] Obtain the real-time monitoring video of the open-pit coal mine, perform frame-by-frame processing on the real-time monitoring video to obtain real-time frames, input the real-time frames into a pre-trained person recognition model to identify the people in the real-time frames, and when a person is in the warning area, regard them as pending personnel;

[0040] When the pending personnel do not have protective measures, mark them as abnormal personnel and send a warning message to prompt the abnormal personnel.

[0041] It is worth noting that by obtaining the real-time monitoring video of the open-pit coal mine and performing frame-by-frame processing to obtain real-time frames, the dynamic situation at the coal mine operation site can be captured in real time, and the video content can be decomposed into multiple independent frames, which is convenient for subsequent processing and analysis; with the help of a trained AI model, information related to people can be efficiently and accurately extracted from each frame, avoiding the errors and delays brought by manual monitoring. In this process, by accurately identifying the position and actions of people, it can be determined in a timely manner whether an employee has entered the preset warning area.

[0042] It can be understood that the protective measures include but are not limited to wearing anti-dust masks, etc.

[0043] In another preferred embodiment of the present invention, the process of pre-training the person recognition model specifically includes:

[0044] Establish a second database, and store the open-pit coal mine monitoring images with labeled people in the second database;

[0045] Based on a deep learning model, establish a person recognition model, and train and verify the person recognition model based on the second database to obtain a pre-trained person recognition model.

[0046] In another preferred embodiment of the present invention, the open-pit coal mine monitoring images with labeled smoke and dust areas stored in the database are manually labeled, and the open-pit coal mine monitoring images with labeled people stored in the second database are manually labeled.

[0047] In another preferred embodiment of the present invention, train the smoke and dust area recognition model and the person recognition model based on the backpropagation algorithm, and verify the smoke and dust area recognition model and the person recognition model based on five-fold cross-validation.

[0048] It should be noted that by training the soot area recognition model and the person recognition model based on the backpropagation algorithm, the accuracy and generalization ability of these two models can be effectively improved. The advantage of the backpropagation algorithm is that it can optimize the model parameters, enabling the model to gradually reduce the prediction error and enhance the accuracy of soot area and person recognition; validating the soot area recognition model and the person recognition model based on five-fold cross-validation can effectively avoid overfitting and improve the stability and generalization ability of the model on different datasets; five-fold cross-validation divides the data into five subsets, using one subset for validation each time and the remaining subsets for training, and finally obtaining a reliable evaluation by averaging the results. This not only ensures the consistency of the model's performance on different data but also improves the applicability and robustness of the model in practical applications.

[0049] In another preferred embodiment of the present invention, in step S3, the following steps are further included:

[0050] Generate coordinate points (a, Pa), where Pa represents the influence value corresponding to the a-th monitoring video frame, and fit the coordinate points to obtain a fitting curve f(t), where t represents time;

[0051] Obtain the monotonicity of the fitting curve. When the fitting curve is monotonically decreasing and the average influence value is less than the influence value threshold, the corresponding target area is not regarded as an abnormal area.

[0052] It can be understood that by extracting the influence value of each frame from the monitoring video and quantifying it into coordinate points, the system can track and analyze the environmental impact at different time points in numerical form; through the fitting curve, the system can associate the influence value of each frame with the time axis to form a continuous influence trend, which not only makes the change process of the soot concentration clearer and more intuitive but also helps the system predict future change trends and provides data support for timely issuing early warning information; by analyzing the change trend of the curve, the system can identify the change pattern of the soot concentration. If the fitting curve shows a monotonically decreasing trend, it indicates that the dust concentration in the environment is continuously decreasing, thus not constituting a dangerous area.

[0053] In another preferred embodiment of the present invention, in step S1, the monitoring video of the open-pit coal mine is obtained based on monitoring equipment, and the posture of the monitoring equipment remains unchanged.

[0054] It is worth noting that by fixing the posture of the monitoring equipment, the stability and consistency of video acquisition can be ensured; the fixed equipment perspective can eliminate the instability of the video image caused by the change of the equipment posture, thus ensuring the continuity and comparability of the image data.

[0055] In another preferred embodiment of the present invention, in step S3, when the number of abnormal personnel exceeds a preset threshold of the number of abnormal personnel, a prompt message is sent to a preset management personnel.

[0056] It can be understood that at this time, it indicates that there are many employees who do not pay attention to personal protection and enter the warning area. Therefore, a prompt message is sent to the management personnel to remind them that the personal protection awareness of the employees is poor, and the management personnel can perform corresponding operations, such as training the employees.

[0057] The above has described in detail an embodiment of the present invention, but the content described is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the present invention application should still fall within the scope covered by the patent of the present invention.

Claims

1. An AI-based open-pit coal mine monitoring video analysis method, characterized in that, Including the following steps: S1: Establish a database, in which monitored images of open-pit coal mines with marked soot areas are stored. Based on a deep learning model, establish a soot area recognition model, and train and verify the soot area recognition model through the database; S2: Obtain the monitored video of the open-pit coal mine within a preset monitoring period, perform frame splitting on the monitored video to obtain monitored video frames, input the monitored video frames into the verified soot area recognition model, and obtain the soot area in the monitored video frames, denoted as the target area; Perform grayscale processing on the monitored video frame to obtain a grayscale frame, determine the grayscale values of the pixel points belonging to the target area, and calculate the theoretical concentration Dys represents a preset standard grayscale value, c represents the dust concentration corresponding to the standard grayscale value, Di represents the grayscale value of the i-th pixel point belonging to the target area, and n represents the total number of pixel points belonging to the target area; S3: Obtain the ratio of the area of the target area to the monitored video frame, denoted as the target ratio b, then the influence value P = η * b * C, where η represents a preset correction coefficient; When the influence value is greater than or equal to the preset influence value threshold, regard it as an outlier, regard the target area corresponding to the outlier as an abnormal area, and regard the intersection of the abnormal areas corresponding to the same monitoring device as the warning area. When it is recognized that an employee enters the warning area, send a warning message for prompt.

2. The method for analyzing monitoring videos of open-pit coal mines based on AI according to claim 1, wherein, In step S3, the process of identifying that a user enters the warning area specifically includes: Obtain the real-time monitored video of the open-pit coal mine, perform frame splitting on the real-time monitored video to obtain real-time frames, input the real-time frames into a pre-trained person recognition model, and identify the person in the real-time frames. When the person is in the warning area, regard it as a pending person; When the pending person has no protection measures, mark it as an abnormal person and send a warning message for prompt to the abnormal person.

3. The method for analyzing open-pit coal mine monitoring video based on AI according to claim 2, characterized in that, The process of pre-training the person recognition model specifically includes: Establish a second database, in which monitored images of open-pit coal mines with marked persons are stored; Based on a deep learning model, establish a person recognition model, and train and verify the person recognition model based on the second database to obtain a pre-trained person recognition model.

4. The method for analyzing surveillance video of open-pit coal mines based on AI according to claim 3, wherein, The monitored images of open-pit coal mines with marked soot areas stored in the database are marked manually, and the monitored images of open-pit coal mines with marked persons stored in the second database are marked manually.

5. The method for analyzing monitoring videos of open-pit coal mines based on AI according to claim 4, characterized in that, Train the soot area recognition model and the person recognition model based on the backpropagation algorithm, and verify the soot area recognition model and the person recognition model based on five-fold cross-validation.

6. The method for analyzing monitoring videos of open-pit coal mines based on AI according to claim 1, wherein In step S3, the following steps are further included: Generate coordinate points (a, Pa), where Pa represents the influence value corresponding to the a-th monitored video frame, and fit the coordinate points to obtain a fitting curve f(t), where t represents time; Obtain the monotonicity of the fitting curve. When the fitting curve is monotonically decreasing and the average influence value is less than the influence value threshold, the corresponding target area is not regarded as an abnormal area.

7. The method for analyzing surveillance video of open-pit coal mines based on AI according to claim 1, wherein In step S1, the monitored video of the open-pit coal mine is obtained based on a monitoring device, and the posture of the monitoring device remains unchanged.

8. The method for analyzing surveillance video of open-pit coal mines based on AI according to claim 2, characterized in that, In step S3, when the number of abnormal persons exceeds the preset abnormal person number threshold, send a prompt message to the preset management personnel.

Images