Mine system monitoring method and device, electronic equipment and storage medium
By using a well-trained mine system monitoring model and feature extraction and fusion modules to identify the mine system status, the problem of low monitoring accuracy caused by human monitoring is solved, and automated and accurate mine system status monitoring is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INFORMATION TECH OPERATION & MAINTENANCE BRANCH OF SHAANXI COAL & NORTHERN SHAANXI MINING CO LTD
- Filing Date
- 2023-11-30
- Publication Date
- 2026-05-01
AI Technical Summary
Existing monitoring methods in mining systems rely on human surveillance, resulting in low monitoring accuracy and high labor intensity.
A mine system monitoring model trained based on sample mine system images and label data is adopted. Through feature extraction, feature fusion and classification modules, the operating status of the mine system is identified, and automated monitoring is achieved.
It enables accurate identification and timely response to the operational status of the mining system, reduces human error, and improves the accuracy and efficiency of monitoring.
Smart Images
Figure CN121963064A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of coal mining technology, and in particular to a method, device, electronic equipment and storage medium for monitoring mining systems. Background Technology
[0002] In the current construction of intelligent mines, the systems that require human monitoring include: (1) Website (platform) type: The visual interface carrier of this type of system is a web page. The web page includes the superior supervision platform. The superior supervision platform supervises the coal mining enterprise based on various data uploaded by the coal mining enterprise. The network status of a certain system is displayed on the web page, and the coal mining enterprise must ensure normal network connection at all times. (2) Client type: The data of this type of system comes from the real-time acquisition of various sensors and the judgment of the system network status. When the client software obtains the sensor data and the system network status, it will convert them into visual icons, graphs, or text and display them on the front-end interface. (3) Video type: This type of system monitors certain specific states of the production environment by acquiring camera images in real time.
[0003] However, all of the above-mentioned monitoring systems require human intervention to respond to changes in the system status, which increases the uncertainty of human intervention and the intensity of human labor, resulting in low accuracy of mine system status monitoring. Summary of the Invention
[0004] This invention provides a method, device, electronic equipment, and storage medium for monitoring mining systems, in order to solve the problem of low accuracy in monitoring the status of mining systems.
[0005] This invention provides a method for monitoring a mining system, comprising:
[0006] Acquire target images of at least one monitored mining system;
[0007] Each of the target images is input into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model; wherein, the mine system monitoring model is trained based on sample mine system images and label data, and the mine system monitoring model is used to identify the operating status of each monitored mine system;
[0008] Based on the aforementioned operating status, each of the monitored mining systems is monitored.
[0009] According to a mining system monitoring method provided by the present invention, the mining system monitoring model includes a feature extraction module, a feature fusion module, and a classification module; the step of inputting each of the target images into the mining system monitoring model to obtain the operating status of each monitored mining system output by the mining system monitoring model includes:
[0010] For each target image, the target image is input to the feature extraction module to obtain at least one feature image output by the feature extraction module; each feature image has a different dimension;
[0011] Each of the aforementioned feature images is input into the feature fusion module to obtain at least one fused image output by the feature fusion module; the scales of the fused images are different.
[0012] The fused images are input into the classification module to obtain the operating status of the monitored mining system corresponding to the target image output by the classification module.
[0013] According to a mining system monitoring method provided by the present invention, the feature extraction module includes a slicing unit, at least one convolutional unit, at least one depth unit, and a pooling unit, wherein the number of convolutional units is the same as the number of depth units;
[0014] The step of inputting the target image into the feature extraction module to obtain at least one feature image output by the feature extraction module includes:
[0015] The target image is input into the slicing unit to obtain a first image output by the slicing unit; the size of the first image is smaller than the size of the target image.
[0016] The first image is input into the first convolutional unit to obtain the first convolutional image output by the first convolutional unit;
[0017] The first convolutional image is input into the first depth unit to obtain the first depth image output by the first depth unit;
[0018] The first depth image is input into the second convolutional unit to obtain the second convolutional image output by the second convolutional unit;
[0019] The second convolutional image is input into the second depth unit to obtain the second depth image output by the second depth unit;
[0020] The second depth image is input into the third convolutional unit to obtain the third convolutional image output by the third convolutional unit;
[0021] The third convolutional image is input into the third depth unit to obtain the third depth image output by the third depth unit;
[0022] The third depth image is input into the fourth convolutional unit to obtain the fourth convolutional image output by the fourth convolutional unit;
[0023] The fourth convolutional image is input into the pooling unit to obtain the pooled image output by the pooling unit;
[0024] The pooled image is input to the fourth depth unit to obtain the fourth depth image output by the fourth depth unit; wherein the second depth image, the third depth image and the fourth depth image are determined as the feature images.
[0025] According to a mining system monitoring method provided by the present invention, the feature fusion module includes at least one convolutional unit, at least one connection layer, at least one depth unit, and at least one upsampling layer;
[0026] The step of inputting each of the feature images into the feature fusion module to obtain at least one fused image output by the feature fusion module includes:
[0027] The fourth depth image is input into the fifth convolutional unit to obtain the fifth convolutional image output by the fifth convolutional unit;
[0028] The fifth convolutional image is input into the first upsampling layer to obtain the first upsampling image output by the first upsampling layer.
[0029] The first upsampled image and the third depth image are input into the first connection layer to obtain the first connection image output by the first connection layer.
[0030] The first connected image is input to the fifth depth unit to obtain the fifth depth image output by the fifth depth unit;
[0031] Based on the fifth depth image, each of the fused images is determined.
[0032] According to a mining system monitoring method provided by the present invention, determining each fused image based on the fifth depth image includes:
[0033] The fifth depth image is input into the sixth convolutional unit to obtain the sixth convolutional image output by the sixth convolutional unit;
[0034] The sixth convolutional image is input into the second upsampling layer to obtain the second upsampling image output by the second upsampling layer.
[0035] The first depth image and the second upsampled image are input into the second connection layer to obtain the second connection image output by the second connection layer;
[0036] The second connected image is input to the sixth depth unit to obtain the sixth depth image output by the sixth depth unit;
[0037] The sixth depth image is input into the seventh convolutional unit to obtain the seventh convolutional image output by the seventh convolutional unit;
[0038] The sixth and seventh convolutional images are input into the third connection layer to obtain the third connection image output by the third connection layer.
[0039] The third connected image is input to the seventh depth unit to obtain the seventh depth image output by the seventh depth unit;
[0040] The seventh depth image is input into the eighth convolutional unit to obtain the eighth convolutional image output by the eighth convolutional unit;
[0041] The eighth convolutional image and the fifth convolutional image are input into the fourth connection layer to obtain the fourth connection image output by the fourth connection layer;
[0042] The fourth connected image is input to the eighth depth unit to obtain the eighth depth image output by the eighth depth unit; wherein the sixth depth image, the seventh depth image and the eighth depth image are determined as the fused image.
[0043] According to a mining system monitoring method provided by the present invention, the target image includes at least one of the following: status icon, equipment indicator light, and mining system name; the tag data includes tags corresponding to the overall interface of the monitored mining system and tags for each object to be monitored in the overall interface.
[0044] According to a mining system monitoring method provided by the present invention, the method further includes:
[0045] When the operating status of the mining system is abnormal, an alarm signal is played or an abnormal information is sent to the terminal; the alarm signal or the abnormal information is used to indicate that the mining system is operating abnormally.
[0046] The present invention also provides a mining system monitoring device, comprising:
[0047] The acquisition module is used to acquire target images of at least one monitored mining system;
[0048] The identification module is used to input each of the target images into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model; wherein, the mine system monitoring model is trained based on sample mine system images and label data, and the mine system monitoring model is used to identify the operating status of each monitored mine system;
[0049] The monitoring module is used to monitor each of the monitored mining systems based on the operating status.
[0050] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the mining system monitoring method described above.
[0051] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the mining system monitoring method as described above.
[0052] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the mining system monitoring method as described above.
[0053] The present invention provides a mining system monitoring method, apparatus, electronic device, and storage medium. The method involves acquiring target images of at least one monitored mining system; inputting each target image into a mining system monitoring model to obtain the operational status of each monitored mining system output by the monitoring model; wherein the mining system monitoring model is trained based on sample mining system images and label data, and is used to identify the operational status of each monitored mining system; and based on the operational status, monitoring is performed on each monitored mining system. Through the mining system monitoring model, accurate identification of the operational status of the monitored mining system is achieved, independent of human factors, thus enabling accurate monitoring of the operational status of the monitored mining system. It also allows for timely response when the system status changes, improving the accuracy and efficiency of mining system operational status monitoring. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0055] Figure 1This is one of the flowcharts of the mining system monitoring method provided by the present invention;
[0056] Figure 2 This is a schematic diagram of the feature extraction module provided by the present invention;
[0057] Figure 3 This is a schematic diagram of the feature fusion module provided by the present invention;
[0058] Figure 4 This is the second flowchart of the mining system monitoring method provided by the present invention;
[0059] Figure 5 This is a schematic diagram of the structure of the mining system monitoring device provided by the present invention;
[0060] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0062] The following is combined Figures 1-4 The present invention describes a mining system monitoring method.
[0063] Figure 1 This is one of the flowcharts of the mining system monitoring method provided by the present invention, such as... Figure 1 As shown, the method includes steps 101-103; wherein,
[0064] Step 101: Obtain a target image of at least one monitored mining system.
[0065] It should be noted that the mining system monitoring method provided by the present invention is applicable to mining system monitoring scenarios. The executing entity of the method can be a mining system monitoring device, such as an electronic device, or a control module in the mining system monitoring device for executing the mining system monitoring method.
[0066] Specifically, the target image is the overall image of the monitored mining system's display interface. The target image includes at least one of the following: status icons, equipment indicator lights, and the mining system name; wherein, the status icons are icons indicating the operating status of the mining system, such as icons for safety monitoring, personnel positioning, video surveillance, water hazard prevention, rock burst prevention, and major equipment; the equipment indicator lights are indicator lights of the equipment displayed on the mining system's display interface, for example, a red indicator light when equipment is faulty and a green indicator light when there is no fault; the mining system name refers to the name of the monitored mining system, such as "Mine Safety Production Risk Monitoring and Early Warning System."
[0067] By capturing the display interface of at least one monitored mining system, target images of each monitored mining system can be obtained.
[0068] Step 102: Input each of the target images into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model; wherein, the mine system monitoring model is trained based on sample mine system images and label data, and the mine system monitoring model is used to identify the operating status of each monitored mine system.
[0069] Specifically, each target image is input into the mine system monitoring model. The model first determines whether it contains the system interface of the monitored mine system. If it does, it identifies the operating status of the monitored mine system, thus obtaining the operating status of each monitored mine system output by the mine system monitoring model. The operating status includes normal or abnormal. For example, if the mine system monitoring model identifies green status icons and equipment indicator lights in the target image, the operating status of the mine system is determined to be normal; if the mine system monitoring model identifies red status icons and equipment indicator lights in the target image, the operating status of the mine system is determined to be abnormal.
[0070] During the training of the mine system monitoring model, sample images of the mine system at different locations and resolutions on the display interface of each monitored mine system are captured. For example, 1000 sample images are acquired, and the image annotation tool (Labelmg) is used to annotate these images, thereby obtaining the sample images and their corresponding label data. To improve the recognition effect, the sample images should simultaneously include both monitored and non-monitored targets, ensuring the diversity of the dataset.
[0071] It should be noted that the sample mine system image also includes at least one of the following: status icons, equipment indicator lights, and the mine system name. After acquiring the sample mine system image, the status icons, equipment indicator lights, and the mine system name included in the sample mine system image are labeled to obtain label data. The label data includes the label corresponding to the overall interface of the monitored mine system and the label for each object to be monitored included in the overall interface. The object to be monitored is a status icon, equipment indicator light, or the mine system name. The label corresponding to the overall interface of the monitored mine system is the label corresponding to the mine system name; for example, the label corresponding to the overall interface of the monitored mine system is "shengju".
[0072] For example, each monitored mining system contains 1000 sample images. When the monitored mining system is a mine safety production risk monitoring and early warning system, the icons for safety monitoring, personnel positioning, video monitoring, water hazard prevention, rock bursts, and major equipment in the sample images are labeled with 0 and 1, respectively, where 0 indicates normal and 1 indicates abnormal, for a total of 12 labels. When the monitored mining system is a coal mine safety information sharing platform, the icons for safety monitoring, personnel positioning, water temperature monitoring, and mine pressure monitoring in the sample images of the coal mine safety information sharing platform are labeled with 0 and 1, respectively, where 0 indicates normal and 1 indicates abnormal, for a total of 8 labels.
[0073] Step 103: Based on the operating status, monitor each of the monitored mining systems.
[0074] Specifically, each monitored mining system can be monitored based on its operational status.
[0075] The mining system monitoring method provided by this invention involves acquiring target images of at least one monitored mining system; inputting each target image into a mining system monitoring model to obtain the operational status of each monitored mining system output by the monitoring model; wherein the mining system monitoring model is trained based on sample mining system images and label data, and is used to identify the operational status of each monitored mining system; based on the operational status, each monitored mining system is monitored. Through the mining system monitoring model, accurate identification of the operational status of the monitored mining system is achieved, independent of human factors, thus enabling accurate monitoring of the operational status of the monitored mining system. It can also respond promptly when the system status changes, improving the accuracy and efficiency of mining system operational status monitoring.
[0076] Optionally, the mine system monitoring model includes a feature extraction module, a feature fusion module, and a classification module; the step of inputting each of the target images into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model includes:
[0077] For each target image, the target image is input to the feature extraction module to obtain at least one feature image output by the feature extraction module; each feature image has a different dimension; each feature image is input to the feature fusion module to obtain at least one fused image output by the feature fusion module; each fused image has a different scale; each fused image is input to the classification module to obtain the operating status of the monitored mining system corresponding to the target image output by the classification module.
[0078] Specifically, for each target image of the monitored mining system, the target image is input into the feature extraction module to obtain at least one feature image output by the feature extraction module; each feature image has a different dimension; then each feature image is input into the feature fusion module to obtain at least one fused image output by the feature fusion module; each fused image has a different scale; then each fused image is input into the classification module to obtain the operating status of the monitored mining system corresponding to the target image output by the classification module.
[0079] The mining system monitoring method provided by this invention, through a feature extraction module, a feature fusion module, and a classification module, achieves accurate identification of the operating status of the monitored mining system, thereby improving the accuracy and efficiency of mining system operating status monitoring.
[0080] Optionally, the feature extraction module includes slicing units, at least one convolutional unit, at least one depth unit, and pooling units, wherein the number of convolutional units and the number of depth units are the same; the step of inputting the target image into the feature extraction module to obtain at least one feature image output by the feature extraction module includes:
[0081] (1) Input the target image into the slicing unit to obtain the first image output by the slicing unit; the size of the first image is smaller than the size of the target image.
[0082] Specifically, by inputting the target image into the slicing unit, the first image output by the slicing unit can be obtained. For example, if the target image is a 640*640*3 image, the first image is a 304*304*12 feature image.
[0083] (2) Input the first image into the first convolution unit to obtain the first convolution image output by the first convolution unit.
[0084] Specifically, by inputting the first image into the first convolutional unit, a first convolutional image can be obtained from the output of the first convolutional unit. The convolutional unit is used to extract feature information from the input image.
[0085] (3) Input the first convolutional image into the first depth unit to obtain the first depth image output by the first depth unit.
[0086] Specifically, by inputting the first convolutional image into the first depth unit, a first depth image output by the first depth unit can be obtained. The depth unit is used to increase the depth and receptive field of the network, thereby improving the feature extraction capability.
[0087] (4) Input the first depth image into the second convolution unit to obtain the second convolution image output by the second convolution unit.
[0088] Specifically, by inputting the first depth image into the second convolutional unit, the second convolutional image output by the second convolutional unit can be obtained.
[0089] (5) Input the second convolutional image into the second depth unit to obtain the second depth image output by the second depth unit.
[0090] Specifically, by inputting the second convolutional image into the second depth unit, the second depth image output by the second depth unit can be obtained.
[0091] (6) Input the second depth image into the third convolution unit to obtain the third convolution image output by the third convolution unit.
[0092] Specifically, by inputting the second depth image into the third convolutional unit, the third convolutional image output by the third convolutional unit can be obtained.
[0093] (7) Input the third convolutional image into the third depth unit to obtain the third depth image output by the third depth unit.
[0094] Specifically, by inputting the third convolutional image into the third depth unit, the third depth image output by the third depth unit can be obtained.
[0095] (8) Input the third depth image into the fourth convolution unit to obtain the fourth convolution image output by the fourth convolution unit.
[0096] Specifically, by inputting the third depth image into the fourth convolutional unit, the fourth convolutional image output by the fourth convolutional unit can be obtained.
[0097] (9) Input the fourth convolutional image into the pooling unit to obtain the pooling image output by the pooling unit.
[0098] Specifically, by inputting the fourth convolutional image into the pooling unit, the pooled image output by the pooling unit can be obtained. The pooling unit is used to capture feature information at different scales.
[0099] (10) Input the pooled image into the fourth depth unit to obtain the fourth depth image output by the fourth depth unit; wherein the second depth image, the third depth image and the fourth depth image are determined as the feature images.
[0100] Specifically, by inputting the pooled image into the fourth depth unit, the fourth depth image output by the fourth depth unit can be obtained;
[0101] Among them, the second depth image, the third depth image, and the fourth depth image are determined as feature images.
[0102] Figure 2 This is a schematic diagram of the feature extraction module provided by the present invention, as shown below. Figure 2 As shown, the feature extraction module includes one slicing unit, four convolutional units, four depth units, and one pooling unit. The target image is input to a slicing unit to obtain a first image output by the slicing unit; the first image is input to a first convolution unit to obtain a first convolutional image output by the first convolution unit; the first convolutional image is input to a first depth unit to obtain a first depth image output by the first depth unit; the first depth image is input to a second convolution unit to obtain a second convolutional image output by the second convolution unit; the second convolutional image is input to a second depth unit to obtain a second depth image output by the second depth unit; the second depth image is input to a third convolution unit to obtain a third convolutional image output by the third convolution unit; the third convolutional image is input to a third depth unit to obtain a third depth image output by the third depth unit; the third depth image is input to a fourth convolution unit to obtain a fourth convolutional image output by the fourth convolution unit; the fourth convolutional image is input to a pooling unit to obtain a pooling image output by the pooling unit; the pooling image is input to a fourth depth unit to obtain a fourth depth image output by the fourth depth unit; wherein, the second depth image, the third depth image, and the fourth depth image are defined as feature images.
[0103] Optionally, the feature fusion module includes at least one convolutional unit, at least one connection layer, at least one depth unit, and at least one upsampling layer;
[0104] Each of the aforementioned feature images is input to the feature fusion module to obtain at least one fused image output by the feature fusion module, including:
[0105] (a) Input the fourth depth image into the fifth convolutional unit to obtain the fifth convolutional image output by the fifth convolutional unit.
[0106] Specifically, by inputting the fourth depth image into the fifth convolutional unit, the fifth convolutional image output by the fifth convolutional unit can be obtained.
[0107] (b) Input the fifth convolutional image into the first upsampling layer to obtain the first upsampling image output by the first upsampling layer.
[0108] Specifically, by inputting the fifth convolutional image into the first upsampling layer, the first upsampled image output by the first upsampling layer can be obtained.
[0109] (c) Input the first upsampled image and the third depth image into the first connection layer to obtain the first connection image output by the first connection layer.
[0110] Specifically, by inputting the first upsampled image and the third depth image into the first connection layer (Concat), the first connected image output by the first connection layer can be obtained.
[0111] (d) Input the first connected image into the fifth depth unit to obtain the fifth depth image output by the fifth depth unit.
[0112] Specifically, by inputting the first connected image into the fifth depth unit, the fifth depth image output by the fifth depth unit can be obtained.
[0113] (e) Based on the fifth depth image, determine each of the fused images.
[0114] Specifically, based on the fifth depth image, each fused image can be further determined.
[0115] Optionally, the specific implementation of step (e) above includes:
[0116] (e-1) Input the fifth depth image into the sixth convolution unit to obtain the sixth convolution image output by the sixth convolution unit.
[0117] Specifically, by inputting the fifth depth image into the sixth convolutional unit, the sixth convolutional image output by the sixth convolutional unit can be obtained.
[0118] (e-2) Input the sixth convolutional image into the second upsampling layer to obtain the second upsampling image output by the second upsampling layer.
[0119] Specifically, by inputting the sixth convolutional image into the second upsampling layer, the second upsampling image output by the second upsampling layer can be obtained.
[0120] (e-3) Input the first depth image and the second upsampled image into the second connection layer to obtain the second connection image output by the second connection layer.
[0121] Specifically, by inputting the first depth image and the second upsampled image into the second connection layer (Concat), the second connection image output by the second connection layer can be obtained.
[0122] (e-4) Input the second connected image into the sixth depth unit to obtain the sixth depth image output by the sixth depth unit.
[0123] Specifically, by inputting the second connected image into the sixth depth unit, the sixth depth image output by the sixth depth unit can be obtained.
[0124] (e-5) Input the sixth depth image into the seventh convolution unit to obtain the seventh convolution image output by the seventh convolution unit.
[0125] Specifically, by inputting the sixth depth image into the seventh convolutional unit, the seventh convolutional image output by the seventh convolutional unit can be obtained.
[0126] (e-6) Input the sixth convolutional image and the seventh convolutional image into the third connection layer to obtain the third connection image output by the third connection layer.
[0127] Specifically, by inputting the sixth and seventh convolutional images into the third connected layer (Concat), the third connected image output by the third connected layer can be obtained.
[0128] (e-7) Input the third connected image into the seventh depth unit to obtain the seventh depth image output by the seventh depth unit.
[0129] Specifically, by inputting the third connected image into the seventh depth unit, the seventh depth image output by the seventh depth unit can be obtained.
[0130] (e-8) Input the seventh depth image into the eighth convolution unit to obtain the eighth convolution image output by the eighth convolution unit.
[0131] Specifically, the seventh depth image is input into the eighth convolutional unit to obtain the eighth convolutional image output by the eighth convolutional unit.
[0132] (e-9) Input the eighth convolutional image and the fifth convolutional image into the fourth connection layer to obtain the fourth connection image output by the fourth connection layer.
[0133] Specifically, by inputting the eighth convolutional image and the fifth convolutional image into the fourth connected layer, the fourth connected image output by the fourth connected layer can be obtained.
[0134] (e-10) The fourth connected image is input to the eighth depth unit to obtain the eighth depth image output by the eighth depth unit; wherein the sixth depth image, the seventh depth image and the eighth depth image are determined as the fused image.
[0135] Specifically, by inputting the fourth connected image into the eighth depth unit, the eighth depth image output by the eighth depth unit can be obtained.
[0136] Among them, the sixth depth image, the seventh depth image, and the eighth depth image were identified as the fused images.
[0137] Figure 3 This is a schematic diagram of the feature fusion module provided by the present invention, as shown below. Figure 3 As shown, the feature fusion module includes 4 convolutional units, 4 connection layers, 4 depth units, and 2 upsampling layers. The fourth depth image is input to the fifth convolutional unit to obtain the fifth convolutional image output by the fifth convolutional unit; the fifth convolutional image is input to the first upsampling layer to obtain the first upsampling image output by the first upsampling layer; the first upsampling image and the third depth image are input to the first connection layer to obtain the first connection image output by the first connection layer; the first connection image is input to the fifth depth unit to obtain the fifth depth image output by the fifth depth unit; the fifth depth image is input to the sixth convolutional unit to obtain the sixth convolutional image output by the sixth convolutional unit; the sixth convolutional image is input to the second upsampling layer to obtain the second upsampling image output by the second upsampling layer; the first depth image and the second upsampling image are input to the second connection layer to obtain the second connection image output by the second connection layer; the second connection image is input to the sixth depth unit... The process involves: inputting the sixth depth image into a sixth depth unit; inputting the sixth depth image into a seventh convolutional unit to obtain the seventh convolutional image; inputting the sixth and seventh convolutional images into a third connection layer to obtain the third connection image; inputting the third connection image into a seventh depth unit to obtain the seventh depth image; inputting the seventh depth image into an eighth convolutional unit to obtain the eighth convolutional image; inputting the eighth and fifth convolutional images into a fourth connection layer to obtain the fourth connection image; and inputting the fourth connection image into an eighth depth unit to obtain the eighth depth image. The sixth, seventh, and eighth depth images are then defined as the fused image.
[0138] Optionally, when the operating status of the mining system is abnormal, an alarm signal is played or an abnormal information is sent to the terminal; the alarm signal or abnormal information is used to indicate that the mining system is operating abnormally.
[0139] Specifically, after obtaining the operational status of each monitored mine system from the mine system monitoring model, an alarm signal can be played or an abnormal information can be sent to the terminal when the operational status of the mine system is abnormal. For example, abnormal information can be pushed to the terminal's WeChat group or WeChat account, where the push of abnormal information can be achieved by linking to a WeChat group chat robot.
[0140] Optionally, when the operating status of the mining system is abnormal, a preset automated processing program can be executed to handle the abnormal operating status.
[0141] The mining system monitoring method provided by this invention identifies the monitored system and elements in the system interface (status icons, equipment indicator lights, mining system name), and makes corresponding processing results (audible and visual alarms, information push, automated processes or tasks) based on the identification results. This reduces human factors and achieves unattended operation, improving the monitoring efficiency and accuracy of the monitored system's operating status. Furthermore, since it does not directly read the data of each monitored system, it avoids network security risks such as data leakage and privacy.
[0142] Figure 4 This is the second flowchart of the mining system monitoring method provided by the present invention, as shown below. Figure 4 As shown, the method includes steps 401-407; wherein,
[0143] Step 401: Run the monitoring program, which can identify the status information of the mining system operation.
[0144] Step 402: Capture a target image of at least one monitored mining system currently displayed on the screen, wherein the target image includes at least one of the following: status icon, equipment indicator light, and mining system name.
[0145] Step 403: Input each target image into the mine system monitoring model. The mine system monitoring model determines whether it contains the system interface of the monitored mine system. If the system interface of the monitored mine system is contained in the target image, proceed to step 404; if the monitored mine system is not contained in the target image, proceed to step 407.
[0146] Step 404: The mine system monitoring model identifies the operating status of each monitored mine system to obtain the operating status of each monitored mine system.
[0147] Step 405: Determine if the running status is normal. If the running status is abnormal, proceed to step 406; if the running status is normal, proceed to step 407.
[0148] Step 406: Play an alarm signal or send an abnormal message to the terminal.
[0149] Step 407: Continue monitoring.
[0150] The following describes the mine system monitoring device provided by the present invention. The mine system monitoring device described below and the mine system monitoring method described above can be referred to in correspondence.
[0151] Figure 5 This is a schematic diagram of the structure of the mining system monitoring device provided by the present invention, as shown below. Figure 5 As shown, the mine system monitoring device 500 includes: an acquisition module 501, an identification module 502, and a monitoring module 503; wherein,
[0152] The acquisition module 501 is used to acquire target images of at least one monitored mining system;
[0153] The identification module 502 is used to input each of the target images into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model; wherein, the mine system monitoring model is trained based on sample mine system images and label data, and the mine system monitoring model is used to identify the operating status of each monitored mine system;
[0154] The monitoring module 503 is used to monitor each of the monitored mining systems based on the operating status.
[0155] The mine system monitoring device provided by this invention acquires target images of at least one monitored mine system; inputs each target image into a mine system monitoring model to obtain the operational status of each monitored mine system output by the mine system monitoring model; wherein, the mine system monitoring model is trained based on sample mine system images and label data, and is used to identify the operational status of each monitored mine system; based on the operational status, each monitored mine system is monitored. Through the mine system monitoring model, accurate identification of the operational status of the monitored mine system is achieved, independent of human factors, thus enabling accurate monitoring of the operational status of the monitored mine system. It can also respond promptly when the system status changes, improving the accuracy and efficiency of mine system operational status monitoring.
[0156] Optionally, the mine system monitoring model includes a feature extraction module, a feature fusion module, and a classification module; the identification module 502 is specifically used for:
[0157] For each target image, the target image is input to the feature extraction module to obtain at least one feature image output by the feature extraction module; each feature image has a different dimension;
[0158] Each of the aforementioned feature images is input into the feature fusion module to obtain at least one fused image output by the feature fusion module; the scales of the fused images are different.
[0159] The fused images are input into the classification module to obtain the operating status of the monitored mining system corresponding to the target image output by the classification module.
[0160] Optionally, the feature extraction module includes a slicing unit, at least one convolutional unit, at least one depth unit, and a pooling unit, wherein the number of convolutional units and the number of depth units are the same; the recognition module 502 is further configured to:
[0161] The target image is input into the slicing unit to obtain a first image output by the slicing unit; the size of the first image is smaller than the size of the target image.
[0162] The first image is input into the first convolutional unit to obtain the first convolutional image output by the first convolutional unit;
[0163] The first convolutional image is input into the first depth unit to obtain the first depth image output by the first depth unit;
[0164] The first depth image is input into the second convolutional unit to obtain the second convolutional image output by the second convolutional unit;
[0165] The second convolutional image is input into the second depth unit to obtain the second depth image output by the second depth unit;
[0166] The second depth image is input into the third convolutional unit to obtain the third convolutional image output by the third convolutional unit;
[0167] The third convolutional image is input into the third depth unit to obtain the third depth image output by the third depth unit;
[0168] The third depth image is input into the fourth convolutional unit to obtain the fourth convolutional image output by the fourth convolutional unit;
[0169] The fourth convolutional image is input into the pooling unit to obtain the pooled image output by the pooling unit;
[0170] The pooled image is input to the fourth depth unit to obtain the fourth depth image output by the fourth depth unit; wherein the second depth image, the third depth image and the fourth depth image are determined as the feature images.
[0171] Optionally, the feature fusion module includes at least one convolutional unit, at least one connection layer, at least one depth unit, and at least one upsampling layer; the recognition module 502 is further configured to:
[0172] The fourth depth image is input into the fifth convolutional unit to obtain the fifth convolutional image output by the fifth convolutional unit;
[0173] The fifth convolutional image is input into the first upsampling layer to obtain the first upsampling image output by the first upsampling layer.
[0174] The first upsampled image and the third depth image are input into the first connection layer to obtain the first connection image output by the first connection layer.
[0175] The first connected image is input to the fifth depth unit to obtain the fifth depth image output by the fifth depth unit;
[0176] Based on the fifth depth image, each of the fused images is determined.
[0177] Optionally, the identification module 502 is further configured to:
[0178] The fifth depth image is input into the sixth convolutional unit to obtain the sixth convolutional image output by the sixth convolutional unit;
[0179] The sixth convolutional image is input into the second upsampling layer to obtain the second upsampling image output by the second upsampling layer.
[0180] The first depth image and the second upsampled image are input into the second connection layer to obtain the second connection image output by the second connection layer;
[0181] The second connected image is input to the sixth depth unit to obtain the sixth depth image output by the sixth depth unit;
[0182] The sixth depth image is input into the seventh convolutional unit to obtain the seventh convolutional image output by the seventh convolutional unit;
[0183] The sixth and seventh convolutional images are input into the third connection layer to obtain the third connection image output by the third connection layer.
[0184] The third connected image is input to the seventh depth unit to obtain the seventh depth image output by the seventh depth unit;
[0185] The seventh depth image is input into the eighth convolutional unit to obtain the eighth convolutional image output by the eighth convolutional unit;
[0186] The eighth convolutional image and the fifth convolutional image are input into the fourth connection layer to obtain the fourth connection image output by the fourth connection layer;
[0187] The fourth connected image is input to the eighth depth unit to obtain the eighth depth image output by the eighth depth unit; wherein the sixth depth image, the seventh depth image and the eighth depth image are determined as the fused image.
[0188] Optionally, the target image includes at least one of the following: status icon, device indicator light, and mining system name; the tag data includes tags corresponding to the overall interface of the monitored mining system and tags for each object to be monitored in the overall interface.
[0189] Optionally, the mine system monitoring device 500 further includes:
[0190] The push module is used to play an alarm signal or send abnormal information to the terminal when the operating status of the mining system is abnormal; the alarm signal or the abnormal information is used to indicate that the mining system is operating abnormally.
[0191] Figure 6 This is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device 600 may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a mine system monitoring method, which includes: acquiring a target image of at least one monitored mine system; inputting each target image into a mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model; wherein the mine system monitoring model is trained based on sample mine system images and label data, and the mine system monitoring model is used to identify the operating status of each monitored mine system; and monitoring each monitored mine system based on the operating status.
[0192] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0193] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the mining system monitoring method provided by the above methods. The method includes: acquiring a target image of at least one monitored mining system; inputting each target image into a mining system monitoring model to obtain the operating status of each monitored mining system output by the mining system monitoring model; wherein the mining system monitoring model is trained based on sample mining system images and label data, and the mining system monitoring model is used to identify the operating status of each monitored mining system; and monitoring each monitored mining system based on the operating status.
[0194] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the mining system monitoring method provided by the above methods. The method includes: acquiring target images of at least one monitored mining system; inputting each target image into a mining system monitoring model to obtain the operating status of each monitored mining system output by the mining system monitoring model; wherein the mining system monitoring model is trained based on sample mining system images and label data, and the mining system monitoring model is used to identify the operating status of each monitored mining system; and monitoring each monitored mining system based on the operating status.
[0195] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for monitoring a mining system, characterized in that, include: Acquire target images of at least one monitored mining system; Each of the target images is input into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model; wherein, the mine system monitoring model is trained based on sample mine system images and label data, and the mine system monitoring model is used to identify the operating status of each monitored mine system; Based on the aforementioned operating status, each of the monitored mining systems is monitored.
2. The mining system monitoring method according to claim 1, characterized in that, The mine system monitoring model includes a feature extraction module, a feature fusion module, and a classification module; the step of inputting each of the target images into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model includes: For each target image, the target image is input to the feature extraction module to obtain at least one feature image output by the feature extraction module; each feature image has a different dimension; Each of the aforementioned feature images is input into the feature fusion module to obtain at least one fused image output by the feature fusion module; the scales of the fused images are different. The fused images are input into the classification module to obtain the operating status of the monitored mining system corresponding to the target image output by the classification module.
3. The mining system monitoring method according to claim 2, characterized in that, The feature extraction module includes a slicing unit, at least one convolutional unit, at least one depth unit, and a pooling unit, wherein the number of convolutional units is the same as the number of depth units; The step of inputting the target image into the feature extraction module to obtain at least one feature image output by the feature extraction module includes: The target image is input into the slicing unit to obtain a first image output by the slicing unit; the size of the first image is smaller than the size of the target image. The first image is input into the first convolutional unit to obtain the first convolutional image output by the first convolutional unit; The first convolutional image is input into the first depth unit to obtain the first depth image output by the first depth unit; The first depth image is input into the second convolutional unit to obtain the second convolutional image output by the second convolutional unit; The second convolutional image is input into the second depth unit to obtain the second depth image output by the second depth unit; The second depth image is input into the third convolutional unit to obtain the third convolutional image output by the third convolutional unit; The third convolutional image is input into the third depth unit to obtain the third depth image output by the third depth unit; The third depth image is input into the fourth convolutional unit to obtain the fourth convolutional image output by the fourth convolutional unit; The fourth convolutional image is input into the pooling unit to obtain the pooled image output by the pooling unit; The pooled image is input to the fourth depth unit to obtain the fourth depth image output by the fourth depth unit; wherein the second depth image, the third depth image and the fourth depth image are determined as the feature images.
4. The mining system monitoring method according to claim 3, characterized in that, The feature fusion module includes at least one convolutional unit, at least one connection layer, at least one depth unit, and at least one upsampling layer; The step of inputting each of the feature images into the feature fusion module to obtain at least one fused image output by the feature fusion module includes: The fourth depth image is input into the fifth convolutional unit to obtain the fifth convolutional image output by the fifth convolutional unit; The fifth convolutional image is input into the first upsampling layer to obtain the first upsampling image output by the first upsampling layer. The first upsampled image and the third depth image are input into the first connection layer to obtain the first connection image output by the first connection layer. The first connected image is input to the fifth depth unit to obtain the fifth depth image output by the fifth depth unit; Based on the fifth depth image, each of the fused images is determined.
5. The mining system monitoring method according to claim 4, characterized in that, The determination of each fused image based on the fifth depth image includes: The fifth depth image is input into the sixth convolutional unit to obtain the sixth convolutional image output by the sixth convolutional unit; The sixth convolutional image is input into the second upsampling layer to obtain the second upsampling image output by the second upsampling layer. The first depth image and the second upsampled image are input into the second connection layer to obtain the second connection image output by the second connection layer; The second connected image is input to the sixth depth unit to obtain the sixth depth image output by the sixth depth unit; The sixth depth image is input into the seventh convolutional unit to obtain the seventh convolutional image output by the seventh convolutional unit; The sixth and seventh convolutional images are input into the third connection layer to obtain the third connection image output by the third connection layer. The third connected image is input to the seventh depth unit to obtain the seventh depth image output by the seventh depth unit; The seventh depth image is input into the eighth convolutional unit to obtain the eighth convolutional image output by the eighth convolutional unit; The eighth convolutional image and the fifth convolutional image are input into the fourth connection layer to obtain the fourth connection image output by the fourth connection layer; The fourth connected image is input to the eighth depth unit to obtain the eighth depth image output by the eighth depth unit; wherein the sixth depth image, the seventh depth image and the eighth depth image are determined as the fused image.
6. The mining system monitoring method according to any one of claims 1 to 5, characterized in that, The target image includes at least one of the following: status icon, device indicator light, and mining system name; the tag data includes tags corresponding to the overall interface of the monitored mining system and tags for each object to be monitored in the overall interface.
7. The mining system monitoring method according to any one of claims 1 to 5, characterized in that, The method further includes: When the operating status of the mining system is abnormal, an alarm signal is played or an abnormal information is sent to the terminal; the alarm signal or the abnormal information is used to indicate that the mining system is operating abnormally.
8. A monitoring device for a mining system, characterized in that, include: The acquisition module is used to acquire target images of at least one monitored mining system; The identification module is used to input each of the target images into the mine system monitoring model to obtain the operating status of each monitored mine system output by the mine system monitoring model; wherein, the mine system monitoring model is trained based on sample mine system images and label data, and the mine system monitoring model is used to identify the operating status of each monitored mine system; The monitoring module is used to monitor each of the monitored mining systems based on the operating status.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the mining system monitoring method as described in any one of claims 1 to 7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the mining system monitoring method as described in any one of claims 1 to 7.