A mine loading area classification method and system based on deep learning
Through the mine loading area classification method based on deep learning, the feature extraction network and full connection layer are used for image processing, which solves the problem of inaccurate feature extraction in traditional methods, and realizes accurate identification of mine loading area and safety status judgment of mine loading area .
Patent Information
- Application Number
- CN202311016998.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2043-08-14
AI Technical Summary
Traditional image processing methods cannot accurately extract key features of mine loading areas, resulting in misjudgment of safety conditions.
Using the mine loading area classification method based on deep learning, the mine scene image is obtained for preliminary processing and feature extraction, a feature extraction network is built to obtain advanced feature maps, and the loading area classification prediction is used using the full connection layer and softmax function to set the threshold value. Judge safety level.
Accurate identification and safety status judgment of mine loading areas are achieved, the ability to extract scene image features of mining areas is improved, and the misjudgment rate is reduced.
Smart Images

Figure CN117274931B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of mining engineering technology, and particularly designs a mine loading area classification method and system based on deep learning. Background Art
[0002] The mine loading area is an important part of the mine production process. The judgment and evaluation of the safety status is of great significance to ensure the smooth progress of mine operations and the safety of miners. The traditional judgment of the safety status of the mine loading area mainly relies on manual on-site inspections. The evaluation results are limited by the experience and subjective judgment of the inspectors, which is prone to misjudgment or omission, and cannot meet the actual needs of mine loading area safety assessment.
[0003] Scene recognition is one of the important tasks in the field of computer vision. With the rapid development of deep learning, the accuracy of scene recognition has been greatly improved. The recognition of mine driving areas is an important part of unmanned mining. It can provide necessary scene semantic information for the perception module and decision-making module of unmanned driving, strengthen the full understanding of the unmanned driving environment in the mine, and ensure the safety and stability of unmanned driving in the mine. At present, manual means are mostly used to classify different areas in the unmanned driving environment of mines. Due to the large volume of scene data collected by autonomous driving, the efficiency of manual classification processing is low and it is difficult to meet the needs. Summary of the invention
[0004] In view of the above-mentioned problems, the present invention is proposed.
[0005] Therefore, the technical problem solved by the present invention is that the traditional image processing method may not be able to accurately extract the key features of the mine loading area image, and may therefore misjudge the safety status of the loading area.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a mine loading area classification method based on deep learning, comprising:
[0007] Acquire the mine scene image, perform preliminary processing on the image, and output the primary feature map;
[0008] Perform feature extraction operations on the primary feature map to obtain a high-level feature map;
[0009] Perform loading area classification prediction on the high-level feature map to obtain the classification result.
[0010] As a preferred solution of the mine loading area classification method based on deep learning described in the present invention, the output step of the primary feature map is:
[0011] Get an image, get a mine scene image with a resolution of 1920*1200;
[0012] Preprocess the image to preprocess the acquired mine scene image; the preprocessing process mainly includes a convolution layer and a maximum pooling layer. In the convolution layer, a convolution kernel is used to perform a convolution operation with a step size of 2. The convolution operation process extracts the preliminary features of the image and reduces the resolution of the image;
[0013] The output of the convolutional layer is further processed using a maximum pooling layer of size 2*2 and stride 2;
[0014] Output the primary feature map. After preprocessing, a primary feature map with a resolution of 480*300 is obtained.
[0015] As a preferred solution of the mine loading area classification method based on deep learning described in the present invention, the step of obtaining the high-level feature map includes:
[0016] Construction of basic units of feature extraction network;
[0017] Construct feature enhancement module;
[0018] Construct a serial structure, cross-layer connection and feature fusion;
[0019] Build a feature extraction network, perform feature extraction, and output high-level feature maps.
[0020] As a preferred solution of the deep learning-based mine loading area classification method of the present invention, the feature extraction network is composed of three basic units, and the characteristics of the basic unit are that the basic unit is composed of a series structure, two cross-layer connection structures and two additive fusion modules;
[0021] The series structure is a basic unit main functional structure, which is formed by sequentially connecting a feature enhancement module and two convolutions;
[0022] The two additive fusion modules and the cross-layer connection structure cooperate to complete the fusion of cross-layer features, wherein the first additive fusion module is located after the first convolution layer and before the second convolution layer; the second additive fusion module is located after the second convolution layer and before the next basic unit;
[0023] The two cross-layer connection structures use different positions in the serial structure as starting points and end points to realize the fusion function of features at different levels. The first cross-layer connection structure is a shortcut connection, which uses the output of the feature enhancement module as the starting point and the first additive fusion module as the end point to cross the first convolution layer, and is connected to the serial structure through the first additive fusion module to realize the fusion of cross-layer features, and the fusion result is continuously transmitted downward along the serial structure; the second cross-layer connection structure is composed of a convolution layer with a convolution kernel size of 3*3 and a stride of 2. It uses the input of the basic unit as the starting point and the second additive fusion module as the end point to cross the feature enhancement module and two convolution layers in the serial structure, and is connected to the serial structure through the second additive fusion module to realize the fusion of cross-layer features, and the fusion result is used as the output structure of the basic unit.
[0024] As a preferred solution of the deep learning-based mine loading area classification method described in the present invention, the feature enhancement module in the series structure of the basic unit is formed by parallel transmission of a shortcut connection branch, a pooling layer branch with a size of 1*1, and three convolution layer branches with sizes of 3*3, 5*5 and 7*7, respectively, and then feature fusion connection is performed through an addition fusion module;
[0025] The convolution kernel size of the first convolution layer in the series structure of the basic unit is 3*3, and the step size is 1; the convolution kernel size of the second convolution layer in the series structure of the basic unit is 3*3, and the step size is 2;
[0026] The convolution kernel size of the convolution layer in the second cross-layer connection structure in the basic unit is 3*3, and the step size is 2;
[0027] All convolutional layers in the basic unit structure include a BN layer and a ReLU layer.
[0028] As a preferred scheme of the deep learning-based mine loading area classification method described in the present invention, the three basic units in the feature extraction network have the same structure and are sequentially connected, the output of the previous unit is the input of the next unit, and the three units in the feature extraction network are named the first unit, the second unit and the third unit according to the sequence.
[0029] The input of the first unit is the primary feature map output by the preprocessing network. The output of the first unit is the input of the second unit, the output of the second unit is the input of the third unit, and the output of the third unit is the output result of the feature extraction network. The three basic units of the feature extraction network extract features from the primary feature map. Due to the different network depths, each unit extracts semantic features at different levels, and finally the third unit outputs a high-level feature map.
[0030] As a preferred solution of the mine loading area classification method based on deep learning described in the present invention, the step of performing loading area classification prediction on the high-level feature map is:
[0031] The high-level feature map is used as input and convolution operation is performed through the first convolution layer of the prediction structure to enhance the features of the image and reduce noise;
[0032] The convolutional feature map is averaged through the pooling layer to further reduce the dimension of the feature and extract the most representative features;
[0033] The pooled feature map is sent to the fully connected layer, which flattens the pooled feature map and transforms it through a linear function to obtain a one-dimensional feature vector. Each element in the one-dimensional feature vector contains the global information of the input image.
[0034] The output of the fully connected layer is nonlinearly mapped through the softmax function, and the output of the fully connected layer is converted into a probability distribution to obtain the probability P that the input image belongs to the "unsafe" category;
[0035] When the transport capacity is at a high level, two thresholds, 0.3 and 0.6, are set to divide the output probability into three intervals, corresponding to the three categories of safety, potential danger and danger;
[0036] When P < 0.3 + ε, the loading area is judged to be safe; in this case, the probability of the loading area being judged to be unsafe is low, so it is judged to be in a safe state;
[0037] When 0.3+ε≤P<0.6+ζ, the loading area is judged to be potentially dangerous; in this case, the probability that the loading area is judged to be unsafe is at a medium level, indicating that there may be some potential safety issues, and therefore it is judged to be in a potentially dangerous state;
[0038] When P ≥ ζ + 0.6, the loading area is judged to be dangerous; in this case, the probability that the loading area is judged to be unsafe is high, indicating that there is a direct safety hazard, so it is judged to be in a dangerous state;
[0039] When the transport capacity is at a low level, two thresholds are set, 0.15 and 0.85;
[0040] When P<0.15+η, the loading area is judged to be safe; in this case, the probability of the loading area being judged to be unsafe is low, so it is judged to be in a safe state;
[0041] When 0.15+η≤P<0.85+θ, the loading area is judged to be potentially dangerous; in this case, the probability of the loading area being judged to be unsafe is at a medium level, indicating that there may be some potential safety issues, and therefore it is judged to be in a potentially dangerous state;
[0042] When P ≥ 0.85 + θ, the loading area is judged to be dangerous; in this case, the probability that the loading area is judged to be unsafe is high, indicating that there is a direct safety hazard, so it is judged to be in a dangerous state;
[0043] Among them, ε, ζ, η, θ are adjustment coefficients;
[0044] When the loading area is classified as safe, the system will provide feedback to the user that the loading area is normal and the loading operation can continue;
[0045] When a loading area is classified as potentially dangerous, the system will provide feedback to the user that there is a potential safety issue in the loading area, and ask the user to be vigilant and check the signal regularly;
[0046] When a loading area is classified as dangerous, the system will provide feedback to the user that there is a safety hazard in the loading area, immediately stop the loading operation and send a signal for investigation.
[0047] In order to solve the above technical problems, the present invention also provides the following technical solutions: a mine loading area classification system based on deep learning, comprising:
[0048] Data acquisition module, used to collect data from various sensors;
[0049] The preprocessing module is used to clean and format the data collected from the data acquisition module;
[0050] Feature extraction module, used to extract useful features from preprocessed data;
[0051] A classification module, for receiving the features obtained from the feature extraction module and using a classification algorithm to determine a safety level of the loading area;
[0052] The feedback module is used to send corresponding feedback information to the user according to the results of the classification module.
[0053] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein the processor implements the steps of any one of the methods of the present invention when executing the computer program.
[0054] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in the present invention when executed by a processor.
[0055] Beneficial effects of the present invention: The deep learning-based mine loading area classification system provided by the present invention proposes to use a deep learning network to identify different driving areas of a mine, and can identify three types of mine driving areas: loading areas, transportation areas, and unloading areas. Compared with the general area classification method, the present invention targets the complexity and particularity of mining scenes, designs basic units to build a feature extraction network to improve the ability to extract image features of mining scenes. The basic units use shortcut connections to retain upper-level information to avoid the loss of global information, and use cross-layer connections to fuse features at different levels, thereby improving the network's perception of target features of different scales. Based on the improvement of feature extraction capabilities, the present invention can effectively extract mining image features and accurately identify different driving areas of mines. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:
[0057] Figure 1 An overall flow chart of a mine loading area classification method based on deep learning provided for the first embodiment of the present invention;
[0058] Figure 2 A structure diagram of a mine loading area classification model of a mine loading area classification method based on deep learning provided in a second embodiment of the present invention;
[0059] Figure 3 A structural diagram of a feature enhancement module of a mine loading area classification method based on deep learning provided in the second embodiment of the present invention. DETAILED DESCRIPTION
[0060] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.
[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0062] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0063] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.
[0064] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0065] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0066] Example 1
[0067] Reference Figure 1 , as an embodiment of the present invention, provides a mine loading area classification method based on deep learning, comprising:
[0068] S1: Obtain the mine scene image, perform preliminary processing on the image, and output the primary feature map.
[0069] Furthermore, the steps to output the primary feature map are:
[0070] Step 1: Get an image, get a mine scene image with a resolution of 1920*1200. Input the acquired mine scene image into the image recognition module for confirmation. First, perform a clarity test on the uploaded image to ensure that the image quality meets the requirements. If the image clarity is insufficient, the system will display an error message, prompting you to upload an image with a clarity that meets the requirements.
[0071] When the image clarity meets the requirements, the system further checks whether the image matches the target area and whether it has features with a resolution of 1920*1200.
[0072] When the image does not match the target area, the system displays an error message and the system will re-collect the image of the target area or manually upload the image of the target area.
[0073] When the image matches the target area, the system confirms that the image meets the requirements and further determines the surrounding environment and transportation capacity of the mine. The judgment formula of the mine transportation capacity C is:
[0074]
[0075] Among them, α, β, γ, δ, ω are weight coefficients, which indicate the influence of each factor on the transportation capacity, S is the terrain slope, is the terrain complexity, R is the road condition, V is the capacity of the transportation tool, W is the climate condition, F is the traffic flow, and the range of S, T, R, W, and F is [0, 1], indicating that each factor is from the lowest to the highest.
[0076] When C ≥ 0.5, the transport capacity is judged to be "high".
[0077] When C < 0.5, the transport capacity is judged to be "low".
[0078] Step 2: Preprocess the image. Preprocess the acquired mine scene image. The preprocessing process mainly includes a convolution layer and a maximum pooling layer. In the convolution layer, convolution kernels of sizes 3*3, 5*5, 7*7 (selected according to specific needs) are used for convolution operation with a step size of 2. The convolution operation process extracts the preliminary features of the image and reduces the resolution of the image.
[0079] The output of the convolutional layer is further processed using a max pooling layer of size 2*2 and stride 2. The pooling operation further reduces the resolution of the image while retaining important information in the image.
[0080] Step 3: Output the primary feature map. After preprocessing, a primary feature map with a resolution of 480*300 is obtained. This primary feature map contains the preliminary feature information in the mine scene image, which is used for subsequent feature extraction and classification prediction.
[0081] S2: Perform feature extraction operations on the primary feature map to obtain a high-level feature map.
[0082] Furthermore, the steps of obtaining a high-level feature map include constructing a basic unit of a feature extraction network, constructing a feature enhancement module, constructing a cascade structure, cross-layer connection and feature fusion, constructing a feature extraction network, performing feature extraction, and outputting a high-level feature map.
[0083] It should be noted that the feature extraction operation is implemented through a feature extraction network, which is composed of three basic units. The characteristics of the basic unit are that the basic unit is composed of a series structure, two cross-layer connection structures and two additive fusion modules.
[0084] The series structure is the main functional structure of the basic unit, responsible for feature extraction. The structure is formed by a feature enhancement module and two convolutions connected in sequence.
[0085] The two additive fusion modules and the cross-layer connection structure work together to complete the fusion of cross-layer features. The first additive fusion module is located after the first convolutional layer and before the second convolutional layer; the second additive fusion module is located after the second convolutional layer and before the next basic unit.
[0086] The two cross-layer connection structures use different positions in the serial structure as the starting point and end point to realize the fusion function of features at different levels. The first cross-layer connection structure is a shortcut connection. This structure uses the output of the feature enhancement module as the starting point and the first additive fusion module as the end point to cross the first convolution layer. Then, it is connected to the serial structure through the first additive fusion module to realize the fusion of cross-layer features. The fusion result is passed down along the serial structure. The second cross-layer connection structure consists of a convolution layer with a convolution kernel size of 3*3 and a stride of 2. This structure uses the input of the basic unit as the starting point and the second additive fusion module as the end point to cross the feature enhancement module and two convolution layers in the serial structure. Then, it is connected to the serial structure through the second additive fusion module to realize the fusion of cross-layer features. The fusion result is used as the output structure of the basic unit.
[0087] It should be noted that the feature enhancement module in the basic unit series structure is composed of a shortcut connection branch, a pooling layer branch with a size of 1*1, and three convolutional layer branches with sizes of 3*3, 5*5 and 7*7, which are transmitted in parallel and then connected through an addition fusion module for feature fusion.
[0088] The convolution kernel size of the first convolution layer in the basic unit series structure is 3*3, and the step size is 1; the convolution kernel size of the second convolution layer in the basic unit series structure is 3*3, and the step size is 2.
[0089] The convolution kernel size of the convolution layer in the second cross-layer connection structure in the basic unit is 3*3 and the stride is 2.
[0090] Moreover, all convolutional layers in the basic unit structure contain BN layers and ReLU layers.
[0091] The three basic units in the feature extraction network have the same structure and are connected sequentially. The output of the previous unit is the input of the next unit. For easy distinction, the three units in the feature extraction network are named the first unit, the second unit, and the third unit according to the order.
[0092] The input of the first unit is the primary feature map output by the preprocessing network, the output of the first unit is the input of the second unit, the output of the second unit is the input of the third unit, and the output of the third unit is the output result of the feature extraction network. The three basic units of the feature extraction network extract features from the primary feature map. Due to the different network depths, each unit extracts semantic features at different levels, and finally the third unit outputs a high-level feature map.
[0093] S3: Perform loading area classification prediction on the high-level feature map to obtain classification results.
[0094] The classification prediction function is realized through the prediction structure, which consists of convolution layer, pooling layer, full connection and softmax. The convolution kernel size of the convolution layer is 3*3 and the step size is 1; the pooling layer adopts average pooling with a size of 2*2 and a step size of 1. The convolution layer and the pooling layer first transform the size of the input features, and then pass the transformed features to the full connection layer and the softmax function to realize the prediction, and finally output the classification results.
[0095] Furthermore, the specific steps for predicting the loading area classification of the high-level feature map are as follows:
[0096] The high-level feature map is taken as input and convolved through the first convolutional layer of the prediction structure to enhance the features of the image and reduce noise.
[0097] The convolutional feature map is averaged through the pooling layer to further reduce the dimension of the feature and extract the most representative features.
[0098] The pooled feature map is sent to the fully connected layer, which flattens the pooled feature map and transforms it through a linear function to obtain a one-dimensional feature vector. Each element in the one-dimensional feature vector contains the global information of the input image.
[0099] The output of the fully connected layer is nonlinearly mapped through the softmax function, and the output of the fully connected layer is converted into a probability distribution to obtain the probability P that the input image belongs to the "unsafe" category.
[0100] When the transport capacity is at a high level, two thresholds, 0.3 and 0.6, are set to divide the output probability into three intervals, corresponding to the three categories of "safe", "potentially dangerous" and "dangerous".
[0101] When P<0.3+ε, the loading area is judged to be safe; in this case, the probability of the loading area being judged to be unsafe is low, so it is judged to be in a safe state.
[0102] When 0.3+ε≤P<0.6+ζ, the loading area is judged to be potentially dangerous; in this case, the probability that the loading area is judged to be unsafe is at a medium level, indicating that there may be some potential safety issues, and therefore it is judged to be in a potentially dangerous state.
[0103] When P ≥ ζ + 0.6, the loading area is judged to be dangerous; in this case, the probability that the loading area is judged to be unsafe is high, indicating that there is a direct safety hazard, and therefore it is judged to be in a dangerous state.
[0104] When the transport capacity is at a low level, two thresholds are set, 0.15 and 0.85.
[0105] When P<0.15+η, the loading area is judged to be safe; in this case, the probability that the loading area is judged to be unsafe is low, so it is judged to be in a safe state.
[0106] When 0.15+η≤P<0.85+θ, the loading area is judged to be potentially dangerous; in this case, the probability that the loading area is judged to be unsafe is at a medium level, indicating that there may be some potential safety issues, and therefore it is judged to be in a potentially dangerous state.
[0107] When P ≥ 0.85 + θ, the loading area is judged to be dangerous; in this case, the probability that the loading area is judged to be unsafe is high, indicating that there is a direct safety hazard, and therefore it is judged to be in a dangerous state.
[0108] Among them, ε, ζ, η, and θ are adjustment coefficients.
[0109] It should be noted that the setting of the above thresholds matches the distribution characteristics of historical data, achieving better risk identification capabilities within the medium probability range, enhancing the system's sensitivity to these risks, and thus taking appropriate preventive measures in advance, making the system more flexible, and relatively easy to adjust the thresholds in situations where the potential danger is low or high, so as to better adapt to different transportation environments and needs. Technical personnel can update the adjustment coefficient based on the historical data and real-time data of the mine transportation capacity C for different transportation environments and needs.
[0110] The above threshold settings match specific security management strategies and reduce false alarm rates. The system will only judge a dangerous state when the probability is relatively high, avoiding frequent alarm triggering, thereby reducing unnecessary interference and focusing on more important risks. More situations are classified as potentially dangerous or dangerous to ensure that potential problems are not missed, even if the risk is relatively small.
[0111] When the loading area is classified as "safe", the system will feedback to the user the signal "the loading area status is normal and loading operations can continue".
[0112] When the loading area is classified as "potentially dangerous", the system will feedback to the user the signal "There are potential safety issues in the loading area, please be vigilant and check regularly".
[0113] When the loading area is classified as "dangerous", the system will feedback to the user the signal "there are safety hazards in the loading area, stop the loading operation immediately and conduct an investigation".
[0114] It should be noted that the specific safety threshold can be manually changed according to the actual mine topography and specific conditions of the loading area.
[0115] This embodiment also provides a mine loading area classification system based on deep learning, including:
[0116] Data acquisition module, used to collect data from various sensors;
[0117] The preprocessing module is used to clean and format the data collected from the data acquisition module;
[0118] Feature extraction module, used to extract useful features from preprocessed data;
[0119] A classification module, for receiving the features obtained from the feature extraction module and using a classification algorithm to determine a safety level of the loading area;
[0120] The feedback module is used to send corresponding feedback information to the user according to the results of the classification module.
[0121] This embodiment also provides a computing device, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement a mine loading area classification method based on deep learning as proposed in the above embodiment.
[0122] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, a deep learning-based mine loading area classification method proposed in the above embodiment is implemented.
[0123] The storage medium proposed in this embodiment and the deep learning-based mine loading area classification method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.
[0124] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory, tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory, magnetic memory, ferroelectric memory, phase change memory, graphene memory, etc. Volatile memory may include random access memory or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory or dynamic random access memory, etc. The database involved in the embodiments provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but is not limited thereto.
[0125] Example 2
[0126] The following is an embodiment of the present invention, which provides a mine loading area classification method based on deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through comparison method.
[0127] Simulate a relatively large mine and collect a large number of scene images to ensure that the images contain a variety of loading area states (such as safe, potentially dangerous, and dangerous). These images are divided into training and test sets in a ratio of 8:2.
[0128] Compare the technical solution invented by our party with a traditional image processing solution. For the traditional solution, use the same training set and test set. Train and test the two solutions, and record various important performance indicators.
[0129] In the above experiment, we divided the mine scene images into three categories: safe, potentially dangerous, and dangerous. The following more detailed simulation data was obtained:
[0130] Table 1 Comparison of simulation data between our invention and traditional solution
[0131] method category Training accuracy Test accuracy Precision Recall F1 Score Our invention Safety 0.98 0.93 0.95 0.94 0.945 Our invention Potential Dangers 0.98 0.92 0.9 0.91 0.905 Our invention Danger 0.97 0.92 0.92 0.94 0.93 Traditional Solution Safety 0.88 0.83 0.85 0.84 0.845 Traditional Solution Potential Dangers 0.9 0.8 0.78 0.82 0.8 Traditional Solution Danger 0.89 0.81 0.82 0.8 0.81
[0132] This simulation data shows that in the classification prediction of each category, the technical solution invented by us can provide higher accuracy, precision, recall and F1 score, which further emphasizes the effectiveness of your technical solution and shows that the technical solution invented by us is better than the traditional solution in accuracy and depth, and can better perform classification prediction of mine loading areas.
[0133] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A mine loading area classification method based on deep learning, characterized in that: include: Acquire the mine scene image, perform preliminary processing on the image, and output the primary feature map; When the image matches the target area, the system confirms that the image meets the requirements and judges the surrounding environment and transportation capacity of the mine. The judgment formula of the mine transportation capacity C is: Among them, α, β, γ, δ, ω are weight coefficients, indicating the influence of each factor on the transportation capacity; S is the terrain slope, is the terrain complexity, R is the road condition, V is the capacity of the transportation tool, W is the climate condition, and F is the traffic flow; the range of S, T, R, W, and F is [0,1], indicating that each factor is from the lowest to the highest; When C ≥ 0.5, the transport capacity is judged to be high level; When C < 0.5, the transport capacity is judged to be low level; Perform feature extraction operations on the primary feature map to obtain a high-level feature map; Perform loading area classification prediction on the high-level feature map to obtain classification results; The step of obtaining the high-level feature map includes: Construction of basic units of feature extraction network; Construct feature enhancement module; Construct a serial structure, cross-layer connection and feature fusion; Build a feature extraction network, perform feature extraction, and output high-level feature maps; The feature extraction network is composed of three basic units, and the characteristics of the basic unit are that the basic unit is composed of a series structure, two cross-layer connection structures and two addition fusion modules; The series structure is a basic unit main functional structure, which is formed by sequentially connecting a feature enhancement module and two convolutions; The two additive fusion modules and the cross-layer connection structure cooperate to complete the fusion of cross-layer features, wherein the first additive fusion module is located after the first convolution layer and before the second convolution layer; the second additive fusion module is located after the second convolution layer and before the next basic unit; The two cross-layer connection structures respectively use different positions in the serial structure as starting points and end points to realize the fusion function of features at different levels. The first cross-layer connection structure is a shortcut connection, which takes the output of the feature enhancement module as the starting point and the first additive fusion module as the end point to cross the first convolution layer, and is connected to the serial structure through the first additive fusion module to realize the fusion of cross-layer features, and the fusion result is continuously transmitted downward along the serial structure; the second cross-layer connection structure is composed of a convolution layer with a convolution kernel size of 3*3 and a stride of 2, which takes the input of the basic unit as the starting point and the second additive fusion module as the end point to cross the feature enhancement module and two convolution layers in the serial structure, and is connected to the serial structure through the second additive fusion module to realize the fusion of cross-layer features, and the fusion result is used as the output structure of the basic unit; The specific steps of performing loading area classification prediction on the high-level feature map are as follows: The high-level feature map is used as input and convolution operation is performed through the first convolution layer of the prediction structure to enhance the features of the image and reduce noise; The convolutional feature map is averaged through the pooling layer to further reduce the dimension of the feature and extract the most representative features; The pooled feature map is sent to the fully connected layer, which flattens the pooled feature map and transforms it through a linear function to obtain a one-dimensional feature vector. Each element in the one-dimensional feature vector contains the global information of the input image. The output of the fully connected layer is nonlinearly mapped through the softmax function to convert the output of the fully connected layer into a probability distribution, and the probability P that the input image belongs to the unsafe category is obtained; When the transport capacity is at a high level, two thresholds, 0.3 and 0.6, are set to divide the output probability into three intervals, corresponding to the three categories of safety, potential danger and danger; When P < 0.3 + ε, the loading area is judged to be safe; in this case, the probability of the loading area being judged to be unsafe is low, so it is judged to be in a safe state; When 0.3+ε≤P<0.6+ζ, the loading area is judged to be potentially dangerous; in this case, the probability that the loading area is judged to be unsafe is at a medium level, indicating that there may be some potential safety issues, and therefore it is judged to be in a potentially dangerous state; When P ≥ ζ + 0.6, the loading area is judged to be dangerous; in this case, the probability that the loading area is judged to be unsafe is high, indicating that there is a direct safety hazard, so it is judged to be in a dangerous state; When the transport capacity is at a low level, two thresholds are set, 0.15 and 0.85; When P<0.15+η, the loading area is judged to be safe; in this case, the probability of the loading area being judged to be unsafe is low, so it is judged to be in a safe state; When 0.15+η≤P<0.85+θ, the loading area is judged to be potentially dangerous; in this case, the probability of the loading area being judged to be unsafe is at a medium level, indicating that there may be some potential safety issues, and therefore it is judged to be in a potentially dangerous state; When P ≥ 0.85 + θ, the loading area is judged to be dangerous; in this case, the probability that the loading area is judged to be unsafe is high, indicating that there is a direct safety hazard, so it is judged to be in a dangerous state; Among them, ε, ζ, η, θ are adjustment coefficients; When the loading area is classified as safe, the system will provide feedback to the user that the loading area is normal and the loading operation can continue; When a loading area is classified as potentially dangerous, the system will provide feedback to the user that there is a potential safety issue in the loading area, and ask the user to be vigilant and check the signal regularly; When a loading area is classified as dangerous, the system will provide feedback to the user that there is a safety hazard in the loading area, immediately stop the loading operation and send a signal for investigation.
2. The method for classifying mine loading areas based on deep learning according to claim 1, characterized in that: The output step of the primary feature map is: Get an image, get a mine scene image with a resolution of 1920*1200; Preprocess the image to preprocess the acquired mine scene image; the preprocessing process includes a convolution layer and a maximum pooling layer. In the convolution layer, a convolution kernel is used to perform a convolution operation with a step size of 2. The convolution operation process extracts the preliminary features of the image and reduces the resolution of the image; The output of the convolutional layer is further processed using a maximum pooling layer of size 2*2 and stride 2; Output the primary feature map. After preprocessing, a primary feature map with a resolution of 480*300 is obtained.
3. The method for classifying mine loading areas based on deep learning according to claim 2, characterized in that: The feature enhancement module in the series structure of the basic unit is composed of a shortcut connection branch, a pooling layer branch with a size of 1*1, and three convolution layer branches with sizes of 3*3, 5*5 and 7*7, which are transmitted in parallel and then connected by feature fusion through an addition fusion module; The convolution kernel size of the first convolution layer in the series structure of the basic unit is 3*3, and the step size is 1; the convolution kernel size of the second convolution layer in the series structure of the basic unit is 3*3, and the step size is 2; The convolution kernel size of the convolution layer in the second cross-layer connection structure in the basic unit is 3*3, and the step size is 2; All convolutional layers in the basic unit structure include a BN layer and a ReLU layer.
4. The method for classifying mine loading areas based on deep learning according to claim 3, characterized in that: The three basic units in the feature extraction network have the same structure and are sequentially connected. The output of the previous unit is the input of the next unit. According to the sequence, the three units in the feature extraction network are named the first unit, the second unit and the third unit respectively. The input of the first unit is the primary feature map output by the preprocessing network. The output of the first unit is the input of the second unit, the output of the second unit is the input of the third unit, and the output of the third unit is the output result of the feature extraction network. The three basic units of the feature extraction network extract features from the primary feature map. Due to the different network depths, each unit extracts semantic features at different levels, and finally the third unit outputs a high-level feature map.
5. A system for implementing the deep learning-based mine loading area classification method according to any one of claims 1 to 4, characterized in that: include: Data acquisition module, used to collect data from various sensors; The preprocessing module is used to clean and format the data collected from the data acquisition module; Feature extraction module, used to extract useful features from preprocessed data; A classification module, for receiving the features obtained from the feature extraction module and using a classification algorithm to determine a safety level of the loading area; The feedback module is used to send corresponding feedback information to the user according to the results of the classification module.
6. A computer device comprising: Memory and processor; The memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 4 when executing the computer program.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.
