Mineral particle identification and classification method based on digital image and deep learning algorithm

Abstract

The present application relates to the field of image processing technology, and discloses a method for identifying and grading mineral particles based on digital images and deep learning algorithms, the method comprising: respectively calculating the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector, and spectrum feature vector of the image to be graded; using an image grading deep learning network model to perform comprehensive processing on multiple feature vectors to obtain a first grading result; and also using a comprehensive evaluation model to obtain a second grading result based on each feature vector and the weight corresponding to each feature vector; wherein the weight corresponding to each feature vector is obtained by real-time calculation by a weight calculation model. The present application first extracts multiple feature vectors of the image to be graded to comprehensively describe the information covered by the image to be graded, and then leverages the intelligent learning advantages of the deep learning model and the comprehensive evaluation model and the feature allocation advantages of the weight calculation model to improve the accuracy of the final grading result of the image to be graded.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for identifying and grading mineral particles based on digital images and deep learning algorithms. Background Art

[0002] Image classification is a fundamental task in computer vision. It involves using algorithms to analyze image content and automatically categorize it into pre-set categories. Traditional image classification methods typically extract a single feature (such as grayscale) or features from multiple categories (such as color and texture). These features are then fed into a classifier to generate the classification result.

[0003] However, traditional image classification methods have poor accuracy in complex scenarios requiring high feature diversity, such as mineral particle identification and classification (such as dust pollution classification and mineral particle size classification). Therefore, there is an urgent need to solve this technical problem. Summary of the Invention

[0004] In response to the above situation, an embodiment of the present application provides a mineral particle identification and classification method based on digital images and deep learning algorithms, which aims to solve the above problems or at least partially solve the above problems.

[0005] The present application provides a method for identifying and grading mineral particles based on digital images and deep learning algorithms, the method comprising:

[0006] Acquire an image containing mineral particles to be classified;

[0007] Obtaining a grayscale feature vector according to the brightness distribution of the image to be classified;

[0008] Obtaining a texture feature vector according to texture details of mineral particles in the image to be classified;

[0009] Obtaining a fractal dimension feature vector according to the local aggregation and global distribution of mineral particles in the image to be classified;

[0010] Obtaining a shape feature vector according to the shape of the mineral particles in the image to be classified;

[0011] Obtaining a frequency spectrum feature vector according to the distribution of mineral particles in the image to be classified in different frequency bands;

[0012] Using a pre-trained image grading deep learning network model to obtain a first grading result according to the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector;

[0013] Calculating the corresponding weights for the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector using a preset weight calculation model;

[0014] A preset comprehensive evaluation model is used to calculate a second classification result based on the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, the spectrum feature vector, and the weights corresponding to each feature vector;

[0015] Calculating a weight of the confidence score of the first grading result, and obtaining a final probability distribution value based on the first grading result, the weight of the confidence score, and the second grading result, and using the final grading result corresponding to the final score range within which the final probability distribution value falls as the final grading result of the image to be graded;

[0016] Wherein, a final score range value corresponds to a final grading result.

[0017] By means of the above technical solution, the embodiment of the present application provides a method for identifying and grading mineral particles based on digital images and deep learning algorithms, which can effectively cope with complex image grading tasks. Specifically, this embodiment first extracts multiple features based on the digital image algorithm (grayscale features reflect the brightness distribution of the image, texture features reflect the texture details of the image, fractal dimension features characterize the complexity, roughness and self-similarity of the image, shape features describe the shape complexity and direction information of the image, and spectrum features reveal the frequency characteristics of the image), and fuses multiple features to obtain a fused feature vector, which can comprehensively and meticulously describe the feature information of the image to be graded. Then, the fused feature vector is processed using a pre-trained image grading deep learning network model to obtain a first grading result. At the same time, a weight calculation model is used to calculate the corresponding weight for each feature vector in real time. Furthermore, a comprehensive evaluation model is used to calculate the second grading result based on each feature vector and the weight corresponding to each feature vector. The final grading result is obtained based on the first grading results and the second grading results. The final grading result combines the intelligent learning advantages of the deep learning network model and the comprehensive evaluation model and the feature allocation advantages of the weight calculation model. Finally, the grading results output by the two paths are combined to determine the final grading result, which reduces the prediction risk caused by environmental interference or single feature interference, significantly enhances the robustness of the system, and makes the final grading result of the image to be graded more accurate and reliable.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 This is a flow chart of a method for identifying and grading mineral particles based on digital images and deep learning algorithms, as provided in an embodiment of the present application;

[0021] Figure 2 This is a flow chart of calculating the texture feature vector of the image to be classified in an embodiment of the method of the present application;

[0022] Figure 3 This is a flow chart of calculating the fractal dimension feature vector of the image to be classified in an embodiment of the method of the present application;

[0023] Figure 4 This is a flowchart of a deep learning network model for training image classification in an embodiment of the method of the present application;

[0024] Figure 5 This is a structural diagram of the image hierarchical deep learning network model in the embodiment of the method of this application;

[0025] Figure 6 This is a flow chart of the weight calculation model in the embodiment of the method of the present application for calculating the weight of each feature vector based on the perception information data;

[0026] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0029] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that such usage is interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "including" and its variations are to be interpreted as open-ended terms meaning "including but not limited to."

[0030] As previously mentioned, traditional image classification methods generally extract a single feature (such as grayscale) or multiple categories of features (such as color and texture) from an image. These features are then input into a classifier to obtain the classification results. However, in complex scenarios such as mineral particle identification and classification (such as dust pollution classification and mineral particle size classification), which require a high level of feature diversity, traditional image classification methods suffer from poor accuracy. Therefore, the present invention proposes a method for mineral particle identification and classification based on digital images and a deep learning algorithm. This application is described in detail below through specific examples.

[0031] To facilitate understanding of this embodiment, a method for identifying and grading mineral particles based on digital images and deep learning algorithms disclosed in an embodiment of the present application is first described in detail. The execution subject of the method for identifying and grading mineral particles based on digital images and deep learning algorithms provided in the embodiment of the present application is generally a computer device with certain computing capabilities. The computer device includes, for example, a terminal device or a server or other processing device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, etc. In some possible implementations, the method for identifying and grading mineral particles based on digital images and deep learning algorithms can be implemented by a processor calling computer-readable instructions stored in a memory.

[0032] Figure 1 The flow chart of the mineral particle identification and classification method based on digital image and deep learning algorithm provided by the embodiment of the present application is shown. Figure 1 It can be seen that the embodiment of the present application includes at least steps S101-S110:

[0033] S101: Acquire an image containing mineral particles to be classified;

[0034] S102: Obtaining a grayscale feature vector according to the brightness distribution of the image to be graded;

[0035] S103: Obtaining a texture feature vector based on texture details of mineral particles in the image to be classified;

[0036] S104: obtaining a fractal dimension feature vector based on the local aggregation and global distribution of mineral particles in the image to be classified;

[0037] S105: Obtaining a shape feature vector according to the shape of the mineral particles in the image to be classified;

[0038] S106: Obtaining a frequency spectrum feature vector based on the distribution of mineral particles in the image to be classified in different frequency bands;

[0039] S107: using a pre-trained image classification deep learning network model to obtain a first classification result according to the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector;

[0040] S108: Calculating corresponding weights for the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector, and spectrum feature vector using a preset weight calculation model;

[0041] S109: using a preset comprehensive evaluation model to calculate a second classification result based on the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector, spectrum feature vector, and the weight corresponding to each feature vector;

[0042] S110: Obtain a final classification result according to the first classification result and the second classification result.

[0043] Specifically, the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector, and spectrum feature vector of the image to be classified are calculated respectively. Among them, the grayscale feature reflects the brightness distribution of the image, the texture feature reflects the texture details of the image, the fractal dimension feature describes the complexity, roughness, and self-similarity of the image, the shape feature describes the shape complexity and directional information of the image, and the spectrum feature reveals the frequency characteristics of the image. Thus, a comprehensive description of the image to be classified is obtained based on multiple feature vectors.

[0044] Before processing each feature vector using a pre-trained deep learning network model for image grading, the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector, and spectral feature vector are first fused to produce a fused feature vector. The fused feature vector is then processed using the deep learning network model for image grading to produce a first classification result. The deep learning network model for image grading is constructed based on a graph neural network teacher model and a Transformer student model. Leveraging the graph neural network teacher model's ability to model structural relationships and the Transformer student model's ability to capture long-range dependencies, an accurate first classification result is achieved.

[0045] Then, the weight calculation model is used to calculate the corresponding weight for each eigenvector in real time. The comprehensive evaluation model then calculates the second classification result based on each eigenvector and the weight corresponding to each eigenvector. That is, the weight calculation model is used to highlight the information expression of important eigenvectors, thereby providing data support for obtaining accurate second classification results.

[0046] Finally, the first grading results and the second grading results are combined to effectively integrate the advantages of different models and deal with the uncertainty in the grading process, so as to obtain the final grading result. That is, the advantages of each model in grading are combined to effectively reduce the limitations of single feature extraction and single model grading, thereby improving the accuracy, robustness and reliability of image grading, and achieving a more accurate and comprehensive image grading effect.

[0047] The above S101-S110 are described in detail below.

[0048] In step S101, the image to be classified contains mineral particles. For example, in open-pit mining operations, processes such as drilling, blasting, shoveling, and transportation generate large amounts of dust, also known as mineral particles. These mineral particles not only pose a serious threat to miners' health and can easily lead to various health problems such as pneumoconiosis and respiratory diseases, but also corrode mining equipment, accelerate equipment wear, and increase the risk of equipment failure. They also place significant pressure on the surrounding ecological environment. In this embodiment, high-resolution, wide-dynamic-range industrial cameras are installed at key dust-generating and diffusion areas in open-pit mines, such as mining operations, haul road intersections, and ore crushing and processing areas. The cameras feature intelligent dimming capabilities. Using a built-in light sensor, they monitor ambient light intensity in real time and automatically adjust parameters such as aperture, shutter speed, and sensitivity to ensure clear, usable images of the mineral particles, i.e., images to be classified, are captured under varying lighting conditions. Furthermore, the industrial cameras are equipped with an automatic cleaning device that regularly cleans dust from the lens surface using compressed air or a mechanical brush to prevent dust accumulation that may affect image quality. A multi-camera array layout can also be used, with the number and location of cameras appropriately arranged based on the size and shape of the monitoring area. For example, in a large mining operation, 10 cameras can be deployed in a grid pattern to ensure that images to be graded are collected from different angles and directions, fully covering the monitoring area and avoiding blind spots.

[0049] In this embodiment, the following preprocessing operations are performed on the obtained image to be classified.

[0050] First, the image to be graded is converted from the RGB color space to the HSV color space, where H represents hue, with a value range of [0,360], and different hue values ​​represent different color categories; S represents saturation, with a value range of [0,1], which reflects the purity of the color; and V represents value, with a value range of [0,1], which indicates the brightness of the color. In practical applications, preset thresholds for the S and V channels can also be used to filter out possible dust areas, thereby reducing the amount of subsequent calculations. For example, when the S channel value is greater than 0.3 and the V channel value is less than 0.8, the pixel is considered to belong to the dust area.

[0051] Then, the image to be graded is denoised. A bilateral filtering algorithm is used for denoising. For example, for each pixel (i, j) in the image, the pixel value after filtering is calculated as follows:

[0052] (1.1)

[0053] Among them, S ij is a neighborhood window centered at (i, j), which can be 3×3 or 5×5 in size. Represents the neighborhood window S ij The internal coordinates are The original grayscale value of the pixel, is the weight coefficient, and the calculation formula is:

[0054] (1.2)

[0055] in, is the spatial standard deviation, which determines the influence of pixel spatial distance on the weight; is the standard deviation of the grayscale domain, which controls the effect of pixel grayscale value differences on the weight.

[0056] After the above preprocessing operation, it is ensured that the image to be graded can better retain the edge and detail information of the image while removing noise.

[0057] In step S102, because the brightness distribution of the image to be classified affects the grayscale average value of the image, the grayscale feature vector is obtained by calculating the grayscale average value of the image to be classified. The specific calculation formula is:

[0058] (2.1)

[0059] Among them, Z ij is the pixel value of the i-th row and j-th column in the image to be classified, and M and N are the number of rows and columns of the image to be classified, respectively.

[0060] In step S103, when calculating the texture feature vector of the image to be classified, it is necessary to consider the influence of environmental factors on the mineral particles in the image to be classified. Figure 2 As shown, the process of calculating the texture feature vector of the image to be classified is shown in steps S201 to S206:

[0061] S201: performing grayscale quantization on each pixel value in the image to be graded to obtain a quantized image;

[0062] S202: Based on the quantized image, gray level co-occurrence matrices at 0°, 45°, 90°, 135° and the main dust diffusion direction are calculated respectively;

[0063] S203: Calculate the gray level co-occurrence matrix eigenvector corresponding to each direction according to the gray level co-occurrence matrix in each direction;

[0064] S204: combining the corresponding gray-level co-occurrence matrix eigenvectors in each direction to obtain a multi-directional joint eigenvector;

[0065] S205: Calculating the local binary pattern feature vector of the quantized image;

[0066] S206: Fusing the gray level co-occurrence matrix eigenvectors, multi-directional joint eigenvectors, and local binary pattern eigenvectors corresponding to each direction to obtain a texture eigenvector.

[0067] Specifically, the image to be graded is first quantized using a uniform quantization method or an adaptive quantization method. This embodiment takes adaptive quantization as an example, and the details are as follows:

[0068] 1. Grayscale histogram calculation: Calculate the frequency of occurrence of each grayscale value in the quantized image to generate a normalized histogram ,in , g represents the gray value of the pixel;

[0069] 2. Use Otsu threshold method to determine the segmentation threshold t;

[0070] 3. Region division and quantization: When g>t, it is a high grayscale area, and the quantization level is set to 8; when t / 2<g≤t, it is a medium grayscale area, and the quantization level is set to 16; when g≤t / 2, it is a low grayscale area, and the quantization level is set to 4.

[0071] Then, the wind speed direction and camera shooting direction of the image to be graded are obtained. The current wind speed measurement value, current temperature, current air pressure, and preset standard temperature and preset standard air pressure corresponding to the image to be graded are substituted into the wind speed correction formula to calculate the corrected wind speed. The wind speed correction formula is based on the ideal gas law and Bernoulli's equation. The specific formula is:

[0072] (3.1)

[0073] Where v is the corrected wind speed, is the anemometer measurement value, that is, the current wind speed measurement value, 、 , both are preset standard environmental parameters, namely the preset standard temperature and the preset standard air pressure, P is the current temperature, and T is the current air pressure.

[0074] Next, the direction offset compensation is calculated based on the current air pressure and the preset standard air pressure. The current measured wind direction is added to the direction offset compensation to obtain the main dust diffusion direction. In this embodiment, the main dust diffusion direction is defined as the angle between the wind speed vector and the x-axis of the camera coordinate system. Specifically, the formula for calculating the main dust diffusion direction is as follows:

[0075] (3.2)

[0076] in, The anemometer measurement direction, that is, the current measured wind direction, Compensates for directional deviation caused by air pressure.

[0077] Based on the quantized image, the gray-level co-occurrence matrix is ​​calculated at 0°, 45°, 90°, 135°, and the main dust diffusion direction. Based on each gray-level co-occurrence matrix, the corresponding gray-level co-occurrence matrix eigenvector is calculated. The method for calculating the gray-level co-occurrence matrix is ​​prior art and will not be described in detail here. Here, the gray-level co-occurrence matrix eigenvector includes, but is not limited to, contrast, correlation, energy, and entropy. The calculation formulas for contrast, correlation, energy, and entropy are prior art and will not be described in detail here.

[0078] Furthermore, after obtaining the corresponding gray level co-occurrence matrix eigenvectors in each direction, the corresponding gray level co-occurrence matrix eigenvectors in each direction are weighted averaged to calculate the multi-directional joint eigenvector. During implementation, the weights of the corresponding gray level co-occurrence matrix eigenvectors in each direction are calculated using the following method:

[0079] For the main dust diffusion direction, the initial weight is calculated using the following adaptive formula: :

[0080] (3.3)

[0081] in, The maximum wind speed of the current scene can be set according to actual needs, for example, , ReLU is the linear rectification function.

[0082] This formula makes the weight of the main dust diffusion direction lower when the wind speed is low, and its weight gradually increases as the wind speed increases, thereby dynamically adjusting the influence of the dust diffusion direction on the texture characteristics; the weights of the traditional four directions remain the same when the wind speed changes, thus realizing the dynamic adjustment of the directional weights as a whole.

[0083] For the remaining four traditional directions, the corresponding initial weights are calculated using the following method: :

[0084] (3.4)

[0085] After obtaining the initial weights of the gray level co-occurrence matrix eigenvectors corresponding to each direction, each initial weight is normalized to obtain the weights of the gray level co-occurrence matrix eigenvectors corresponding to each direction.

[0086] Finally, the corresponding gray-level co-occurrence matrix eigenvectors in each direction and their corresponding weights are multiplied and added together to obtain the multi-directional joint eigenvector.

[0087] Meanwhile, a method for calculating the local binary pattern feature vector of the quantized image is as follows.

[0088] 1. Image segmentation: The quantized image is divided into 16 × 16 pixel sub-blocks with an overlap ratio of 50% to preserve edge information.

[0089] 2. LBP feature calculation: For the central pixel (x, y) of each sub-block, calculate the LBP value of its 8 neighboring pixels according to the following formula:

[0090] (3.5)

[0091] Among them, g c is the gray value of the center pixel, g n is the grayscale value of the neighborhood pixel, and s(·) is the sign function, which is used to indicate that if x ≥ 0, s(x) = 1; otherwise, it is 0.

[0092] 3. Histogram statistics: For the LBP value of each sub-block, a 256-level histogram is counted and normalized to obtain the local texture feature vector.

[0093] 4. Concatenate the local texture feature vectors of each sub-block to obtain the local binary pattern feature vector of the quantized image.

[0094] Finally, the gray-level co-occurrence matrix eigenvectors, multi-directional joint eigenvectors and local binary pattern eigenvectors are fused to obtain the texture eigenvector. The fusion process is as follows:

[0095] 1. Feature standardization: Z-score normalization is performed on the eigenvectors of the gray-level co-occurrence matrix (16 dimensions), the multi-directional joint eigenvector (4 dimensions), and the local binary pattern eigenvector (256 dimensions).

[0096] 2. PCA dimensionality reduction: First, for all samples of the gray level co-occurrence matrix eigenvector, multi-directional joint eigenvector and local binary pattern eigenvector, calculate the mean of each feature dimension and obtain the mean vector Then, the covariance matrix C is calculated according to the following formula:

[0097] (3.6)

[0098] Where N is the total number of samples, X i is the gray-level co-occurrence matrix eigenvector, multi-directional joint eigenvector and local binary pattern eigenvector of the i-th sample.

[0099] Perform eigenvalue decomposition on C and select the principal components according to the cumulative contribution rate to ensure that more than 95% of the information is retained.

[0100] 3. Weight determination: According to the variance contribution rate of the principal component, the weight coefficient is determined by the following formula , , :

[0101] (3.7)

[0102] Among them, here 、 、 are used to represent the weight coefficients of the gray-level co-occurrence matrix eigenvector, the multi-directional joint eigenvector, and the local binary pattern eigenvector, respectively. is the principal component eigenvalue, and k is the number of principal components.

[0103] 4. Fusion feature calculation:

[0104] (3.8)

[0105] in, Represents texture eigenvector, GLCM_dyn represents each gray-level co-occurrence matrix eigenvector, GLCM_multi represents multi-directional joint eigenvector, and LBP_hist represents local binary pattern eigenvector.

[0106] It can be seen from this that this embodiment first uses the wind speed correction formula obtained based on the ideal gas law and the Bernoulli equation to correct the wind speed measurement value and eliminate the interference of air pressure and temperature on the wind speed; then, through wind direction offset compensation, the accurate main dust diffusion direction is calculated; through the linear rectification function, the main weight corresponding to the main dust diffusion direction is dynamically generated, thereby directionally strengthening the texture pattern dominated by dust diffusion and suppressing the interference of texture patterns in other directions; finally, the eigenvectors of each grayscale co-occurrence matrix, the multi-directional joint eigenvector and the local binary pattern eigenvector are combined to construct a texture feature vector that is sensitive to dust concentration distribution and resistant to environmental interference, thereby improving the reliability of the pollution level recognition results of the image classification deep learning network model under complex atmospheric conditions of low temperature and low pressure.

[0107] In step S104, the box counting method is used to calculate the fractal dimension feature vector of the image to be classified, such as Figure 3 As shown, the specific steps are shown as S301 to S304:

[0108] S301: Divide the image to be graded into a plurality of first sub-images, assign a corresponding first weight to each first sub-image, and assign a second weight to the image to be graded;

[0109] S302: Calculating local fractal dimension feature data of each first sub-image, where the local fractal dimension feature data is used to reflect the aggregation of mineral particles in the first sub-image;

[0110] S303: Calculating global fractal dimension feature data of the image to be classified, where the global fractal dimension feature data is used to reflect the global distribution of mineral particles in the image to be classified;

[0111] S304: Calculate a fractal dimension feature vector according to the local fractal dimension feature data and the corresponding first weight, the global fractal dimension feature data and the corresponding second weight of each first sub-image.

[0112] Specifically, the image to be classified is first evenly divided according to a grid of a preset size to obtain a plurality of second sub-images. The preset size can be set according to actual needs and is not limited in this embodiment. For example, the preset size is 4×4. By evenly dividing the image to be classified according to a grid of this size, 16 second sub-images can be obtained.

[0113] Then, the grayscale entropy of each second sub-image is calculated using the following formula:

[0114] (4.1)

[0115] in, represents the probability of the i-th gray level appearing, and L is the total number of gray levels.

[0116] Grayscale entropy characterizes the complexity of the second sub-image. Higher grayscale entropy indicates a denser dust concentration within the corresponding second sub-image. Therefore, the second sub-images can be screened based on their grayscale entropy to determine the sub-images to be encrypted. For example, the grayscale entropies can be sorted from highest to lowest, and the second sub-images corresponding to the top 20% of grayscale entropies can be selected as the sub-images to be encrypted.

[0117] Next, further grid division can be performed for each sub-image to be encrypted until the sub-image to be encrypted meets a preset termination condition. The preset termination condition can be that the grid size reaches a minimum resolution, or that the grayscale entropy of each second sub-image falls below a preset threshold. The preset threshold can be set according to actual needs and is not limited in this embodiment.

[0118] For local fractal dimension feature data, for example, the image to be graded can be divided into 8×8 grids to obtain 64 first sub-images. For each first sub-image, the box counting method is used to cover the dust area with squares of different side lengths, and the number of covered boxes is counted. ; Fitting by least squares method and The slope of the straight line is used to obtain the local fractal dimension characteristic data .

[0119] For global fractal dimension feature data , is to use the box counting method for the entire image to be classified. The calculation process is similar to that of the local fractal dimension feature data and will not be repeated here.

[0120] Finally, a weighted fusion strategy can be adopted, that is, the fractal dimension feature vector is calculated using the following formula:

[0121] (4.2)

[0122] in, represents the fractal dimension eigenvector, represents the first weight, In specific implementation, the weight coefficients of local and global fractal dimensions can be determined by analyzing the mining area sample data through multiple linear regression.

[0123] As can be seen, this embodiment takes into account the local aggregation and global distribution of mineral particles. By calculating the global fractal dimension of the image to be classified and the local fractal dimension of each local area in the image, and then combining the first weight corresponding to each local fractal dimension with the second weight corresponding to the global fractal dimension, a fractal dimension feature vector is calculated. This fractal dimension feature vector reflects both the macroscopic and microscopic features of the image to be classified, providing richer and more accurate fractal dimension feature information for dust pollution level classification tasks.

[0124] In step S105, the Hu moment method is used to calculate the shape feature vector of the image to be classified. The Hu moment is a set of invariant moments calculated based on the geometric moments of the image, and has translation, rotation and scaling invariance. For example, for the binarized image to be classified, its p+q order geometric moment is defined as:

[0125] (5.1)

[0126] Where x and y are the image pixel coordinates, is the pixel value (0 or 1 in a binary image), p and q are non-negative integers. By calculating a series of geometric moments, the Hu moment can be obtained, such as , etc., among which is the normalized central moment. The value of Hu moment can reflect the shape characteristics of dust particles, such as circularity and elongation.

[0127] In step S106, the image to be graded is converted to the frequency domain by the Fourier transform algorithm. The Fourier transform formula is:

[0128] (6.1)

[0129] in, The image to be graded is at coordinate The pixel value at is the transformed frequency domain image, and is the frequency domain coordinate, By analyzing the frequency domain image, the energy information of the frequency domain image in the first frequency band, the second frequency band, and the third frequency band is extracted to obtain the spectrum feature vector.

[0130] The first frequency band, the second frequency band, and the third frequency band are three non-overlapping frequency range values ​​arranged in ascending order. In this embodiment, the first frequency band, the second frequency band, and the third frequency band are respectively referred to as the low frequency band, the middle frequency band, and the high frequency band. That is, the energy information of the low frequency, middle frequency, and high frequency components of the frequency domain image is obtained. Specifically:

[0131] Low frequency band ( , is the energy of the low frequency radius threshold) ;

[0132] Mid-band ( , is the energy of the intermediate frequency radius threshold) ;

[0133] High frequency band ( ) .

[0134] It can be seen from this that this embodiment captures multiple aspects of information such as brightness distribution, structural morphology, and frequency components of the image to be classified by extracting multi-dimensional feature vectors such as grayscale, texture, fractal dimension, shape, and spectrum of the image to be classified, and reduces the influence of environmental interference on a single feature through feature complementarity.

[0135] In step S107, the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector and spectrum feature vector of the image to be graded are first fused to obtain a fused feature vector. During fusion, each feature vector can be first weighted serially fused, that is, different weights are assigned according to the importance of the features, and then spliced ​​into a high-dimensional vector; then, the principal component analysis method is used to reduce the dimensionality of the spliced ​​high-dimensional feature vector, and the original features are converted into a set of linearly independent principal components through linear transformation to eliminate redundant information between features. This method can not only preserve the integrity of multiple features, but also eliminate feature redundancy through dimensionality reduction, thereby improving the efficiency of subsequent deep learning models.

[0136] Then, the fused feature vector is processed using the pre-trained image grading deep learning network model to obtain the first grading result.

[0137] like Figure 4 As shown, the training process of the image classification deep learning network model is shown in steps S401 to S405:

[0138] S401, obtaining a data set, wherein the data set includes a plurality of sample mineral particle images, and the label of each sample mineral particle image is the level of the image;

[0139] S402, using the graph neural network teacher model, processing each sample mineral particle image separately to obtain the corresponding node feature vector;

[0140] S403, using the Transformer student model, processing the feature vectors of each node to obtain the classification prediction results corresponding to each sample mineral particle image;

[0141] S404: Using a preset loss function, based on each classification prediction result and the corresponding sample label, calculate each loss value;

[0142] S405. Using each loss value, update the parameters of the image classification deep learning network model to be trained to obtain a trained image classification deep learning network model.

[0143] Specifically, a large number of images to be classified, covering various dust concentrations, were collected at different times and locations within the mine. Each image was labeled with a dust concentration level (e.g., low, medium, and high, represented by 0, 1, and 2, respectively). The images to be classified were then preprocessed by normalizing them, mapping pixel values ​​from [0, 255] to [0, 1] using the following mapping formula:

[0144] (7.1)

[0145] in, is the original pixel value, is the minimum pixel value of the image to be classified, is the maximum pixel value of the image to be classified.

[0146] The images to be graded are then cropped and scaled to uniformly adjust the images to a fixed size, such as 224×224 pixels, to meet the input requirements of the initial image grading network model.

[0147] Finally, the dataset is expanded by performing operations such as flipping (horizontally or vertically), rotation (rotation range [-15°, 15°]), and brightness adjustment (brightness change range [0.8, 1.2]) to increase data diversity and improve the generalization ability of the model.

[0148] After obtaining the pre-processed images to be classified, each image to be classified can be used to train the image classification deep learning network model to be trained.

[0149] like Figure 5 As shown in the figure, the structure of the image classification deep learning network model mainly includes the graph neural network teacher model and the Transformer student model. Therefore, when updating the parameters of the image classification deep learning network model to be trained:

[0150] 1. Build a graph structure: Divide the image to be graded into multiple pixel blocks (e.g., 16×16 pixel blocks), with each pixel block as a node. Node features can be information such as the mean and variance of the pixel values ​​of the pixel block. Build a graph structure , where N is the number of nodes and d is the feature dimension. Then, the adjacency matrix is ​​constructed based on the spatial adjacency relationship between nodes. , the edge between nodes corresponding to adjacent pixel blocks is 1, otherwise it is 0.

[0151] 2. Input the multiple pixel blocks from the previous step into one or more Graph Convolutional Network (GCN) layers, such as layers 3-5. The data processing logic of the GCN layer is as follows:

[0152] (7.2)

[0153] in, is the output feature vector of node v in layer l, is the activation function, 、 are the normalized diagonal elements of the degree matrix, is the set of neighbor nodes of node v, is the normalized adjacency matrix element (including self-loops, such as , I is the identity matrix, representing the connection weight between nodes u and v), is the weight matrix of layer l.

[0154] 3. Use the fully connected layer and Softmax function to process the output of the last GCN layer to obtain the classification prediction of each node:

[0155] (7.3)

[0156] in, is the output layer weight matrix, and L is the total number of GCN layers.

[0157] 4. Take the node features output by the graph neural network teacher model as input, and map the feature dimensions to the input dimensions of the Transformer student model through the linear projection layer :

[0158] (7.4)

[0159] in, is the projection matrix.

[0160] 5. Use the multi-head self-attention layer to process the input feature vector. The core formula of the multi-head self-attention mechanism is:

[0161] (7.5)

[0162] (7.6)

[0163] (7.7)

[0164] Among them, Q, K, and V are query, key, and value matrices respectively, which are composed of input features Linear transformation is obtained; is the dimension of the key; h is the number of heads; is the projection matrix.

[0165] 6. Use the LayerNorm layer to normalize the output of the multi-head self-attention layer and then input it into the multi-layer perceptron (MLP) layer. The calculation logic of the MLP layer is:

[0166] (7.8)

[0167] in, is the weight matrix, is the bias, is the activation function and x is the output of the LayerNorm layer.

[0168] 7. Use the Softmax function to process the output of the MLP layer to obtain the classification prediction result.

[0169] Next, the loss function is used to calculate the loss values ​​based on the classification prediction results and corresponding sample labels output by the Transformer student model; and then the loss values ​​are used to update the parameters of the image classification deep learning network model to be trained.

[0170] In some embodiments, the preset loss function is:

[0171] (7.9)

[0172] in, is a set of labeled nodes, is the true label of node v, is the output feature of the student model for node v, is the probability distribution predicted by the student model for node v.

[0173] In some other embodiments, the preset loss function is:

[0174] (7.10)

[0175] in, is the classification loss term, is the microstructure distillation loss term, is the macrostructure distillation loss term, is the multi-scale feature fusion loss term, is a preset balance parameter; and ,in, is a set of labeled nodes, is the true label of node v, is the output feature of the student model for node v, is the probability distribution predicted by the student model for node v , where E is the set of edges in the graph, is the probability distribution predicted by the teacher model for the head node v, is the probability distribution predicted by the student model for the tail node u, is the preset temperature parameter; ,in, , , is the L1 distance between the features of the head node u and the tail node v in the student model, is the L1 distance between the features of the head node u and the tail node v in the teacher model; ; where K is the number of scales, is the edge feature at the k-th scale, , is the average value of edge features at multiple scales, yes and KL divergence of .

[0176] In this embodiment, , used to control the relative importance of each loss term. During implementation, it can be initially set to 0.5 and adjusted during model training based on model performance. The temperature parameter can be set to 1. The number of scales can be set based on actual needs and is not limited in this embodiment.

[0177] It can be seen that the loss function provided in this embodiment integrates classification loss, microstructure distillation loss, macrostructure distillation loss and multi-scale feature fusion loss. Among them, the classification loss can measure the difference between the prediction result of the Transformer student model and the true label, and guide the model to learn in the correct classification direction. The microstructure distillation loss enables the Transformer student model to learn the local distribution information of the edge level of the graph neural network teacher model. The macrostructure distillation loss is used to match the distribution at the graph level, which can help the Transformer student model learn the macrostructure information of the graph neural network teacher model. The multi-scale feature fusion loss further integrates the structural information of different scales, which can ensure the consistency of features at different scales and enhance the model's ability to learn multi-scale information of the image. It can be seen that with the loss function provided in this embodiment, the model can not only accurately classify, but also comprehensively learn the structural features of the graph data, thereby significantly improving the model performance.

[0178] When updating, you can first randomly initialize the weight matrices of the graph neural network teacher model and the Transformer student model, such as using the Xavier initialization method, and determine the initialization value range according to the shape of the weight matrix and the type of activation function to ensure that the parameters are reasonably distributed in the early stage of training to avoid gradient disappearance or explosion. You can choose an optimizer, such as the Adam optimizer, and the learning rate Set to 1e-4, the first-order moment estimates the decay rate , the second-order moment estimate decay rate , numerical stability parameter . Set the number of training rounds, such as epoch=100, batch size During the training process, in each epoch, data is taken from the training set in batches and input into the graph neural network teacher model and the Transformer student model. The total loss value is calculated , calculate the gradient through the back-propagation algorithm and update the model parameters. Taking the Adam optimizer as an example, the parameter update formula is:

[0179] (7.11)

[0180] (7.12)

[0181] (7.13)

[0182] (7.14)

[0183] (7.15)

[0184] in, is the parameter value of the model at the current moment, t is the number of training steps, The first moment estimate at step t, is the second-order moment estimate at step t, is the gradient of the total loss with respect to the parameters, is the bias correction term.

[0185] After each training round, the model performance is evaluated on the validation set, and the learning rate is adjusted based on the validation set performance. After training is completed, the trained model is evaluated on the test set, and indicators such as accuracy, recall, and F1 value are calculated. Specifically:

[0186] Accuracy: , TP is the number of true positives, TN is the number of true negatives, FP is the number of false positives, and FN is the number of false negatives;

[0187] Recall: ;

[0188] F1 value: , .

[0189] Finally, evaluate model performance based on the calculated metrics. If accuracy, recall, and F1 scores are low, analyze the causes and make improvements. For example, check whether data labeling is accurate, data augmentation is reasonable, the model structure is appropriate, and the loss function weights are appropriate. Based on the analysis results, adjust the model structure, re-label the data, optimize the loss function weights or data augmentation methods, and retrain and evaluate the model until satisfactory performance is achieved.

[0190] As can be seen, the image grading deep learning network model training method provided in this embodiment introduces a teacher-student model framework. A graph neural network (GNN) serves as the teacher model to extract node feature vectors from sample mineral particle images. A Transformer network, acting as the student model, then processes these feature vectors and outputs classification predictions. This design combines the structural advantages of GNNs in capturing the spatial relationships between mineral particles in an image with the sequential processing capabilities of the Transformer in modeling long-range dependencies. The node feature vectors of the teacher model provide a more structured input for the student model, enabling it to learn the topological relationships and contextual information between mineral particles. The Transformer's self-attention mechanism further enhances its ability to model local features and global relationships. The student model parameters are optimized using a preset loss function, and the trained student model is ultimately used as the final classifier. This approach avoids the computational overhead of directly using complex GNNs while retaining their advantages in structural feature extraction. It also leverages the sequential processing capabilities of the Transformer to improve classification accuracy. This is particularly true when processing images of mineral particles with complex spatial relationships and morphological features. The Transformer can more effectively capture the interactions between particles and global features, significantly improving the accuracy and robustness of mineral image grading.

[0191] In step S108, the weight calculation model calculates the corresponding weight for each eigenvector in real time, which can be calculated based on the information perception data, or calculated by calculating the Shapley value of each eigenvector, or even based on the information perception data and the Shapley value of each eigenvector.

[0192] like Figure 6 As shown, when the weight corresponding to each feature vector is calculated based on the information perception data, the specific calculation steps are shown in S501 to S508:

[0193] S501: Calculate the attention weight of each feature vector through the image hierarchical deep learning network model;

[0194] S502: Acquire an actual environmental parameter group, which is environmental information when the image to be graded is collected. The environmental information includes at least temperature, humidity, dust concentration, and wind direction.

[0195] S503: Calculate a third weight of each eigenvector based on the attention weight of each eigenvector and the actual environment parameter group;

[0196] S504: Obtain a preset mapping table, wherein the mapping table stores multiple sets of standard environmental parameter groups, each set of standard environmental parameter groups including environmental information and initial weights of each feature vector under the environmental information;

[0197] S505: Selecting the standard environment parameter group with the highest similarity to the actual environment parameter group in the mapping table as the target environment parameter group;

[0198] S506: Calculating the reliability index of the target environmental parameter group based on the actual environmental parameter group;

[0199] S507: Calculate a fourth weight of each eigenvector based on the reliability index and the initial weight of each eigenvector in the target environment parameter group;

[0200] S508: Calculate the final weight value of each eigenvector based on the third weight and the fourth weight of each eigenvector.

[0201] Specifically, combined with the multi-head attention matrix in the image classification deep learning network model , calculate the attention weight of each feature vector , the calculation formula is:

[0202] (8.1)

[0203] Wherein, H is the number of attention heads, N is the length of the feature sequence, and i here corresponds to any one of the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector, and spectrum feature vector. In this embodiment, when i=1, i=2, i=3, i=4, and i=5, they correspond to the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector, and spectrum feature vector, respectively.

[0204] Vectorize the environmental information to obtain the actual environmental parameter group E, , Represents different types of environmental information.

[0205] Based on attention weight And the actual environment parameter group E, construct the joint input group Z: .

[0206] Input the joint input group Z into the weight calculation model to obtain the third weight w of each feature vector i :

[0207] (8.2)

[0208] (8.3)

[0209] in, represents the activation function, W1 and W2 represent the weight matrices of the first and second layers of the image classification deep learning network model respectively, b1 and b2 represent the bias vectors of the first and second layers of the image classification deep learning network model respectively, and the third weight w i is After normalization, formula 8.3 is the normalization function.

[0210] The above mapping table stores multiple sets of standard environmental parameter groups. Each set of standard environmental parameter groups includes environmental information and the initial weights of each feature vector under the environmental information, such as temperature less than -5°C, air pressure greater than 100ug / m 3 When performing the initial weighting, each eigenvector is assigned an initial weight of 0.3-0.5. This initial weighting is obtained by building an expert database in advance. To ensure the accuracy of the initial weights, a mapping model can also be set up in advance. The mapping model automatically adjusts the initial weights of each eigenvector based on the actual environmental parameter set to ensure the accuracy of the obtained initial weights.

[0211] Then, from the mapping table, the standard environmental parameter group with the highest similarity to the actual environmental parameter group is selected as the target environmental parameter group, and the reliability index of the target environmental parameter group is calculated based on the actual environmental parameter group. The higher the similarity between the actual environmental parameter group and the target environmental parameter group, the greater the reliability index. The reliability index is represented by r. Finally, according to the reliability index r and the initial weight of each eigenvector in the target environment parameter group, the fourth weight W of each eigenvector is calculated. i,env :

[0212] (8.4)

[0213] Among them, W i,0 Represents the initial weight of each eigenvector.

[0214] Finally, for each eigenvector, the third weight and the fourth weight of each eigenvector are calculated to obtain the corresponding final weight value W i,final :

[0215] (8.5)

[0216] in, Indicates the fusion coefficient, which is set in advance.

[0217] It can be seen from this that when calculating the weight corresponding to each eigenvector based on the information perception data, the influence of the information expression of each eigenvector in the image to be classified on the weight is taken into account. At the same time, the influence of environmental parameters on the weight of each eigenvector is also considered. The final weight value of each eigenvector is obtained by combining the dual influencing factors to ensure the accuracy of the final weight value.

[0218] When calculating the corresponding weights based on the Shapley values ​​of each eigenvector: first combine the grayscale eigenvector, texture eigenvector, fractal dimension eigenvector, shape eigenvector and spectrum eigenvector to define a feature set, and then , calculate its contribution Shapley value by permutation and combination method , the calculation formula is:

[0219] (8.6)

[0220] Among them, F represents the set of all available features, S represents a subset of the feature set F, and does not include the current feature f i , Indicates that after adding features based on feature subset S , the predictive performance of the model, represents the prediction performance of the model when using feature subset S.

[0221] When the weight of each eigenvector is calculated based on the information perception data and the Shapley value of each eigenvector, the final weight value W of each eigenvector is calculated. i,final After the shapley value, the product of the two is used as the weight of each eigenvector, or the final weight value W i,final and shapley values ​​are assigned different weights again, with the final weight value W i,final The sum of the shapley values ​​is multiplied by the corresponding weights respectively, and the two products are added together to obtain the sum as the weight of each eigenvector. This embodiment does not limit the specific calculation method.

[0222] In step S109, the comprehensive evaluation model can directly multiply the grayscale feature vector, texture feature vector, fractal dimension feature vector, shape feature vector and spectrum feature vector, as well as their respective corresponding weights, and then add the multiplication results to obtain a comprehensive evaluation index. Alternatively, after obtaining the product corresponding to each feature vector, the product corresponding to each feature vector and the preset error term can be added to obtain a comprehensive evaluation index. The preset error term is determined based on the regression analysis algorithm and experimental data, such as by experimentally collecting a data set containing feature vectors (grayscale, texture, fractal dimension, shape, spectrum, etc.) and corresponding real comprehensive evaluation indexes, linearly combining the feature vectors according to preset weights, and comparing them with the real comprehensive evaluation index to obtain a residual (i.e., the true value minus the weighted sum of the features). The residual reflects information not captured by the model (such as noise, unmodeled features or interactions). The residual is statistically processed by regression analysis (such as calculating the mean and verifying unbiasedness) to finally obtain the error term. The specific calculation formula is:

[0223] (9.1)

[0224] in, represents the grayscale feature vector, represents the texture feature vector, represents the fractal dimension eigenvector, represents the shape feature vector, represents the spectrum feature vector, e represents the preset error term, is the weight of each eigenvector.

[0225] Based on the obtained comprehensive evaluation index, determine the score range value that the comprehensive evaluation index falls into. If a score range value is set in advance with a unique corresponding grading result, the grading result corresponding to the score range value that the comprehensive evaluation index falls into is used as the second grading result.

[0226] In step S110 , a final classification result is obtained by fusing the first classification result and the second classification result.

[0227] First, calculate the confidence of the first classification result. The calculation formula is:

[0228] (10.1)

[0229] Among them, P dl Represents the probability distribution vector of the first classification result output by the image classification deep learning network model, C represents the total number of classification categories, c represents the category index, H(P dl ) represents the entropy of the probability distribution, which is used to characterize the uncertainty of the first classification result.

[0230] Then, the confidence is normalized to obtain the confidence score. The normalization formula is:

[0231] (10.2)

[0232] The larger the confidence score, the more credible the first classification result is.

[0233] Then, calculate the fifth weight corresponding to the confidence score using the following formula:

[0234] (10.3)

[0235] in, represents the fifth weight, is a learnable parameter. In this embodiment, .

[0236] Secondly, the second-level classification results output by the comprehensive evaluation model are converted into probability distribution vectors. Finally, the first classification result and the second classification result are integrated, that is, the final probability distribution value is calculated based on the probability distribution vector of the first classification result, the fifth weight, and the probability distribution vector of the second classification result. The calculation formula is:

[0237] (10.4)

[0238] Based on the final probability distribution value P final , determine the final probability distribution value P final The final score range value falls into, and a final score range value is also set in advance with a unique corresponding final classification result, then the final probability distribution value P final The final grading result corresponding to the final score range value falls into is used as the final grading result of the image to be graded.

[0239] In summary, the implementation principle of a mineral particle identification and grading method based on digital images and deep learning algorithms in an embodiment of the present application is as follows: first, multiple features of the image to be graded are extracted, and the multiple features are used to comprehensively and meticulously describe the feature information of the image to be graded. Then, the multiple features are fused to obtain a fused feature vector, and the fused feature vector is processed using an image grading deep learning network model to obtain a first grading result. At the same time, a weight calculation model is used to calculate the corresponding weight for each feature vector in real time based on information perception data and / or the Shapley value of each feature vector. Furthermore, a comprehensive evaluation model is used to calculate the second grading result based on each feature vector and the weight corresponding to each feature vector. Finally, the first grading result and the second grading result are fused again to obtain the final grading result. The final grading result combines the intelligent learning advantages of the deep learning network model and the comprehensive evaluation model and the feature allocation advantages of the weight calculation model, combines the grading results output by the two paths, reduces the prejudgment risk caused by environmental interference, significantly enhances the robustness of the system, and makes the final grading result of the image to be graded more accurate and reliable.

[0240] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned embodiments of the mineral particle identification and grading method based on digital images and deep learning algorithms.

[0241] In an exemplary embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7 The electronic device 700 shown includes a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, for example, via a bus 702. Optionally, the electronic device 700 may further include a transceiver 704. It should be noted that in actual applications, the number of transceivers 704 is not limited to one, and the structure of the electronic device 700 does not constitute a limitation on the embodiments of the present application.

[0242] Processor 701 may be a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 701 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

[0243] Bus 702 may include a path for transmitting information between the above components. Bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 702 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0244] The memory 703 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0245] The memory 703 is used to store computer program codes for executing the solution of the present application, and the execution is controlled by the processor 701. The processor 701 is used to execute the computer program codes stored in the memory 703 to implement the contents shown in the above method embodiments.

[0246] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0247] Based on the same inventive concept, an embodiment of the present application also provides a storage medium, which stores a computer program, wherein the computer program is configured to execute the mineral particle identification and grading method based on digital images and deep learning algorithms of any of the above embodiments when running.

[0248] Those skilled in the art will clearly understand that the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they will not be further described here.

[0249] Those skilled in the art will understand that the technical solution of the present application, in essence, or in whole or in part, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of program instructions that cause an electronic device (such as a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application when the program instructions are executed. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0250] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware related to program instructions (such as electronic devices such as personal computers, servers, or network devices), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of an electronic device, the electronic device executes all or part of the steps of the methods described in the various embodiments of the present application.

[0251] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that, within the spirit and principles of the present application, they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate from the protection scope of the present application.

Claims

1. A method for identifying and grading mineral particles based on digital images and deep learning algorithms, characterized in that: The method comprises: Acquire an image containing mineral particles to be classified; Obtaining a grayscale feature vector according to the brightness distribution of the image to be classified; Obtaining a texture feature vector according to texture details of mineral particles in the image to be classified; Obtaining a fractal dimension feature vector according to the local aggregation and global distribution of mineral particles in the image to be classified; Obtaining a shape feature vector according to the shape of the mineral particles in the image to be classified; Obtaining a frequency spectrum feature vector according to the distribution of mineral particles in the image to be classified in different frequency bands; Using a pre-trained image grading deep learning network model to obtain a first grading result according to the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector; Calculating the corresponding weights for the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector using a preset weight calculation model; A preset comprehensive evaluation model is used to calculate a second classification result based on the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, the spectrum feature vector, and the weights corresponding to each feature vector; Calculating a weight of the confidence score of the first grading result, and obtaining a final probability distribution value based on the first grading result, the weight of the confidence score, and the second grading result, and using the final grading result corresponding to the final score range within which the final probability distribution value falls as the final grading result of the image to be graded; Wherein, a final score range value corresponds to a final grading result.

2. The method according to claim 1, characterized in that The step of obtaining a texture feature vector based on texture details of mineral particles in the image to be classified includes: quantizing each pixel value in the image to be graded to obtain a quantized image; Based on the quantized image, gray level co-occurrence matrices in the directions of 0°, 45°, 90°, and 135° are calculated respectively; According to the gray level co-occurrence matrix in each direction, the gray level co-occurrence matrix eigenvector corresponding to each direction is calculated; Combining the gray-level co-occurrence matrix eigenvectors corresponding to each direction to obtain a multi-directional joint eigenvector; Calculating a local binary pattern feature vector of the quantized image; The gray level co-occurrence matrix eigenvectors, the multi-directional joint eigenvectors and the local binary pattern eigenvectors corresponding to each direction are fused to obtain the texture eigenvectors.

3. The method according to claim 2, characterized in that The combining of the gray level co-occurrence matrix eigenvectors corresponding to each direction to obtain a multi-directional joint eigenvector includes: Obtaining the wind speed direction and camera shooting direction of the shooting location of the image to be graded; determining a main diffusion direction of mineral particles in the image to be classified according to the wind speed direction at the shooting location and the shooting direction of the camera; Determine the direction of the gray level co-occurrence matrix that has the smallest angle with the main diffusion direction, and use the gray level co-occurrence matrix eigenvector corresponding to the direction as the target gray level co-occurrence matrix eigenvector; Calculate the weights of the gray-level co-occurrence matrix eigenvectors corresponding to each direction, and calculate the multi-directional joint eigenvector based on the gray-level co-occurrence matrix eigenvectors corresponding to each direction and the corresponding weights; wherein the weight of the target gray-level co-occurrence matrix eigenvector is greater than the weights of the gray-level co-occurrence matrix eigenvectors corresponding to other directions.

4. The method according to claim 1, wherein The fractal dimension feature vector is obtained according to the local aggregation and global distribution of the mineral particles in the image to be classified, including: Dividing the image to be classified into a plurality of first sub-images, assigning a corresponding first weight to each of the first sub-images, and assigning a second weight to the image to be classified; Calculating local fractal dimension feature data of each of the first sub-images, wherein the local fractal dimension feature data is used to reflect the aggregation of mineral particles in the first sub-image; Calculating global fractal dimension feature data of the image to be classified, wherein the global fractal dimension feature data is used to reflect the global distribution of mineral particles in the image to be classified; The fractal dimension feature vector is obtained by calculation according to the local fractal dimension feature data and the corresponding first weight, the global fractal dimension feature data and the corresponding second weight of each of the first sub-images.

5. The method according to claim 1, characterized in that The obtaining of a frequency spectrum feature vector according to the distribution of mineral particles in the image to be classified in different frequency bands includes: Converting the image to be classified into a frequency domain image by a Fourier transform algorithm; Energy information of the frequency domain image in a first frequency band, a second frequency band, and a third frequency band are extracted to obtain a spectrum feature vector; wherein the first frequency band, the second frequency band, and the third frequency band are three non-overlapping frequency range values ​​arranged in ascending order.

6. The method according to claim 1, characterized in that The image grading deep learning network model is constructed based on a graph neural network teacher model and a Transformer student model; The image grading deep learning network model trained in advance obtains a first grading result according to the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector, including: fusing the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector to obtain a fused feature vector; The first grading result is obtained by jointly processing the fused feature vector through the graph neural network teacher model and the Transformer student model.

7. The method according to claim 1, characterized in that The using of a preset weight calculation model to calculate the corresponding weights for the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector includes: The weight corresponding to each eigenvector is calculated based on the information perception data and / or the Shapley value of each eigenvector; wherein the information perception data at least includes the mineral particle information in the image to be classified and the environmental information when the image to be classified is collected.

8. The method according to claim 7, characterized in that If the weight corresponding to each eigenvector is calculated based on the information perception data, the method includes: Calculating the attention weight of each feature vector through the image hierarchical deep learning network model; Acquire an actual environmental parameter group, where the actual environmental parameter group is environmental information when the image to be graded is collected, and the environmental information includes at least temperature, humidity, dust concentration, and wind direction; Calculating a third weight of each eigenvector based on the attention weight of each eigenvector and the actual environment parameter group; Obtaining a preset mapping table, wherein the mapping table stores multiple sets of standard environmental parameter groups, each set of the standard environmental parameter groups including environmental information and initial weights of each feature vector under the environmental information; In the mapping table, the standard environment parameter group having the highest similarity to the actual environment parameter group is selected as the target environment parameter group; Calculating the reliability index of the target environmental parameter group according to the actual environmental parameter group; Calculating a fourth weight of each eigenvector according to the reliability index and the initial weight of each eigenvector in the target environmental parameter group; The final weight value of each eigenvector is calculated based on the third weight and the fourth weight of each eigenvector.

9. The method according to claim 8, characterized in that The preset comprehensive evaluation model is used to calculate the second classification result according to the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, the spectrum feature vector, and the weight corresponding to each feature vector, including: Multiplying the grayscale feature vector, the texture feature vector, the fractal dimension feature vector, the shape feature vector, and the spectrum feature vector by their respective corresponding final weight values ​​to obtain products corresponding to the feature vectors; Adding the product corresponding to each eigenvector and a preset error term to obtain a comprehensive evaluation index; wherein the preset error term is determined based on a regression analysis algorithm and experimental data, and the experimental data at least includes the grayscale eigenvector, the texture eigenvector, the fractal dimension eigenvector, the shape eigenvector, and the spectrum eigenvector; Determining a score range within which the comprehensive evaluation index falls, wherein the score range corresponds to a grading result; The grading result corresponding to the score range within which the comprehensive evaluation index falls is used as the second grading result.

10. The method according to claim 1, characterized in that The final probability distribution value is calculated by the following calculation formula: , where P dl represents the probability distribution vector of the first classification result, Represents the weight of the confidence score, P ce The probability distribution vector representing the second classification result.

Images