Intelligent sorting method and system for coal gangue and coal quality based on image hyperspectral technology
By combining image hyperspectral technology and acoustic testing technology, the multi-dimensional characteristics of coal gangue and coal quality were analyzed using deep learning models, which solved the problems of low accuracy and poor adaptability of existing sorting methods, and achieved efficient and accurate coal gangue and coal quality sorting.
Patent Information
- Application Number
- CN202411648448.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2025-03-21
- Estimated Expiration
- 2044-11-19
AI Technical Summary
The existing coal gangue and coal quality sorting methods have problems such as low sorting accuracy, poor adaptability, and insufficient real-time performance, which is difficult to meet the needs of fast and accurate sorting on the industrial site.
Intelligent sorting method based on image hyperspectral technology and acoustic wave testing technology is adopted, and multi-dimensional features are fusion analysis combined with deep learning models to achieve efficient and accurate sorting of coal gangue and coal quality.
It significantly improves the sorting accuracy of coal gangue and coal quality, achieves high recognition accuracy in complex environments, improves sorting efficiency and quality, and reduces production costs.
Smart Images
Figure CN119152377B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of coal processing, and in particular to a method and system for intelligently sorting coal gangue and coal quality based on image hyperspectral technology. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] The existing methods for sorting coal gangue and coal quality mainly include manual experience identification, chemical analysis, image analysis sorting, and sorting combined with machine learning algorithms. Identification based on manual experience is greatly affected by subjective factors, and is prone to misjudgment and omission, resulting in low sorting accuracy and difficulty in meeting the needs of coal quality classification; chemical analysis is a non-real-time analysis method, which is complex and time-consuming to operate and difficult to adapt to the high requirements of industrial sites for rapid analysis; mechanical sorting technology is often based on the physical property differences between coal and coal gangue, with low sorting accuracy, poor adaptability to different types of coal, and difficulty in achieving coal quality sorting; image analysis sorting method generally uses ordinary images, processes images, extracts features for sorting, and provides a non-contact solution, but image acquisition relies on a fixed light source and image information is limited, which affects recognition accuracy and makes it difficult to accurately distinguish between coal gangue and coal of different qualities; machine learning algorithms have been applied to a certain extent in the field of sorting, but there are also some problems. Machine learning algorithms have high requirements for the quality and richness of the collected information, so they have poor adaptability and need to further improve the information collection method to improve the anti-interference ability of the collection. In addition, machine learning algorithms lack a real-time feedback mechanism and cannot adjust the sorting strategy and model parameters in time according to changes in coal gangue and coal characteristics, resulting in affected sorting efficiency and stability. Summary of the invention
[0004] In order to solve the above problems, the present invention proposes a method and system for intelligent sorting of coal gangue and coal quality based on image hyperspectral technology, which combines hyperspectral imaging technology and acoustic wave testing technology to extract acoustic wave, spectrum, texture and spatial features in real time, and uses a deep learning model to perform fusion analysis on multi-dimensional features. At the same time, combined with the adaptability of the online learning mechanism model and the stability of the sorting process, efficient and accurate sorting of coal gangue and coal quality can be achieved, thereby improving production efficiency and quality.
[0005] In some embodiments, the following technical solutions are adopted:
[0006] A method for intelligently sorting coal gangue and coal quality based on image hyperspectral technology, comprising:
[0007] Acquire a hyperspectral image on the transmission line of the sample to be tested, convert the hyperspectral image into a grayscale image, and extract edge information of the sample to be tested;
[0008] The spectral image of each sample to be tested is segmented by edge thinning method, and the position information of each sample to be tested is extracted at the same time; the dynamic position of each sample to be tested is determined by combining the temporal and spatial characteristics of sample movement;
[0009] Transmitting sound waves to each sample to be tested, and receiving sound wave signals after passing through each sample to be tested;
[0010] Preprocess the spectral image and acoustic wave signal of each sample to be tested, extract the spectral features, texture features, spatial features and acoustic wave features respectively, perform multi-dimensional feature fusion, and form a comprehensive feature vector;
[0011] The comprehensive feature vector is input into the trained gangue recognition model to obtain the recognition results of gangue and coal.
[0012] As a further solution, the spectral image of each sample to be tested is segmented by edge thinning method, specifically:
[0013] If only one pixel in the 8-neighborhood of a pixel belongs to the edge, the pixel is retained; otherwise, the pixel is deleted; according to this process, each pixel along the edge is judged, and the skeleton line of the sample to be tested is obtained based on the retained pixels;
[0014] Create a labeled image of the same size as the grayscale image, initialize the pixel values of all pixels in the labeled image to 0, traverse each pixel, perform connectivity detection on pixels with a pixel value of 0, record the connected domain to which each pixel belongs in the labeled image, and mark the connected domain as a mask; perform bitwise operations on each pixel value and mask of the original image to segment a single image of each object in the original image.
[0015] As a further solution, the texture features of each sample to be tested are extracted. The specific process is as follows:
[0016] A multi-scale convolutional neural network is used to extract features from the preprocessed spectral image, wherein the multi-scale convolutional neural network includes a plurality of convolutional layers, and each convolutional layer outputs a texture feature map; all texture feature maps are weightedly fused to obtain a fused texture feature map;
[0017] Calculate the gradient information of each pixel in the fused texture feature map, and determine the attention weight of each pixel according to the gradient information; use the attention weight to weighted refine each pixel in the fused texture feature map to obtain a weighted texture feature map;
[0018] For the weighted texture feature map, the gradient information of each pixel is recalculated, and the attention weight of each pixel is re-determined based on the gradient information to obtain the weighted texture feature map again; the process is repeated until the set requirements are met; and finally the texture features of the sample to be tested are obtained.
[0019] As a further solution, the spatial features of each sample to be tested are extracted. The specific process is as follows:
[0020] The spectral image of each sample to be tested is processed using a hole convolutional neural network model to obtain the spatial characteristics of each sample to be tested;
[0021] The dilated convolutional neural network model includes an input layer, three dilated convolutional layers, a feature fusion layer and an output layer connected in sequence; the input layer receives the spectral image of each sample to be tested. I ,in I is a three-dimensional tensor represented as , H is the height of the image, W is the width of the image, C is the number of channels of the image;
[0022] Each dilated convolution layer uses dilated convolutions with different dilation rates to capture spectral images. I Spatial information of different scales, output feature map F n , n =1,2,3;
[0023] After the spectral image is processed by three layers of hole convolution layers, the output feature map is fused by the feature fusion layer. F n Fusion is performed to obtain the spatial characteristics of each sample to be tested F f , and output through the output layer.
[0024] As a further solution, multi-dimensional feature fusion is performed to form a comprehensive feature vector, specifically:
[0025] Convert the spectral features into a fixed-length feature vector v spec ; The texture features and spatial features are converted into feature vectors through global average pooling v texture and v spatial ; Convert the acoustic wave features into feature vectors v sonic ;
[0026] The texture feature vector v texture and spatial eigenvectors v spatial The feature vector is obtained by concatenating and learning the internal feature weights using the self-attention mechanism. v self-attention ;
[0027] The feature vector v spec , the dimensionally adjusted feature vector v sonic And the feature vector v self-attention , respectively input into the multi-layer perceptron MLP to obtain a comprehensive feature vector that integrates all modal information v fused .
[0028] In other embodiments, the following technical solutions are adopted:
[0029] An intelligent sorting system for coal gangue and coal quality based on image hyperspectral technology, comprising:
[0030] A data acquisition module, used to acquire a hyperspectral image on the transmission line of the sample to be tested, convert the hyperspectral image into a grayscale image, and extract edge information of the sample to be tested;
[0031] The spatiotemporal feature extraction module is used to segment the spectral image of each sample to be tested by edge thinning method, and extract the position information of each sample to be tested; combined with the spatiotemporal characteristics of sample movement, the dynamic position of each sample to be tested is determined;
[0032] The acoustic wave signal extraction module is used to transmit acoustic waves to each sample to be tested and receive the acoustic wave signal after passing through each sample to be tested;
[0033] The comprehensive feature extraction module is used to pre-process the spectral image and acoustic wave signal of each sample to be tested, respectively extract the spectral features, texture features, spatial features and acoustic wave features, perform multi-dimensional feature fusion, and form a comprehensive feature vector;
[0034] The coal identification module is used to input the comprehensive feature vector into the trained gangue identification model to obtain the identification results of gangue and coal.
[0035] The coal quality identification module is used to perform secondary fusion of the multi-dimensional features identified as coal, and input the fused features into the trained coal quality identification model to obtain the coal quality identification results; the coal quality identification results include high, medium and low.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] (1) Based on hyperspectral imaging technology, the present invention combines acoustic wave testing technology and deep learning models to perform fusion analysis on the multi-dimensional features of coal and coal gangue, which can more comprehensively describe the characteristics of coal and coal gangue. By introducing an adaptive light source adjustment system and an edge refinement algorithm, the anti-interference ability of feature extraction is improved, and the sorting accuracy of coal gangue and coal quality is significantly improved, so that the model can still maintain a high recognition accuracy in complex environments.
[0038] (2) The present invention realizes the organic combination of gangue sorting and coal quality sorting. By performing one-time imaging acquisition and adopting a hierarchical fusion neural network structure to perform two multi-dimensional feature fusions, the results of gangue sorting and coal quality sorting are obtained respectively, and the two sorting tasks are completed at one time. Not only the gangue is effectively separated, but also the quality of coal is accurately sorted, thereby improving the sorting efficiency, reducing the production cost, and having a high economic benefit.
[0039] (3) The present invention uses a multi-scale convolutional neural network to capture texture information of different scales, avoiding the limitations of single-scale feature extraction, so that the model can better learn the complex texture features of gangue and coal. Then, weighted refinement is used to further enhance the texture features. By continuously updating the attention weights, the model pays more attention to areas with drastic texture changes, thereby improving the distinguishability and robustness of the features, allowing the model to maintain a high recognition accuracy in complex environments.
[0040] (4) The present invention can better adapt to the changes in the characteristics of coal and coal gangue by updating the model parameters in real time through the online learning mechanism. The continuity and stability of the sorting process can be ensured through abnormal data detection. Combined with the online sorting execution and control system, the online sorting of coal gangue and coal quality can be carried out according to the results of the recognition model, which can realize real-time, fast and accurate recognition and sorting, and improve production efficiency and quality.
[0041] Other features and advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 This is a flow chart of a method for intelligently separating gangue and coal quality based on image hyperspectral technology in an embodiment of the present invention;
[0043] Figure 2 A schematic diagram of a comprehensive feature vector in an embodiment of the present invention;
[0044] Figure 3 Schematic diagram of the structure of the coal gangue identification model in the embodiment of the present invention;
[0045] Figure 4Schematic diagram of the process of edge thinning algorithm in an embodiment of the present invention. DETAILED DESCRIPTION
[0046] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.
[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0048] Embodiment 1
[0049] In one or more embodiments, a method for intelligently sorting coal gangue and coal quality based on image hyperspectral technology is disclosed, combined with Figure 1 , including:
[0050] S101: Acquire a hyperspectral image on the transmission line of the sample to be tested, convert the hyperspectral image into a grayscale image, and extract edge information of the sample to be tested.
[0051] In this embodiment, the optimal threshold range of reflectivity is determined based on the optimal working illumination range of the hyperspectral imaging device. The hyperspectral imaging device and the adaptive light source adjustment system are placed above the conveyor belt. The adaptive light source adjustment system monitors the reflectivity of the surface of the sample to be tested and the light conditions of the surrounding environment through sensors, and processes the collected data to evaluate whether the current lighting conditions meet the optimal requirements of hyperspectral imaging. For example, it can be determined whether the reflectivity is within the optimal threshold range of reflectivity; according to the judgment result, the current or voltage input of the light source is adjusted to change the brightness of the light source. After adjusting the light, the hyperspectral imaging device is used to collect the hyperspectral image of the target object in real time. At this time, multiple samples being transmitted on the conveyor belt are collected.
[0052] The collected image information is converted into a grayscale image, the Canny operator is used for edge detection, the Gaussian filter is used to smooth the image, and the amplitude and direction of the image gradient are calculated to determine the initial edge information.
[0053] S102: Segment the spectral image of each sample to be tested by edge thinning method, and extract the position information of each sample to be tested; determine the dynamic position of each sample to be tested by combining the spatiotemporal characteristics of the object movement.
[0054] In this embodiment, the initial rough edge detected by the Canny operator is refined, and combined with Figure 4 Based on the determined initial edge information, each pixel along the edge is judged: if only one pixel in the 8-neighborhood of a pixel belongs to the edge, the pixel is retained; otherwise, the pixel is deleted; this process is repeated for each pixel along the edge until no pixels are deleted, and finally the skeleton line of the sample to be tested is obtained to more accurately identify the position and shape of coal gangue and improve the segmentation accuracy.
[0055] Morphological processing is performed on the binary image after edge detection to close the disconnected edges, remove small noise points, and smooth the edges, thereby creating a marked image of the same size as the binary image, which is also a grayscale image.
[0056] Initialize the pixel value of all pixels of the marked image to 0; then, traverse each pixel of the marked image. When encountering a pixel with a pixel value of 0, it means that the pixel has not been visited, and then the connectivity check is performed. Use the depth-first search (DFS) algorithm, starting from the pixel, search along the 4-neighborhood or 8-neighborhood of the image, and mark all connected pixels with the same number. Whenever a new connected domain is found, mark all pixels of the connected domain with a unique non-zero value in the marked image; in this way, each non-zero value in the marked image represents a different connected domain. After the traversal is completed, the marked image records the connected domain to which each pixel belongs, and marks these connected domains as masks. By performing bit operations on the pixel values of the original image and the mask, the pixel values in the connected domain marked in the mask can be retained, and other pixel values can be set to 0, thereby segmenting a single image of each sample to be tested in the original image.
[0057] The connected domain is marked as a mask, and each single image of the sample to be tested in the original image is segmented by performing bit operations on the original image and the mask. For each segmented single image of the sample to be tested, the position information of its bounding box is calculated, and the leftmost, rightmost, topmost and bottom four boundary points of the sample to be tested are obtained by traversing the segmented single image of the sample to be tested, and then the coordinates, width and height of the upper left corner are calculated. Combined with time series analysis, the spatiotemporal characteristics of the sample to be tested during its movement on the conveyor belt are extracted to determine the dynamic position of the sample to be tested.
[0058] S103: transmitting sound waves to each sample to be tested, and receiving sound wave signals after passing through each sample to be tested.
[0059] After determining the dynamic position of the target object, the automatic capture system is triggered to lock and track the target sample to be tested, the sound wave transmitter is started to transmit sound waves to the locked target sample to be tested, and the sound wave receiver is used to receive the sound wave signal reflected from the target sample to be tested.
[0060] S104: Preprocess the spectral image and acoustic wave signal of each sample to be tested, extract spectral features, texture features, spatial features and acoustic wave features respectively, perform multi-dimensional feature fusion, and form a comprehensive feature vector.
[0061] In this embodiment, combined with Figure 2 , the collected hyperspectral images, acoustic wave signals and corresponding dynamic position information of the samples to be tested are integrated to establish the hyperspectral image, acoustic wave and dynamic position data structure of the samples to be tested.
[0062] Specifically, the acoustic wave signal and the segmented original hyperspectral image are preprocessed respectively. The preprocessing of the acoustic wave signal includes filtering, denoising, signal enhancement, etc. The image preprocessing process includes grayscale conversion to simplify image data, brightness contrast adjustment to improve visual effects, sharpening to enhance edges and details, and median filtering to remove noise; spectral preprocessing includes smoothing to reduce data noise, derivative calculation to highlight spectral features, normalization to eliminate intensity changes, wavelet transform for feature extraction and denoising, and least squares convolution smoothing to further reduce noise interference and improve the quality of spectral data and the accuracy of analysis.
[0063] (1) The process of extracting acoustic wave features is as follows: preprocess the acoustic wave signal, such as filtering, denoising and signal enhancement, to improve the signal quality. Then, extract the time domain features of the acoustic wave signal, such as statistical features such as peak value, valley value, average value, variance, and periodic features such as frequency, period, and pulse width. At the same time, the frequency domain features of the acoustic wave signal can also be extracted, such as spectrum, short-time Fourier transform (STFT) and wavelet transform. The extracted acoustic wave features can reflect the physical properties and internal structure of coal gangue and coal, such as hardness, density and internal structure, so as to effectively distinguish coal gangue and coal and improve the sorting accuracy.
[0064] (2) The spectral feature extraction process is as follows: extract the original spectral data of the hyperspectral image, perform first-order differential calculation and standardization, further use the continuum removal transform to process the standardized spectral data, and extract spectral features from the spectral data.
[0065] By performing the first-order differentiation on the spectral curve, a first-order differential spectral curve can be obtained, highlighting the slope change of the spectral curve; standardizing the spectral curve after the first-order differentiation can eliminate the influence of external factors such as lighting conditions, making the spectral curve more comparable, and then performing a continuum removal transformation on the standardized spectral curve to remove the baseline drift of the spectral curve and highlight the absorption characteristics.
[0066] (3) The process of extracting texture features is as follows:
[0067] A multi-scale convolutional neural network (MSCNN) is used to extract features from the preprocessed spectral image. The multi-scale convolutional neural network contains a network of multiple convolutional layers. Each convolutional layer uses convolution kernels of different sizes to extract texture features of different scales, and uses ReLU as the activation function.
[0068] Assume that MSCNN contains L convolutional layers, l The convolutional layers use a size of k l The convolution kernel extracts texture features, then l The feature map output by the convolutional layer is expressed as F l ,in l =1,2,3,…, L .
[0069] The texture feature maps of different scales are fused to obtain the fused texture feature map F f .
[0070] The specific fusion method can use weighted summation or weighted average, and the weights can be adjusted according to the importance of different features to the classification task. The following formula can be used for weighted summation fusion:
[0071]
[0072] in, The fused texture feature map at the pixel point and Channel k The value of For the l The convolutional layer outputs a feature map at pixel points and Channel k The value of For the l The weights of the convolutional layer output features; among them, the channel k is a dimension of the feature map, which represents the The above is a collection of feature information extracted by different convolutional layers and convolution kernels of different sizes.
[0073] The weighting process is to weight each pixel in each feature map. , each feature map has a pixel value at that position. Pixels at the same position will correspond to different weights in each feature map. For each pixel position , the pixel values of different feature maps at this position are combined with their corresponding weights, and the value of the pixel position is obtained after weighting.
[0074] In order to further highlight the texture features, this embodiment is a fused texture feature map Assign weights to each pixel point; first, calculate the fused texture feature map Gradient information , the larger the gradient value, the more drastic the texture change, and the higher the corresponding pixel weight. As the attention weight of each pixel ; Use attention weights The fused texture feature map Perform weighted refinement to obtain a weighted texture feature map Repeat the weighted refinement process for the weighted texture feature map until the set number of iterations is reached or the convergence condition is met, thereby obtaining the final multi-scale texture feature map. .
[0075] Each iteration builds on the results of the previous iteration to gradually approach a better or more refined solution. Over multiple iterations, the value of each pixel is gradually adjusted to better reflect the structure and features of its neighborhood.
[0076] The formula for weighted refinement in this embodiment is as follows:
[0077]
[0078] in, It is After iterative weighted refinement, the texture feature map is at pixel point and Channel k The value of The fused texture feature map at the pixel point and Channel k The value of Pixel In the channel k The attention weight, and Represents pixel The offset during iterative weighted refinement, n is the convolution kernel size.
[0079] This embodiment uses a multi-scale convolutional neural network to capture texture information of different scales, avoiding the limitations of single-scale feature extraction, so that the model can better learn the complex texture features of gangue and coal. Then, weighted refinement is used to further strengthen the texture features. By continuously updating the attention weights, the model pays more attention to areas with drastic texture changes, thereby improving the distinguishability and robustness of the features, so that the model can still maintain a high recognition accuracy in complex environments.
[0080] (4) The process of extracting spatial features is as follows:
[0081] The spectral image of each sample to be tested is processed using a hole convolutional neural network model to obtain the spatial characteristics of each sample to be tested;
[0082] The dilated convolutional neural network model includes an input layer, three dilated convolutional layers, a feature fusion layer, and an output layer connected in sequence; the input layer receives the spectral image of each sample to be tested. I ,in I is a three-dimensional tensor represented as , H is the height of the image, W is the width of the image, C is the number of channels of the image.
[0083] The model consists of three layers of dilated convolution layers, each of which uses dilated convolutions with different dilation rates to capture spatial information of different scales. n The dilation rate of the layer dilated convolution is d n , the convolution kernel is K n , then n Output feature map of the layer F n It can be expressed as:
[0084] ;
[0085] in, I n-1 It is n -1 layer output, Conv d n Indicates the use of expansion rate d n The dilated convolution operation.
[0086] After the spectral image is processed by three layers of hole convolution layers, the feature map is output through the feature fusion layer. F 1 , F 2 , F 3 Fusion is performed to obtain the spatial characteristics of each sample to be tested F f And output through the output layer.
[0087] The three-layer hole convolution layer in this embodiment can expand the receptive field of the convolution kernel, enabling the model to capture a wider range of image information, thereby better understanding the overall shape and structure of the object.
[0088] In this embodiment, multi-dimensional feature fusion is performed on the acoustic wave features, spectral features, texture features and spatial features extracted above to form a comprehensive feature vector. The specific process is as follows:
[0089] Preprocess the extracted features: convert the spectral features into a fixed-length feature vector v spec ; Through global average pooling, texture features and spatial features are converted into feature vectors respectively v texture and v spatial ; Form the acoustic wave features into feature vectors v sonic .
[0090] Texture feature vector v texture and spatial eigenvectors v spatial Through splicing and fusion, we get v concat , and use the self-attention mechanism layer to learn the internal feature weights to obtain the feature vector v self-attention .
[0091] v concat =[ v texture ; v spatial ];
[0092] v self-attention =Self-Attention( v concat );
[0093] Among them, Self-Attention represents the self-attention mechanism function.
[0094] Then, the feature vector v spec , eigenvector v sonic And the feature vector v self-attention The input is sent to the multi-layer perceptron MLP, which integrates the feature vectors from different modalities, applies the ReLU activation function for nonlinear transformation, and outputs a comprehensive feature vector that integrates all modal information. v fused .
[0095] v cross-modal =[ vspec ; v self-attention ; v sonic ];
[0096] v fused =MLP( v cross-modal );
[0097] Among them, MLP represents a multi-layer perceptron function, which includes multiple fully connected layers and ReLU activation functions. v cross-modal Represents the input features of the multilayer perceptron MLP.
[0098] Since both texture features and spatial features are closely related to the shape and structure of an object, splicing the two first can highlight this correlation, allowing the model to better learn the relationship between texture and spatial features, thereby more accurately describing the shape and structure of the object, enhancing the model's expressiveness, and improving recognition accuracy. In addition, if all features are directly spliced together, it may lead to excessive feature dimensions, increase model complexity, and may also lead to redundancy and conflict between features of different modalities, thereby affecting training efficiency and generalization ability; this embodiment splices spatial features and texture features first, which can reduce feature dimensions, reduce model complexity, effectively reduce feature redundancy, reduce conflicts, and improve model training efficiency and generalization ability.
[0099] S105: Input the comprehensive feature vector into the trained gangue recognition model to obtain the recognition results of gangue and coal.
[0100] In this embodiment, the training data of the gangue recognition model is coal sample data with gangue or coal labels; these data may include information such as hyperspectral images, acoustic signals, texture features, and spatial features. Through training and adjusting model parameters, the model can accurately identify gangue and coal, and through continuous training and adjustment, the model can automatically learn the complex features of gangue and coal, improving the accuracy and generalization ability of recognition.
[0101] Combination Figure 3The gangue recognition model of this embodiment includes an input layer, a classification layer, a position regression layer and an output layer connected in sequence; wherein the input layer receives a comprehensive feature vector that integrates spectral features, texture features, spatial features and acoustic wave features, and the classification layer obtains the probability distribution of each sample to be tested belonging to gangue or coal based on the comprehensive feature vector; the classification layer specifically includes two fully connected layers and a Softmax layer, the fused feature vector is connected to the two fully connected layers, the probabilities of coal and gangue are calculated respectively, the output of the fully connected layer is connected to the Softmax layer, the output of the fully connected layer is converted into a probability distribution, and the probability distribution of each object belonging to gangue or coal is output; the position regression layer outputs the position information of each sample to be tested; the output layer outputs the classification result and position information of each sample to be tested.
[0102] According to the results of the identification model, the actuator is directed by a high-precision control system to perform online sorting of gangue and coal.
[0103] S106: The multi-dimensional features identified as coal are fused for a second time, and the fused features are input into a trained coal quality identification model to obtain a coal quality identification result.
[0104] After the gangue and coal are separated, the multi-dimensional features of the separated coal, such as spectral features, texture features, spatial features, and acoustic wave features, are secondary fused, and the trained coal quality recognition model is used to achieve coal quality separation.
[0105] The structure of the coal quality recognition model is basically the same as that of the coal gangue recognition model. The training data of the coal quality recognition model is sample data of different coal qualities with coal quality labels. These data may include hyperspectral images, acoustic signals, texture features, spatial features, etc. The coal quality recognition model recognizes the quality of coal, so it is necessary to set the fully connected layer of the classification layer according to the classification level. For example, if the quality of coal is divided into high, medium, and low, the classification layer needs three fully connected layers.
[0106] Of course, the gangue identification model and the coal quality identification model can also adopt the existing neural network structure, such as CNN.
[0107] In this embodiment, an incremental learning method is used to implement online learning. Incremental learning allows the system to regularly receive new training data and use these data to continuously update the model, thereby adapting to changes in the characteristics of different quality coal and coal gangue.
[0108] For the spectral characteristics, texture characteristics, spatial characteristics and acoustic wave characteristics of each pixel point in the coal spectral image after sorting, the statistical quantities such as the mean value, variance or standard deviation of each feature are calculated respectively, and the pixel points whose statistical quantities exceed the normal range are judged as abnormal data points.
[0109] This embodiment can identify abnormal data points that are significantly different from normal data by analyzing the statistical characteristics of the data. Once abnormal data is detected, the system will immediately start the sorting strategy adjustment mechanism and dynamically adjust the parameters of the model, such as classification thresholds or feature weights, to adapt to data changes. This method of dynamically adjusting parameters can effectively deal with abnormal situations and ensure the continuity and stability of the sorting process.
[0110] Embodiment 2
[0111] In one or more embodiments, a coal gangue and coal quality intelligent sorting system based on image hyperspectral technology is disclosed, comprising:
[0112] A data acquisition module, used to acquire a hyperspectral image on the transmission line of the sample to be tested, convert the hyperspectral image into a grayscale image, and extract edge information of the sample to be tested;
[0113] The spatiotemporal feature extraction module is used to segment the spectral image of each sample to be tested by edge thinning method, and extract the position information of each sample to be tested; combined with the spatiotemporal characteristics of sample movement, the dynamic position of each sample to be tested is determined;
[0114] The acoustic wave signal extraction module is used to transmit acoustic waves to each sample to be tested and receive the acoustic wave signal after passing through each sample to be tested;
[0115] A comprehensive feature extraction module is used to pre-process the spectral image and acoustic wave signal of each sample to be tested, respectively extract spectral features, texture features, spatial features and acoustic wave features, and form a comprehensive feature vector;
[0116] The coal identification module is used to input the comprehensive feature vector into the trained gangue identification model to obtain the identification results of gangue and coal.
[0117] The coal quality identification module is used to perform secondary fusion of the multi-dimensional features identified as coal, and input the fused features into the trained coal quality identification model to obtain the coal quality identification results; the coal quality identification results include high, medium and low.
[0118] The specific implementation of the above modules is the same as that in Example 1 and will not be described in detail.
[0119] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A method for intelligent separation of coal gangue and coal quality based on image hyperspectral technology, characterized in that: include: Acquire a hyperspectral image on the transmission line of the sample to be tested, convert the hyperspectral image into a grayscale image, and extract edge information of the sample to be tested; The spectral image of each sample to be tested is segmented by edge thinning method, and the position information of each sample to be tested is extracted at the same time; Combined with the temporal and spatial characteristics of sample movement, the dynamic position of each sample to be tested is determined; Transmitting sound waves to each sample to be tested, and receiving sound wave signals after passing through each sample to be tested; Preprocess the spectral image and acoustic wave signal of each sample to be tested, extract the spectral features, texture features, spatial features and acoustic wave features respectively, perform multi-dimensional feature fusion, and form a comprehensive feature vector; Inputting the comprehensive feature vector into a trained gangue recognition model to obtain recognition results of gangue and coal; Among them, multi-dimensional feature fusion is performed to form a comprehensive feature vector, specifically: Convert the spectral features into a fixed-length feature vector v spec ; The texture features and spatial features are converted into feature vectors through global average pooling v texture and v spatial ; Convert the acoustic wave features into feature vectors v sonic ; Texture feature vector v texture and spatial eigenvectors v spatial The feature vector is obtained by concatenating and learning the internal feature weights using the self-attention mechanism. v self-attention ; The feature vector v spec , the dimensionally adjusted feature vector v sonic And the feature vector v self-attention , respectively input into the multi-layer perceptron MLP to obtain a comprehensive feature vector that integrates all modal information v fused .
2. The method for intelligent separation of coal gangue and coal quality based on image hyperspectral technology according to claim 1, characterized in that: Also includes: The multi-dimensional features identified as coal are fused for a second time, and the fused features are input into a trained coal quality identification model to obtain a coal quality identification result; the coal quality identification result includes high, medium and low.
3. The method for intelligent separation of coal gangue and coal quality based on image hyperspectral technology according to claim 1, characterized in that: The spectral image of each sample to be tested is segmented by the edge thinning method, specifically: If only one pixel in the 8-neighborhood of a pixel belongs to the edge, the pixel is retained; otherwise, the pixel is deleted; according to this process, each pixel along the edge is judged, and the skeleton line of the sample to be tested is obtained based on the retained pixels; Create a labeled image of the same size as the grayscale image, initialize the pixel values of all pixels in the labeled image to 0, traverse each pixel, perform connectivity detection on pixels with a pixel value of 0, record the connected domain to which each pixel belongs in the labeled image, and mark the connected domain as a mask; perform bitwise operations on each pixel value and mask of the original image to segment a single image of each object in the original image.
4. The method for intelligent separation of coal gangue and coal quality based on image hyperspectral technology according to claim 1, characterized in that: Extract the location information of each sample to be tested, specifically: Traverse the segmented object image to obtain the leftmost, rightmost, topmost, and bottommost boundary points of the object, and then calculate the coordinates, width, and height of the upper left corner. Combined with time series analysis, the spatiotemporal characteristics of the object moving on the conveyor belt are extracted to determine the dynamic position of the object.
5. The method for intelligently sorting coal gangue and coal quality based on image hyperspectral technology according to claim 1, characterized in that: Extract the texture features of each sample to be tested. The specific process is as follows: A multi-scale convolutional neural network is used to extract features from the preprocessed spectral image, wherein the multi-scale convolutional neural network includes a plurality of convolutional layers, and each convolutional layer outputs a texture feature map; all texture feature maps are weightedly fused to obtain a fused texture feature map; Calculate the gradient information of each pixel in the fused texture feature map, and determine the attention weight of each pixel based on the gradient information; Using the attention weights, weighted refinement is performed on each pixel in the fused texture feature map to obtain a weighted texture feature map; For the weighted texture feature map, the gradient information of each pixel is recalculated, and the attention weight of each pixel is re-determined according to the gradient information to obtain the weighted texture feature map again; Repeat this process until the set requirements are met; ultimately the texture characteristics of the sample to be tested are obtained.
6. The method for intelligently sorting coal gangue and coal quality based on image hyperspectral technology according to claim 1, characterized in that: Extract the spatial features of each sample to be tested. The specific process is as follows: The spectral image of each sample to be tested is processed using a hole convolutional neural network model to obtain the spatial characteristics of each sample to be tested; The dilated convolutional neural network model includes an input layer, three dilated convolutional layers, a feature fusion layer and an output layer connected in sequence; the input layer receives the spectral image of each sample to be tested. I ,in I is a three-dimensional tensor represented as , H is the height of the image, W is the width of the image, C is the number of channels of the image; Each dilated convolution layer uses dilated convolutions with different dilation rates to capture spectral images. I Spatial information of different scales, output feature map F n , n =1,2,3; After the spectral image is processed by three layers of hole convolution layers, the output feature map is fused by the feature fusion layer. F n Fusion is performed to obtain the spatial characteristics of each sample to be tested F f , and output through the output layer.
7. The method for intelligently sorting coal gangue and coal quality based on image hyperspectral technology according to claim 1, characterized in that: Also includes: Adopt incremental learning method to realize online learning of coal gangue identification model and coal quality identification model; For the spectral features, texture features, spatial features and acoustic wave features of each pixel point in the spectral image of the sorted coal, the statistics of each feature are calculated respectively, and the pixel points whose statistics exceed the set normal range are judged as abnormal data points.
8. An intelligent coal gangue and coal quality sorting system based on image hyperspectral technology, characterized in that: include: A data acquisition module, used to acquire a hyperspectral image on the transmission line of the sample to be tested, convert the hyperspectral image into a grayscale image, and extract edge information of the sample to be tested; The spatiotemporal feature extraction module is used to segment the spectral image of each sample to be tested by edge thinning method, and extract the position information of each sample to be tested; combined with the spatiotemporal characteristics of sample movement, the dynamic position of each sample to be tested is determined; The acoustic wave signal extraction module is used to transmit acoustic waves to each sample to be tested and receive the acoustic wave signal after passing through each sample to be tested; The comprehensive feature extraction module is used to pre-process the spectral image and acoustic wave signal of each sample to be tested, respectively extract the spectral features, texture features, spatial features and acoustic wave features, perform multi-dimensional feature fusion, and form a comprehensive feature vector; A coal identification module, used for inputting the comprehensive feature vector into a trained gangue identification model to obtain identification results of gangue and coal; Among them, multi-dimensional feature fusion is performed to form a comprehensive feature vector, specifically: Convert the spectral features into a fixed-length feature vector v spec ; The texture features and spatial features are converted into feature vectors through global average pooling v texture and v spatial ; Convert the acoustic wave features into feature vectors v sonic ; Texture feature vector v texture and spatial eigenvectors v spatial The feature vector is obtained by concatenating and learning the internal feature weights using the self-attention mechanism. v self-attention ; The feature vector v spec , the dimensionally adjusted feature vector v sonic And the feature vector v self-attention , respectively input into the multi-layer perceptron MLP to obtain a comprehensive feature vector that integrates all modal information v fused .
9. The intelligent coal gangue and coal quality sorting system based on image hyperspectral technology as claimed in claim 8, characterized in that: Also includes: The coal quality identification module is used to perform secondary fusion of the multi-dimensional features identified as coal, and input the fused features into the trained coal quality identification model to obtain the coal quality identification results; the coal quality identification results include high, medium and low.
Citation Information
Patent Citations
Intelligent dry separator for three products
CN106733721A
Internet of things system
WO2023030513A1