Coal gangue identification method and system based on hyperspectral imaging and laser ranging

Abstract

The present invention relates to the technical field of gangue classification and identification, and in particular to a gangue identification method and system based on hyperspectral imaging and laser ranging, the method comprising: mapping hyperspectral image data onto a three-dimensional point cloud of laser ranging data to obtain a mapping result; using a trained network model to extract multi-scale spatial features and multi-scale spectral features from the mapping result; fusing the extracted features, and then generating candidate target areas based on the fused features; classifying the candidate target areas to obtain a classification result of gangue; constructing a three-dimensional model of the coal surface based on laser ranging data; generating a triangular mesh model for the three-dimensional model of the coal surface; smoothing the triangular mesh model to obtain a reconstructed model; determining the particle size of the coal block based on the reconstructed model; and identifying the petrographic features of the coal based on the spectral data to distinguish different types of coal. The accuracy and efficiency of gangue sorting are improved.

Description

Technical Field

[0001] The invention relates to the technical field of coal gangue classification and identification, and in particular to a coal gangue identification method and system based on hyperspectral imaging and laser ranging. Background Art

[0002] Raw coal will be mixed with a large amount of gangue during the mining process, such as top coal caving and thin coal seam mining, which will cause the top and bottom rocks of the working face coal seam to be mixed with a large amount of raw coal during the mining process, which will not only increase the cost of coal preparation, but also pollute the environment with the gas emitted from combustion. Therefore, it is urgent to sort coal and gangue. With the rapid development of artificial intelligence technology, the coal preparation industry is moving towards the direction of simplified process, intelligent equipment, and refined sorting. Intelligent sorting of coal gangue has become an important task in the coal mining field, including the identification of coal and gangue, analysis of coal block particle size, and real-time identification of foreign matter.

[0003] Chinese invention patent, CN113570542A A method for detecting coal gangue under occlusion based on machine vision, using Mask RCNN to segment the visible part of coal or gangue that is not occluded from the collected image; using the Completion-GAN network to generate the occluded invisible part according to the visible part segmented from the image, and then obtain the complete image; using the complete image for detection to obtain the recognition result.

[0004] Chinese invention patent, CN114627109A A method and process for coal gangue classification and recognition based on image enhancement and deep learning, the method includes coal gangue image acquisition and image enhancement processing, making a data set for the coal gangue image after the coal gangue image enhancement processing, building and improving the LeNet network and training the obtained data set, and obtaining a coal gangue classification and recognition result model through neural network training.

[0005] The inventors found that the existing methods commonly used for gangue sorting mainly include image recognition, ray recognition, heavy medium coal separation, flotation, dry coal separation, etc., which have problems such as low sorting efficiency, low sorting accuracy and serious environmental pollution. Summary of the invention

[0006] In order to overcome the deficiencies of the prior art, the present invention provides a coal gangue identification method and system based on hyperspectral imaging and laser ranging, which improves the accuracy and efficiency of coal gangue sorting.

[0007] On the one hand, a method for identifying coal gangue based on hyperspectral imaging and laser ranging is provided, including: acquiring a hyperspectral image and laser ranging data of a target object; mapping the hyperspectral image data onto a three-dimensional point cloud of laser ranging data to obtain a mapping result; using a trained network model to extract multi-scale spatial features and multi-scale spectral features from the mapping result; fusing the extracted features, and then generating candidate target areas based on the fused features; classifying the candidate target areas to obtain classification results of coal gangue; constructing a three-dimensional model of the coal surface based on the laser ranging data; generating a triangular mesh model for the three-dimensional model of the coal surface; smoothing the triangular mesh model to obtain a reconstructed model; determining the particle size of the coal block based on the reconstructed model; identifying the petrographic characteristics of the coal based on the spectral data to distinguish different types of coal; and detecting the heavy metal content in the coal.

[0008] On the other hand, a gangue identification system based on hyperspectral imaging and laser ranging is provided, including: an acquisition module, which is configured to: acquire a hyperspectral image and laser ranging data of a target object; map the hyperspectral image data onto a three-dimensional point cloud of the laser ranging data to obtain a mapping result; a classification module, which is configured to: extract multi-scale spatial features and multi-scale spectral features from the mapping result using a trained network model; fuse the extracted features, and then generate candidate target areas based on the fused features; classify the candidate target areas to obtain a classification result of the gangue; a particle size analysis module, which is configured to: construct a three-dimensional model of the coal surface based on the laser ranging data; generate a triangular mesh model for the three-dimensional model of the coal surface; smooth the triangular mesh model to obtain a reconstructed model; determine the particle size of the coal block based on the reconstructed model; an identification module, which is configured to: identify the petrographic characteristics of the coal based on the spectral data to distinguish different types of coal; and detect the heavy metal content in the coal.

[0009] The above technical solution has the following advantages or beneficial effects: the present invention combines hyperspectral imaging technology and laser ranging technology. Laser ranging data has a natural complementarity with hyperspectral imaging. Laser detection technology can effectively make up for the shortcomings of hyperspectral remote sensing imaging technology in the spatial dimension, and can better combine the spatial information and spectral information of the study area.

[0010] The present invention can not only distinguish between coal and gangue based on mineral components, but also obtain the volume and surface characteristics of gangue. Based on the fused data, it makes up for the limitations of single data source application, realizes complementary advantages, and improves the accuracy and efficiency of gangue sorting.

[0011] The present invention identifies coal gangue to ensure low gangue content of raw coal and provides gangue material for filling goaf; it realizes real-time identification of harmful objects such as waste wood, plastic pipes, anchor nuts and other foreign matters mixed in ore on the belt to control product quality; it performs precise particle size analysis on coal blocks to provide a reference basis for the control of crushers and conveyor belts to improve product quality, ensure equipment safety and reduce production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0013] Figure 1 This is a flow chart of the method of embodiment 1. DETAILED DESCRIPTION

[0014] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0015] Embodiment 1: Figure 1 As shown, a gangue identification method based on hyperspectral imaging and laser ranging includes: S101: acquiring a hyperspectral image and laser ranging data of a target object; mapping the hyperspectral image data onto a three-dimensional point cloud of laser ranging data to obtain a mapping result; S102: extracting multi-scale spatial features and multi-scale spectral features from the mapping result using a trained network model; fusing the extracted features, and then generating candidate target areas based on the fused features; classifying the candidate target areas to obtain classification results of gangue; S103: constructing a three-dimensional model of the coal surface based on the classification results of the gangue and the laser ranging data; generating a triangular mesh model for the three-dimensional model of the coal surface; smoothing the triangular mesh model to obtain a reconstructed model; determining the particle size of the coal block based on the reconstructed model; S104: identifying the petrographic characteristics of the coal based on the spectral data to distinguish different types of coal; and detecting the heavy metal content in the coal.

[0016] Furthermore, the S101: acquiring a hyperspectral image and laser ranging data of a target object comprises: acquiring an image of the target object by a hyperspectral camera to obtain a hyperspectral image of the target object; performing laser ranging on the target object by a laser rangefinder to obtain laser ranging data of the target object; the target object refers to a mixture of mined coal, coal gangue and impurities; the impurities include: waste wood, plastic conduits or anchor nuts.

[0017] Furthermore, the S101: after acquiring the hyperspectral image and laser ranging data of the target object, before mapping the hyperspectral image data to the three-dimensional point cloud of the laser ranging data, also includes: using a principal component analysis algorithm to reduce the dimension of the hyperspectral image; and performing noise removal, downsampling and data registration processing on the laser ranging data.

[0018] Furthermore, the laser ranging data is subjected to noise removal, downsampling and data registration processing, specifically including: using a filtering algorithm to clean up the noise, using a voxel grid to downsample the point cloud, and using an iterative closest point algorithm ICP (Iterative Closest Point) to register the point cloud data of different scanning angles to form a three-dimensional point cloud model.

[0019] The voxel grid formula is: ;in, is the number of points in each voxel, are the coordinates of each point within the voxel; It is the centroid of the points in each voxel (i.e., the representative point of the voxel). The centroid of each voxel is used to replace all the points in the voxel, thereby reducing the density of the point cloud data and the amount of calculation.

[0020] Iterative Closest Point algorithm Iterative Closest Point is used to align point cloud data from different perspectives, find the best rotation matrix and translation vector, and achieve point cloud registration. The best point cloud registration is found by minimizing the distance between each point and its nearest point: ;in, and are the corresponding point pairs, R is the rotation matrix describing the rotation of the point cloud, is the translation vector describing the translation of the point cloud, is the minimum distance, For the points.

[0021] Furthermore, the S101: mapping the hyperspectral image data to the three-dimensional point cloud of the laser ranging data to obtain a mapping result includes: determining the same position points in the hyperspectral image data and the three-dimensional point cloud data, performing a geometric transformation on the hyperspectral data and the three-dimensional point cloud data so that they are aligned in the same coordinate system; and mapping the information of each point in the point cloud to the hyperspectral image.

[0022] Furthermore, the S102: using the trained network model to extract multi-scale spatial features and multi-scale spectral features from the mapping results, wherein the trained network model is implemented using a Swin Transformer network.

[0023] Furthermore, S102: using the trained network model, extracting multi-scale spatial features and multi-scale spectral features from the mapping results, wherein the training process of the trained network model includes: constructing a training set, the training set being a hyperspectral image with a known classification label; inputting the training set into a Swin Transformer network, training the Swin Transformer network, and stopping the training when the loss function value of the Swin Transformer network no longer decreases, or the number of iterations exceeds a set number, to obtain a trained Swin Transformer network.

[0024] Furthermore, the loss function of the Swin Transformer network is specifically expressed as: ;in, is the total loss function, is the number of samples, C is the number of categories, It is The samples belong to The label of the class, It is The samples belong to The predicted probability of the class, is the regularization weight coefficient, which is used to balance the classification error and model complexity. are model parameters.

[0025] Furthermore, the S102: using the trained network model to extract multi-scale spatial features and multi-scale spectral features from the mapping results, includes: using the trained network model to divide the hyperspectral image in the mapping results into a plurality of image blocks; for each image block, respectively extracting spatial features of different scales and spectral features of different scales.

[0026] Furthermore, the hyperspectral image in the mapping result is divided into a plurality of image blocks, including: converting the input image into a plurality of image blocks of set sizes, each image block containing spatial and spectral information, and each image block having a size of (P, P, C), where (P, P) is the spatial size of the image block, and C is the number of spectral channels; the divided image blocks form a data cube of (N, P, P, C), where N is the total number of patches.

[0027] Among them, the linear layer Convert each image patch into a feature representation : , is the spectral data of the image patch, is the matrix of linear transformation. After passing through the linear layer, the spectral and spatial information of each image block is integrated into a feature vector of fixed size.

[0028] Furthermore, for each image block, spatial features of different scales and spectral features of different scales are extracted respectively, including: adopting a hierarchical structure to gradually extract spectral and spatial features through multiple stages, each stage including multiple Swin Transformer Blocks; the window multi-head self-attention mechanism (windows multi-head self-attention, W-MSA) of each Swin Transformer Block is used to extract low-level features, and the shifted window multi-head self-attention mechanism (shifted windows multi-head self-attention, SW-MSA) is used to capture high-level features, and finally, multi-scale information is captured by gradually fusing low-level features and high-level features.

[0029] After the window self-attention, the moving window attention breaks the original independence between windows by sliding the window, introduces the interaction of moving windows, and further processes the global information; wherein, for each window self-attention mechanism W-MSA, it is expressed as: ; is the self-attention mechanism within the window, are the query vector, key vector and value vector of the image patch respectively, is the dimension of the key vector K. By offsetting the window by a certain number of pixels, the offset feature is obtained , the moving window attention mechanism performs the self-attention mechanism SW-MSA operation: .

[0030] It should be understood that the Swin Transformer network is used to simultaneously process the spatial and spectral feature information of the hyperspectral image: the hyperspectral image is divided into small patches, a data cube with three dimensions of width, height and simplified spectral dimension is created, and the spectral information of each small patch is converted into the feature representation required by the Transformer model through a linear layer. This step combines the spectral dimension and the spatial dimension through a linear transformation, preparing for the subsequent self-attention mechanism.

[0031] A hierarchical structure is adopted to gradually extract spectral and spatial features through multiple stages, the self-attention mechanism within the window is used to process local features, the moving window attention mechanism is used to capture global information, and finally multi-scale information is captured by gradually fusing low-level and high-level features.

[0032] Furthermore, the S102: fusing the extracted features includes: the features after serial splicing and fusion are: ; and are low-level and high-level features, Represents a splicing operation, It is the feature after concatenation in the channel dimension.

[0033] After multiple levels of feature extraction, the spatial details captured at the low level and the global spectral information at the high level are integrated together to enhance the model's ability to express complex spectral features. Useful spectral and spatial features are extracted from the output features, and different features are fused to enhance the model's ability to express complex spectral features. The concatenated features are fused to integrate information, and the attention mechanism is used to dynamically adjust the weights of different features to further enhance key information.

[0034] After splicing Perform global average pooling to obtain the global information vector of each channel: ; is the global information vector, represents the feature map height, represents the feature map width, represents the total channels of the feature map, represents the row coordinates of the feature map, represents the column coordinates of the feature map, Indicates channels (1≤ ≤ ).

[0035] The channel description vector after aggregation Through the fully connected layer and nonlinear activation function, the attention weight of each channel is generated: ; is the attention weight corresponding to each channel, and are the weight matrices of the two fully connected layers, is the scaling factor, usually set to 4, is the Sigmoid activation function, output weight between 0 and 1, represents the activation function, is the global information vector mentioned above.

[0036] The generated channel weights Reassign to the concatenated feature map In , each channel is weighted: ; It is the attention-weighted feature (fused feature), which has the same size as the original feature, but each channel is assigned a different weight.

[0037] Furthermore, the S102: generating candidate target regions based on the fused features, includes: generating region proposals based on a region proposal network (RPN): predicting the bounding box of each candidate region through regression analysis, and performing bounding box regression and region proposal on the feature map.

[0038] Specifically, a sliding window mechanism is used to generate anchor boxes, where the center position of each sliding window is the anchor point, the region generation network RPN generates multiple anchor boxes, and a convolutional layer is used to perform bounding box regression on each anchor box, where the bounding box regression predicts four coordinate values: , Indicates the offset of the horizontal coordinate of the center point of the anchor frame. Indicates the offset of the vertical coordinate of the center point of the anchor box. represents the change in width of the anchor box, represents the change in the height of the anchor box, Represents the horizontal coordinate of the center point of the anchor box, Represents the ordinate of the center point of the anchor box, represents the width of the anchor box, Indicates the height of the anchor box.

[0039] Furthermore, the S102: classifying the candidate target area to obtain the classification result of coal gangue includes: inputting the candidate target area into the classification network, and the classification network uses the convolution layer and the pooling layer to extract the feature vector of the candidate target area as ,The classification network outputs the final gangue classification and foreign matter identification results. ;in, and are the weights and biases of the classification layer; represents the probability of each category, represents the feature vector after convolution pooling processing, The function converts the output into a probability distribution.

[0040] The final output results include the category label of each area and the corresponding confidence level and bounding box position, realizing the classification of coal gangue and foreign body detection.

[0041] Predict the probability of each anchor box containing the target, and classify the target category and background category through the classifier. Based on the bounding box regression and region classification results, the region proposal is screened, and non-maximum suppression (NMS) is used to remove redundant boxes with high overlap, retain the best region proposal, and apply the regression prediction bias to adjust the anchor box to obtain the most accurate region proposal as the target region.

[0042] Further, the S103: constructing a three-dimensional model of the coal surface based on the classification results of the coal gangue and the laser ranging data, including: forming point cloud data in a three-dimensional coordinate system based on the distance (depth information) and corresponding angle information of each point measured in the laser ranging data, each point cloud data including coordinate: ; ; ;in, is the measured distance, Vertical angle, is the horizontal angle.

[0043] Furthermore, the S103: generating a triangular mesh model for the three-dimensional model of the coal surface includes: using a Delaunay triangulation algorithm to connect adjacent points to form a triangular mesh model, and using a surface interpolation method to fill in the unscanned blank areas to generate a continuous surface model.

[0044] Specifically, for a point set in a point cloud , Delaunay triangulation searches for a triangular mesh such that for any triangle, its inscribed circle does not contain any other points.

[0045] Furthermore, the step S103: smoothing the triangular mesh model to obtain a reconstructed model includes: using a Laplacian Smoothing algorithm to smooth the triangular mesh model to obtain a reconstructed model, and removing sharp corners of irregular surfaces to make the model smoother.

[0046] The Laplacian Smoothing algorithm is implemented by adjusting the position of each point to be close to the average position of its neighboring points: ;in, is the current point position, Yes The set of neighboring points of is the smoothing factor.

[0047] Further, the S103: determining the particle size of the coal block based on the reconstructed model includes: S103-1: calculating the surface area of ​​the triangular mesh, wherein the surface area calculation is to analyze the particle size of the coal block by calculating the total surface area of ​​the triangular mesh, and calculating the surface area of ​​the triangular mesh for: ;in, and are the vectors of the two sides of the triangle.

[0048] S103-2: Combine the triangle with the normal vector of the surface, calculate the volume using the integral method, and correct it by the convex and concave features of the surface area; specifically, calculate the normal vector of each point : ; Calculate the unadjusted volume : ;in, Represents the centroid of the triangle; calculates the curvature of the surface, and the curvature is calculated as: ;in, represents the square of the length of the tangent vector on the surface, Represents the normal vector in Changes in direction, express and The angle between the tangent vectors, Represents the normal vector in and Interaction between directions, Indicated in The square of the length of the tangent vector in the direction, Represents the normal vector in Changes in direction, Represents curvature.

[0049] Then use the curvature , distinguish between convex and concave areas. If the curvature value is positive, it means that the current area is convex outward; if the curvature value is negative, it means that the current area is concave inward; if the curvature value is zero, it means that the current area is flat.

[0050] According to the curvature of each area, combined with the surface area data of the triangular mesh, the volume of each area is calculated; for concave surfaces, the concave volume is calculated : ; where S is the area of ​​the triangle mesh, is the depression depth.

[0051] For convex surfaces, calculate the bulge volume : ; where S is the area of ​​the triangle mesh, is the protrusion height.

[0052] Finally, for the concave and convex areas, the final coal volume is obtained: .

[0053] S103-3: Analyze the particle size of the coal block using specific surface area calculation according to the reconstructed model; ;in, represents the specific surface area of ​​the coal block, represents the surface area of ​​the coal block, Indicates the volume of coal; when When it is greater than the set threshold, the current coal block is identified as a large-size coal block; When it is less than the set threshold, the current coal block is identified as a small-size coal block (the threshold can be set according to the actual needs of different mining applications).

[0054] Furthermore, the S103 further includes: S103-4: classifying the surface features by clustering algorithm to achieve quantitative analysis of particle size distribution; specifically, using clustering analysis to calculate the surface area ,volume , specific surface area and curvature parameters , normalize the data to ensure that the data is on a similar scale; select hierarchical clustering for analysis, first select Euclidean distance to measure the distance, and form a distance matrix D. For two coal block samples and The eigenvectors are and , the Euclidean distance is:

[0055] .

[0056] Find the smallest value in the distance matrix and determine the two clusters to be merged; connect them by minimizing the total variance after cluster merging to merge the two clusters into a new cluster; select two clusters in each merging process and To merge, the total variance of the merged cluster is: ; is the variance of the merged clusters, is the centroid of cluster C, The feature vector representing each sample in the cluster. Based on the distance matrix and merge information, a dendrogram is constructed.

[0057] Furthermore, the S103-4: classifying the surface features by clustering algorithm to achieve quantitative analysis of particle size distribution specifically includes: (1) first using Euclidean distance to calculate the distance between samples, and treating each sample as an independent cluster, and creating an initial cluster list; (2) finding the minimum value in the distance matrix to determine the two clusters to be merged; (3) merging the two clusters into a new cluster according to the rule of minimizing cluster merging; (4) calculating the distance between the new cluster and the remaining clusters based on the two merged clusters, and updating the distance matrix; (5) recording each merged cluster and its distance at the time of merging to form a merging history; (6) continuing to merge and update the distance matrix until all samples are merged to reach a preset number of clusters (determined by actual working conditions and requirements); (7) drawing the merging history as a dendrogram, selecting a horizontal cutting line at the position where the merging height is higher than the set threshold on the dendrogram, and obtaining the preset number of clusters to analyze the effectiveness of the clustering.

[0058] Furthermore, the S104: based on the spectral data, identifying the petrographic characteristics of the coal to distinguish different types of coal includes: S104-11: calculating the reflectivity of each coal sample in different bands: ;in, is the reflected light intensity, is the intensity of incident light; if the reflectivity of the coal sample is lower than the first set threshold, it indicates that the type of the current coal sample is anthracite; if the reflectivity of the coal sample is higher than the second set threshold, it indicates that the type of the current coal sample is lignite; if the reflectivity of the coal sample is higher than the first set threshold but lower than the second set threshold, it indicates that the type of the current coal sample is bituminous coal.

[0059] S104-12: Calculate the absorption rate , and find the minimum point in the absorbance data. The minimum point is the absorption valley. Locate the absorption valley. For wavelength If the absorption valley and ,but To absorb the valley, and are adjacent wavelengths.

[0060] Calculate the depth D, width W and absorption symmetry of the absorption valley : ; is the maximum absorption rate around the absorption valley; is the absorption rate at the valley position. ; and They are the wavelengths on both sides of the absorption valley to reach a certain absorption rate threshold; ; and They are the width of the left and right sides of the absorption valley respectively; if the absorption valley is located around 1400nm and 1900nm, it means that the current coal type is lignite.

[0061] S104-13: Calculate the spectral slope: ; and There are two wavelengths and at the reflectivity; if the reflectivity slope is higher than the first set threshold, it indicates that the type of the current coal sample is lignite; if the reflectivity slope is lower than the second set threshold, it indicates that the type of the current coal sample is anthracite; if the reflectivity slope of the coal sample is lower than the first set threshold but higher than the second set threshold, it indicates that the type of the current coal sample is bituminous coal.

[0062] S104-14: Normalize the reflectivity, absorption valley depth, absorption valley width, absorption valley symmetry and spectral slope features and convert them to the same scale; perform weighted fusion on the normalized features, test the impact of different weight allocations on classification performance based on cross-validation, and optimize the weight allocation, assign weights to each feature, and obtain the fused features .

[0063] S104-15: Based on the fused feature vector and the coefficient vector for each category Compute linear combinations , input the result of the linear combination into the Softmax function and convert it into a probability value: ; Yes Category The linear score of Yes Category The probability of It is the exponential sum of all category scores, which is used for normalization and ultimately determines the final coal category based on the category with the highest probability.

[0064] Furthermore, the S104: detecting the heavy metal content in coal includes: using a hyperspectral spectrum to detect the heavy metal content in coal, and the specific steps are as follows: S104-21: receiving spectral data of different coal samples and performing smoothing filtering, and using an X-ray fluorescence spectrometer to measure the metal content in the coal samples as training labels; the metal content refers to the content of heavy metals such as lead, cadmium, and arsenic; S104-22: performing principal component analysis on the spectral data, selecting the front principal components; S104-23: Use the training data to train a random forest regression model, , is the final prediction of the model, is the number of decision trees, It is Trees for input The prediction result; the training data includes: principal components and corresponding metal content labels.

[0065] S104-24: In each decision tree, the core of tree construction is to minimize the mean square error (MSE) by splitting nodes. For a given node, the sample set , select the split feature and split point When , use minimizing MSE as the splitting criterion: , It is a sample Tags, It is a sample set The mean label value of .

[0066] S104-25: Use mean square error to evaluate the performance of the random forest regression model, and further optimize the prediction performance of the random forest regression model by adjusting parameters; the parameters include: the number of trees, maximum depth, minimum number of split samples, and learning rate.

[0067] S104-26: Apply the trained random forest regression model to the hyperspectral data of the new coal sample, and predict the metal content in the coal sample by inputting the hyperspectral data of the new coal sample.

[0068] Embodiment 2: This embodiment provides a gangue identification system based on hyperspectral imaging and laser ranging, including: an acquisition module, which is configured to: acquire a hyperspectral image and laser ranging data of a target object; map the hyperspectral image data onto a three-dimensional point cloud of laser ranging data to obtain a mapping result; a classification module, which is configured to: extract multi-scale spatial features and multi-scale spectral features from the mapping result using a trained network model; fuse the extracted features, and then generate candidate target areas based on the fused features; classify the candidate target areas to obtain a classification result of the gangue; a particle size analysis module, which is configured to: construct a three-dimensional model of the coal surface based on laser ranging data; generate a triangular mesh model for the three-dimensional model of the coal surface; smooth the triangular mesh model to obtain a reconstructed model; determine the particle size of the coal block based on the reconstructed model; an identification module, which is configured to: identify the lithologic features of the coal based on the spectral data to distinguish different types of coal; and detect the heavy metal content in the coal. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A coal gangue identification method based on hyperspectral imaging and laser ranging is characterized by: include: Acquire hyperspectral images and laser ranging data of target objects; Map the hyperspectral image data onto the three-dimensional point cloud of the laser ranging data to obtain the mapping result; The trained network model is used to extract multi-scale spatial features and multi-scale spectral features from the mapping results; the extracted features are fused, and then candidate target areas are generated based on the fused features; the candidate target areas are classified to obtain the classification results of coal gangue; Based on the classification results of coal gangue and the laser ranging data, a three-dimensional model of the coal surface is constructed; and a triangular mesh model is generated for the three-dimensional model of the coal surface; The triangular mesh model is smoothed to obtain a reconstructed model; based on the reconstructed model, the particle size of the coal block is determined; Among them, based on the classification results of coal gangue and laser ranging data, a three-dimensional model of the coal surface is constructed, including: based on the distance and corresponding angle information of each point measured in the laser ranging data, point cloud data in a three-dimensional coordinate system is formed, and each point cloud data contains coordinate: ; ; ;in, is the measured distance, Vertical angle, is the horizontal angle; The three-dimensional model of the coal surface is used to generate a triangular mesh model, including: using a Delaunay triangulation algorithm to connect adjacent points to form a triangular mesh model, and using a surface interpolation method to fill in the blank areas that have not been scanned to generate a continuous surface model; The triangular mesh model is smoothed to obtain a reconstructed model, including: using a Laplacian Smoothing algorithm to smooth the triangular mesh model to obtain a reconstructed model; The particle size of the coal block is determined based on the reconstruction model, including: S103-1: Calculate the surface area of ​​the triangular mesh, where the surface area calculation is to analyze the particle size of the coal block by calculating the total surface area of ​​the triangular mesh. for: ;in, and are the vectors of the two sides of the triangle; S103-2: Combine the triangle with the normal vector of the surface, calculate the volume using the integral method, and correct it by the convex and concave features of the surface area; specifically, calculate the normal vector of each point : ; Calculate the unadjusted volume : ;in, Represents the centroid of the triangle; calculates the curvature of the surface, and the curvature is calculated as: ;in, represents the square of the length of the tangent vector on the surface, Represents the normal vector in Changes in direction, express and The angle between the tangent vectors, Represents the normal vector in and Interaction between directions, Indicated in The square of the length of the tangent vector in the direction, Represents the normal vector in Changes in direction, represents curvature; Then use the curvature , distinguish between convex and concave areas. If the curvature value is positive, it means that the current area is convex outward; if the curvature value is negative, it means that the current area is concave inward; if the curvature value is zero, it means that the current area is flat; According to the curvature of each area, combined with the surface area data of the triangular mesh, the volume of each area is calculated; for concave surfaces, the concave volume is calculated : ; where S is the area of ​​the triangle mesh, is the depth of the depression; For convex surfaces, calculate the bulge volume : ; where S is the area of ​​the triangle mesh, is the protrusion height; Finally, for the concave and convex areas, the final coal volume is obtained: ; S103-3: Analyze the particle size of the coal block using specific surface area calculation according to the reconstructed model; ;in, represents the specific surface area of ​​the coal block, represents the surface area of ​​the coal block, Indicates the volume of coal; when When it is greater than the set threshold, the current coal block is identified as a large-size coal block; When it is less than the set threshold, the current coal block is identified as a small-size coal block; S103-4: Classify the surface features by clustering algorithm to achieve quantitative analysis of particle size distribution; specifically, use cluster analysis to calculate the surface area ,volume , specific surface area and curvature parameters , normalize the data to ensure that the data is on a similar scale; select hierarchical clustering for analysis, first select Euclidean distance to measure the distance, and form a distance matrix D. For two coal block samples and The eigenvectors are and , the Euclidean distance is: ； Find the smallest value in the distance matrix and determine the two clusters to be merged; connect them by minimizing the total variance after cluster merging to merge the two clusters into a new cluster; select two clusters in each merging process and To merge, the total variance of the merged cluster is: ; is the variance of the merged clusters, is the centroid of cluster C, The feature vector representing each sample in the cluster; construct a dendrogram based on the distance matrix and merge information; The S103-4: classifying the surface features by clustering algorithm to achieve quantitative analysis of particle size distribution, specifically includes: (1) first using Euclidean distance to calculate the distance between samples, and treating each sample as an independent cluster, and creating an initial cluster list; (2) finding the minimum value in the distance matrix to determine the two clusters to be merged; (3) merging the two clusters into a new cluster according to the rule of minimizing cluster merging; (4) calculating the distance between the new cluster and the remaining clusters based on the two merged clusters, and updating the distance matrix; (5) recording each merged cluster and its distance at the time of merging to form a merging history; (6) continuing to merge and update the distance matrix until all samples are merged to reach a preset number of clusters; (7) drawing the merging history as a dendrogram, selecting a horizontal cutting line at the position where the merging height is higher than the set threshold on the dendrogram, and obtaining the preset number of clusters to analyze the effectiveness of the clustering; Based on spectral data, the petrographic characteristics of coal are identified to distinguish different types of coal; the heavy metal content in coal is detected.

2. The method for identifying coal gangue based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: After acquiring the hyperspectral image and laser ranging data of the target object, before mapping the hyperspectral image data to the three-dimensional point cloud of the laser ranging data, it also includes: using a principal component analysis algorithm to reduce the dimension of the hyperspectral image; and performing noise removal, downsampling and data registration processing on the laser ranging data.

3. The method for identifying coal gangue based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: Map the hyperspectral image data onto the 3D point cloud of the laser ranging data to obtain the mapping results, including: Determine the same position points in the hyperspectral image data and the three-dimensional point cloud data, perform geometric transformation on the hyperspectral data and the three-dimensional point cloud data, and align them in the same coordinate system; map the information of each point in the point cloud to the hyperspectral image.

4. The method for identifying coal gangue based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: The trained network model is used to extract multi-scale spatial features and multi-scale spectral features from the mapping results, wherein the trained network model is implemented using the Swin Transformer network; The loss function of the Swin Transformer network is specifically expressed as: ； in, is the total loss function, is the number of samples, C is the number of categories, It is The samples belong to The label of the class, It is The samples belong to The predicted probability of the class, is the regularization weight coefficient, which is used to balance the classification error and model complexity. are model parameters; The trained network model is used to extract multi-scale spatial features and multi-scale spectral features from the mapping results, including: the trained network model is used to divide the hyperspectral image in the mapping results into a number of image blocks; for each image block, spatial features of different scales and spectral features of different scales are extracted respectively; For each image block, spatial features of different scales and spectral features of different scales are extracted respectively, including: adopting a hierarchical structure to gradually extract spectral and spatial features through multiple stages, each stage including multiple SwinTransformer Blocks; the window self-attention mechanism W-MSA of each Swin Transformer Block is used to extract low-level features, and the moving window attention mechanism SW-MSA is used to capture high-level features, and finally, multi-scale information is captured by gradually fusing low-level features and high-level features.

5. The method for identifying coal gangue based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: The extracted features are fused, including: The features after serial splicing and fusion are: ； and are low-level and high-level features, Represents a splicing operation, It is the feature after splicing in the channel dimension; After splicing Perform global average pooling to obtain the global information vector of each channel: ； is the global information vector, represents the feature map height, represents the feature map width, represents the total channels of the feature map, represents the row coordinates of the feature map, represents the column coordinates of the feature map, Indicates channels; The channel description vector after aggregation Through the fully connected layer and nonlinear activation function, the attention weight of each channel is generated: ； is the attention weight corresponding to each channel, and are the weight matrices of the two fully connected layers, is the scaling factor, is the Sigmoid activation function, output weight between 0 and 1, represents the activation function, is the global information vector; The generated channel weights Reassign to the concatenated feature map In , each channel is weighted: ； It is the feature after attention weighting.

6. The method for identifying coal gangue based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: Classify the candidate target areas and obtain the classification results of coal gangue, including: The candidate target area is input into the classification network, and the classification network uses convolutional layer and pooling layer to extract the feature vector of the candidate target area. , the classification network outputs the final gangue classification and foreign body identification results: ;in, and are the weights and biases of the classification layer; represents the probability of each category, represents the feature vector after convolution pooling processing, The function converts the output into a probability distribution; The final output results include the category label of each area and the corresponding confidence level and bounding box position, realizing the classification of coal gangue and foreign body detection.

7. The method for identifying coal gangue based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: Based on spectral data, the petrographic characteristics of coal are identified to distinguish different types of coal; Detection of heavy metal content in coal, including: Calculate the reflectance of each coal sample in different bands; calculate the absorbance; calculate the spectral slope; Normalize the reflectivity, absorption valley depth, absorption valley width, absorption valley symmetry and spectral slope features and convert them to the same scale; perform weighted fusion on the normalized features, and test the impact of different weight allocations on classification performance based on cross-validation, thereby optimizing the weight allocation, assigning weights to each feature, and obtaining the fused features. ; According to the fused feature vector and the coefficient vector for each category Compute linear combinations , the result of the linear combination is input into the Softmax function and converted into a probability value; finally, the final coal type is determined based on the category with the highest probability.

8. The method for identifying coal gangue based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: Detecting the heavy metal content in coal includes: using hyperspectral to detect the heavy metal content in coal. The specific steps are as follows: Receive spectral data of different coal samples and perform smoothing and filtering, and use an X-ray fluorescence spectrometer to measure the metal content in the coal samples as training labels; the metal content refers to the content of heavy metals such as lead, cadmium, and arsenic; Perform principal component analysis on the spectral data and select principal components; Train a random forest regression model using the training data, , is the final prediction of the model, is the number of decision trees, It is Trees for input The prediction result; the training data includes: principal components and corresponding metal content labels; In each decision tree, the core of tree construction is to minimize the mean square error by splitting nodes. For a given node, the sample set , select the split feature and split point When , use minimizing MSE as the splitting criterion: , It is a sample Tags, It is a sample set The label mean of ; The performance of the random forest regression model is evaluated using mean square error, and the prediction performance of the random forest regression model is further optimized by adjusting parameters; the parameters include: the number of trees, the maximum depth, the minimum number of split samples, and the learning rate; The trained random forest regression model is applied to the hyperspectral data of new coal samples, and the metal content in the coal samples is predicted by inputting the hyperspectral data of the new coal samples.

9. A coal gangue identification system based on hyperspectral imaging and laser ranging using the coal gangue identification method based on hyperspectral imaging and laser ranging as claimed in claim 1, characterized in that: include: An acquisition module is configured to: acquire a hyperspectral image and laser ranging data of a target object; Map the hyperspectral image data onto the three-dimensional point cloud of the laser ranging data to obtain the mapping result; The classification module is configured to: extract multi-scale spatial features and multi-scale spectral features from the mapping results using the trained network model; fuse the extracted features, and then generate candidate target areas based on the fused features; classify the candidate target areas to obtain the classification results of coal gangue; The particle size analysis module is configured to: construct a three-dimensional model of the coal surface based on the classification results of the coal gangue and the laser ranging data; and generate a triangular mesh model for the three-dimensional model of the coal surface; The triangular mesh model is smoothed to obtain a reconstructed model; based on the reconstructed model, the particle size of the coal block is determined; The identification module is configured to: identify the petrographic characteristics of coal based on spectral data to distinguish different types of coal; and detect the heavy metal content in the coal.

Images