Coal rock carbon content detection method and system based on multi-feature image spectroscopy technology

Abstract

The present invention belongs to the technical field of coal rock carbon content detection, and specifically discloses a coal rock carbon content detection method and system based on multi-feature image spectroscopy technology, the method comprising: constructing a database of typical coal rock types and their corresponding characteristic spectra and texture features; textually annotating the image category of each pixel point in a mixed sample spectral image; using the data in the database as source domain data, and using the hyperspectral data of the coal rock sample to be tested as target domain data, and realizing coal rock classification of the coal rock sample to be tested in the target domain through transfer learning; for the identified coal sample, identifying whether each pixel point on the spectral image of the coal sample contains carbon content; for the pixel point with carbon content, using a trained carbon content prediction model to obtain the carbon content of each pixel point on the spectral image of the coal sample, thereby ensuring rapid and accurate detection of the carbon content of the coal rock.

Description

Technical Field

[0001] The present invention relates to the technical field of coal rock carbon content detection, and in particular to a coal rock carbon content detection method and system based on multi-feature image spectroscopy technology. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Qualitative identification of coal and quantitative analysis of coal quality play an important role in coal processing, selection and washing, and coal quality assessment. Among them, how to efficiently and accurately identify coal rocks and detect the carbon content of coal rocks is a key link in achieving the above qualitative identification and quantitative analysis.

[0004] The current traditional identification and analysis methods have the disadvantages of high manual participation, low detection accuracy and low intelligence level; the static image recognition method of coal and rock in the visible light range only roughly identifies the surface of coal and rock based on characteristics such as color, gloss, and texture, and has low reliability; the single infrared spectroscopy technology will even have the phenomenon of different substances having the same spectrum in the same spectral band, thereby interfering with coal and rock identification and accurate detection of coal and rock carbon content; and the traditional hyperspectral image classification technology ignores the relationship between adjacent spatial contexts, making it difficult to effectively identify coal and rock and detect carbon content. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a method and system for detecting the carbon content of coal rocks based on multi-feature image spectroscopy technology. By constructing a database of typical coal rock image texture features and characteristic spectral information, based on the influence of carbon content on typical coal rock spectral characteristics, the extraction strategy of spatial features and spectral features is improved, and computer vision image features, visible-near infrared spectral information and text label features are integrated to ensure the rapid and accurate detection of the carbon content of coal rocks.

[0006] In some embodiments, the following technical solutions are adopted:

[0007] A method for detecting carbon content in coal rock based on multi-feature image spectroscopy technology, comprising:

[0008] Collect the original spectral information of typical coal and rock samples and extract characteristic spectra; use multi-scale grayscale co-occurrence matrix to extract texture features of the grayscale image corresponding to the extracted characteristic spectra; build a database of typical coal and rock types and their corresponding characteristic spectra and texture features;

[0009] Mix coal and rock samples in different proportions, annotate the image category of each pixel in the spectral image of the mixed sample with text, and add it to the database;

[0010] The data in the database are used as source domain data, and the hyperspectral data of the coal and rock samples to be tested are used as target domain data. The fusion features of the source domain data and the target domain data are extracted using a feature fusion network, and the coal and rock classification of the coal and rock samples to be tested in the target domain is achieved through transfer learning.

[0011] For the identified coal samples, a three-dimensional convolutional neural network is used to extract the spatial-spectral joint features of the original spectral image, and based on the spatial-spectral joint features, whether each pixel point on the spectral image of the coal sample contains carbon content is identified; for pixels with carbon content, the trained carbon content prediction model is used to obtain the carbon content of each pixel point on the spectral image of the coal sample.

[0012] As an optional solution, before collecting the original spectral information of typical coal and rock samples, spectral data is collected on the original block samples, and the spectral data is corrected for black and white by calculating the relative reflectance.

[0013] As an optional solution, extract the characteristic spectrum of the original spectrum information, specifically:

[0014] Represent the original spectral information in matrix form , each row of the matrix represents a pixel, each column represents a band, and there are m rows and n columns in total;

[0015] Assign different initial weights to all bands according to the importance characteristics of the data;

[0016] Calculate each band The variance of the reflectivity of all pixels in the image is calculated, and the band with the largest variance is selected as the starting band; a band subspace S containing the starting band is constructed;

[0017] Calculate each of the remaining bands based on the weights Projection value to band subspace S , put the band with the largest projection value into the band subspace S and use it as the starting band for the next projection;

[0018] Calculate the projection error of each band and update the weight of each band according to the projection error;

[0019] Based on the updated weights, continue to calculate the projection value of each band outside the band subspace S to the band subspace S, put the band with the largest projection value into the band subspace S, and use it as the starting band for the next projection; repeatedly calculate the projection error of each band and update the weights;

[0020] Repeat the above process until k set number of characteristic bands F={ }, as the characteristic spectrum of the original spectral information.

[0021] As an optional solution, a multi-scale gray-level co-occurrence matrix is ​​used to extract texture features from the gray-scale image corresponding to the extracted characteristic spectrum. The specific process is as follows:

[0022] Represent different scales with different pixel distances and directions;

[0023] Under a certain set scale, the corresponding gray level co-occurrence matrix is ​​calculated to obtain the frequency of occurrence of pixel pairs with gray level values ​​i and j in the gray level image; the gray level co-occurrence matrix is ​​normalized, and multiple texture features such as contrast, energy, entropy, homogeneity and correlation are extracted from the normalized gray level co-occurrence matrix, and multiple texture features are combined into a complete texture feature vector;

[0024] For each scale, the texture feature vector is extracted respectively, so as to obtain the texture feature vector of the grayscale image at multiple scales.

[0025] As an optional solution, the influence of carbon content on the spectral characteristics of coal rock is specifically as follows:

[0026] The two types of spectral features, spectral slope and absorption valley depth, were parameterized, and regression fitting analysis was performed on the relationship between carbon content and the two types of spectral features, spectral slope and absorption valley depth. When the carbon content is greater than the set threshold, the two types of spectral features, spectral slope of each pixel point in the coal and rock spectral images and absorption valley depth of the characteristic band, will show the same change trend; at this time, coal and rock cannot be distinguished by spectral features alone. Therefore, the image category of each pixel point in the mixed sample spectral image is annotated with text to increase the amount of image information in the database.

[0027] As an optional solution, the feature fusion network includes a spectral encoder, an image encoder and a text encoder. The spectral encoder extracts the deep spectral features of the input spectral data, the image encoder obtains the enhanced spatial features of the input image texture features, and the text encoder converts the input annotation data into text features; the deep spectral features, enhanced spatial features and text features are positionally encoded respectively to achieve feature alignment and unify them into the same representation space, and then the three types of features are input into a Transformer-based feature fusion module to output the fused features of the source domain data or the target domain data.

[0028] As an optional solution, the coal and rock classification of the coal and rock samples to be tested in the target domain can be realized through transfer learning, specifically:

[0029] The source domain data contains labeled feature samples, and the target domain data contains unlabeled feature samples. Use all labeled samples to train an initial nearest neighbor classifier. For each labeled sample, calculate the distance metric between its features, find the nearest m neighbors and classify them. Then use the initial nearest neighbor classifier to predict the unlabeled samples, record the prediction results and confidence, and add the unlabeled samples with the highest prediction confidence as pseudo-label samples to the labeled sample set.

[0030] Update the labeled sample set to include the initial labeled samples and the unlabeled samples that have just been pseudo-labeled, and retrain the nearest neighbor classifier using the expanded sample set;

[0031] Repeat the above steps to continue classifying the remaining unlabeled samples and expand the labeled sample set in each iteration until all confidence levels no longer increase. At this point, the iteration is considered complete and the unlabeled samples have been classified.

[0032] As an optional solution, the three-dimensional convolutional neural network specifically includes:

[0033] A 3D convolution layer, three 3D convolution layers of the front Bayesian network, an average pooling layer and a fully connected layer are connected in sequence; wherein the activation function of the 3D convolution layer of the front Bayesian network introduces a parameter a in the negative region, which can dynamically adjust the slope of the negative region;

[0034] The three-dimensional spectral data is input into the three-dimensional convolutional neural network, and the spatial-spectral joint features are extracted through multiple layers of 3D convolutional layers. The spatial-spectral joint features are sampled through the average pooling layer to reduce the size of the feature map; the spatial-spectral joint features output by the average pooling layer are input into the fully connected layer for comprehensive processing of high-dimensional features to obtain the category probability distribution of whether each pixel point has carbon content, and finally the classification result of whether each pixel point on the spectral image of the coal sample contains carbon content is output through the output layer.

[0035] In some embodiments, the following technical solutions are adopted:

[0036] A coal rock carbon content detection system based on multi-feature image spectroscopy technology, comprising:

[0037] The feature extraction module is used to collect the original spectral information of typical coal and rock samples and extract the characteristic spectrum; use the multi-scale grayscale co-occurrence matrix to extract the texture features of the grayscale image corresponding to the extracted characteristic spectrum; and build a database of typical coal and rock types and their corresponding characteristic spectra and texture features;

[0038] A text annotation module is used to mix coal and rock samples in different proportions, annotate the image category of each pixel in the spectral image of the mixed sample, and add it to the database;

[0039] A coal-rock classification module is used to use the data in the database as source domain data and the hyperspectral data of the coal-rock samples to be tested as target domain data, respectively extract fusion features of the source domain data and the target domain data using a feature fusion network, and achieve coal-rock classification of the coal-rock samples to be tested in the target domain through transfer learning;

[0040] The carbon content prediction module is used to extract the spatial-spectral joint features of the original spectral image for the identified coal samples using a three-dimensional convolutional neural network, and to identify whether each pixel point on the spectral image of the coal sample contains carbon content based on the spatial-spectral joint features; for pixels with carbon content, the trained carbon content prediction model is used to obtain the carbon content of each pixel point on the spectral image of the coal sample.

[0041] In some embodiments, the following technical solutions are adopted:

[0042] A terminal device includes a processor and a memory, wherein the processor is used to implement instructions; the memory is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the above-mentioned coal rock carbon content detection method based on multi-feature image spectroscopy technology.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) Based on mixed sampling tests of coal and rock with different mixing ratios and different types, the present invention analyzes the influence of carbon content changes on the spectral curves of coal and rock, summarizes the influence of carbon content on the spectral characteristics of typical coal and rock, and integrates it into the hyperspectral image classification task as prior knowledge, providing new knowledge basis for the distinction between coal and rock, and improving the reliability of coal and rock classification.

[0045] (2) The present invention integrates characteristic spectra, texture features and text features, and realizes coal and rock classification of coal and rock samples to be tested in the target domain through transfer learning, which can improve the interpretability and cross-domain adaptability of traditional coal and rock classification models. The introduction of text features can better utilize the spectrum and image data in hyperspectral image data, make up for the shortcomings of single modality data, and thus enhance the adaptability of coal and rock classification models to small sample data.

[0046] (3) The present invention extracts characteristic spectra by updating the weights of the bands and extracts texture features using a multi-scale grayscale co-occurrence matrix. It can select more representative characteristic spectral bands and extract texture feature information in the image more comprehensively, thereby enriching the data quality in the constructed typical coal rock spectrum-image database.

[0047] (4) The present invention uses a three-dimensional convolutional neural network to extract the spatial and spectral joint features of hyperspectral images, and then captures the local spatial dependency between pixels. It can better adapt to the complex relationship of spectral-spatial feature interaction in hyperspectral data, and pre-places a Bayesian network and improves the activation function in the convolution layer to improve the convergence speed, generalization ability and expression ability of the pixel classification model.

[0048] (5) The carbon content prediction model proposed in the present invention predicts the carbon content of each pixel point by measuring the reflectivity at a characteristic wavelength that has a greater correlation with the carbon content characteristics in the spectral curve of the carbon content pixel point. The model can be quickly deployed based on a third-party library to achieve rapid and accurate detection of the carbon content of coal rocks.

[0049] Other features and advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of a method for detecting carbon content in coal rock based on multi-feature image spectroscopy technology in an embodiment of the present invention;

[0051] Figure 2 Schematic diagram of a multi-feature fusion process in an embodiment of the present invention;

[0052] Figure 3 Schematic diagram of the three-dimensional convolutional neural network structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0054] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0055] Embodiment 1

[0056] In one or more embodiments, a method for detecting carbon content in coal based on multi-feature image spectroscopy technology is disclosed, combined with Figure 1 , specifically including the following process:

[0057] S101: Collect original spectral information of typical coal and rock samples and extract characteristic spectra; use a multi-scale grayscale co-occurrence matrix to extract texture features of the grayscale images corresponding to the extracted characteristic spectra; construct a database of typical coal and rock types and their corresponding characteristic spectra and texture features (hereinafter referred to as a typical coal and rock spectrum-image database).

[0058] In this embodiment, anthracite, bituminous coal, and lignite are selected as typical coal types, and mudstone, shale, sandstone and other coal-bearing rocks are selected as typical rock types.

[0059] The selected typical coal and rock samples are crushed and pressed, and then made into cake-shaped samples and placed on the hyperspectral acquisition platform to collect the original spectra of the typical coal and rock samples.

[0060] As an optional solution, in order to remove the influence of changes in the intensity of the light source in the acquisition environment and the noise of the sensor itself, original block samples of typical coal and rock are used as calibration samples on the hyperspectral acquisition platform for black and white calibration.

[0061] Specifically, block the halogen light source, record the dark current noise of the sensor, and obtain black field data ; Secondly, use a standard white board to obtain the sensor's response to the ideal reflected light and record its white field data under the current lighting conditions ; Spectral data is collected on the original bulk sample, and the data is corrected for black and white by calculating the relative reflectance.

[0062] Among them, the following formula can be used to calculate the relative reflectivity R:

[0063] ;

[0064] Where: To collect samples at a wavelength of Spectral measurement value at ; It is the dark current response of the sensor under no light conditions, that is, black field data; is the spectral response of the base plate under the same conditions, that is, the white field data.

[0065] The ordinate of the original spectrum curve is reflectivity. In this embodiment, the calculated relative reflectivity R is used to replace the reflectivity to achieve black and white correction.

[0066] In this embodiment, the Savitzky-Golay (SG) convolution smoothing method is used to smooth the original spectral data of the collected typical coal and rock samples, remove the abnormal data in the noise band, and perform preliminary dimensionality reduction.

[0067] In this embodiment, the characteristic spectrum, that is, the main characteristic band, in the original spectral information is extracted to further reduce the dimension of the spectral data. The specific process is as follows:

[0068] S1011: Represent the original spectral information in matrix form , each row of the matrix represents a pixel, each column represents a band, and there are m rows and n columns in total; that is, the spectral data matrix , m is the number of pixels, and n is the number of bands.

[0069] S1012: Initialize and assign different initial weights to all bands according to the importance characteristics of the data, that is, the weight vector is .

[0070] S1013: Each band Contains data from multiple pixels and calculates each band The variance of the reflectance of all pixels in is calculated, and the band with the largest variance value is selected as the starting band; a band subspace S containing the starting band is constructed.

[0071] S1014: Calculate each remaining band based on the weight Projection value to band subspace S , the band with the largest projection value Put it into the band subspace S, that is , and the band Serves as the starting band for the next projection.

[0072] In this embodiment, the remaining bands Gradually project it into the selected band subspace S, using the weights Calculate the projection values ​​of the remaining bands , specifically:

[0073] ;

[0074] Where: is the band that is not in the band subspace S during each projection process, S is the band subspace, is the inner product between two types of bands, is the weight.

[0075] S1015: Calculate the projection error of each band, and update the weight of each band according to the projection error.

[0076] Among them, the projection error , weighted according to the projection error The updates are as follows:

[0077] .

[0078] S1016: Based on the updated weights, continue to calculate the projection value of each band outside the band subspace S on the band subspace S, put the band with the largest projection value into the band subspace S, and use it as the starting band for the next projection; repeatedly calculate the projection error of each band and update the weights.

[0079] S1017: Repeat the process of S1016 until k set number of characteristic bands F={ }, as the characteristic spectrum of the original spectral information.

[0080] In this embodiment, a multi-scale grayscale co-occurrence matrix (MS-GLCM) is used to extract texture features of the grayscale images corresponding to the main characteristic bands of the typical coal rock selected above; the specific process is as follows: the spectral image of the main characteristic band is gray-converted, the scale is changed by changing the size of the observation window, the grayscale co-occurrence matrix is ​​applied at each scale to extract grayscale texture features, and different grayscale feature matrices are normalized to obtain the grayscale texture features of the image at multiple scales.

[0081] Specifically, the size of the original hyperspectral image after grayscale conversion is 256×256, the grayscale value range is [0,255], and different pixel distances d=1, 2, 3 are selected, and different directions are selected. =0°, 45°, 90°, 135° to represent different scales, select a specific distance and direction, and calculate the corresponding gray-level co-occurrence matrix:

[0082]

[0083] Thus, the number of pixel pairs (p, q) and (r, s) in the image at a given pixel distance d and direction is counted. The frequency of occurrence of pixel pairs with grayscale values ​​i and j is shown in the figure below.

[0084] Then normalize the gray-level co-occurrence matrix:

[0085]

[0086] The values ​​in the gray-level co-occurrence matrix represent the joint probability distribution of pixel pairs, and then texture features such as contrast, energy, entropy, homogeneity, and correlation are extracted from the normalized gray-level co-occurrence matrix.

[0087] Finally, for each scale (different d and ), repeat the steps of gray-level co-occurrence matrix calculation and texture feature extraction to obtain multiple texture features at each scale, and combine multiple texture features into a complete feature vector to obtain the grayscale texture features of the image at multiple scales.

[0088] In this embodiment, the two types of data, the extracted main feature band and the image texture feature, are simply combined into one data set, and then the linear discriminant analysis (LDA) is used to reduce the dimension of the data set. The feature vector after dimension reduction is the data after feature layer fusion; in addition, since the numerical ranges of the main feature band and the image texture feature are different, they need to be normalized when performing a simple combination operation, that is, the maximum-minimum normalization is used to scale the two types of feature values ​​to the range of [0, 1]; wherein, the main body of data in the typical coal rock spectrum-image database is: typical coal rock atlas fusion data after feature layer fusion, which is stored in the form of a multi-dimensional feature vector, including spectral information: the reflectivity of each pixel point in the main feature band, which can characterize the spectral characteristics of the material, and texture information: the texture structure characteristics of each pixel point, reflecting the spatial structure and pattern of the image.

[0089] S102: Mix coal and rock samples in different proportions, perform spectral curve analysis on the mixed samples with different carbon contents, obtain the influence of carbon content on the spectral characteristics of coal and rock, and manually annotate the pixel image data categories in a typical coal and rock spectrum-image database.

[0090] In this embodiment, three types of coal, namely anthracite, bituminous coal and lignite, and three types of coal-bearing rocks, namely mudstone, shale and sandstone, are mixed in pairs with different mixing ratios to prepare samples. Various mixed samples are crushed and mixed evenly to form cake-shaped samples. Spectral data are collected on a hyperspectral acquisition platform to analyze the change phenomenon of the spectral curve under different carbon contents. The test results show that with the increase of carbon content, the spectral slope of the near-infrared band will change accordingly, and the absorption valley depth of certain characteristic bands will also change. Therefore, the two types of spectral features, namely the spectral slope and the absorption valley depth, are parameterized, and the relationship between them and the carbon content is subjected to regression fitting analysis to obtain the correlation law between the carbon content and the two types of spectral features, which is specifically as follows:

[0091] S g =αx β ;

[0092] V h =γe δx +ζ;

[0093] Where x is the carbon content, S g is the spectral slope, V h is the absorption valley depth at the characteristic band, and α, β, γ, δ and ζ are all constants obtained through regression fitting.

[0094] The correlation law obtained based on the above relationship is as follows: the spectral curves of the pixel images of coal and rock have certain differences in the absorption valley depth and spectral slope of the characteristic band; however, when the carbon content is greater than the set threshold, the two types of spectral features, spectral slope and characteristic band absorption valley depth in the pixel images of coal and rock, will gradually show the same change trend, which will affect the accuracy of the subsequent cross-domain classification learning task; therefore, when encountering coal and rock samples in this situation, the single spectral feature information of the two cannot distinguish whether the category of the corresponding pixel image is coal or rock; since the typical coal-rock spectrum-image database is clear about coal or rock, for this situation where it is difficult to distinguish the type of coal and rock from the spectral features, the category of the pixel image is manually annotated according to whether the actual sample belongs to coal or rock, that is, the image category of the pixel is annotated in the form of text description, and the text feature is obtained, and is integrated into the cross-domain classification learning task as language prior knowledge; for example: "coal", "rock"; thereby avoiding the interference of single spectral feature information and widening the information volume of the annotated data in the typical coal-rock spectrum-image database.

[0095] S103: Using the data in the typical coal rock spectrum-image database as the source domain data and the hyperspectral data of the coal rock sample to be tested as the target domain data, respectively using the feature fusion network to extract the fusion features of the source domain data and the target domain data, and realizing the coal rock classification of the coal rock sample to be tested in the target domain through transfer learning.

[0096] In this embodiment, combined with Figure 2 The multi-level feature fusion network of spectrum-image-text includes a spectrum encoder, an image encoder and a text encoder. The spectrum encoder extracts the deep spectrum features of the input spectrum data, the image encoder obtains the enhanced spatial features of the input image texture features, and the text encoder converts the input annotation data into text features; the deep spectrum features, enhanced spatial features and text features are positionally encoded respectively to achieve feature alignment and unify them into the same representation space, and then the three types of features are input into the feature fusion module based on Transformer to output the ternary fusion features of the hyperspectral image.

[0097] Specifically, deep spectral features refer to high-level spectral features obtained by mapping low-level spectral features through the deep learning network embedded in the image encoder. This type of feature combines the details of spectral information and the expressive power of the deep learning model. It can capture global information, hidden patterns and complex relationships, and effectively improve the accuracy of tasks such as target classification and segmentation. Its role in the network can be understood as an enhanced representation of the input spectral data.

[0098] Strengthening spatial features means that the input hyperspectral image texture feature data is extracted in layers through an image encoder, and the edge texture is extracted, thereby eliminating redundant information in the edge area, maintaining the spatial integrity of the texture information, and realizing spatial enhancement of the input image texture feature data in the network.

[0099] The specific steps of feature alignment are: first, standardize the deep spectral features, enhanced spatial features, and text features to ensure that the three can be compared at the same scale; then, based on the cross-modal autoencoder, encode the spectral features, spatial features, and text features into a shared latent space, and then reconstruct the three back to the original modality through the decoder, so as to embed the three types of features into a shared representation space, thus achieving feature alignment;

[0100] The specific form of the ternary fusion feature is a multimodal feature tensor that contains spectral, spatial, and textual information. For example, in the Transformer-based feature fusion module, the input data batch size is Bs, the deep spectral feature uses each spectral band as the input token, the number of which is h, the enhanced spatial feature uses different pixel areas of the image as the input token, the number of which is b, and the text feature uses the serialized data directly as the input token, the number of which is t, and the token embedding dimension is Ed. The three types of tokens are mixed and input into the Transformer-based feature fusion module to obtain the sequence length Seq=h+b+t, and finally the fused multimodal feature tensor is output, and its data form is M=(Bs, Seq, Ed).

[0101] In this embodiment, the annotated data in the typical coal rock spectrum-image database is used as the source domain data, and the hyperspectral data of the coal rock samples to be identified in the actual working conditions is used as the target domain data. Figure 2 In this paper, the two types of data are first dimensionally aligned and input into the constructed spectrum-image-text multi-level feature fusion network respectively, and the spectrum-image-text ternary fusion features of the source domain and target domain data are output; secondly, the distance between the labeled and unlabeled samples in the ternary fusion features is evaluated, and small sample learning of the source domain or target domain data is performed. Finally, the nearest neighbor classifier and domain discriminator are used to classify the unlabeled samples in the target domain data, thereby realizing cross-domain classification of hyperspectral images and obtaining the labeled classification results of coal and rock under actual working conditions.

[0102] Among them, in the cross-domain classification task, the source domain and target domain data are respectively input into the spectrum, image, and text channels of the multi-level feature fusion network for ternary feature extraction and embedding, and then input into the spectrum, image, and text encoders in the network for internal feature fusion, and finally output their respective spectrum-image-text ternary fusion features at the output end of the network.

[0103] The purpose of small sample learning is to solve the problem of class inconsistency between source domain and target domain data. It creates support sets and query sets for source domain and target domain data. In the training phase of the small sample learning task model, the model parameters are updated based on the Euclidean distance metric. By continuously reducing the loss function of the source domain and target domain query set samples, the accuracy of the small sample learning model is optimized.

[0104] The source domain data contains a large number of labeled feature samples, and there are a small number of labeled feature samples in the target domain. All the labeled samples are used to train an initial nearest neighbor classifier. For each labeled sample, the distance metric between its features is calculated, and the nearest m neighbors are found and classified; then the initial nearest neighbor classifier is used to predict the unlabeled samples, and the prediction results and confidence levels are recorded. The unlabeled samples with the highest prediction confidence levels are added to the labeled sample set as pseudo-labeled samples; secondly, the labeled sample set is updated to include the initial labeled samples and the unlabeled samples that have just been pseudo-labeled, and the nearest neighbor classifier is retrained using the expanded sample set; finally, the above steps are repeated to continue classifying the remaining unlabeled samples, and the labeled sample set is expanded in each iteration until all confidence levels no longer increase. At this time, the iteration is considered to be completed and the unlabeled samples have been classified.

[0105] The purpose of the domain discriminator is to distinguish whether the labeled sample belongs to the source domain or the target domain data. It improves the recognition accuracy by reducing the domain adversarial loss function, thereby effectively distinguishing the characteristics of the source domain data and the target domain data, and realizing the classification of unlabeled samples in the target domain data, thereby realizing coal and rock recognition under small samples.

[0106] S104: For the identified coal sample, a three-dimensional convolutional neural network is used to extract the spatial-spectral joint features of the original spectral image, and based on the spatial-spectral joint features, each pixel point on the spectral image of the coal sample is identified to determine whether it contains carbon content.

[0107] In this embodiment, the three-dimensional convolutional neural network has the ability to extract three-dimensional information, taking two of the dimensions as spatial dimensions and the other dimension as spectral dimension, so as to directly extract the joint features of space and spectrum, namely, the spatial-spectral joint features, for the original hyperspectral image; obtain the contextual relationship between neighboring pixels, and obtain the classification result of whether each pixel on the spectral image of the coal sample contains carbon content.

[0108] Combination Figure 3The structure of the three-dimensional convolutional neural network (3DCNN) includes: a 3D convolution layer connected in sequence, three 3D convolution layers of the front Bayesian network, an average pooling layer and a fully connected layer; among them, the activation function of the 3D convolution layer of the front Bayesian network can dynamically adjust the slope of the negative area by introducing the parameter a in the negative area, which solves the problem of information loss in the negative area of ​​the traditional activation function ReLU, thereby improving the model convergence speed, generalization ability and expression ability.

[0109] In this embodiment, the input of the three-dimensional convolutional neural network (3DCNN) is usually a hyperspectral data cube, which includes three dimensions: a two-dimensional spatial dimension and a one-dimensional spectral dimension. For example, the input data is a cube of size (H, W, B), where H is the image height, W is the image width, B is the number of spectral bands, and each pixel corresponds to a spectral vector of (1, 1, B); then the convolution kernel of the 3DCNN convolution layer slides in the three-dimensional data space to extract the spatial-spectral joint features of the neighborhood pixel area; the spatial-spectral joint features are input to the pooling layer for sampling to reduce the size of the feature map, and then the joint features after convolution and pooling operations are input to the fully connected layer for comprehensive processing of high-dimensional features, and these features are matched with category labels of different substances; finally, the data is input to the output layer of the model, and a category probability distribution is output for each pixel, and the category with the highest probability is used as the final classification result of the pixel, thereby obtaining the classification result of whether each pixel contains carbon content.

[0110] S105: For the pixel points with carbon content, the trained carbon content prediction model is used to obtain the carbon content of each pixel point on the spectral image of the coal sample.

[0111] In this embodiment, according to the classification results of the hyperspectral image, the characteristic wavelength with a large correlation with the carbon content characteristics in the spectral curve of the pixel point with carbon content is analyzed, and a carbon content prediction model of the relationship between the reflectivity at the characteristic wavelength and the carbon content is obtained based on the partial least squares regression algorithm.

[0112] Specifically, the carbon content prediction model is constructed by maximizing the correlation between the reflectance at the characteristic wavelength and the carbon content. First, data is collected and prepared, including the known reflectance at the characteristic wavelength and the measured value of the carbon content at each pixel point, and the two are established in pairs. Data sets are then divided into training sets and test sets at a ratio of 80% and 20%; then the correlation between the reflectance at the characteristic wavelength of the spectrum and the carbon content is extracted by partial least squares regression (PLSR), which can be quickly called through the sklearn package; finally, the prediction model is trained using the data of the training set and the test set, and evaluation indicators such as mean square error (MSE) and R² are used to measure the prediction effect, and the hyperparameters of the model are adjusted through cross-validation to prevent overfitting and underfitting of the model, thereby establishing a carbon content prediction model between the reflectance at the characteristic wavelength and the carbon content.

[0113] The input of the carbon content prediction model is the reflectivity at the characteristic wavelength of the spectral curve of the carbon content pixel. The carbon content of each pixel is obtained through the carbon content prediction model, and each pixel is colorized according to the calculation result, that is, the carbon content distribution is visualized, realizing the mapping operation of the carbon content in the hyperspectral image, thereby detecting the carbon content value in the coal.

[0114] This embodiment integrates characteristic spectra, texture features and text features, and realizes coal and rock classification of coal and rock samples to be tested in the target domain through transfer learning, thereby improving the interpretability and cross-domain adaptability of traditional coal and rock classification models. The introduction of text features can better utilize the spectrum and image data in hyperspectral image data, make up for the shortcomings of single modality data, and thus enhance the adaptability of coal and rock classification models to small sample data.

[0115] For coal samples, a three-dimensional convolutional neural network is used to jointly extract the joint spatial-spectral features of the hyperspectral image, thereby capturing the local spatial dependency between pixels. This can better adapt to the complex relationship of spectral-spatial feature interaction in hyperspectral data and accurately predict whether each pixel has carbon content.

[0116] For pixels with carbon content, a carbon content prediction model is constructed based on the relationship between the reflectivity at the characteristic wavelength and the carbon content to achieve accurate prediction of the carbon content data of each pixel.

[0117] Embodiment 2

[0118] In one or more embodiments, a coal rock carbon content detection system based on multi-feature image spectroscopy technology is disclosed, comprising:

[0119] The feature extraction module is used to collect the original spectral information of typical coal and rock samples and extract the characteristic spectrum; use the multi-scale grayscale co-occurrence matrix to extract the texture features of the grayscale image corresponding to the extracted characteristic spectrum; and build a database of typical coal and rock types and their corresponding characteristic spectra and texture features;

[0120] A text annotation module is used to mix coal and rock samples in different proportions, annotate the image category of each pixel in the spectral image of the mixed sample, and add it to the database;

[0121] A coal-rock classification module is used to use the data in the database as source domain data and the hyperspectral data of the coal-rock samples to be tested as target domain data, respectively extract fusion features of the source domain data and the target domain data using a feature fusion network, and achieve coal-rock classification of the coal-rock samples to be tested in the target domain through transfer learning;

[0122] The carbon content prediction module is used to extract the spatial-spectral joint features of the original spectral image for the identified coal samples using a three-dimensional convolutional neural network, and to identify whether each pixel point on the spectral image of the coal sample contains carbon content based on the spatial-spectral joint features; for pixels with carbon content, the trained carbon content prediction model is used to obtain the carbon content of each pixel point on the spectral image of the coal sample.

[0123] The specific implementation of the above modules is the same as that in Example 1 and will not be described in detail.

[0124] Embodiment 3

[0125] In one or more embodiments, a terminal device is disclosed, which includes a processor and a memory, the processor is used to implement instructions; the memory is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executing the coal rock carbon content detection method based on multi-feature image spectroscopy technology described in Example 1.

[0126] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0127] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0128] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0129] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for detecting carbon content in coal based on multi-feature image spectroscopy technology, characterized in that: include: Collect the original spectrum information of typical coal and rock samples and extract characteristic spectra; The multi-scale grayscale co-occurrence matrix is ​​used to extract texture features from the grayscale image corresponding to the extracted characteristic spectrum; a database of typical coal and rock types and their corresponding characteristic spectra and texture features is constructed; Mix coal and rock samples in different proportions, annotate the image category of each pixel in the spectral image of the mixed sample with text, and add it to the database; The data in the database are used as source domain data, and the hyperspectral data of the coal and rock samples to be tested are used as target domain data. The fusion features of the source domain data and the target domain data are extracted using a feature fusion network, and the coal and rock classification of the coal and rock samples to be tested in the target domain is achieved through transfer learning. For the identified coal samples, a three-dimensional convolutional neural network is used to extract the spatial-spectral joint features of the original spectral image, and based on the spatial-spectral joint features, each pixel point on the spectral image of the coal sample is identified to determine whether it contains carbon content; for pixels with carbon content, the trained carbon content prediction model is used to obtain the carbon content of each pixel point on the spectral image of the coal sample; The feature fusion network includes a spectral encoder, an image encoder and a text encoder. The spectral encoder extracts the deep spectral features of the input spectral data, and the image encoder obtains the enhanced spatial features of the input image texture features. The text encoder converts the input annotated data into text features; the deep spectral features, enhanced spatial features and text features are positionally encoded respectively to achieve feature alignment and unify them into the same representation space. The three types of features are then input into the Transformer-based feature fusion module to output the fused features of the source domain data or the target domain data.

2. The method for detecting carbon content in coal based on multi-feature image spectroscopy technology according to claim 1, characterized in that: Before collecting the original spectral information of typical coal and rock samples, spectral data are collected on the original block samples, and the spectral data are corrected for black and white by calculating the relative reflectance.

3. The method for detecting carbon content in coal based on multi-feature image spectroscopy technology according to claim 1, characterized in that: Extract the characteristic spectrum of the original spectrum information, specifically: Represent the original spectral information in matrix form , each row of the matrix represents a pixel, each column represents a band, and there are m rows and n columns in total; Assign different initial weights to all bands according to the importance characteristics of the data; Calculate each band The variance of the reflectivity of all pixels in the image is calculated, and the band with the largest variance is selected as the starting band; a band subspace S containing the starting band is constructed; Calculate each of the remaining bands based on the weights Projection value to band subspace S , put the band with the largest projection value into the band subspace S and use it as the starting band for the next projection; Calculate the projection error of each band and update the weight of each band according to the projection error; Based on the updated weights, continue to calculate the projection value of each band outside the band subspace S to the band subspace S, put the band with the largest projection value into the band subspace S, and use it as the starting band for the next projection; repeatedly calculate the projection error of each band and update the weights; Repeat the above process until k set number of characteristic bands F={ }, as the characteristic spectrum of the original spectral information.

4. The method for detecting carbon content in coal based on multi-feature image spectroscopy technology according to claim 1, characterized in that: The multi-scale gray-level co-occurrence matrix is ​​used to extract texture features from the gray-level image corresponding to the extracted characteristic spectrum. The specific process is as follows: Represent different scales with different pixel distances and directions; Under a certain set scale, the corresponding gray-level co-occurrence matrix is ​​calculated to obtain the frequency of occurrence of pixel pairs with gray-level values ​​i and j in the gray-level image; Normalizing the gray-level co-occurrence matrix, extracting multiple texture features including contrast, energy, entropy, homogeneity and correlation from the normalized gray-level co-occurrence matrix, and combining the multiple texture features into a complete texture feature vector; For each scale, the texture feature vector is extracted respectively, so as to obtain the texture feature vector of the grayscale image at multiple scales.

5. The method for detecting carbon content in coal based on multi-feature image spectroscopy technology according to claim 1, characterized in that: The influence of carbon content on the spectral characteristics of coal rock is specifically as follows: The two types of spectral features, spectral slope and absorption valley depth, were parameterized, and regression fitting analysis was performed on the relationship between carbon content and the two types of spectral features, spectral slope and absorption valley depth. When the carbon content is greater than the set threshold, the two types of spectral features, spectral slope of each pixel point in the coal and rock spectral images and absorption valley depth of the characteristic band, will show the same change trend; at this time, coal and rock cannot be distinguished by spectral features alone. Therefore, the image category of each pixel point in the mixed sample spectral image is annotated with text to increase the amount of image information in the database.

6. The method for detecting carbon content in coal based on multi-feature image spectroscopy technology according to claim 1, characterized in that: The coal and rock classification of the coal and rock samples to be tested in the target domain is realized through transfer learning, specifically: The source domain data contains labeled feature samples, and the target domain data contains unlabeled feature samples. Use all labeled samples to train an initial nearest neighbor classifier. For each labeled sample, calculate the distance metric between its features, find the nearest m neighbors and classify them. Then use the initial nearest neighbor classifier to predict the unlabeled samples, record the prediction results and confidence, and add the unlabeled samples with the highest prediction confidence as pseudo-label samples to the labeled sample set; Update the labeled sample set to include the initial labeled samples and the unlabeled samples that have just been pseudo-labeled, and retrain the nearest neighbor classifier using the expanded sample set; Repeat the above steps to continue classifying the remaining unlabeled samples and expand the labeled sample set in each iteration until all confidence levels no longer increase. At this point, the iteration is considered complete and the unlabeled samples have been classified.

7. The method for detecting carbon content in coal based on multi-feature image spectroscopy technology according to claim 1, characterized in that: The three-dimensional convolutional neural network specifically includes: A 3D convolution layer, three 3D convolution layers of the front Bayesian network, an average pooling layer and a fully connected layer are connected in sequence; wherein the activation function of the 3D convolution layer of the front Bayesian network introduces a parameter a in the negative region, which can dynamically adjust the slope of the negative region; The three-dimensional spectral data is input into the three-dimensional convolutional neural network, and the spatial-spectral joint features are extracted through multiple layers of 3D convolutional layers. The spatial-spectral joint features are sampled through the average pooling layer to reduce the size of the feature map; the spatial-spectral joint features output by the average pooling layer are input into the fully connected layer for comprehensive processing of high-dimensional features to obtain the category probability distribution of whether each pixel point has carbon content, and finally the classification result of whether each pixel point on the spectral image of the coal sample contains carbon content is output through the output layer.

8. A coal rock carbon content detection system based on multi-feature image spectroscopy technology, characterized in that: include: Feature extraction module, used to collect original spectrum information of typical coal and rock samples and extract characteristic spectra; The multi-scale grayscale co-occurrence matrix is ​​used to extract texture features from the grayscale image corresponding to the extracted characteristic spectrum; a database of typical coal and rock types and their corresponding characteristic spectra and texture features is constructed; A text annotation module is used to mix coal and rock samples in different proportions, annotate the image category of each pixel in the spectral image of the mixed sample, and add it to the database; A coal-rock classification module is used to use the data in the database as source domain data and the hyperspectral data of the coal-rock samples to be tested as target domain data, respectively extract fusion features of the source domain data and the target domain data using a feature fusion network, and achieve coal-rock classification of the coal-rock samples to be tested in the target domain through transfer learning; The carbon content prediction module is used to extract the spatial-spectral joint features of the original spectral image for the identified coal sample using a three-dimensional convolutional neural network, and identify whether each pixel point on the spectral image of the coal sample contains carbon content based on the spatial-spectral joint features; for the pixel points with carbon content, the trained carbon content prediction model is used to obtain the carbon content of each pixel point on the spectral image of the coal sample; The feature fusion network includes a spectral encoder, an image encoder and a text encoder. The spectral encoder extracts the deep spectral features of the input spectral data, and the image encoder obtains the enhanced spatial features of the input image texture features. The text encoder converts the input annotated data into text features; the deep spectral features, enhanced spatial features and text features are positionally encoded respectively to achieve feature alignment and unify them into the same representation space. The three types of features are then input into the Transformer-based feature fusion module to output the fused features of the source domain data or the target domain data.

9. A terminal device, comprising a processor and a memory, wherein the processor is used to implement instructions; and the memory is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the coal rock carbon content detection method based on multi-feature image spectroscopy technology as described in any one of claims 1-7.

Images