Solid mineral product quality parameter detection method based on NIR and granularity domain generalization

Abstract

The invention discloses a solid mineral product quality parameter detection method based on NIR and granularity domain generalization, and relates to the technical field of solid mineral product detection.The method comprises the steps that near infrared spectrum data of solid mineral product samples of various granularities are obtained, and an original spectrum data set is constructed; identifying an absorption peak change sensitive area of the near infrared spectrum data and extracting an absorption structure spectrum; randomly disturbing the near infrared spectrum data to generate a structure disturbance spectrum, and fusing the structure disturbance spectrum with an absorption structure spectrum to obtain an enhanced spectrum so as to expand the original spectrum data set to obtain an enhanced spectrum data set; training by utilizing the enhanced spectrum data set to obtain a solid mineral product quality parameter detection model; and predicting the near infrared spectrum data of the to-be-detected solid mineral product sample by using the solid mineral product quality parameter detection model, and outputting a quality parameter prediction value. According to the method, the quality parameters of solid mineral products with different granularities can be rapidly and accurately detected, and the robustness and the granularity domain generalization ability of the model are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of solid mineral product testing technology, and in particular to a method for detecting solid mineral product quality parameters based on NIR and particle size domain generalization. Background Technology

[0002] Mineral products, as vital resources supporting national economic development, hold an irreplaceable position in energy supply, industrial production, and strategic reserves. Solid mineral products, such as coal and bauxite, are not only basic industrial raw materials but also key commodities in global trade. With the increasingly frequent flow of international resources, the quality parameters of solid mineral products have become crucial criteria for upholding fair trade, strengthening market supervision, and ensuring the reliability of the supply chain.

[0003] Currently, the quality parameter detection of solid mineral products largely relies on traditional methods such as chemical analysis and physical testing. These methods suffer from problems such as long detection cycles, heavy reliance on manual labor, high costs, and poor adaptability, making it difficult to meet the application requirements of high-frequency circulation and rapid grading of modern mineral products. In recent years, data-driven modern spectroscopic detection technologies have shown significant advantages in non-destructive testing and real-time response. Existing research has attempted to use X-ray fluorescence spectroscopy, laser-induced breakdown spectroscopy, and Raman spectroscopy to achieve mineral identification and elemental quantification; however, these methods typically suffer from high sample preparation requirements, expensive instruments, and limited application scenarios. In contrast, near-infrared spectroscopy (NIRS) offers advantages such as fast detection speed, non-destructive testing, low cost, and portability, making it an ideal technical approach for detecting mineral product quality parameters.

[0004] However, significant differences exist in the scattering intensity and baseline morphology of mineral products of different particle sizes: fine-grained mineral products exhibit smoother near-infrared spectral baselines and sharper absorption peaks; while coarse-grained mineral products show enhanced scattering, resulting in a raised near-infrared spectral baseline, broadened absorption peaks, and reduced absorption depth. This "style difference" in near-infrared spectroscopy leads to a significant degradation in the detection performance of solid mineral quality parameter detection models trained on samples of a particular particle size on mineral products of other particle sizes, exhibiting a clear cross-particle size domain shift problem. Therefore, improving the particle size domain generalization performance of quality parameter detection methods is a crucial issue that urgently needs to be addressed when solid mineral product samples are limited. Summary of the Invention

[0005] This application addresses the aforementioned problems and technical needs by proposing a method for detecting quality parameters of solid mineral products based on NIR and particle size domain generalization. The technical solution of this application is as follows: A method for detecting quality parameters of solid mineral products based on NIR and particle size domain generalization includes the following steps: Near-infrared spectral data of different solid mineral samples with multiple particle sizes were obtained, and the original spectral dataset was constructed, with each particle size including multiple solid mineral samples. For any near-infrared spectral data, identify the sensitive region of absorption peak change in the near-infrared spectral data and extract the absorption structure spectrum. The absorption structure spectrum indicates the absorption structure information of the chemical composition of the solid mineral product sample. The sensitive region of absorption peak change is the spectral region in the near-infrared spectral data where the absorption peak change meets the predetermined conditions. The near-infrared spectral data is randomly perturbed to generate a structure perturbation spectrum, and the absorption structure spectrum is fused with the structure perturbation spectrum to obtain an enhanced spectrum; the original spectral dataset is then augmented using the enhanced spectrum samples to obtain an enhanced spectral dataset. A network architecture for a solid mineral product quality parameter detection model was constructed and trained using an enhanced spectral dataset. The solid mineral product quality parameter detection model is used to predict the quality parameters of solid mineral products after feature extraction from near-infrared spectral data. Near-infrared spectral data of solid mineral product samples to be inspected are obtained and input into the trained solid mineral product quality parameter detection model to predict and output the predicted values ​​of solid mineral product quality parameters.

[0006] A further technical solution involves identifying sensitive regions for changes in absorption peaks in near-infrared spectral data, including: Frequency domain feature decomposition is performed on near-infrared spectral data to obtain high-frequency and low-frequency components. The high-frequency components characterize the components in the near-infrared spectral data that change rapidly with wavelength, while the low-frequency components characterize the components in the near-infrared spectral data that change slowly with wavelength. The high-frequency components of near-infrared spectral data are reconstructed to obtain the high spectrum, and the high spectrum is then divided into... A continuous spectral band, the high spectrum characterizes the chemical bond composition and absorption characteristics of solid mineral products, and the high spectrum is available at any wavelength. amplitude at , It is the first wavelength point. j Layer high frequency components, It is a high-frequency initiation layer. J It is the highest frequency layer; Calculate the average response intensity for each spectral band and identify the spectral bands with the average response intensity in the top p% as the sensitive regions for absorption peak changes, using integer parameters. >1.

[0007] A further technical solution involves calculating the average response intensity of any spectral band i. include: Determine any wavelength point within spectral band i Response intensity value at , It is the high spectrum at the wavelength point The amplitude at that point, , , All are weighting coefficients. express The first derivative, express The second derivative; The average response intensity of spectral band i is determined based on the response intensity values ​​at each wavelength point within spectral band i. , It is the length of spectral segment i.

[0008] Its further technical solution is to divide the high spectrum into The continuous spectral bands include: Calculate the second and third derivatives at each wavelength point in the near-infrared spectral data, and determine the wavelength points where the second derivative is 0 and the third derivative is not 0 as the spectral segment division points; The high spectrum is divided according to the spectral segment division points. A continuous spectral band is formed by any two adjacent spectral band division points and all wavelength points between them.

[0009] The further technical solution is that the extracted absorption structure spectrum includes: In the near-infrared spectral data, all wavelengths in the spectral bands with average response intensity in the top p% are marked as 1. All wavelengths in the spectral bands with average response intensity not in the top p% but belonging to a preset typical absorption band are also marked as 1. All wavelengths in the remaining spectral bands are marked as 0, thus obtaining the high-response spectral band mask. ; Using high response spectral band mask The absorbance at the wavelength points marked as 1 is extracted from the near-infrared spectral data, the absorbance at the wavelength points marked as 0 is set to 0, and the absorption structure spectrum is obtained by arranging the wavelength points in order.

[0010] A further technical solution involves generating a structural perturbation spectrum by randomly perturbing near-infrared spectral data, including: Another near-infrared spectral dataset is randomly selected from the original spectral dataset, and the near-infrared spectral data is divided according to the spectral band division points. A series of continuous spectral segments are used, and the order of each spectral segment is randomly rearranged. The rearranged spectral segments are then combined to form the structural perturbation spectrum.

[0011] A further technical solution involves fusing the absorption structure spectrum with the structure perturbation spectrum, including: Linear interpolation is performed between the absorption structure spectrum and the structure perturbation spectrum to obtain the enhancement spectrum at any wavelength. absorbance at , It is the absorption structure spectrum at the wavelength point absorbance at that point The structural perturbation spectrum at the wavelength point absorbance at that point These are the interpolation coefficients.

[0012] A further technical solution involves constructing a solid mineral product quality parameter detection model based on an MLP or Transformer model. The trained solid mineral product quality parameter detection model includes: The network parameters of the solid mineral product quality parameter detection model are initialized, and the model is trained using an enhanced spectral dataset. The loss function is then minimized. A detection model for quality parameters of solid mineral products was obtained; loss function ,in, It is the balance coefficient. It is a regression loss. It is structural consistency loss; regression loss This is used to measure the prediction accuracy of a solid mineral product quality parameter detection model on raw near-infrared spectral data. Higher prediction accuracy and lower regression loss are associated with better prediction accuracy. The smaller the value, the less structural consistency is lost. This is used to measure the stability of the prediction results of a solid mineral product quality parameter detection model before and after structural disturbance. The higher the stability of the prediction results before and after structural disturbance, the lower the structural consistency loss. The smaller.

[0013] The further technical solution is that the enhanced spectral dataset includes original near-infrared spectral samples and enhanced spectral samples. The original near-infrared spectral samples include original near-infrared spectral data and its corresponding quality parameter labels. The enhanced spectral samples include enhanced spectra constructed based on the original near-infrared spectral data and the quality parameter labels corresponding to the original near-infrared spectral data. Regression loss , N It is the total number of original near-infrared spectral samples in the original spectral dataset. It is the near-infrared spectral data of the i-th original near-infrared spectral sample in the original spectral dataset. It is the quality parameter label for the i-th raw near-infrared spectrum sample. yes Predicted values ​​of quality parameters; Structural consistency loss , K It is the near-infrared spectral data of the i-th original near-infrared spectral sample. The corresponding total number of enhanced spectra, It is the near-infrared spectral data of the i-th original near-infrared spectral sample. The corresponding j-th enhancement spectrum, yes Predicted values ​​of quality parameters It represents the 2-norm.

[0014] A further technical solution involves using the discrete wavelet transform method to perform frequency domain feature decomposition on near-infrared spectral data to obtain the high-frequency and low-frequency components in the near-infrared spectral data. The higher the resolution of the near-infrared spectral data, the more decomposition layers the discrete wavelet transform can achieve.

[0015] The beneficial technical effects of this application are: This application discloses a method for detecting quality parameters of solid mineral products based on NIR and grain size domain generalization. By decomposing raw near-infrared spectral data into low-frequency components characterizing the global spectral morphology trend and high-frequency components characterizing the local spectral details, it achieves bidirectional decoupling of the "structural features-style features" of near-infrared spectroscopy. This method can stably extract macroscopic peak shapes and local absorption peak details that characterize chemical composition, reducing domain shifts caused by grain size, scattering, or instrument differences. By identifying absorption peak change-sensitive regions in the high-frequency domain and combining perturbation mechanisms to generate enhanced spectra in different grain size domains, it can effectively expand the grain size domain spectral dataset under the condition of samples with limited grain size. This provides important data support for cross-grain size domain modeling of solid mineral product quality parameter detection models and helps improve the generalization and regression performance of solid mineral product quality parameter detection models under different grain sizes and sampling conditions.

[0016] Considering that in one-dimensional near-infrared spectroscopy, the wavelength axis strictly corresponds to the absorption behavior of specific chemical groups, and that existing enhancement methods designed for traditional convolutional neural networks (CNNs) can destroy the physical meaning of spectral absorption peaks through local pruning, channel transformation, or arbitrary style perturbation, leading to the invalidation of the one-to-one correspondence between absorbance and chemical composition, making it easy for solid mineral product quality parameter detection models to learn pseudo-textures unrelated to chemical structure. To address this issue, this application employs MLP or Transformer models to construct solid mineral product quality parameter detection models. These models divide the spectrum into continuous band tokens and utilize self-attention mechanisms to capture long-distance dependencies, achieving stronger "shape bias" and characterizing the cooperative changes and long-distance structural relationships between absorption peaks. Combined with the token construction method in this application, which directly divides the original near-infrared spectral data into original spectral band-level representation units, and directly uses the original absorbance fragments of the spectrum as modeling units, while maintaining the integrity of physical semantics, it captures the global peak shape and structural relationships of the spectrum through token dependency modeling, thereby achieving a higher level of shape modeling capability and generalization stability than traditional convolutional models.

[0017] By analyzing the amplitude, slope, and curvature of the high-frequency structure, a multi-index response intensity function is constructed to quantify the high-frequency change intensity (i.e., absorption rate) at each wavelength point. Based on the absorption peak change sensitive region extracted from the response intensity, only the absorption structure region related to the composition characteristics is retained, which can truly reflect the position and intensity changes of the absorption band in the near-infrared spectrum. This helps to obtain enhanced spectra with real chemical structures and is of great significance for improving the accuracy and reliability of solid mineral product quality parameter detection models. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method for testing the quality parameters of solid mineral products.

[0019] Figure 2 This is a near-infrared spectrum of coal samples of different particle sizes in an example.

[0020] Figure 3 This is a schematic diagram of the absorption structure spectrum extracted from an example.

[0021] Figure 4 This is a schematic diagram of the generated structural perturbation spectrum in an example. Detailed Implementation

[0022] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0023] This application discloses a method for detecting quality parameters of solid mineral products based on NIR and particle size domain generalization. Please refer to [link / reference]. Figure 1 The flowchart shown illustrates the specific steps of this method as follows: Step 1: Obtain near-infrared spectral data of solid mineral samples with various particle sizes and construct the original spectral dataset.

[0024] Solid mineral samples are samples of solid mineral products suitable for near-infrared spectroscopy detection, such as bauxite or coal samples. Near-infrared spectral data of solid mineral samples of various particle sizes are collected using a near-infrared spectral acquisition instrument. Each particle size includes multiple solid mineral samples, and each solid mineral sample corresponds to one near-infrared spectral data point. The near-infrared spectral acquisition instrument used can be customized according to actual needs. This application uses the Antaris II Fourier transform near-infrared spectrometer from the Molecular Spectroscopy Division of Thermo Fisher Scientific to collect near-infrared spectra of multiple coal samples of different particle sizes, as shown below. Figure 2 As shown, the horizontal axis represents the wavelength, and the vertical axis represents the absorbance. Figure 2 (a) shows the near-infrared spectral data of different coal samples with a diameter of 0.2 mm. Figure 2(b) shows the near-infrared spectral data of different coal samples at 1 mm.

[0025] The solid mineral samples collected using near-infrared spectroscopy instruments have different particle sizes, and the particle size of the collected samples should be within a preset particle size threshold. This is to ensure the feasibility of constructing the original spectral dataset, and to facilitate the collection of near-infrared spectral data under conditions of limited samples and equipment, thus ensuring the acquisition of effective near-infrared spectral data. The particle size threshold can be customized according to the actual application scenario. The particle size threshold in this application is 6 mm, meaning that when collecting near-infrared spectral data from solid mineral samples, the particle size of the solid mineral samples should be within the particle size range of 0 to 6 mm.

[0026] A raw near-infrared spectral dataset is constructed using the collected raw near-infrared spectral data. Each raw near-infrared spectral sample in the dataset includes raw near-infrared spectral data and its corresponding quality parameter label. The following example, using coal as the solid mineral product to be inspected, illustrates the method and process of constructing the raw spectral dataset. Specifically, 400 coal samples are collected, originating from import and export batches at a port within a certain time period, covering multiple countries and regions. The raw spectral dataset includes various particle sizes (0.2mm, 1mm, 3mm, 6mm). Near-infrared spectral data are collected for each coal sample, and its corresponding quality parameter label is determined, including moisture, volatile matter, and calorific value. When the solid mineral product to be inspected is another type, the method for constructing the raw spectral dataset can be found here.

[0027] Step 2: For any near-infrared spectral data, identify the sensitive region of absorption peak change in the near-infrared spectral data and extract the absorption structure spectrum. The absorption structure spectrum indicates the absorption structure information of the chemical composition of the solid mineral product sample.

[0028] The absorption peak change-sensitive region refers to a sub-spectrum containing absorption peaks sensitive to chemical composition characteristics and structural details. These regions are strongly correlated with chemical structure and can reflect changes in the position, intensity, and shape of absorption peaks in the spectrum, representing key structural features for quality parameter detection. For near-infrared spectral data, the absorption peak change-sensitive region is the spectral region where absorption peak changes meet predetermined conditions.

[0029] Due to the limited number of solid mineral product samples and sampling conditions, the near-infrared spectral data in the collected original spectral dataset cannot cover all granularity domains of solid mineral products. If only the original spectral dataset is used to train the solid mineral product quality parameter detection model, the model's granularity domain generalization ability will be severely insufficient, failing to achieve true cross-granularity domain quality parameter prediction. Therefore, expanding and enhancing the original spectral dataset is the key and foundation for improving the granularity domain generalization performance of the solid mineral product quality parameter detection model.

[0030] Existing spectral enhancement methods, such as smoothing, noise addition, and random stretching, while improving the diversity of training data, only indirectly enhance the model's edge detection ability. They cannot distinguish between "true spectral structures related to chemical absorption" and "pseudo-spectral structures caused by scattering and instrument noise," making it difficult for the model to learn stable and transferable structural features. Therefore, to ensure the reliability of the constructed enhanced spectrum and to ensure that the enhanced spectrum contains chemical information related to quality parameters, this application analyzes the near-infrared spectral data of solid mineral products and identifies sensitive regions of absorption peak changes based on spectral absorption characteristics.

[0031] In one embodiment, identifying sensitive regions for changes in absorption peaks in near-infrared spectral data includes: (1) Frequency domain feature decomposition was performed on the near-infrared spectral data to obtain the high-frequency and low-frequency components in the near-infrared spectral data. The high-frequency components characterize the components in the near-infrared spectral data that change rapidly with wavelength. They mainly originate from the rising and falling edges of absorption peaks, the inflection point of peak shape, the sharpness, and the local structural changes between multiple absorption bands. These changes are determined by the absorption behavior of chemical bond vibrations and reflect the chemical composition. The low-frequency components characterize the components in the near-infrared spectral data that change slowly with wavelength. They mainly originate from baseline drift, overall lifting and bending caused by particle size scattering, instrument differences, and unstable light sources. These changes are mainly caused by physical factors (rather than chemical absorption) and reflect scattering, baseline, and instrument changes.

[0032] Since the high-frequency components of near-infrared spectral data contain key chemical absorption structure information, multi-scale decomposition is used to divide the near-infrared spectral data into multiple high-frequency detail components and low-frequency approximation components to fully extract the high-frequency features, thereby separating the chemical structure from the baseline scattering information. Any frequency domain feature decomposition method can be used to extract the high-frequency and low-frequency components. In one embodiment, the discrete wavelet transform (DWT) method is used to perform frequency domain feature decomposition on the near-infrared spectral data to obtain the high-frequency and low-frequency components. The higher the resolution of the near-infrared spectral data, the more decomposition levels the discrete wavelet transform can achieve.

[0033] The DWT method is used to decompose the near-infrared spectrum into multi-scale components, separating the spectrum into approximate (low-frequency) and multi-layer detail (high-frequency) components. This allows for the separation of "baseline / scattering" and "absorption peak / local detail" information at different scales, serving as the basis for subsequent absorption peak response extraction and enhancement. Assuming at any wavelength... The original near-infrared spectral data at the location are Its sampling points are Discrete wavelet transform includes a pair of orthogonal filters and a low-pass filter. High-pass filter ,in, and The lengths are all The filter coefficients are from Control. Based on the wavelet orthogonality constraint, we can let This ensures that the filter performs complementary low-frequency and high-frequency decomposition of the signal. Based on this, the multi-level decomposition relationship of the learnable filter can be written as:

[0034] in, It is the first j The values ​​of the low-frequency approximation coefficients at different discrete locations in the layer. n It is the discrete-time index of the input signal, corresponding to The locations of each sampling point, This is the raw near-infrared spectral data. Indicates the first j The low-frequency approximation coefficients of layer +1 Indicates the first j +1 layer high-frequency detail coefficient, k It is the discrete-time index of the output signal.

[0035] Perform on each near-infrared spectrum J Layered discrete wavelet transform, at any wavelength point Near-infrared spectral data at [location] It can be represented as:

[0036] in, It is the first J The low-frequency components of the layer reflect the overall morphology and variation trend of the near-infrared spectrum; It is the first j High-frequency components of the layer reflect the local fluctuations and absorption peak structure of the near-infrared spectrum. Through this multi-scale decomposition, low-frequency components related to particle size and scattering, as well as high-frequency details related to chemical structure, can be obtained separately, providing a foundation for subsequent structural feature extraction and enhancement.

[0037] This application selects the Daubechies-4 (db4) wavelet as the mother wavelet. db4 possesses excellent time-frequency localization characteristics, effectively separating the smooth baseline and absorption peak details in the near-infrared spectra of solid mineral products. (Number of decomposition layers) J The choice of is related to the spectral resolution. The higher the resolution of the near-infrared spectral data, the more decomposition levels the discrete wavelet transform can achieve. The specific value can be customized according to the actual application requirements, and is generally set to . J= 3~4, to balance scattering trends and chemical structure scale.

[0038] (2) The high-frequency components of the near-infrared spectral data are reconstructed to obtain the high spectrum, and the high spectrum is divided into... A continuous spectral band, the high spectrum characterizes the chemical bond composition and absorption characteristics of solid mineral products, and the high spectrum is available at any wavelength. amplitude at , It is the first wavelength point. j Layer high frequency components, It is a high-frequency initiation layer. J It is the highest frequency layer (i.e., the decomposition layer number).

[0039] The high-frequency spectrum retains only local absorption peaks and detailed structural variations, primarily reflecting overtones and combination band absorptions of chemical bonds (such as OH, CH, and NH vibrations), thus revealing the chemical composition information of the sample. Low-frequency components mainly reflect the macroscopic spectral style caused by particle size and scattering, such as baseline drift and scattering elevation. Therefore, to preserve the high-frequency structure related to chemical absorption, the high-order components of the high-frequency components in the near-infrared spectral data are reconstructed to obtain the high-frequency spectrum, with a high-frequency initiation layer... The value of is determined based on experimental results, and is generally taken as . =3~4, that is, retaining the detailed components above the 3rd and 4th layers, thus highlighting the absorption peaks and their shape changes. Through this reconstruction, what is obtained is essentially a spectrum that has removed most of the low-frequency baseline and mainly contains high-frequency absorption structure information, which can reflect the position and intensity changes of the absorption band.

[0040] To reduce the sensitivity of the high-frequency response to random noise, this embodiment divides the spectrum into bands. However, since the absorption peak distribution in near-infrared spectra is usually not uniform, the dense absorption peak regions and the gentle scattering regions exhibit a significant imbalance along the wavelength axis. Therefore, it is necessary to adaptively divide the spectrum according to the spectral structure characteristics of near-infrared spectra. The high-frequency spectrum is divided into... The continuous spectral bands include: Calculate the second and third derivatives at each wavelength point in the near-infrared spectral data, and determine the wavelength points where the second derivative is 0 and the third derivative is not 0 as the spectral segment division points; divide the high spectrum according to the spectral segment division points. A continuous spectral band is formed by any two adjacent spectral band division points and all wavelength points between them.

[0041] Considering that the second derivative can characterize the curvature change of a spectrum, its extreme points correspond to the peaks, valleys, or inflection points of the spectrum. The third derivative is the rate of change of curvature, that is, whether the curvature near a point decreases, increases, changes from positive to negative, or changes from negative to positive. It is used to characterize the trend of curvature at its maxima / minimum and inflection points, and its value can characterize the trend of curvature changing with wavelength. The second derivative is calculated for near-infrared spectra, and points where the second derivative is 0 and the third derivative is not 0 are taken as structural boundaries, i.e., the inflection points (peaks or valleys) of absorption peaks. When the second derivative is 0 and the third derivative is not 0 at a certain wavelength, it indicates that the curvature near that point changes from positive to negative or from negative to positive, corresponding to the structural boundary position where the spectrum changes from rising to falling or from falling to rising. Using these boundaries as spectral segmentation points, the spectrum can be dynamically divided into... A set of spectral bands containing absorption peak structural features Each spectral band extends from one segmentation point to the next, ensuring that it contains the rising or falling edge of the absorption peak. Adaptive segmentation enables the segment-level average response intensity to accurately reflect the overall changes in the local structure of that segment, improving the accuracy of the response spectral band in locating the absorption peak region and significantly enhancing the specificity and stability of the spectral mask.

[0042] (3) Calculate the average response intensity of each spectral band, and determine the spectral bands with the average response intensity in the top p% as the sensitive regions for absorption peak changes, using integer parameters. >1.

[0043] To quantify the intensity of high-frequency changes at each wavelength, this invention further defines a multi-index response intensity function based on the high-frequency structural spectrum. This function reflects the intensity of high-frequency changes (i.e., the rate of absorption change) at each wavelength. Since the frequency domain amplitude reflects the strength of the frequency component change: large high-frequency amplitudes indicate dramatic peak changes, clear structural details, and strong chemical structural information; small high-frequency amplitudes indicate flat peaks and weak chemical structural information; large low-frequency amplitudes indicate large baseline rise, strong scattering, obvious morphological change trends, and correlation with particle size / instrument variations; small low-frequency amplitudes indicate stable morphological change trends and weak scattering. Therefore, the response intensity can be calculated based on the amplitude. Furthermore, in addition to using the absolute value of the high-frequency amplitude, the rate of change information of the first and second derivatives is introduced to simultaneously characterize the abrupt slope changes and curvature changes of the high-frequency spectrum.

[0044] Calculate the average response intensity of any spectral band i Includes: determining any wavelength point within spectral band i Response intensity value at , It is the high spectrum at the wavelength point The amplitude at that point, , , All are weighting coefficients. express The first derivative, express The second derivative; determine the average response intensity of spectral band i based on the response intensity values ​​at each wavelength point within spectral band i. , It is the length of spectral segment i. express Is in Wavelength points within.

[0045] Larger This indicates that the region contains drastic absorption variations or peak characteristics, with smaller... This indicates that the segment is relatively flat and may only contain the baseline. Calculating the response intensity of this spectral segment is equivalent to estimating "structural importance" in the spectral space. Quantifying the "structural importance" of each spectral segment can guide the determination of sensitive regions for absorption peak changes in raw near-infrared spectral data. The value of p% can be selected based on experimental results and experience, typically between 70% and 80%, which preserves the main absorption features while avoiding the misselection of too many noisy segments. Therefore, the predetermined condition for determining the sensitive region for absorption peak changes is that the average response intensity of the spectral segments is within the top p%, meaning that the average response intensity of each spectral segment belonging to the sensitive region for absorption peak changes exceeds a predetermined threshold.

[0046] Furthermore, absorption structure spectra can be extracted from the original near-infrared spectrum based on the sensitive regions of absorption peak changes. This is because some weak absorption bands, although having low high-frequency energies, still possess significant chemical meaning. For example, the stretching or combinatory bands of hydroxyl (OH) commonly found in solid mineral products (approximately 7000-9000 cm⁻¹) are important. -1 4500-5500cm -1 ) and the methyl (CH) combination band (approximately 4300-4400 cm) -1 These regions typically have weak absorption intensities, and relying solely on response intensity thresholds may lead to the incorrect exclusion of such regions. To avoid the loss of critical structural spectral segments, this embodiment further introduces a set of characteristic spectral segments driven by prior chemical knowledge, in addition to the high-response spectral segments. , It is the first in the set of characteristic spectral bands q A pre-defined typical absorption band. Q This refers to the preset total number of typical absorption bands, which can be pre-set based on the typical absorption bands of known chemical groups. Specifically, the extracted absorption structure spectrum includes: In the near-infrared spectral data, all wavelengths in the spectral bands with average response intensity in the top p% are marked as 1. All wavelengths in the spectral bands with average response intensity not in the top p% but belonging to a preset typical absorption band are also marked as 1. All wavelengths in the remaining spectral bands are marked as 0, thus obtaining the high-response spectral band mask. , This represents the average response strength in the top p% of the responses:

[0047] Final spectral mask It can simultaneously cover strong-response absorption bands (data-driven) and weak-response but chemically significant characteristic bands (knowledge-driven), thereby improving the integrity of structural interval selection.

[0048] In 10000-4000 cm -1 Within this region, spectral bands with high average response intensity are typically concentrated in characteristic absorption bands, such as 6900 cm⁻¹. -1 (OH first harmonic), 5200 cm -1 (Water combination zone), 4300-4700 cm -1 (CH combination band); regions with lower response intensity correspond to smooth baselines or non-characteristic regions (e.g., 9500-8500 cm⁻¹). -1 Therefore, spectral mask The selected spectral bands are actually a collection of chemical structure sensitive bands.

[0049] Using high response spectral band mask The absorbance at wavelengths marked as 1 is extracted from near-infrared spectral data, and the absorbance at wavelengths marked as 0 is set to 0. The absorption structure spectra are then obtained by arranging the wavelengths in order. The absorption structure spectra at any wavelength... absorbance at .

[0050] The extraction process for the above absorption structure spectrum can be referred to Figure 3 By performing frequency domain feature decomposition and reconstruction through DWT to obtain high spectrum, and then using high spectrum to locate structural regions, while retaining the original near-infrared spectrum values, this approach balances "structure-oriented" and "semantic preservation".

[0051] Step 3: Randomly perturb the near-infrared spectral data to generate a structure perturbation spectrum, and fuse the absorption structure spectrum with the structure perturbation spectrum to obtain an enhanced spectrum; use the enhanced spectrum samples to expand the original spectral dataset to obtain an enhanced spectral dataset.

[0052] Coal samples of different particle sizes exhibit significant "style differences" in near-infrared spectra: finer-particle samples typically have smoother baselines and clearer absorption peaks; while coarser-particle samples, due to enhanced scattering and path effects, show overall baseline elevation, absorption peak broadening, and intensity weakening. These differences are non-chemical variations, causing significant distribution shifts between different particle size domains, and are one of the challenges in domain generalization. To mitigate the overfitting of solid mineral product quality parameter detection models to specific particle size styles, this application, based on the obtained high-response spectral mask, generates samples with different "spectral styles" by performing structural perturbations and mixing within local spectral intervals, thereby improving the robustness of solid mineral product quality parameter detection models to particle size differences.

[0053] Please refer to Figure 4 The random perturbation process shown, which generates a structured perturbation spectrum from near-infrared spectral data, includes: randomly selecting another near-infrared spectral data set from the original spectral dataset, and dividing the near-infrared spectral data into segments according to the spectral segmentation points in step 2. A series of continuous spectral bands are randomly rearranged to disrupt the continuity of their absorption structure. The rearranged spectral bands are then combined to form the structure perturbation spectrum. The structure perturbation spectrum is defined at any wavelength. absorbance at , This is another near-infrared spectral data at the wavelength point. absorbance at that point This indicates a random rearrangement operation of the spectral index.

[0054] The structural perturbation spectrum obtained in this way retains the amplitude statistical characteristics of the high-frequency components, meaning the values ​​of each absorbance sampling point are not modified. However, because the wavelength order is disrupted, the original continuity, peak position, peak width, and edge morphology of the absorption peaks are completely destroyed, resulting in the loss of the overall peak structure. Therefore, the Shuffle sample is close to the original spectrum in terms of amplitude distribution, but completely different in terms of overall morphological trend, retaining only a rough amplitude statistical characteristic.

[0055] To further obtain near-infrared spectra of samples with other particle sizes besides existing solid mineral samples, the fusion of absorption structure spectra and structure perturbation spectra includes: Linear interpolation is performed between the absorption structure spectrum and the structure perturbation spectrum to obtain the enhancement spectrum at any wavelength. absorbance at , It is the absorption structure spectrum at the wavelength point absorbance at that point The structural perturbation spectrum at the wavelength point absorbance at that point These are interpolation coefficients. The specific value is set based on the experimental results.

[0056] This operation preserves some of the original structural regions while injecting structural perturbations from other samples, effectively increasing the shape diversity of training samples and helping to improve the domain robustness of the solid mineral product quality parameter detection model. By introducing random structural perturbations at the spectral level, the solid mineral product quality parameter detection model can see multiple "shape" changes during the training phase; at the same time, fixing the quality parameter labels of structural regions allows the solid mineral product quality parameter detection model to focus its learning on absorbing structural (semantic information) while ignoring perturbation bias; thus achieving feature learning with structural invariance and improving generalization performance in different granular domains.

[0057] Step 4: Construct the network architecture of the solid mineral product quality parameter detection model and train it using the enhanced spectral dataset. The solid mineral product quality parameter detection model is used to predict the quality parameters of solid mineral products after feature extraction from near-infrared spectral data.

[0058] The structural perturbation operation generates a large number of enhanced spectral samples with different morphological changes for each original near-infrared spectral sample, significantly expanding the distribution of the training data in both the "absorption structural domain" and the "structural perturbation domain". These samples form an expanded spectral manifold, enabling the model to learn more robust regression mappings on samples of different granularities (even unknown granularities).

[0059] The enhanced spectral dataset includes original near-infrared spectral samples and enhanced spectral samples. The original near-infrared spectral samples include the original near-infrared spectral data and its corresponding quality parameter labels, while the enhanced spectral samples include the enhanced spectra constructed based on the original near-infrared spectral data and the corresponding quality parameter labels of the original near-infrared spectral data.

[0060] Considering that near-infrared spectroscopy is a typical one-dimensional long-sequence signal, its information contains significant long-range dependencies. The spacing and combination relationships between absorption peaks often reflect the cooperative absorption characteristics between chemical groups. Therefore, capturing the "global shape pattern" of the spectrum is particularly important for quality regression. Traditional convolutional neural networks extract local features from the signal through a sliding window of the convolution kernel. Their output is only the locally filtered response value, which is a feature-level representation. In the extraction process, some original morphological information and absolute absorption intensity relationships are lost. This makes it difficult for CNN models to maintain stable feature interpretability and chemical-physical consistency in scenarios across granularity or instruments. However, Transformer and MLP models based on token input have the advantage of global dependency modeling, which is beneficial for capturing the global peak shape and structural relationships of near-infrared spectra. Therefore, this application constructs a solid mineral product quality parameter detection model based on MLP or Transformer models, and trains the solid mineral product quality parameter detection model using an enhanced spectral dataset, including: The network parameters of the solid mineral product quality parameter detection model are initialized, and the model is trained using an enhanced spectral dataset. The loss function is then minimized. A solid mineral product quality parameter detection model is obtained. The training objective of the solid mineral product quality parameter detection model is to ensure the accuracy of the quality parameter prediction results of the main regression task, while enhancing the consistency of the model's output on the original samples and the enhanced samples (structural perturbation samples), thereby achieving structural invariance learning.

[0061] Specifically, the loss function for model training ,in, It is a balancing coefficient that is continuously optimized during model training. It is typically set to a smaller value in the early stages of training (e.g., ...). =0.1) to ensure the convergence of the main task, and then gradually increase it to enhance the structural robustness of learning; It is a regression loss. It is structural consistency loss; regression loss This is used to measure the prediction accuracy of a solid mineral product quality parameter detection model on raw near-infrared spectral data. Higher prediction accuracy and lower regression loss are associated with better prediction accuracy. The smaller the value, the less structural consistency is lost. This is used to measure the stability of the prediction results of a solid mineral product quality parameter detection model before and after structural disturbance. The higher the stability of the prediction results before and after structural disturbance, the lower the structural consistency loss. The smaller.

[0062] The regression loss term ensures that the model learns the correct spectral-quality parameter mapping relationship on the original data distribution, and it is the core constraint of the overall training objective. Regression Loss , N It is the total number of original near-infrared spectral samples in the original spectral dataset. It is the near-infrared spectral data of the i-th original near-infrared spectral sample in the original spectral dataset. It is the quality parameter label for the i-th raw near-infrared spectrum sample. yes Predicted values ​​of quality parameters.

[0063] To enhance structural invariance, this application designs a structural consistency loss in the spectral domain for each original near-infrared spectral sample. ,generate K One enhanced spectrum This requires the model to maintain consistency between the two predictions. The loss constraint ensures the model's output remains stable before and after structural perturbations, causing the network to tend to learn and absorb structure-dominant features while ignoring style changes caused by granularity. Specifically, this is the structural consistency loss. , K It is the near-infrared spectral data of the i-th original near-infrared spectral sample. The corresponding total number of enhanced spectra, It is the near-infrared spectral data of the i-th original near-infrared spectral sample. The corresponding j-th enhancement spectrum, yes Predicted values ​​of quality parameters It represents the 2-norm.

[0064] This training mechanism achieves two types of invariance through joint constraints: 1) structural invariance, ensuring the model's output remains stable under shape perturbations; and 2) style robustness, enabling the model to adapt to spectral morphology changes caused by different particle sizes. In the context of near-infrared spectroscopy, this is equivalent to making the model focus on chemical absorption characteristics while ignoring particle size-related scattering differences, thereby achieving robust predictions for samples with unknown particle sizes.

[0065] A solid mineral product quality parameter detection model is trained using near-infrared spectral data collected from solid mineral product samples of various particle sizes. Subsequently, when the particle size of any sample corresponding to any near-infrared spectral data to be tested falls within the particle size domain, the trained solid mineral product quality parameter detection model can be used for quality detection processing. Furthermore, the quality parameters of the solid mineral product obtained after quality detection processing exhibit high accuracy; that is, the accuracy of the quality parameters obtained through rapid detection is minimally affected by the sample particle size. In other words, the solid mineral product quality parameter detection model of this application is adaptable to near-infrared spectral data of different sample particle sizes and can effectively achieve rapid detection, improving the generalization ability of quality detection processing for near-infrared spectral data.

[0066] Training configuration parameters during model training may include: using the Adam optimizer, setting the initial learning rate to 0.001, employing the ReduceLROnPlateau learning rate decay strategy, and dynamically adjusting the learning rate based on the loss function. The batch size is 64, and the maximum number of iterations is set to 150. Specifically, after training the model once using the enhanced spectral dataset, the loss is calculated, and backpropagation is performed to optimize the model.

[0067] Step 5: Obtain the near-infrared spectral data of the solid mineral product sample to be tested, and input it into the trained solid mineral product quality parameter detection model to predict and output the predicted value of the solid mineral product quality parameter.

[0068] When testing the quality parameters of a solid mineral product sample, the quality parameters are determined based on the provided near-infrared spectral data. For example, if the sample is coal, the quality parameter can be the content of volatile matter in the sample. When the sample is of other types, the corresponding quality parameters correspond to the type of sample. The details of the corresponding quality parameters can be found here and will not be elaborated further.

[0069] The above descriptions are merely preferred embodiments of this application, and this application is not limited to the above embodiments. It is understood that other improvements and variations that can be directly derived or conceived by those skilled in the art without departing from the spirit and concept of this application should be considered to be included within the protection scope of this application.

Claims

1. A method for detecting quality parameters of solid mineral products based on NIR and particle size domain generalization, characterized in that, The method for detecting the quality parameters of solid mineral products includes: Near-infrared spectral data of different solid mineral samples with multiple particle sizes were obtained, and the original spectral dataset was constructed, with each particle size including multiple solid mineral samples. For any near-infrared spectral data, identify the sensitive region of absorption peak change in the near-infrared spectral data, and extract the absorption structure spectrum. The absorption structure spectrum indicates the absorption structure information of the chemical composition of the solid mineral product sample, and the sensitive region of absorption peak change indicates the spectral region in the near-infrared spectral data where the absorption peak change meets the predetermined conditions. The near-infrared spectral data is randomly perturbed to generate a structure perturbation spectrum, and the absorption structure spectrum is fused with the structure perturbation spectrum to obtain an enhanced spectrum; the original spectral dataset is expanded using the enhanced spectrum samples to obtain an enhanced spectral dataset; A network architecture for a solid mineral product quality parameter detection model is constructed, and the model is trained using an enhanced spectral dataset. The solid mineral product quality parameter detection model is used to predict the quality parameters of solid mineral products after feature extraction from near-infrared spectral data. Near-infrared spectral data of solid mineral product samples to be inspected are obtained and input into the trained solid mineral product quality parameter detection model to predict and output the predicted values ​​of solid mineral product quality parameters.

2. The method for detecting quality parameters of solid mineral products according to claim 1, characterized in that, Sensitive regions for identifying changes in absorption peaks in near-infrared spectral data include: Frequency domain feature decomposition is performed on near-infrared spectral data to obtain high-frequency and low-frequency components. The high-frequency components characterize the components in the near-infrared spectral data that change rapidly with wavelength, while the low-frequency components characterize the components in the near-infrared spectral data that change slowly with wavelength. The high-frequency components of near-infrared spectral data are reconstructed to obtain the high spectrum, and the high spectrum is then divided into... A continuous spectral band, the high spectrum characterizes the chemical bond composition and absorption characteristics of solid mineral products, and the high spectrum is available at any wavelength. amplitude at , It is the first wavelength point. j Layer high frequency components, It is a high-frequency initiation layer. J It is the highest frequency layer; Calculate the average response intensity for each spectral band and identify the spectral bands with the average response intensity in the top p% as the sensitive regions for absorption peak changes, using integer parameters. >

1.

3. The method for detecting quality parameters of solid mineral products according to claim 2, characterized in that, Calculate the average response intensity of any spectral band i include: Determine any wavelength point within spectral band i Response intensity value at , It is the high spectrum at the wavelength point The amplitude at that point, , , All are weighting coefficients. express The first derivative, express The second derivative; The average response intensity of spectral band i is determined based on the response intensity values ​​at each wavelength point within spectral band i. , It is the length of spectral segment i.

4. The method for detecting quality parameters of solid mineral products according to claim 2, characterized in that, Divide the high spectrum into The continuous spectral bands include: Calculate the second and third derivatives at each wavelength point in the near-infrared spectral data, and determine the wavelength points where the second derivative is 0 and the third derivative is not 0 as the spectral segment division points; The high spectrum is divided according to the spectral segment division points. A continuous spectral band is formed by any two adjacent spectral band division points and all wavelength points between them.

5. The method for detecting quality parameters of solid mineral products according to claim 2, characterized in that, The extracted absorption structure spectra include: In the near-infrared spectral data, all wavelengths in the spectral bands with average response intensity in the top p% are marked as 1. All wavelengths in the spectral bands with average response intensity not in the top p% but belonging to a preset typical absorption band are also marked as 1. All wavelengths in the remaining spectral bands are marked as 0, thus obtaining the high-response spectral band mask. ; Using high response spectral band mask The absorbance at the wavelength points marked as 1 is extracted from the near-infrared spectral data, the absorbance at the wavelength points marked as 0 is set to 0, and the absorption structure spectrum is obtained by arranging the wavelength points in order.

6. The method for detecting quality parameters of solid mineral products according to claim 4, characterized in that, Random perturbation of near-infrared spectral data to generate structural perturbation spectra includes: Another near-infrared spectral data set is randomly selected from the original spectral dataset, and the near-infrared spectral data is divided according to the spectral segmentation points. A series of continuous spectral segments are used, and the order of each spectral segment is randomly rearranged. The rearranged spectral segments are then combined to form the structural perturbation spectrum.

7. The method for detecting quality parameters of solid mineral products according to claim 1, characterized in that, The fusion of the absorption structure spectrum and the structure perturbation spectrum includes: Linear interpolation is performed between the absorption structure spectrum and the structure perturbation spectrum to obtain the enhancement spectrum at any wavelength. absorbance at , It is the absorption structure spectrum at the wavelength point absorbance at that point The structural perturbation spectrum at the wavelength point absorbance at that point These are the interpolation coefficients.

8. The method for detecting quality parameters of solid mineral products according to claim 1, characterized in that, A solid mineral product quality parameter detection model is constructed based on an MLP or Transformer model, and the trained solid mineral product quality parameter detection model includes: The network parameters of the solid mineral product quality parameter detection model are initialized, and the model is trained using an enhanced spectral dataset. The loss function is then minimized. The solid mineral product quality parameter detection model is obtained; The loss function ,in, It is the balance coefficient. It is a regression loss. It is structural consistency loss; regression loss This is used to measure the prediction accuracy of a solid mineral product quality parameter detection model on raw near-infrared spectral data. Higher prediction accuracy and lower regression loss are associated with better prediction accuracy. The smaller the value, the less structural consistency is lost. This is used to measure the stability of the prediction results of a solid mineral product quality parameter detection model before and after structural disturbance. The higher the stability of the prediction results before and after structural disturbance, the lower the structural consistency loss. The smaller.

9. The method for detecting quality parameters of solid mineral products according to claim 7, characterized in that, The enhanced spectral dataset includes original near-infrared spectral samples and enhanced spectral samples. The original near-infrared spectral samples include original near-infrared spectral data and their corresponding quality parameter labels. The enhanced spectral samples include enhanced spectra constructed based on the original near-infrared spectral data and the quality parameter labels corresponding to the original near-infrared spectral data. The regression loss , N It is the total number of original near-infrared spectral samples in the original spectral dataset. It is the near-infrared spectral data of the i-th original near-infrared spectral sample in the original spectral dataset. It is the quality parameter label for the i-th raw near-infrared spectrum sample. yes Predicted values ​​of quality parameters; The structural consistency loss , K It is the near-infrared spectral data of the i-th original near-infrared spectral sample. The corresponding total number of enhanced spectra, It is the near-infrared spectral data of the i-th original near-infrared spectral sample. The corresponding j-th enhancement spectrum, yes Predicted values ​​of quality parameters This represents the 2-norm.

10. The method for detecting quality parameters of solid mineral products according to claim 2, characterized in that, The near-infrared spectral data is decomposed in the frequency domain using the discrete wavelet transform method to obtain the high-frequency and low-frequency components in the near-infrared spectral data. The higher the resolution of the near-infrared spectral data, the more decomposition layers the discrete wavelet transform has.

Images