A rapid detection method for effective components of compound fertilizer by near infrared spectroscopy

Abstract

The application discloses a kind of composite fertilizer effective component near-infrared spectrum rapid detection method, belong to the cross technical field of agricultural detection technology, spectral analysis and artificial intelligence.It includes the original near-infrared spectrum data of composite fertilizer sample and its corresponding standard chemical determination value, constructs multilevel spectral database, constructs rapid detection model based on the improved one-dimensional convolution neural network model optimized by improved particle swarm optimization algorithm, based on the preset data set, the rapid detection model is trained, the near-infrared spectrum data to be measured is input into the rapid detection model after training, and the effective component content of composite fertilizer is obtained.The application constructs high diversity spectral database by data enhancement, improves the robustness of model against physical interference;Improved one-dimensional convolution neural network can accurately mine deep spectral features;Improved particle swarm optimization algorithm is introduced to optimize network initial parameters, improve training stability and convergence efficiency, realize the detection of composite fertilizer effective component.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of agricultural testing technology, spectral analysis, and artificial intelligence, specifically to a rapid near-infrared spectral detection method for the effective components of compound fertilizers. Background Technology

[0002] The content of nitrogen (N), phosphorus (P2O5), and potassium (K2O) in compound fertilizers is a core indicator for measuring their quality. Currently, although the chemical testing methods stipulated by national standards (such as the Kjeldahl method) are accurate, they have inherent defects such as complex procedures, time-consuming and labor-intensive processes, sample destruction, and chemical pollution, which cannot meet the needs of rapid monitoring on the production line or on-site screening in the field.

[0003] Near-infrared spectroscopy, as a rapid and non-destructive analytical technique, has been introduced into this field. However, its practical application faces three major bottlenecks: First, the complex matrix of compound fertilizers, and even small changes in its physical state (such as particle size, density, and moisture) can lead to significant nonlinear distortions in the spectrum, resulting in severely insufficient prediction accuracy and robustness of traditional linear correction models (such as partial least squares, PLS). Second, establishing robust models requires a large number of samples covering various variations, but the cost of calibrating actual chemical values ​​is high, leading to a scarcity of high-quality training data. Existing data augmentation methods are mostly limited to simple numerical transformations, making it difficult to simulate real physical disturbances or generate chemically reasonable samples. Finally, while deep learning models (such as one-dimensional convolutional neural networks) possess powerful potential for learning nonlinear features, their performance is highly dependent on randomly initialized parameters, making them prone to getting trapped in local optima, and their convergence process is unstable. Furthermore, how to balance multi-scale feature extraction and key information focusing in model design still needs optimization.

[0004] To address the aforementioned issues, there is an urgent need for a rapid near-infrared spectroscopy method for detecting the effective components of compound fertilizers, which can solve the problems associated with traditional methods. Summary of the Invention

[0005] The purpose of this invention is to provide a rapid near-infrared spectral detection method for the effective components of compound fertilizers. A highly diverse spectral database is constructed through both physical and implicit data augmentation, significantly improving the model's robustness against physical interference. An improved one-dimensional convolutional neural network is designed to accurately mine deep spectral features. An improved particle swarm optimization algorithm is introduced to optimize the network's initial parameters, effectively improving training stability and convergence efficiency. The entire method achieves rapid, high-precision end-to-end detection from spectrum to components, providing reliable technical support for on-site monitoring of compound fertilizer quality.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A rapid near-infrared spectroscopy method for detecting effective components in compound fertilizers includes:

[0008] Step 1: Collect raw near-infrared spectral data of compound fertilizer samples and their corresponding standard chemical determination values ​​to construct a multi-level spectral database;

[0009] Step 2: Construct a fast detection model based on an improved one-dimensional convolutional neural network model optimized by an improved particle swarm optimization algorithm. The improved one-dimensional convolutional neural network includes an input layer, a first convolutional layer, an ECA layer, a first max pooling layer, a second convolutional layer, a second max pooling layer, a third convolutional layer, a third max pooling layer, a fourth convolutional layer, an Inception layer, a fourth max pooling layer, a fully connected layer, and an output layer.

[0010] Step 3: Train the fast detection model based on the pre-set dataset;

[0011] Step 4: Input the near-infrared spectral data to be tested into the trained rapid detection model to obtain the effective component content of the compound fertilizer.

[0012] Further, in step 1, the raw near-infrared spectral data of the compound fertilizer samples and their corresponding standard chemical determination values ​​are collected to construct a multi-level spectral database, specifically as follows:

[0013] Collect the raw near-infrared spectral data of compound fertilizer samples and their corresponding standard chemical determination values, and use them as raw data pairs;

[0014] The original data pairs are processed using a strategy that combines physical-guided generation with implicit feature space enhancement to generate enhanced spectral data and their corresponding component labels. The enhanced spectral data and their corresponding component labels are then merged to form a multi-level spectral database.

[0015] Furthermore, the physical guidance generation strategy is as follows:

[0016] Based on the Kubelka-Munk physical optics model, the original near-infrared spectral data is transformed into an optical feature space, and enhanced spectral data simulating different physical states are generated by using a conditional generative adversarial network and combining preset physical state parameters.

[0017] Furthermore, the implicit feature space enhancement strategy is specifically as follows:

[0018] The original spectrum is mapped to a low-dimensional latent space based on a variational autoencoder. New latent variables are generated by performing linear interpolation or adding controllable noise that conforms to its distribution in the low-dimensional latent space. The data is then reconstructed into enhanced spectral data by a decoder.

[0019] Furthermore, the first, second, third, and fourth convolutional layers are all two-layer one-dimensional convolutions. The first convolutional layer has 16 kernels, a kernel size of 16, a stride of 2, and uses the ReLU activation function. The second convolutional layer has 32 kernels, a kernel size of 8, a stride of 2, and uses the ReLU activation function. The third convolutional layer has 64 kernels, a kernel size of 4, a stride of 2, and uses the ReLU activation function. The fourth convolutional layer has 128 kernels, a kernel size of 2, a stride of 1, and uses the ReLU activation function. The stride of the first, second, third, and fourth max-pooling layers is 2.

[0020] Furthermore, the improved particle swarm optimization algorithm includes the following improvements: adding a selection strategy based on the quartiles of particle fitness values ​​to maintain the quality of the population; and introducing an adaptive perturbation strategy based on the ratio of individual fitness values ​​to the average fitness value of the population to enhance the algorithm's ability to escape local optima.

[0021] Furthermore, in step 3, the fast detection model is trained based on a pre-set dataset, specifically as follows:

[0022] The multi-level spectral database is divided into a training set, a validation set, and a test set. Both the training set and the validation set contain enhanced spectral data and their strictly corresponding component labels.

[0023] An improved particle swarm optimization algorithm is used to globally optimize the key initial parameters of the fast detection model to obtain optimized initial weights and biases.

[0024] Starting with optimized initial model weights and biases, the fast detection model is trained under supervision using the training set. During training, multi-task weighted mean square error is used as the loss function, and the validation set is used to monitor the training process and implement an early stopping strategy to select the optimal model.

[0025] The performance of the optimal model is evaluated using a test set to obtain the final prediction model after training.

[0026] Furthermore, the supervised training employs the AdamW optimizer, along with a cosine annealing preheating learning rate scheduling strategy.

[0027] Furthermore, the multi-task weighted mean square error loss function is a weighted sum of the mean square errors of each component prediction, and the weights are set according to the content range or prediction difficulty of each component.

[0028] Furthermore, the early stopping strategy is as follows: when the loss function value on the validation set no longer decreases within a series of preset training cycles, training is terminated and the model parameters are rolled back to the state where the validation set performance is optimal.

[0029] In summary, the present invention has at least one of the following beneficial technical effects:

[0030] 1. Fundamental Improvement in Data Quality and Diversity: A multi-level spectral database is systematically constructed by fusing a dual strategy of physically guided generation and implicit feature space enhancement. This strategy not only expands the data scale but, more importantly, injects prior physical knowledge (simulating real physical state interference) and the inherent distribution laws of the data (generating new samples with continuous and reasonable chemical composition). This fundamentally solves the problem of weak model generalization ability caused by the limited actual calibration samples and the variability of physical states, laying a solid foundation for training highly robust models.

[0031] 2. Superior Feature Extraction and Focusing Capabilities: The improved one-dimensional convolutional neural network model extracts spectral features from contours to details progressively through a multi-level convolutional structure (first to fourth convolutional layers). An ECA (Effective Channel Attention) layer is introduced, enabling the model to adaptively enhance key feature channels related to nitrogen, phosphorus, and potassium components while suppressing noise. An Inception layer is used to fuse convolutional features at different scales in parallel, ensuring the model can simultaneously capture subtle local changes and global contextual information. This structural design significantly enhances the model's ability to deeply mine and intelligently focus on complex and overlapping spectral information.

[0032] 3. Model Training Starting Point and Stability Optimization: An improved particle swarm optimization algorithm is applied to the parameter initialization stage of the deep learning model. This algorithm efficiently performs global optimization through a quartile-based selection strategy and an adaptive perturbation strategy based on the individual-to-group fitness ratio, providing the network with a significantly better starting point than random initialization. This effectively avoids the problem of getting stuck in local optima and slow convergence in the early stages of model training, ensuring rapid and stable training and improving the performance ceiling of the final model.

[0033] 4. The training process is scientifically rigorous, ensuring reliable generalization performance: The entire training process employs advanced strategies such as multi-task weighted loss functions, the AdamW optimizer, and cosine annealing warm-up learning rate scheduling to guarantee training efficiency. Independent validation set monitoring and early stopping strategies effectively prevent overfitting. The final model is evaluated on a completely independent test set, ensuring the objectivity and unbiasedness of performance validation, and the resulting model possesses strong generalization capabilities for practical applications.

[0034] 5. Integrated end-to-end process, in line with the trend of intelligent detection: This invention forms a complete, closed-loop, and highly automated solution from data preparation, model building, optimization initialization to training and verification. This method requires no complex preprocessing, deeply integrating artificial intelligence algorithms into the entire spectral analysis process, achieving rapid end-to-end detection from spectral input to component content output. It is particularly suitable for embedding in portable devices or online detection systems, contributing to the development of smart agriculture. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0036] Figure 2 A schematic diagram of the improved one-dimensional convolutional neural network model structure;

[0037] Figure 3 This is a schematic diagram of the ECA layer structure. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0039] like Figure 1 As shown, this invention provides a rapid near-infrared spectroscopy method for detecting the effective components of compound fertilizers, comprising:

[0040] Step 1: Collect raw near-infrared spectral data of compound fertilizer samples and their corresponding standard chemical determination values ​​to construct a multi-level spectral database;

[0041] Step 2: Construct a fast detection model based on an improved one-dimensional convolutional neural network model optimized by an improved particle swarm optimization algorithm. The improved one-dimensional convolutional neural network includes an input layer, a first convolutional layer, an ECA layer, a first max pooling layer, a second convolutional layer, a second max pooling layer, a third convolutional layer, a third max pooling layer, a fourth convolutional layer, an Inception layer, a fourth max pooling layer, a fully connected layer, and an output layer.

[0042] Step 3: Train the fast detection model based on the pre-set dataset;

[0043] Step 4: Input the near-infrared spectral data to be tested into the trained rapid detection model to obtain the effective component content of the compound fertilizer.

[0044] In step 1, raw near-infrared spectral data of compound fertilizer samples and their corresponding standard chemical determination values ​​are collected to construct a multi-level spectral database, specifically as follows:

[0045] Step 101: Collect the raw near-infrared spectral data of the compound fertilizer sample and its corresponding standard chemical determination value, and use it as the raw data pair;

[0046] Step 102: Process the original data pairs based on a strategy that combines physical-guided generation with implicit feature space enhancement to generate enhanced spectral data and their corresponding component labels. Merge the enhanced spectral data and their corresponding component labels to form a multi-level spectral database.

[0047] First, it should be noted that the data collection process in step 101 is a conventional technical method, which will not be elaborated on here. The focus will be on step 102, which is as follows:

[0048] Define the data collected in step 101 as the input for step 102:

[0049] Original near-infrared diffuse reflectance spectral matrix X original : Dimension is [N samples L wavelengths ]. Wherein, N samples L represents the number of representative compound fertilizer samples initially collected. wavelengths This represents the number of wavelength points collected within a specific wavelength range (e.g., 900-1700nm).

[0050] Corresponding standard chemical truth matrix Y original : Dimension is [N samples [3], each row vector represents the mass percentage of total nitrogen (N), available phosphorus (P2O5), and potassium oxide (K5O) in the corresponding sample, accurately determined by national standard chemical methods (such as Kjeldahl method, vanadium molybdenum yellow colorimetric method, and flame photometry). This matrix constitutes the original label for supervised learning.

[0051] Processing flow:

[0052] This step enhances the original data pair (X) using the following two core modules. original Y original The data is processed to generate large-scale augmented data, namely:

[0053] 1. Physics-guided generative enhancement

[0054] The aim is to explicitly simulate near-infrared spectral nonlinear distortions caused by changes in sample physical properties (such as particle size distribution, bulk density, and ambient humidity), thereby improving the model's generalization ability to physical disturbances in real-world applications. Specifically, this includes:

[0055] (1) Physical Optical Feature Extraction: First, each raw diffuse reflectance spectrum R is input into a physical converter based on the Kubelka-Munk (KM) theory. This converter performs the conversion from the observed diffuse reflectance R to the KM function F(R) = (1-R), which is more directly related to the intrinsic optical parameters of the sample (absorption coefficient K and scattering coefficient S). 2 The calculation of / (2R). This step transforms the spectral data into a feature space that is more clearly influenced by the physical state.

[0056] (2) Conditional Generative Adversarial Network (cGAN) Modeling and Generation: Construct a conditional generative adversarial network. During the training phase, the original data pairs (F(R), P) are used. original The network is trained using P(R), where P(R) = ... original The physical state parameters of the original samples (which can be estimated or set as baseline values) are used. The generator G learns the mapping from optical features F(R) and physical conditions P to the observed spectrum R. During the enhancement phase, the trained generator G is fixed. For each original spectrum, the KM features F(R) are... i ), by systematically enumerating a set of physical state parameters {P} covering the actual range of variation. target_j (e.g., different particle size indices, density simulation values), and compare them with F(R) i The inputs are combined with the generator G to generate a series of simulations in P. target_j Physically plausible enhanced spectra X measured under certain conditions phys_ij The chemical composition of these generated spectra is labeled Y. phys_ij Y inherits from its source sample i original_i Because changes in physical state do not alter chemical composition.

[0057] 2. Implicit Feature Space Augmentation

[0058] The aim is to generate new samples with continuous and reasonable chemical composition and subtle spectral variations by interpolating and perturbing the feature space through mining the inherent manifold structure of the original spectral data. Specifically, this includes:

[0059] (1) Construction of spectral semantic latent space: A variational autoencoder (VAE) is used to construct the original spectrum X. original Unsupervised learning is performed. The encoder of the VAE compresses the high-dimensional spectrum into a low-dimensional, continuous latent variable z, assuming that z follows a multivariate Gaussian distribution (outputting the mean vector μ and variance vector σ). The decoder is then responsible for reconstructing the spectrum from the latent variable z.

[0060] (2) Latent space manipulation and sample generation:

[0061] Linear interpolation: In the latent space, the latent variable z of two different samples is randomly selected. a and zb Calculate its linear interpolation points , where λ is a random number in the interval (0, 1). Let z interp Input to the decoder to generate a new spectrum x interp Its corresponding chemical component label y interp Set as This enables a continuous transition in chemical composition.

[0062] Controlled noise injection: Random noise is added to the latent variable distribution mean μ of a single sample within a range controlled by its standard deviation σ, resulting in... , where ε follows a standard normal distribution, ⊙ denotes element-wise multiplication, and α is a scaling factor controlling the intensity of the perturbation. Let z new The input decoder generates a new spectrum x that is similar to the original sample but with subtle, reasonable variations. aug Its chemical composition label y aug Inherit the tags directly from the source sample.

[0063] 3. Data fusion, standardization, and output:

[0064] (1) Data merging: Merging the original spectral data X original Physically guided augmented dataset X phys and dataset X generated by implicit feature space augmentation sem The samples are merged to form a preliminary enhanced spectral set X. total Simultaneously, the corresponding label matrix Y is... original Y phys Y sem Merge into Y total .

[0065] (2) Standardization: For the merged X total A comprehensive normalization process (e.g., using Standard Normal Transform (SNV)) is performed to eliminate baseline drift and light scattering effects, resulting in the final spectral data matrix X. final The corresponding component label matrix Y final It remains unchanged.

[0066] (3) Database structure: consisting of X final and Y final The resulting multi-level spectral database is the final output of this step. The sample size M of this database is... samples Much larger than the original sample size N samples Each sample consists of an enhanced spectral data line and its precise labeling of the content of the three active ingredients.

[0067] In step 2, a fast detection model is constructed based on an improved one-dimensional convolutional neural network model optimized by the improved particle swarm optimization algorithm, specifically as follows:

[0068] First, the input and output of the fast detection model are explained. The input is preprocessed near-infrared spectral data from a multi-level spectral database, with a dimension of [1, L], where L is the number of spectral wavelengths. For batch processing, the dimension is [Batch]. Size The output of [L] is a three-dimensional regression vector. , respectively, represent the mass percentages of total nitrogen, available phosphorus (as P2O5), and potassium oxide (as K2O) predicted by the model.

[0069] The improved one-dimensional convolutional neural network model will be introduced next, and its structure is as follows: Figure 2 As shown, they include:

[0070] The improved one-dimensional convolutional neural network includes an input layer, a first convolutional layer, an ECA layer, a first max pooling layer, a second convolutional layer, a second max pooling layer, a third convolutional layer, a third max pooling layer, a fourth convolutional layer, an Inception layer, a fourth max pooling layer, a fully connected layer, and an output layer.

[0071] The first, second, third, and fourth convolutional layers are all two-layer one-dimensional convolutions. The first convolutional layer has 16 kernels, a kernel size of 16, a stride of 2, and uses the ReLU activation function. The second convolutional layer has 32 kernels, a kernel size of 8, a stride of 2, and uses the ReLU activation function. The third convolutional layer has 64 kernels, a kernel size of 4, a stride of 2, and uses the ReLU activation function. The fourth convolutional layer has 128 kernels, a kernel size of 2, a stride of 1, and uses the ReLU activation function. The stride of the first, second, third, and fourth max-pooling layers is 2.

[0072] 1. Input layer: Receives raw or preprocessed spectral vectors.

[0073] 2. First Convolutional Layer: Consists of two cascaded one-dimensional convolutional layers. The first sub-layer has 16 kernels of size 16 and a stride of 2, using the ReLU activation function; the second sub-layer maintains the same configuration. This layer is responsible for extracting preliminary spectral contour features with a relatively large receptive field.

[0074] 3. ECA Layer (Effective Channel Attention Layer): Embedded immediately after the first convolutional layer. This module calculates an importance weight for each channel of the input feature map. It first obtains the global descriptor for each channel through one-dimensional global average pooling, then generates a channel weight vector through a cross-channel interaction implemented by one-dimensional convolution without dimensionality reduction. After normalization using the Sigmoid function, the feature map is reweighted. This layer enables the model to adaptively enhance feature channels sensitive to nitrogen, phosphorus, and potassium, while suppressing irrelevant or noisy channels. Its structural diagram is shown below. Figure 3 As shown.

[0075] 3. First max pooling layer: with a stride of 2, used to downsample the attention-weighted features, reduce data dimensionality and enhance the translation invariance of features.

[0076] 4. Second Convolutional Layer: The structure is the same as the first convolutional layer, but the parameters are upgraded. It consists of two cascaded one-dimensional convolutional layers with 32 kernels of size 8, a stride of 2, and the activation function ReLU. This layer extracts more refined spectral patterns in a higher-dimensional feature space.

[0077] 5. Second max pooling layer: step size is 2, further downsampling.

[0078] 6. Third Convolutional Layer: Further deepening feature extraction, this layer consists of two cascaded one-dimensional convolutional layers with 64 kernels of size 4, a stride of 2, and the activation function ReLU. This layer is dedicated to capturing subtle changes in the local spectrum.

[0079] 7. Third max pooling layer: step size is 2.

[0080] 8. Fourth Convolutional Layer: Consists of two cascaded one-dimensional convolutional layers with 128 kernels of size 2 and a stride of 1. The activation function is ReLU. The smaller kernels and stride of 1 allow the model to fine-tune and integrate high-dimensional features at deeper network layers.

[0081] 9. Inception Layer: This layer is crucial for the model to achieve multi-scale feature fusion. It contains four parallel branches:

[0082] Branch 1: 1×1 convolution, used for cross-channel information integration and dimensionality reduction.

[0083] Branch 2: Cascaded 1×1 and 3×1 convolutions capture local adjacent features.

[0084] Branch 3: Cascaded 1×1 and 5×1 convolutions capture a wider range of contextual features.

[0085] Branch 4: 3×1 max pooling followed by 1×1 convolution provides a downsampled feature abstraction.

[0086] The outputs of all branches are concatenated along the channel dimension, fusing spectral feature information at different scales.

[0087] 10. Fourth max pooling layer: with a stride of 2, performs final spatial downsampling on the rich features output by the Inception layer.

[0088] 11. Fully Connected Layers: After flattening the high-dimensional feature map output from the pooling layers, one or more fully connected layers are connected. These layers are responsible for nonlinearly combining and mapping the extracted abstract spectral features, preparing for the final regression prediction. Dropout layers are typically introduced to prevent overfitting.

[0089] 12. Output layer: A linear layer with 3 neurons that directly outputs the predicted values ​​of nitrogen, phosphorus and potassium.

[0090] The improved particle swarm optimization algorithm will be described in detail below:

[0091] It should be noted that since the standard particle swarm optimization (IPSO) algorithm is existing technology, it will not be described in detail here. Instead, its improvements will be discussed in detail below:

[0092] Using fixed inertia weights in the standard Particle Swarm Optimization (PSO) algorithm can easily lead to local optima. Therefore, many researchers have focused on improving the search capabilities of PSO to ensure good local search ability in the early stages of iteration and good global search ability in the later stages. This invention, therefore, adds a quartile-based selection strategy to the standard PSO algorithm.

[0093] First, the initial population is determined, and the fitness value of each particle is calculated. Second, the particles are sorted in ascending order based on their fitness values, and their corresponding positions and velocities are indexed. Finally, only individuals with fitness values ​​from the lowest to the third quartile are updated, while individuals with fitness values ​​from the third quartile to the highest are discarded. This ensures good population fitness and maintains population diversity.

[0094] The adaptability of genetic algorithms is mainly reflected in the fact that individuals in the population can adjust crossover and mutation probabilities according to different fitness levels. When the algorithm gets stuck in a local optimum, this operator will adjust the positions of all individuals according to certain probabilities. p m Variations occur, but the individual's historical best position remains unchanged. In statistics, the difference between the third quartile and the first quartile is also called the interquartile range (IQR). The IQR value... d IQRThe smaller the value, the more concentrated the data, and the more concentrated the fitness of the population. When the fitness value of the mutated individual... f ≤ Average fitness value f avg When the value is zero, it indicates that the overall fitness of the population is good. In this case, a smaller weight should be assigned to preserve the goodness. This invention sets an adaptive perturbation probability, calculated using the following formula:

[0095] (1)

[0096] In the formula, the subscript IQP represents the interquartile range; f max and f avg These represent the maximum fitness and average fitness of the population. f It indicates the fitness of an individual.

[0097] The relative search state of a particle within a population can be characterized by the ratio of its own fitness value to the population's average fitness value. For a minimization problem, a larger ratio indicates that the particle is far from the population's optimal position, while a smaller ratio indicates that the particle is close to the population's optimal position. An adaptive perturbation operation can be used to ensure that the perturbation force is larger when the particle is far from its global optimum and smaller when the distance is small, thereby achieving the goal of approaching the population's optimal position. Therefore, this invention uses an adaptive perturbation operation based on the ratio of the individual's own fitness value to the population's average fitness value after the selection operation. The perturbation method is as follows:

[0098] (2)

[0099] (3)

[0100] In the formula, ξ It adapts to changes in the fitness ratio.

[0101] Next, this invention will describe the specific steps of the improved PSO algorithm, which include:

[0102] (1) Algorithm parameter initialization;

[0103] (2) Initialize the position of the particle x i and speed v i ;

[0104] (3) Calculate the fitness function value of each particle and find the historical best value of each particle in the current search process. pbest ;

[0105] (4) Find the global optimal solution obtained by searching all particles. gbest ;

[0106] (5) Adjust the position and velocity of each particle;

[0107] (6) Perform selection operations and adaptive perturbation operations;

[0108] (7) Calculate the fitness value of each particle in the population, compare it with the individual historical best and the global historical best, and update the optimal solution;

[0109] (8) Determine whether the algorithm meets the termination condition. If it does, the algorithm stops; otherwise, return to step (3).

[0110] In step 3, the fast detection model is trained based on a pre-set dataset, specifically as follows:

[0111] Step 301: Divide the multi-level spectral database into a training set, a validation set, and a test set. Both the training set and the validation set contain enhanced spectral data and their strictly corresponding component labels, wherein:

[0112] (1) Training set: Used for iterative updates and learning of model weights. It typically accounts for 60%-70% of the total number of samples in the database and includes the spectral matrix X. train and label matrix Y train .

[0113] (2) Validation set: Used to monitor the model's performance on unseen data in real time during training, and to implement hyperparameter fine-tuning and early stopping strategies accordingly. It typically accounts for 15%-20% and includes (X... val Y val ).

[0114] (3) Test set: Used to objectively evaluate the generalization ability of the model after all training is completed. It typically accounts for 15%-20% and includes (X... test Y test ).

[0115] The partitioning process must employ stratified random sampling to ensure that the training, validation, and test sets maintain consistency with the overall statistical characteristics of the original database in terms of sample source, physical state, and numerical distribution of component labels, thereby avoiding bias introduced by data partitioning. This validation set must be completely independent of the fixed validation set used to improve the fitness evaluation of the particle swarm optimization algorithm to ensure the fairness of the evaluation.

[0116] Step 302: Based on the improved particle swarm optimization algorithm, the key initial parameters of the fast detection model are globally optimized to obtain the optimized initial weights and biases of the model, specifically:

[0117] 1. Parameter Encoding and Population Initialization: The parameters of the model to be optimized are expanded and concatenated into a high-dimensional real-valued vector, which serves as the position of a "particle" in the improved particle swarm optimization algorithm. A certain number of particles are randomly initialized to form the initial population;

[0118] 2. Fitness Function Design: The particle position is decoded into a specific set of network parameters, and a brand-new network model is initialized using these parameters. Then, the network is trained iteratively (e.g., 5-10 epochs) a small, fixed validation set (pre-defined from the database in step 1, different from the validation set used in subsequent training). After training, the reciprocal of the multi-task weighted mean square error of the network on the validation set is calculated as the fitness value of the particle. The smaller the error, the higher the fitness.

[0119] 3. After the algorithm terminates, the globally optimal particle ( gbest By decoding the position vector, a set of initial network parameters optimized globally can be obtained. This set of parameters is used as the initial state of the fast detection model, thus obtaining a training version of the fast detection model with an excellent starting point.

[0120] Step 303: Starting with the optimized initial weights and biases of the model, supervise the training of the fast detection model using the training set. During training, multi-task weighted mean square error is used as the loss function, and the validation set is used to monitor the training process and implement an early stopping strategy to select the optimal model. Specifically:

[0121] Starting with the optimized model output from step 302, using the training set (X) train Y train ) and validation set (X val Y val The supervised training process includes the following steps:

[0122] 1. Loss function definition: A multi-task weighted mean squared error loss function is adopted, with the total loss L... total Defined as the weighted sum of the predicted values ​​of each component and the mean squared errors of the true labels, it is: ;

[0123] Among them, w N w P w K The weighting coefficients can be set based on the common content range (scale) of total nitrogen, available phosphorus, and potassium oxide in actual compound fertilizers or the prediction difficulty shown in historical modeling, so that the model pays balanced attention to all three regression tasks during training.

[0124] 2. Optimizer and Learning Rate Scheduling Configuration: The AdamW optimizer is used. AdamW decouples the weight decay term from the standard Adam algorithm, enabling more effective regularization and preventing overfitting. The initial learning rate is set to a small value (e.g., 0.001 or 0.0003).

[0125] 3. Use a cosine annealing preheating learning rate scheduling strategy. In the early stages of training, the learning rate linearly increases from a very small value to the initial learning rate (warm-up phase), which helps stabilize the training. After warm-up, the learning rate smoothly decays from the initial value to near zero according to a cosine function. This scheduling method facilitates fine-tuning of the model's parameters in the later stages of training, promoting stable convergence.

[0126] 4. Training loop and validation monitoring:

[0127] During each training cycle, batches of data are sampled sequentially or randomly from the training set for forward propagation, loss calculation, backpropagation, and parameter updates.

[0128] After each training cycle, the complete validation set (X) is used. val Y val Perform a forward propagation and calculate the verification loss.

[0129] 5. Early Stopping Strategy Implementation: Employ an early stopping strategy. Set a "patience" value. When the validation set loss does not decrease within a certain number of training epochs consecutively reaching the "patience" value, it is determined that the model performance has ceased to improve, and continued training may lead to overfitting to the training set. At this point, the training cycle is terminated immediately.

[0130] Step 304: Evaluate the performance of the optimal model using the test set to obtain the final prediction model after training, specifically:

[0131] After training is terminated due to the early stopping strategy, the model from the last training cycle is not used. Instead, the model checkpoint saved when the model performed best on the validation set is loaded and restored. This model is considered the best candidate model obtained during the training phase.

[0132] Subsequently, this optimal model was applied to a completely independent test set (X). test Y test On the test set, predictive metrics such as root mean square error and coefficient of determination R are calculated. 2 The model's true generalization ability is obtained through methods such as mean absolute error. This evaluation result is used to confirm whether the model's performance meets the standards for practical applications.

[0133] To verify the effectiveness, practicality, and authenticity of this invention, the following description is provided in conjunction with specific embodiments.

[0134] 1. Sample and Data Preparation

[0135] Compound fertilizer samples were collected from major fertilizer manufacturers in Inner Mongolia Autonomous Region and surrounding areas, covering different brands, formulas (such as N-P-K ratios of 15-15-15, 17-17-17, etc.), and production batches, totaling 300 original samples. The diffuse reflectance spectra of each sample were acquired using a Fourier transform near-infrared spectrometer in the 900-1700 nm wavelength range. Simultaneously, the total nitrogen, available phosphorus, and potassium oxide contents of each sample were strictly determined according to national standard methods as standard chemical values.

[0136] 2. Construction of a multi-level spectral database

[0137] The aforementioned 300 original data pairs are input into the data augmentation process described in this invention. First, using a physically guided generation strategy (based on KM theory and conditional generative adversarial networks), spectral variations under five different granularity distributions and three humidity conditions are simulated to generate 4500 enhanced spectra. Second, using an implicit feature space augmentation strategy (based on variational autoencoders), a large amount of linear interpolation and noise injection are performed in the latent space to generate 4500 enhanced spectra. All generated spectra are merged with the original spectra and standardized using Standard Normal Variable Transform (SNV), ultimately constructing a multi-level spectral database containing approximately 9300 samples, each sample containing one spectral data point and three corresponding component ground truth values.

[0138] 3. Model building and initialization optimization

[0139] An improved one-dimensional convolutional neural network model was constructed according to the structure described in this invention. Subsequently, an improved particle swarm optimization algorithm was applied to optimize the key initial parameters of the model. The particle swarm size was set to 50, with 100 iterations, and a fixed independent validation set containing 300 samples was used to evaluate the fitness. After IPSO optimization, a set of optimized initial model weights and biases was obtained.

[0140] 4. Model Training and Evaluation

[0141] The constructed multi-level spectral database was randomly divided into training, validation (for training monitoring), and test sets in a 60%:20%:20% ratio. Supervised training was performed on the training set using the optimized initial model parameters as a starting point. The AdamW optimizer was used during training, with an initial learning rate of 0.0003 and a cosine annealing preheating strategy applied. The loss function was a multi-task loss weighted by the mean squared errors of total nitrogen, available phosphorus, and potassium oxide predictions in a 1:1:1 ratio. An early stopping patience value of 20 rounds was set.

[0142] The training process automatically terminates after approximately 150 epochs due to the early stopping mechanism. The model checkpoint with the lowest loss on the validation set is selected as the optimal model.

[0143] 5. Results and Validation

[0144] The optimal model was finally tested on a test set (approximately 1860 samples) that was not involved in training or hyperparameter tuning, to evaluate its true generalization performance.

[0145] Test results show that the prediction accuracy of the method of this invention for the three effective components of compound fertilizer is significantly higher than that of the traditional partial least squares method. Specifically, the determination coefficient R for total nitrogen prediction is higher. 2 The R² value reached above 0.98, and the root mean square error of prediction was below 0.25%; the R² value for effective phosphorus prediction was above 0.98. 2 The R² value reached above 0.96, and the root mean square error of prediction was below 0.30%; the R² value predicted by potassium oxide was above 0.96. 2 The accuracy reaches above 0.97, and the root mean square error of prediction is below 0.28%. The model's prediction time for a single sample is in the millisecond range, fully meeting the requirements for rapid detection.

[0146] Furthermore, comparative experiments revealed that if only the original 300 samples were used to train the model (even with the same network structure), the prediction error increased significantly. Moreover, if traditional data augmentation methods (such as adding Gaussian noise) were used, the model's robustness to changes in physical state was significantly inferior to the method of this invention. This fully validates the effectiveness and superiority of the multi-level spectral database construction strategy and the model initialization optimization strategy in this invention.

[0147] This embodiment fully implements the method proposed in this invention, from data preparation, enhancement, model construction, optimization to training and validation, the entire process is clear and repeatable. Experimental results fully demonstrate that this method can utilize limited actual calibration samples and, through advanced data enhancement and model optimization techniques, construct a high-precision, highly robust, and rapid detection model, effectively solving key challenges in near-infrared spectroscopy detection of compound fertilizers, and possesses high practical value and promising prospects for widespread application.

[0148] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0150] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0151] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0152] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A rapid near-infrared spectroscopy method for detecting the effective components of compound fertilizer, characterized in that, include: Step 1: Collect raw near-infrared spectral data of compound fertilizer samples and their corresponding standard chemical determination values ​​to construct a multi-level spectral database, specifically as follows: Collect the raw near-infrared spectral data of compound fertilizer samples and their corresponding standard chemical determination values, and use them as raw data pairs; The original data pairs are processed using a strategy combining physical-guided generation and implicit feature space enhancement to generate enhanced spectral data and their corresponding component labels. These enhanced spectral data and their corresponding component labels are then merged to form a multi-level spectral database. Specifically, the physical-guided generation strategy is as follows: Based on the Kubelka-Munk physical optics model, the raw near-infrared spectral data is transformed into an optical feature space. Then, using a conditional generative adversarial network and pre-defined physical state parameters, enhanced spectral data simulating different physical states are generated. The implicit feature space augmentation strategy is as follows: The original spectrum is mapped to a low-dimensional latent space based on the variational autoencoder. New latent variables are generated by performing linear interpolation or adding controllable noise that conforms to its distribution in the low-dimensional latent space. The data is then reconstructed into enhanced spectral data by the decoder. Step 2: Construct a fast detection model based on an improved one-dimensional convolutional neural network model optimized by an improved particle swarm optimization algorithm. The improved one-dimensional convolutional neural network includes an input layer, a first convolutional layer, an ECA layer, a first max-pooling layer, a second convolutional layer, a second max-pooling layer, a third convolutional layer, a third max-pooling layer, a fourth convolutional layer, an Inception layer, a fourth max-pooling layer, a fully connected layer, and an output layer. The improved particle swarm optimization algorithm specifically includes: adding a selection strategy based on the quartiles of particle fitness values ​​to maintain population quality; and introducing an adaptive perturbation strategy based on the ratio of individual fitness values ​​to the average fitness value of the population to enhance the algorithm's ability to escape local optima. Step 3: Train the fast detection model based on the pre-set dataset; Step 4: Input the near-infrared spectral data to be tested into the trained rapid detection model to obtain the effective component content of the compound fertilizer.

2. The method for rapid near-infrared spectroscopy detection of effective components in compound fertilizer according to claim 1, characterized in that, The first, second, third, and fourth convolutional layers are all two-layer one-dimensional convolutions. The first convolutional layer has 16 kernels, a kernel size of 16, a stride of 2, and uses the ReLU activation function. The second convolutional layer has 32 kernels, a kernel size of 8, a stride of 2, and uses the ReLU activation function. The third convolutional layer has 64 kernels, a kernel size of 4, a stride of 2, and uses the ReLU activation function. The fourth convolutional layer has 128 kernels, a kernel size of 2, a stride of 1, and uses the ReLU activation function. The stride of the first, second, third, and fourth max-pooling layers is 2.

3. The method for rapid near-infrared spectroscopy detection of effective components in compound fertilizer according to claim 2, characterized in that, In step 3, the fast detection model is trained based on a pre-set dataset, specifically as follows: The multi-level spectral database is divided into a training set, a validation set, and a test set. Both the training set and the validation set contain enhanced spectral data and their strictly corresponding component labels. An improved particle swarm optimization algorithm is used to globally optimize the key initial parameters of the fast detection model to obtain optimized initial weights and biases. Starting with optimized initial model weights and biases, the fast detection model is trained under supervision using the training set. During training, multi-task weighted mean square error is used as the loss function, and the validation set is used to monitor the training process and implement an early stopping strategy to select the optimal model. The performance of the optimal model is evaluated using a test set to obtain the final prediction model after training.

4. The method for rapid near-infrared spectroscopy detection of effective components in compound fertilizer according to claim 3, characterized in that, The supervised training employs the AdamW optimizer, combined with a cosine annealing preheating learning rate scheduling strategy.

5. The method for rapid near-infrared spectroscopy detection of effective components in compound fertilizer according to claim 4, characterized in that, The multi-task weighted mean square error loss function is a weighted sum of the mean square errors of each component prediction, with the weights set according to the content range or prediction difficulty of each component.

6. The method for rapid near-infrared spectroscopy detection of effective components in compound fertilizer according to claim 5, characterized in that, The early stopping strategy is as follows: when the loss function value on the validation set no longer decreases within a series of preset training cycles, training is terminated and the model parameters are rolled back to the state where the validation set performance is optimal.

Images