Corn nitrogen content accurate detection method in vegetative growth period based on deep learning
By using a handheld near-ground transmission hyperspectral imager and a deep learning model, combined with ring partitioning and customized spectral indices, the destructive and external interference problems of nitrogen content detection in maize have been solved, enabling rapid and stable nitrogen content detection during the vegetative growth period of maize and supporting precision fertilization in the field.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANDONG AGRICULTURAL UNIVERSITY
- Filing Date
- 2026-03-19
- Publication Date
- 2026-07-14
AI Technical Summary
Existing methods for detecting nitrogen content in maize suffer from problems such as destructive sampling, long detection cycles, inability to achieve real-time monitoring, susceptibility of signals to external interference, insufficient feature fusion, and lack of detection during the nutrient growth period.
Hyperspectral images were acquired using a handheld near-ground transmission hyperspectral imager. By combining annular partitioning, horizontal segmentation, and customized spectral indices, a spatial-spectral fusion feature set was constructed. Detection was performed using a deep learning model combining 2DCNN-ResNet50 and 1DCNN with SE attention mechanism.
It enables non-destructive and rapid detection of nitrogen content during the vegetative growth stage of maize, improves the stability of the detection signal and the ability to mine features, supports continuous monitoring of nitrogen status in the field, and is suitable for the V6-V12 vegetative growth stage of maize.
Smart Images
Figure CN122391081A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of agricultural information technology, specifically to a method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning. Background Technology
[0002] Nitrogen is a core nutrient element required for the growth and development of maize. It participates in the synthesis of various biological macromolecules in maize and plays a key role in physiological processes such as photosynthesis and dry matter accumulation. Accurate detection of leaf nitrogen content during the vegetative growth period of maize is the foundation for achieving on-demand application of nitrogen fertilizer and carrying out precision nutrient management in maize fields.
[0003] Currently, the main methods for detecting nitrogen content in corn are traditional chemical detection methods and spectroscopic detection methods. Traditional chemical detection methods require destructive sampling of corn plants. After the samples are dried, crushed, and digested, the detection is performed using instruments. The operation process is cumbersome and the detection cycle is long. It is impossible to achieve real-time and dynamic monitoring of nitrogen content during the growth process of corn. Moreover, the plants lose their growth value after sampling, making them unsuitable for continuous field testing.
[0004] Spectroscopic detection, as a non-destructive detection technology, has been widely used in crop nutrient status detection. Among them, near-ground hyperspectral imaging technology can simultaneously acquire spatial and spectral information of crop leaves, making it the mainstream technology for crop nutrient detection. However, existing methods for detecting nitrogen content in maize based on hyperspectral imaging still have many shortcomings: First, they only use a single spectral index or average spectral feature for detection, ignoring the nitrogen-related features contained in the spatial morphology, texture, and regional spectral differences of leaves, making it difficult to achieve fine differentiation of different nitrogen levels. Second, some detection methods use reflectance hyperspectral imaging, and the detection signal is easily interfered with by external factors such as changes in leaf surface illumination, dust adhesion, and wax layer status, affecting the stability of the detection. Third, existing detection methods combined with deep learning mostly use general convolutional neural network models, without customized design for the spatial-spectral features of maize leaves, resulting in insufficient feature fusion and insufficient feature mining ability and detection specificity of the model. Fourth, most methods focus on nitrogen content detection in the later stages of maize growth, lacking the design of detection methods for nitrogen status in the early stages of maize vegetative growth, and failing to provide timely data support for field lateral fertilization. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a deep learning-based method for accurate detection of nitrogen content in maize during its vegetative growth stage. This method uses a handheld near-ground transmission hyperspectral imager to acquire hyperspectral images in the 460-900nm band. Transmission imaging penetrates the interior of the leaves, avoiding surface interference and improving signal stability. Preprocessing eliminates equipment noise and scanning distortion through reference calibration, NDVI threshold background removal, and column alignment shape correction. Feature extraction employs annular partitioning, horizontal block division, and midrib region spectral ratios, combined with classical and customized spectral indices to construct a spatial-spectral fusion feature set. The deep learning model integrates 2DCNN-ResNet50, 1DCNN, and SE attention mechanisms. Through training / validation / test set partitioning, D'Agostino normality test, Levene variance homogeneity test, and Adam optimizer iteration, nitrogen content is graded and quantitatively detected.
[0006] To solve the above-mentioned technical problems, this invention provides the following technical solution: a method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning, which includes the following specific steps:
[0007] S1: Set up maize cultivation experiments with different nitrogen treatment levels, determine the target leaves of the experimental samples during the vegetative growth period of maize, and use the digestion-inductively coupled plasma atomic emission spectrometry method to determine the true value of nitrogen content in the leaves and plants of the experimental samples, and use the true value of nitrogen content as the label data for model training.
[0008] S2: Using a handheld near-ground transmission hyperspectral imaging device, the target leaf is scanned in vivo during the vegetative growth period of corn to acquire hyperspectral transmission images in the visible-near-infrared band.
[0009] S3: Perform reference calibration, background removal, and blade shape correction sequentially on the original hyperspectral transmission image to obtain a standardized effective hyperspectral image;
[0010] S4: Based on the spatial partitioning principle of circular partitioning and horizontal block partitioning, the hyperspectral effective image is processed to extract the spatial features of the leaves; the hyperspectral effective image is processed by combining classical spectral indices and customized spectral indices to generate a spectral heatmap and extract the spectral features of the leaves; the spatial features and spectral features are fused to construct a spatial-spectral fusion feature set;
[0011] S5: Build a deep learning model that integrates two-dimensional convolutional neural networks, one-dimensional convolutional neural networks and attention mechanisms. Input the spatial-spectral fusion feature set into the deep learning model, use the true value of nitrogen content as the label, and train and optimize the deep learning model.
[0012] S6: After preprocessing and feature extraction, the hyperspectral transmission image of the corn leaf to be tested is input into the trained deep learning model, and the model outputs the nitrogen content grading results and quantitative detection values of the corn leaf to be tested.
[0013] Furthermore, in S3, for reference calibration: via formula Calculate the calibrated hyperspectral image, where, For the calibrated hyperspectral image, This is the original hyperspectral transmission image. A white reference image of the incident light transmission spectrum. The image is a black reference image with zero transmission. For background removal: the Normalized Difference Vegetation Index (NDVI) thresholding method is used to locate the region of interest (ROI) of the leaf. Pixels with an NDVI value < 0.4 are identified as background and set to zero. The effective pixels of the ROI of the leaf are retained to eliminate background noise. For leaf shape correction: to address the leaf distortion problem caused by discontinuous operation or leaf orientation differences during scanning, column alignment is used for correction. The offset distance from the leaf pixel to the mask center is calculated column by column. Based on the offset distance, the leaf mask and the original hyperspectral transmission image are realigned so that the leaf center coincides with the mask center.
[0014] Furthermore, in step S3, the specific steps for correcting the blade shape using the column alignment method are as follows: A blade mask is generated based on the region of interest of the blade, and the horizontal coordinate of the mask center in the image coordinate system is calibrated as follows: Then, the leaf mask is traversed column by column, and the effective pixels of the leaf in each column are extracted. The center x-coordinate of the effective pixels in each column is calculated. And through the formula Solve the first Column correction offset Then, based on this offset, use the formula... The first part of the blade mask and the original hyperspectral transmission image The x-coordinates of all pixels in each column are translated and corrected to realign the blade mask with each column of the original hyperspectral transmission image, ensuring that the center of each effective pixel in the blade coincides with the center x-coordinate of the mask. It is the x-coordinate of the center of the blade mask in the image coordinate system. It is an image column index, with values ranging from 1 to the total number of image columns. , It is the first The center x-coordinate of the effective pixels of the column leaf. It is the first Column correction offset, It is the first before correction The x-coordinate of the column pixels, It is the corrected number The x-coordinate of the column pixels.
[0015] Furthermore, in S4, for extracting leaf spatial features: based on the physiological gradient features of the midrib-leaf margin and leaf base-leaf tip of maize leaves, three methods are used—ring partitioning, horizontal block division, and proportional feature extraction—to extract multi-dimensional spatial features:
[0016] Annular partitioning: Centered on the centroid of the blade, the blade is divided into 10 annular regions with a scaling ratio of 0.1-0.9. Overlapping regions are removed by XOR operation, and the pixel statistical features of each annular region are extracted.
[0017] Horizontal segmentation: Divide the leaf blade evenly into 10 horizontal blocks along the direction from the leaf base to the leaf tip, and extract the pixel statistical features of each horizontal block;
[0018] Proportional characteristics: Taking the annular region where the midrib of the leaf is located as a reference, the spectral ratio of the other 9 annular regions to the midrib region is calculated to extract spatial proportional characteristics.
[0019] Furthermore, in step S4, for extracting leaf spectral features: classic spectral indices related to plant nitrogen physiology are selected, and combined with customized spectral indices and single-band features, a spectral heatmap is generated and multi-dimensional spectral features are extracted.
[0020] Classical spectral indices: Select at least 6 classical spectral indices related to plant nitrogen from NDVI, PRI, NRI, CARI, PBI, ARI, PSRI, and NPCI, generate corresponding spectral heatmaps based on effective hyperspectral images, and extract the features of classical spectral indices.
[0021] Customized spectral index: Sixteen characteristic wavelengths are selected from the 460-900nm band, and a customized spectral index heatmap is generated by randomly combining normalized difference-type calculation formulas and ratio-type calculation formulas to extract customized spectral index features;
[0022] Single-band features: Extract single-band transmission spectral features of 16 characteristic wavelengths.
[0023] Furthermore, in S4, the specific steps for fusing spatial features and spectral features are as follows: the extracted spatial features and spectral features of the corn leaves are uniformly regularized in terms of pixel dimension, so that the spatial features and spectral features of the corn leaves are completely matched with the pixel coordinates and total number of pixels of the effective hyperspectral image, and the dimensions of the spatial feature vector and spectral feature vector corresponding to each pixel are kept fixed. Then, taking a single pixel of the effective hyperspectral image as the basic unit, the multidimensional spatial feature vector and multidimensional spectral feature vector corresponding to each pixel are sequentially concatenated to form the spatial-spectral fusion feature vector exclusive to that pixel. The spatial-spectral fusion feature vectors of all pixels are integrated sequentially according to the pixel arrangement order of the image to construct a spatial-spectral fusion feature set covering the entire corn leaf.
[0024] Furthermore, in S4, the normalized difference type calculation formula is: The formula for ratio-type operations is: ,in, Wavelength Transmission thermogram of maize leaves at [location]. Wavelength Transmission spectral thermogram of maize leaves at [location]; It is any two different characteristic wavelengths selected from the 460-900nm band.
[0025] Furthermore, in S5, the structure of the deep learning model includes a spatial feature extraction branch, a spectral feature extraction branch, an attention fusion layer, and a prediction layer. The connection relationship between each part is as follows: the output ends of the spatial feature extraction branch and the spectral feature extraction branch are both connected to the input end of the attention fusion layer, and the output end of the attention fusion layer is connected to the input end of the prediction layer.
[0026] The spatial feature extraction branch is a 2DCNN-ResNet50 network, which is used to perform convolution and pooling on the feature maps after spatial partitioning to extract the spatial morphology and texture features of the leaves.
[0027] The spectral feature extraction branch is a 1DCNN network, which is used to perform convolution processing on the one-dimensional spectral sequence of the spectral index heatmap to extract spectral correlation features between bands.
[0028] The attention fusion layer includes a channel attention mechanism (SE) module, which is used to adaptively weight and fuse spatial features and spectral features to generate fused features.
[0029] The prediction layer is a fully connected layer used to reduce the dimensionality and map the fused features, and output the nitrogen content grading results and quantitative detection values, respectively.
[0030] Furthermore, in S5, the specific steps for model training and optimization are as follows: normalize the spatial-spectral fusion feature set, verify the normality of the feature set using the D'Agostino and Pearson test, and verify the homogeneity of variance of the feature set using the Levene test; divide the verified feature set into training set, validation set, and test set in a 7:2:1 ratio, use mean squared error (MSE) as the quantitative detection loss function and cross-entropy as the hierarchical detection loss function, and use the Adam optimizer to iteratively optimize the deep learning model; simultaneously, use the validation set loss to determine early stopping, and combine it with the Dropout layer to suppress model overfitting, thus completing the training and optimization of the deep learning model.
[0031] Compared with existing technologies, this deep learning-based method for accurate detection of nitrogen content in maize during its vegetative growth stage has the following advantages:
[0032] This invention enables non-destructive detection of nitrogen content in maize leaves during the vegetative growth stage using a handheld near-ground transmission hyperspectral imaging device. It supports continuous monitoring of nitrogen status during growth, solving the problems of destructive sampling, cumbersome procedures, and long cycles associated with traditional chemical detection methods. The transmission imaging mode penetrates the leaf interior to capture biochemical spectral information, effectively avoiding interference from external factors such as changes in leaf surface light, dust adhesion, and wax layer conditions, thus improving the stability of the detection signal. By combining spatial partitioning principles (ring partitioning and horizontal block partitioning) with spectral heatmaps generated from classic and customized spectral indices, a spatial-spectral fusion feature set is constructed to fully explore nitrogen-related features related to leaf spatial morphology, regional distribution, and spectral band correlation. Through a customized deep learning model integrating two-dimensional convolutional neural networks, one-dimensional convolutional neural networks, and attention mechanisms, adaptive weighted fusion of spatial and spectral features is achieved, enhancing feature mining capabilities and detection specificity. Ultimately, this results in a simple and rapid field nitrogen nutrient status detection method, providing reliable technical support for precise nitrogen fertilizer application and field nutrient management in maize. It is applicable to the V6-V12 vegetative growth stage of maize and has industrial applicability for promotion to other gramineous crops.
[0033] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from an examination of the following, or may be learned from the practice of the invention. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0035] Figure 1 This is a flowchart of a deep learning-based method for accurate detection of nitrogen content during the vegetative growth stage of maize.
[0036] Figure 2 This is a flowchart of step S4 in a deep learning-based method for accurate detection of nitrogen content during the vegetative growth stage of maize.
[0037] Figure 3 This is a flowchart of step S5 in a deep learning-based method for accurate detection of nitrogen content during the vegetative growth stage of maize. Detailed Implementation
[0038] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.
[0039] This invention provides a deep learning-based method for accurate detection of nitrogen content in maize during its vegetative growth stage. A handheld near-ground transmission hyperspectral imager acquires hyperspectral images in the 460-900nm band, employing transmission imaging to penetrate the leaf interior, avoiding surface interference and improving signal stability. Preprocessing steps include reference calibration, NDVI threshold background removal, and column alignment shape correction to eliminate equipment noise and scanning distortion. Feature extraction utilizes annular partitioning, horizontal segmentation, and midrib region spectral ratios, combined with classic and customized spectral indices to construct a spatial-spectral fusion feature set. The deep learning model integrates 2DCNN-ResNet50, 1DCNN, and SE attention mechanisms. Through training / validation / test set partitioning, D'Agostino normality test, Levene variance homogeneity test, and Adam optimizer iteration, nitrogen content is graded and quantitatively detected.
[0040] like Figure 1 As shown, S1: Set up maize cultivation experiments with different nitrogen treatment levels, determine the target leaves of the experimental samples during the vegetative growth period of maize, and use the digestion-inductively coupled plasma atomic emission spectrometry method to determine the true value of nitrogen content in the leaves and plants of the experimental samples, and use the true value of nitrogen content as the label data for model training.
[0041] A maize cultivation experiment was conducted. Based on the actual nitrogen nutrition status in the maize field, a gradient of nitrogen treatment levels was set up, including low nitrogen, medium nitrogen, high nitrogen and a nitrogen-free blank control group. Each treatment group was replicated to ensure that the nitrogen content gradient of the experimental samples covered the full range of nitrogen deficiency, suitability and excess during the vegetative growth period of maize. During the experiment, the cultivation conditions other than nitrogen, such as water, light and soil fertility, were kept consistent to eliminate interference from irrelevant variables.
[0042] During the vegetative growth period of maize (such as the jointing stage and the large trumpet stage), functional leaves on maize plants are selected as target leaves. To ensure sample consistency, leaves with fixed nodes and the same growth status are selected from each maize plant as test samples, and the test samples are numbered and marked.
[0043] The true nitrogen content of the experimental samples was determined by digestion-inductively coupled plasma atomic emission spectrometry (ICP-AES): The labeled leaf and plant samples were dried, pulverized, and digested before being pretreated. The nitrogen content of the treated samples was determined by ICP-AES. Each sample was measured three or more times, and the average value of the multiple measurements was taken as the true nitrogen content of the experimental sample. The sample number was associated with the corresponding true nitrogen content to establish a label database for model training. The true nitrogen content was used as the label data for subsequent deep learning model training and optimization.
[0044] S2: Using a handheld near-ground transmission hyperspectral imaging device, the target leaf is scanned in vivo during the vegetative growth period of corn to acquire hyperspectral transmission images in the visible-near-infrared band.
[0045] In the natural field environment during the vegetative growth period of maize, a handheld near-ground transmission hyperspectral imaging device was used to perform in vivo non-destructive scanning of the selected target leaves. During the scanning process, the device's detection probe was kept in close contact with the leaf surface, and the scanning speed was kept uniform to avoid interference such as light refraction and leaf movement caused by improper operation. Hyperspectral transmission images of the target leaves in the visible to near-infrared band were acquired. The images simultaneously contain the spatial morphology information of the leaves and the spectral information of each band. After the scanning was completed, the acquired hyperspectral transmission images were associated with the corresponding test sample number and synchronized to the label database to ensure that each hyperspectral transmission image has a corresponding true value label of nitrogen content.
[0046] S3: Perform reference calibration, background removal, and blade shape correction sequentially on the original hyperspectral transmission image to obtain a standardized effective hyperspectral image;
[0047] White reference image of incident light transmission spectrum and a black reference image with zero transmission Then through the formula The calibrated hyperspectral image was calculated, where For the calibrated hyperspectral image, This is the original hyperspectral transmission image; this calibration operation eliminates spectral deviations caused by equipment noise, changes in ambient light, and differences in optical paths, making hyperspectral images acquired at different times and under different environments comparable.
[0048] The Normalized Difference Vegetation Index (NDVI) thresholding method was used to locate the region of interest (ROI) on the leaf: First, based on the calibrated hyperspectral image, the NDVI value of each pixel was calculated. Pixels with an NDVI value < 0.4 were set as background pixels, and the pixel values of these pixels were set to zero. Only the effective pixels in the leaf area were retained. This operation eliminated noise generated by background factors such as soil, air, weeds, and supports, and extracted the hyperspectral information of the pure leaf.
[0049] To address blade distortion and center offset issues caused by discontinuous operation and differences in blade orientation during scanning, a column alignment method is used for correction to eliminate blade undulation and distortion along its length. The specific steps are as follows:
[0050] Based on the obtained region of interest of the blade, a binary blade mask is generated, and the x-coordinate of the mask center in the image coordinate system is denoted as . ;
[0051] Traverse the blade mask column by column, using column indexes. The value ranges from 1 to the total number of columns in the image. Valid leaf pixels are extracted column by column, and the center x-coordinate of each valid pixel in each column is calculated. ;
[0052] Through formula Solve the first Column correction offset Based on the correction offset Through formula The first part of the blade mask and the original hyperspectral transmission image The x-coordinates of all pixels in each column are translated and corrected to realign the blade mask with each column of the original hyperspectral transmission image, ensuring that the center of each effective pixel in the blade is aligned with the x-coordinate of the mask center. After the above operations, a hyperspectral image with various interferences eliminated and morphology standardized is obtained.
[0053] S4: Based on the spatial partitioning principles of circular partitioning and horizontal block division, the effective hyperspectral image is processed to extract leaf spatial features; the effective hyperspectral image is processed by combining classical spectral indices and customized spectral indices to generate a spectral heatmap and extract leaf spectral features; the spatial features and spectral features are fused to construct a spatial-spectral fusion feature set, such as... Figure 2 As shown, the specific operation is as follows:
[0054] Based on the physiological gradient characteristics of the midrib-leaf margin and leaf base-leaf tip of maize leaves, three methods—ring partitioning, horizontal segmentation, and proportional feature extraction—were employed to extract multi-dimensional spatial features of the leaves. These three types of features complement each other, fully exploring the spatial morphology and texture information of the leaves. Specifically:
[0055] Annular partitioning: First, determine the centroid of the leaf in the effective hyperspectral image. Using this centroid as the center, divide the image into 10 annular regions with a scaling gradient of 0.1-0.9. Remove overlapping pixels of adjacent annular regions by XORing the image to avoid feature redundancy. Extract the pixel statistical features of each annular region, including mean, variance, standard deviation, skewness, kurtosis, etc., as the annular partitioning features.
[0056] Horizontal segmentation: Divide the leaf blade evenly into 10 horizontal blocks along the length from the leaf base to the leaf tip, ensuring that the number of pixels in each horizontal block is roughly equal; extract the above pixel statistical features of each horizontal block as horizontal segmentation features.
[0057] Proportional feature extraction: Taking the annular region where the midrib of the leaf is located as the reference region, calculate the spectral ratios of the other 9 annular regions and the reference region (including the mean ratio, maximum ratio, minimum ratio, etc. of the transmission spectra of each band), extract this type of spatial proportional feature, reflect the differences in the spectral spatial distribution of different regions of the leaf, and integrate the above annular partition features, horizontal block features, and spatial proportional features to obtain a multidimensional spatial feature set of the leaf.
[0058] Classical spectral indices related to plant nitrogen physiology were selected, and combined with customized spectral indices and single-band features to generate various spectral heatmaps and extract multi-dimensional spectral features, fully exploring the correlation between leaf spectra and nitrogen content. Specifically:
[0059] Classical spectral index feature extraction: Select at least 6 classical spectral indices closely related to plant nitrogen metabolism from NDVI, PRI, NRI, CARI, PBI, ARI, PSRI, and NPCI. Based on the spectral information of each band of the effective hyperspectral image, calculate the classical spectral index value of each pixel and generate the corresponding classical spectral index heatmap. Extract the pixel statistical features and texture features of the heatmap as classical spectral index features.
[0060] Customized spectral index feature extraction: 16 characteristic wavelengths are selected from the visible-near infrared band of 460-900nm. Two different characteristic wavelengths are randomly selected, and then processed using a normalized difference calculation formula. Sum of ratio calculation formulas Perform calculations (where) Wavelength Transmission thermogram of maize leaves at [location]. Wavelength Transmission spectral thermogram of maize leaves at [location]; It generates a customized spectral index heatmap by selecting any two different characteristic wavelengths from the 460-900nm band; and extracts the pixel statistical features and texture features of the heatmap as customized spectral index features.
[0061] Single-band feature extraction: The single-band transmission spectral features at the 16 characteristic wavelengths mentioned above are directly extracted, including pixel values, mean, variance, and peak values at each wavelength, as single-band features. The above classical spectral index features, customized spectral index features, and single-band features are integrated to obtain a multidimensional spectral feature set of the leaf.
[0062] The extracted spatial and spectral features are fused at the pixel level to ensure that the fused features simultaneously contain both the spatial morphological information and the spectral physiological information of the leaves. The specific steps are as follows:
[0063] The spatial and spectral features of the leaf are uniformly regularized in terms of pixel dimension. The dimension of the features and the arrangement of pixels are adjusted so that the spatial features, spectral features and the pixel coordinates and total number of pixels of the effective hyperspectral image are completely matched. The dimension of the spatial feature vector and spectral feature vector corresponding to each pixel remains fixed, eliminating the deviation between the feature dimension and the pixel position.
[0064] Using a single pixel of the effective hyperspectral image as the basic fusion unit, the multidimensional spatial feature vector and the multidimensional spectral feature vector corresponding to each pixel are sequentially concatenated to form the spatial-spectral fusion feature vector exclusive to that pixel.
[0065] The spatial-spectral fusion feature vectors of all pixels are integrated sequentially according to the pixel row and column order of the effective hyperspectral image to construct a spatial-spectral fusion feature set covering the entire corn leaf.
[0066] S5: Build a deep learning model that integrates two-dimensional convolutional neural networks, one-dimensional convolutional neural networks and attention mechanisms. Input the spatial-spectral fusion feature set into the deep learning model, use the true value of nitrogen content as the label, and train and optimize the deep learning model.
[0067] The overall model structure includes a spatial feature extraction branch, a spectral feature extraction branch, an attention fusion layer, and a prediction layer. The connections between these parts are as follows: the outputs of both the spatial and spectral feature extraction branches are connected to the input of the attention fusion layer, and the output of the attention fusion layer is connected to the input of the prediction layer. The functions and network design of each part are as follows:
[0068] Spatial feature extraction branch: A 2DCNN-ResNet50 network is adopted, consisting of convolutional layers, pooling layers, and residual blocks. The convolutional layers perform convolution operations on the feature maps after spatial partitioning to extract the local spatial morphology and texture features of the blade. The pooling layers reduce the feature dimension and reduce the amount of computation. The residual blocks solve the gradient vanishing problem of deep networks, ensuring the effective extraction of deep features, and finally outputting the high-dimensional spatial features of the blade.
[0069] Spectral feature extraction branch: A 1DCNN network is used, consisting of multiple one-dimensional convolutional layers and pooling layers. Convolution operations are performed on the one-dimensional spectral sequence of the spectral index heatmap to capture the spectral correlation features, spectral variation patterns and nitrogen-related spectral response features between different bands, and finally output the high-dimensional spectral features of the leaf.
[0070] Attention Fusion Layer: The channel attention mechanism SE module is adopted. It performs global information aggregation on each channel of spatial and spectral features through squeezing operation, and performs adaptive weighting on different feature channels through activation operation. It performs adaptive weighted fusion on the spatial features output by the spatial feature extraction branch and the spectral features output by the spectral feature extraction branch, highlighting key features that are highly correlated with nitrogen content, suppressing irrelevant and invalid features, and generating spatial-spectral fusion features with higher fusion degree.
[0071] Prediction layer: A fully connected layer is used, consisting of multiple fully connected neurons; the fusion features output by the attention fusion layer are reduced in dimension and nonlinearly mapped, and finally output in two ways: the grading results of nitrogen content in maize leaves (such as low nitrogen, medium nitrogen, and high nitrogen levels) and the quantitative detection value.
[0072] After preprocessing the spatial-spectral fusion feature set, the dataset is divided proportionally, and a reasonable loss function and optimization strategy are set to complete the model training and optimization. Simultaneously, multiple measures are taken to suppress model overfitting and improve the model's generalization ability, such as... Figure 3 As shown, the specific steps are as follows:
[0073] Feature set preprocessing: The constructed spatial-spectral fusion feature set is normalized to map all feature values to the [0, 1] interval, eliminating the dimensional differences between different feature dimensions and improving model training efficiency; the D'Agostino and Pearson test is used to verify the normality of the normalized feature set, and the Levene test is used to verify the homogeneity of variance of the feature set, ensuring the statistical validity of the feature set and removing abnormal feature samples.
[0074] Dataset partitioning: The statistically validated feature set is randomly divided into a training set, a validation set, and a test set in a ratio of 7:2:1. The training set is used for iterative updates of the model's parameters, the validation set is used for hyperparameter tuning and early stopping detection, and the test set is used for evaluating the model's final detection performance.
[0075] Loss function and optimizer settings: Mean squared error is used as the loss function for quantitative nitrogen content detection to measure the numerical deviation between the model's predicted quantitative value and the true value; cross-entropy is used as the loss function for nitrogen content classification detection to measure the class deviation between the model's predicted classification result and the true level; the Adam optimizer is used to iteratively optimize the deep learning model, adjusting the learning rate, batch size, and number of iterations according to the model training situation to ensure rapid model convergence.
[0076] Overfitting suppression and early stopping criteria: A Dropout layer is added before the fully connected layer of the model to randomly discard some neurons, avoiding overfitting of the model to the training set; validation set loss is used for early stopping criteria. If the loss value of the validation set does not decrease for several consecutive rounds during model training, the model training is stopped immediately to avoid the decline in generalization ability caused by overtraining. After the above training and optimization, a deep learning model for detecting nitrogen content in maize leaves with high detection accuracy and high generalization ability is obtained.
[0077] S6: After preprocessing and feature extraction, the hyperspectral transmission image of the corn leaf to be tested is input into the trained deep learning model, and the model outputs the nitrogen content grading results and quantitative detection values of the corn leaf to be tested.
[0078] During the vegetative growth period of maize, target leaves of maize plants to be tested are selected, and hyperspectral transmission images in the visible-near-infrared band are collected. The collected original hyperspectral transmission images are then subjected to reference calibration, background removal, and leaf shape correction in sequence to obtain standardized effective hyperspectral images.
[0079] Spatial and spectral features are extracted from the effective hyperspectral images, and pixel-level spatial-spectral feature fusion is performed to construct a spatial-spectral fusion feature set for the sample to be tested.
[0080] The spatial-spectral fusion feature set of the sample to be tested is input into the trained deep learning model. After spatial feature extraction, spectral feature extraction, adaptive weighted fusion and dimensionality reduction mapping, the model directly outputs the nitrogen content classification results and quantitative detection values of the maize leaves to be tested, so as to realize the accurate detection of nitrogen content during the vegetative growth period of maize.
[0081] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.
Claims
1. A method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning, characterized in that, The method includes the following specific steps: S1: Set up maize cultivation experiments with different nitrogen treatment levels, determine the target leaves of the experimental samples during the vegetative growth period of maize, and use the digestion-inductively coupled plasma atomic emission spectrometry method to determine the true value of nitrogen content in the leaves and plants of the experimental samples, and use the true value of nitrogen content as the label data for model training. S2: Using a handheld near-ground transmission hyperspectral imaging device, the target leaf is scanned in vivo during the vegetative growth period of corn to acquire hyperspectral transmission images in the visible-near-infrared band. S3: Perform reference calibration, background removal, and blade shape correction sequentially on the original hyperspectral transmission image to obtain a standardized effective hyperspectral image; S4: Based on the spatial partitioning principle of annular partitioning and horizontal block partitioning, the effective hyperspectral image is processed to extract the spatial features of the leaves; By combining classical and customized spectral indices, hyperspectral images are processed to generate spectral heatmaps, and leaf spectral features are extracted. Spatial and spectral features are fused to construct a spatial-spectral fusion feature set. S5: Build a deep learning model that integrates two-dimensional convolutional neural networks, one-dimensional convolutional neural networks and attention mechanisms. Input the spatial-spectral fusion feature set into the deep learning model, use the true value of nitrogen content as the label, and train and optimize the deep learning model. S6: After preprocessing and feature extraction, the hyperspectral transmission image of the corn leaf to be tested is input into the trained deep learning model, and the model outputs the nitrogen content grading results and quantitative detection values of the corn leaf to be tested.
2. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 1, characterized in that, In S3, for reference calibration: via formula Calculate the calibrated hyperspectral image, where, For the calibrated hyperspectral image, This is the original hyperspectral transmission image. A white reference image of the incident light transmission spectrum. The image is a black reference image with zero transmission. For background removal: the Normalized Difference Vegetation Index (NDVI) thresholding method is used to locate the region of interest (ROI) of the leaf. Pixels with an NDVI value < 0.4 are identified as background and set to zero. The effective pixels of the ROI of the leaf are retained to eliminate background noise. For leaf shape correction: to address the leaf distortion problem caused by discontinuous operation or differences in leaf orientation during the scanning process, column alignment is used for correction. The offset distance from the leaf pixel to the mask center is calculated column by column. Based on the offset distance, the leaf mask and the original hyperspectral transmission image are realigned so that the leaf center coincides with the mask center.
3. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 2, characterized in that, In step S3, the specific steps for blade shape correction using the column alignment method are as follows: A blade mask is generated based on the region of interest of the blade, and the horizontal coordinate of the mask center in the image coordinate system is calibrated. Then, the leaf mask is traversed column by column, and the effective pixels of the leaf in each column are extracted. The center x-coordinate of the effective pixels in each column is calculated. And through the formula Solve the first Column correction offset ; Then, based on this offset, use the formula... The first part of the blade mask and the original hyperspectral transmission image The x-coordinates of all pixels in each column are translated and corrected to realign the blade mask with each column of the original hyperspectral transmission image, ensuring that the center of each effective pixel in the blade coincides with the center x-coordinate of the mask. It is the x-coordinate of the center of the blade mask in the image coordinate system. It is an image column index, with values ranging from 1 to the total number of image columns. , It is the first The center x-coordinate of the effective pixels of the column leaf. It is the first Column correction offset, It is the first before correction The x-coordinate of the column pixels, It is the corrected number The x-coordinate of the column pixels.
4. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 1, characterized in that, In step S4, for extracting leaf spatial features: based on the physiological gradient features of the midrib-leaf margin and leaf base-leaf tip of maize leaves, three methods are used—ring partitioning, horizontal block division, and proportional feature extraction—to extract multi-dimensional spatial features. Annular partitioning: Centered on the centroid of the blade, the blade is divided into 10 annular regions with a scaling ratio of 0.1-0.
9. Overlapping regions are removed by XOR operation, and the pixel statistical features of each annular region are extracted. Horizontal segmentation: Divide the leaf blade evenly into 10 horizontal blocks along the direction from the leaf base to the leaf tip, and extract the pixel statistical features of each horizontal block; Proportional characteristics: Taking the annular region where the midrib of the leaf is located as a reference, the spectral ratio of the other 9 annular regions to the midrib region is calculated to extract spatial proportional characteristics.
5. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 1, characterized in that, In step S4, for extracting leaf spectral features: Classical spectral indices related to plant nitrogen physiology were selected, and combined with customized spectral indices and single-band features to generate spectral heatmaps and extract multi-dimensional spectral features. Classical spectral indices: Select at least 6 classical spectral indices related to plant nitrogen from NDVI, PRI, NRI, CARI, PBI, ARI, PSRI, and NPCI, generate corresponding spectral heatmaps based on effective hyperspectral images, and extract the features of classical spectral indices. Customized spectral index: Sixteen characteristic wavelengths are selected from the 460-900nm band, and a customized spectral index heatmap is generated by randomly combining normalized difference-type calculation formulas and ratio-type calculation formulas to extract customized spectral index features; Single-band features: Extract single-band transmission spectral features of 16 characteristic wavelengths.
6. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 1, characterized in that, In step S4, the specific steps for fusing spatial and spectral features are as follows: the extracted spatial and spectral features of the corn leaves are uniformly regularized in terms of pixel dimension, so that the spatial and spectral features of the corn leaves are completely matched with the pixel coordinates and total number of pixels of the effective hyperspectral image, and the dimensions of the spatial feature vector and spectral feature vector corresponding to each pixel are kept fixed. Then, taking a single pixel of the effective hyperspectral image as the basic unit, the multidimensional spatial feature vector and multidimensional spectral feature vector corresponding to each pixel are sequentially concatenated to form the spatial-spectral fusion feature vector exclusive to that pixel. The spatial-spectral fusion feature vectors of all pixels are integrated sequentially according to the pixel arrangement order of the image to construct a spatial-spectral fusion feature set covering the entire corn leaf.
7. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 5, characterized in that, In S4, the normalized difference type operation formula is: The formula for ratio-type operations is: ,in, Wavelength Transmission thermogram of maize leaves at [location]. Wavelength Transmission spectral thermogram of maize leaves at [location]; It is any two different characteristic wavelengths selected from the 460-900nm band.
8. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 1, characterized in that, In S5, the structure of the deep learning model includes a spatial feature extraction branch, a spectral feature extraction branch, an attention fusion layer, and a prediction layer. The connection relationship between each part is as follows: the outputs of the spatial feature extraction branch and the spectral feature extraction branch are both connected to the input of the attention fusion layer, and the output of the attention fusion layer is connected to the input of the prediction layer. The spatial feature extraction branch is a 2DCNN-ResNet50 network, which is used to perform convolution and pooling on the feature maps after spatial partitioning to extract the spatial morphology and texture features of the leaves. The spectral feature extraction branch is a 1DCNN network, which is used to perform convolution processing on the one-dimensional spectral sequence of the spectral index heatmap to extract spectral correlation features between bands. The attention fusion layer includes a channel attention mechanism (SE) module, which is used to adaptively weight and fuse spatial features and spectral features to generate fused features. The prediction layer is a fully connected layer used to reduce the dimensionality and map the fused features, and output the nitrogen content classification results and quantitative detection values, respectively.
9. The method for accurate detection of nitrogen content in maize during its vegetative growth stage based on deep learning according to claim 1, characterized in that, In step S5, the specific steps for model training and optimization are as follows: The spatial-spectral fusion feature set is normalized; the D'Agostino and Pearson tests are used to verify the normality of the feature set; the Levene test is used to verify the homogeneity of variance of the feature set; the validated feature set is divided into training, validation, and test sets in a 7:2:1 ratio; the mean squared error (MSE) is used as the quantitative detection loss function; the cross-entropy is used as the hierarchical detection loss function; and the Adam optimizer is used to iteratively optimize the deep learning model. Simultaneously, the validation loss is used to determine early stopping, and the Dropout layer is combined to suppress model overfitting, thus completing the training and optimization of the deep learning model.