Corn leaf nitrogen content detection method based on spectrum symbolization aggregation operator

Abstract

The invention discloses a corn leaf nitrogen content detection method based on a spectrum symbolization aggregation operator, and relates to the technical field of agricultural remote sensing and spectrum analysis. The invention aims to solve the problem of low accuracy of existing corn leaf nitrogen content detection. The method comprises the following steps: acquiring a hyperspectral image NewHyperImage of a corn leaf to be detected and an RGB (Red, Green, Blue) image NewRgbImage of the corn leaf to be detected, acquiring spectral data to be predicted by utilizing the NewHyperImage and the NewRgbImage, inputting the spectral data to be predicted into a nitrogen content detection model Model to obtain a nitrogen content predicted value PredictedValues, and acquiring a final prediction result NitrogenContent by utilizing the nitrogen content predicted value PredictedValues. The method is used for detecting the nitrogen content of the corn leaves.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural remote sensing and spectral analysis, and particularly relates to a corn leaf nitrogen content detection method based on a spectral symbolization aggregation operator. BACKGROUND

[0002] In agricultural production and food security, nitrogen is one of the most important nutrient elements for crop growth and development, and has a decisive influence on yield and quality. As a major food crop in China, the nitrogen use efficiency of corn is directly related to the level of precision fertilization, which is related to the sustainable development of agriculture and environmental protection. Through the detection of corn leaf nitrogen content by hyperspectral technology, non-destructive, rapid and high-precision evaluation of crop nutrient status can be achieved, which provides a scientific basis for precision fertilization and intelligent agricultural decision-making. Therefore, developing a high-precision and automated corn leaf nitrogen content detection method has important theoretical significance and application value.

[0003] The current commonly used corn nitrogen content detection methods mainly include two types: the first type is a nitrogen content detection method based on traditional spectral index, such as using normalized difference vegetation index (NDVI), red edge index (RE), difference vegetation index (DVI) and the like to represent leaf nitrogen content. This kind of method is simple to realize, but these spectral indices are sensitive to environmental and light conditions, and have limited feature expression capability, so the accuracy of corn leaf nitrogen content detection is low under different growth periods and complex field environments. The second type is a method based on deep learning to realize corn leaf nitrogen content detection, such as convolutional neural network (CNN), residual network (ResNet) and the like, which can automatically extract features from corn leaf spectral data and achieve high accuracy under large-scale sample conditions. However, the existing corn nitrogen content detection methods based on deep learning still have the following problems: on the one hand, the convolution kernel size of the network is fixed, which is difficult to balance local fine granularity and global slow trend at the same time, resulting in unstable corn leaf nitrogen content detection results in different growth periods, thereby causing low overall accuracy of corn leaf nitrogen content detection; on the other hand, the dimension of corn leaf hyperspectral data is extremely high (hundreds of bands), and direct input into the network will cause redundancy, overfitting and lack of interpretability, thereby resulting in low accuracy of corn leaf nitrogen content detection. SUMMARY

[0004] The present application aims to solve the problem of low accuracy of existing corn leaf nitrogen content detection, and proposes a corn leaf nitrogen content detection method based on a spectral symbolization aggregation operator.

[0005] A corn leaf nitrogen content detection method based on a spectral symbolization aggregation operator, comprising the following steps: Obtain the hyperspectral image NewHyperImage of the corn leaf to be tested and the RGB image NewRgbImage of the corn leaf to be tested, obtain the to-be-predicted spectral data by using the NewHyperImage and the NewRgbImage, input the to-be-predicted spectral data into the nitrogen content detection model Model, obtain the nitrogen content prediction value PredictedValues, and obtain the final prediction result NitrogenContent by using the nitrogen content prediction value PredictedValues.

[0006] Further, the nitrogen content detection model Model is obtained by the following method: S1, a region spectral data table obtaining operator is constructed, the input of the region spectral data table obtaining operator is a corn leaf hyperspectral image HyperImage and a corn leaf RGB image RgbImage, the region spectral data table obtaining operator obtains a hyperspectral region block by using the HyperImage and the RgbImage, obtains the reflectivity of each region corn leaf corresponding to the region corn leaf, and stores each region corn leaf reflectivity and the corresponding nitrogen content label to a region spectral data table RegionSpectraTable; S2, a corn leaf spectral preprocessing module is constructed, the corn leaf spectral preprocessing module constructs a training set TrainSet, a validation set ValSet and a test set TestSet by using the RegionSpectraTable; S3, a corn leaf spectral symbolization aggregation module is constructed, the input of the corn leaf spectral symbolization aggregation module is a region spectral tensor RegionSpectraTensor, and the output is a symbolization aggregation vector set; S4, a corn leaf region association context modulation module is constructed, the input of the corn leaf region association context modulation module is a symbolization aggregation vector set, and the output is a global context aggregation feature vector ; S5, the corn leaf spectral symbolization aggregation module and the corn leaf region association context modulation module are called to construct a detection model, the input of the detection model is the preprocessed spectral data, and the output is a nitrogen content prediction value; S6, the training set TrainSet, the validation set ValSet and the test set TestSet are used to train, verify and test the detection model respectively, and a nitrogen content detection model Model is obtained.

[0007] Further, the constructing region spectrum data table obtaining operator in the S1, the region spectrum data table obtaining operator input is the corn leaf hyperspectral image HyperImage and the corn leaf RGB image RgbImage, the region spectrum data table obtaining operator obtains the hyperspectral region block using HyperImage and RgbImage, obtains the reflectivity of the corresponding region corn leaf of each hyperspectral region block, and the reflectivity of each region corn leaf is stored to the region spectrum data table RegionSpectraTable with the corresponding nitrogen content label, specifically: S101, input the corn leaf hyperspectral image HyperImage and the corn leaf RGB image RgbImage, obtain the width Width, the height Height and the total number of bands of HyperImage ; initialize matrix SingleImage=zero matrix; Wherein, the matrix SingleImage is the matrix same as the height and width of the corn leaf hyperspectral image HyperImage; S102, read the image data of each band of the hyperspectral image HyperImage in turn, simultaneously obtain the minimum value TempMin and the maximum value TempMax in the image data of the current band, and the image data of the current band is normalized using TempMin and TempMax, specifically:

[0008] Wherein, is the normalized image data of the current band, is the image data of the current band, is the minimum value in the image data of the current band, and TempMax is the maximum value in the image data of the current band; S103, accumulate the normalized image data of each band to SingleImage, and update SingleImage, let SingleImage=updated SingleImage, obtain the fusion spectrum image FusionImage using SingleImage:

[0009] Wherein, is the total number of bands of the corn leaf hyperspectral image HyperImage; S104, obtain the normalized vegetation index using the near-infrared band and the red band of each pixel of HyperImage, and obtain the vegetation candidate region mask NdviMask using the normalized vegetation index, specifically: Firstly, the near-infrared band and the red band of each pixel in the HyperImage are obtained, and the normalized difference vegetation index of each pixel is obtained by using the near-infrared band and the red band of each pixel, specifically as follows:

[0010] wherein, NDVI is the normalized difference vegetation index of the current pixel, NIR is the near-infrared band of the current pixel, R is the red band of the current pixel; Then, the normalized difference vegetation index NDVI of the current pixel is compared with the vegetation index threshold value, if the current pixel is assigned a value of 1; otherwise, the current pixel is assigned a value of 0, and finally the vegetation candidate region mask NdviMask is obtained. wherein, NDVIth is the vegetation index threshold value; S105, a preliminary leaf mask LeafMask is obtained by using the FusionImage, the corn leaf hyperspectral image HyperImage and the corn leaf RGB image, specifically as follows: S105-1, a multi-source feature set FeatureSet is obtained by using the FusionImage, the corn leaf hyperspectral image HyperImage and the corn leaf RGB image, specifically as follows: The RGB image RgbImage is converted into HSV and Lab color spaces respectively, so as to obtain color features and brightness features; A matrix HyperImageNDVI is formed by using the normalized difference vegetation index of each pixel in the corn leaf hyperspectral image HyperImage; The texture operator is used to process the fusion spectral image FusionImage, and the texture features TextureC of the leaf region are extracted; The RGB image RgbImage is converted into a gray image, the Canny operator is used to perform edge detection on the gray image of RgbImage, and a leaf contour structure feature map EdgeMap is obtained; The multi-source feature set FeatureSet is formed by using the fusion spectral image FusionImage, the texture features TextureC, the contour structure feature map EdgeMap and HyperImageNDVI; S105-2, a composite feature map CompositeFeature is obtained by weighting, superimposing and normalizing the multi-source feature FeatureSet; S105-3, Otsu threshold segmentation and morphological operation are performed on the composite feature map CompositeFeature to obtain a preliminary leaf mask LeafMask; S106, a final leaf mask FinalLeafMask is obtained by using the preliminary leaf mask LeafMask and the vegetation candidate region mask NdviMask, M spectral region blocks are obtained within the range of the FinalLeafMask, corn leaf reflectivity parameters of each waveband in each spectral region block are obtained, and a nitrogen content label is set for the corn leaf reflectivity parameters of each waveband in each spectral region block; S107, the nitrogen content label and all corn leaf reflectivity parameters in each spectral region block are stored into a region spectral data table RegionSpectraTable.

[0011] Further, in the S106, the final leaf mask FinalLeafMask is obtained by using the preliminary leaf mask LeafMask and the vegetation candidate region mask NdviMask, M spectral region blocks are obtained within the range of the FinalLeafMask, corn leaf reflectivity parameters of each waveband in each spectral region block are obtained, and a nitrogen content label is set for the corn leaf reflectivity parameters of each waveband in each spectral region block, and specifically: S106-1, a logical AND operation is performed on the preliminary leaf mask LeafMask and the vegetation candidate region mask NdviMask to obtain the final leaf mask FinalLeafMask; S106-2, M points are randomly selected as region sampling positions within the range of the FinalLeafMask, and a RegionSize hyperspectral region block of the RegionSize is extracted; S106-3, a final leaf mask value of each pixel in the hyperspectral region block is obtained, it is determined whether the current pixel is a valid pixel by using the final leaf mask value of each pixel, the number of valid pixels in the hyperspectral region block is obtained, and corn leaf reflectivity parameters of each waveband in each hyperspectral region block are obtained by using the number of valid pixels in each hyperspectral region block; The determination of whether the current pixel is a valid pixel by using the final leaf mask value of each pixel is specifically: If the final leaf mask value of the current pixel is 1, it indicates that the current pixel is a valid pixel; if the final leaf mask value of the current pixel is 0, it indicates that the current pixel is an invalid pixel; The obtaining of the corn leaf reflectivity parameters of each waveband in each hyperspectral region block by using the number of valid pixels in each hyperspectral region block is specifically:

[0012] in, It is the first The first hyperspectral region block Reflectance of corn leaves in each wavelength band It is the first The total number of effective pixels in each hyperspectral region block It is the first Effective pixel labels in each hyperspectral region block It is the first The first hyperspectral region block The effective pixel in the first The original values ​​of each band, , These are hyperspectral region block labels; S106-4. Set nitrogen content labels for the corn leaf reflectance parameters of each band in each spectral region block.

[0013] Furthermore, the corn leaf spectral preprocessing module in S2 utilizes RegionSpectraTable to construct a training set TrainSet, a validation set ValSet, and a test set TestSet, specifically as follows: S201. The corn leaf reflectance parameter CornLeafR in RegionSpectraTable is smoothed and transformed by a filter to obtain the smoothed spectral data. S202. Perform a standard normal transformation on the spectral data to obtain the transformed spectral data, specifically as follows:

[0014] in, It is the first Spectral data, It is spectral data The mean, It is spectral data Standard deviation; S203. Global standardization is used to process the spectral data after the standard normal variable transformation, scaling the spectral data to zero mean and unit variance to obtain preprocessed spectral data. S204. Perform a logarithmic transformation on the nitrogen content label and then standardize the logarithmically transformed nitrogen content label to obtain a preprocessed nitrogen content label. S205. The preprocessed spectral data and the preprocessed nitrogen content labels are combined to form a dataset, and the dataset is divided into a training set TrainSet, a validation set ValSet, and a test set TestSet.

[0015] Further, the corn leaf spectrum symbolization aggregation module in S3, the corn leaf spectrum symbolization aggregation module input is region spectrum tensor RegionSpectraTensor, output is a set of symbolization aggregation vectors, specifically: S301, input region spectrum tensor RegionSpectraTensor; S302, embedding mapping is carried out on each row sequence in region spectrum tensor RegionSpectraTensor, and embedding matrix is obtained :

[0016]

[0017] Wherein, is the first l row of RegionSpectraTensor, is a sigmoid function, is a weight, is a bias, is the embedding matrix of the first l band; S303, the L band is symbolized into symbol aggregation units, and the embedding matrix E is used to obtain symbolization assignment matrix ; The element in the A row and the k column of matrix l is specifically:

[0018] Wherein, is the assignment weight of the first k symbol aggregation unit to the first l band, is the first symbol prototype vector, is a scaling factor, is a band label, is the embedding of the first l band, is the embedding of the first band, , is a symbol aggregation unit label, is the total number of symbol aggregation units; S304, symbolization assignment matrix An encoded aggregated vector set is acquired, specifically:

[0019]

[0020] wherein, is an encoded aggregated vector, , , is a constant, is an encoded aggregated vector.

[0021] Further, the corn leaf region association context modulation module in S4 is inputted with the encoded aggregated vector set and outputted with a global context aggregated feature vector , specifically: S401, input the encoded aggregated vector set ; S402, channel and spatial context modulation is performed on to obtain channel attention weights:

[0022] wherein, GAP is a global average pooling, GMP is a global maximum pooling, is a RELU activation function, and are weight matrices, is a channel attention weight; S403, spatial attention weights are obtained by using , specifically:

[0023] wherein, represents a one-dimensional convolution operator with a convolution kernel size of , AvgPool represents an average pooling, MaxPool represents a maximum pooling, is a spatial attention weight; S404, an enhanced region feature matrix is obtained by using the channel attention weight and the spatial attention weight:

[0024] wherein, is an enhanced region feature matrix, is an element-wise weighting operation; S405, different region feature weighted results are obtained by using the enhanced region feature matrix : ​​

[0025]

[0026]

[0027]

[0028]

[0029] in, It is an activation function. , and It is a linear mapping matrix. It is the dimension of K. It is the first The weighted result of regional characteristics, It is the first The weighted result of regional characteristics, It is a query vector. It is a key vector. It is a value vector; S406. Obtain the global context aggregation feature vector, specifically:

[0030]

[0031]

[0032]

[0033]

[0034] in, It is a global context aggregation feature vector. It is the first The network weight of each region For the weight vector, For the first The first-class feature representation of each region It is the first The second type of mapping feature representation of each region It is a region feature vector. It is an element-wise multiplication operation. and It's weight. and It is a bias.

[0035] Further, the corn leaf spectrum symbolization aggregation module and the corn leaf region associated context modulation module in S5 construct a detection model, the input of the detection model is the preprocessed spectrum data, and the output is a nitrogen content prediction value, specifically: S501, input the preprocessed spectrum data into the adaptive feature pyramid AFP to obtain a feature representation fused with multi-scale information :

[0036] wherein, is a convolution with a kernel size of , is a convolution with a kernel size of , is a convolution with a kernel size of , is the preprocessed spectrum data; S502, call the corn leaf spectrum symbolization aggregation module to obtain a symbolization aggregation vector set; the input of the corn leaf spectrum symbolization aggregation module is RegionSpectraTensor= , and the output is the symbolization aggregation vector set CharacterS; S503, call the corn leaf region associated context modulation module; the input of the corn leaf region associated context modulation module is the symbolization aggregation vector set =CharacterS; and the output is a global context aggregation feature vector ; S504, input the global context aggregation feature vector into a final regressor to obtain a nitrogen content prediction value:

[0037] wherein, , and are weight matrices, is an activation function.

[0038] Further, the detection model is trained, validated and tested by using a training set TrainSet, a validation set ValSet and a test set TestSet respectively in S6 to obtain a nitrogen content detection model Model, specifically: S601, call the detection model, train the detection model by using the training set TrainSet, monitor the training of the detection model by using the validation set ValSet, stop the iterative training by using an early stopping strategy, and obtain a trained detection model; S602, test the trained detection model by using the test set TestSet to obtain a coefficient of determination R2. , root mean square error , mean absolute error , if , or , the trained detection model is output as a nitrogen content detection model Model; otherwise, S601 is returned to retrain.

[0039] Further, the NewHyperImage and the NewRgbImage are used to obtain the to-be-predicted spectral data, the to-be-predicted spectral data is input into the nitrogen content detection model Model, the nitrogen content prediction value PredictedValues is obtained, and the final prediction result NitrogenContent is obtained by using the nitrogen content prediction value PredictedValues, and specifically: S701, input the to-be-measured corn leaf hyperspectral image NewHyperImage and the to-be-measured corn leaf RGB image NewRgbImage; S702, call the regional spectral data table acquisition operator to process the NewHyperImage and the NewRgbImage, and acquire the regional spectral data table NewSpectraTable; S703, pre-process each spectral data in the NewSpectraTable to obtain the pre-processed spectral data ProcessedNewSpectra; The pre-processing of each spectral data in the NewSpectraTable includes: smoothing and derivative transformation by using a filter, standard normal variable transformation SNV processing, and global standardization processing; S704, input the ProcessedNewSpectra into the nitrogen content detection model Model, and output the nitrogen content prediction value PredictedValues; S705, inverse standardization and inverse logarithmic transformation are performed on the PredictedValues to obtain the final nitrogen content value NitrogenContent and output.

[0040] The beneficial effects of the present application are: This invention first performs multi-scale denoising at the spectral input, thereby preserving the original spectral shape while improving the robustness of local features. Then, a spectral symbolic aggregation operator adaptively aggregates the high-dimensional spectrum into a finite number of symbolic aggregation units, capturing the differential contributions of different band groups to nitrogen content. A regional correlation context modulation mechanism is introduced into the maize leaf regional correlation context modulation module to weighted aggregate features from different leaf regions, highlighting the feature expression of nitrogen-sensitive regions. Simultaneously, cross-regional interactive coding and channel-spectral position attention mechanisms are employed to enhance the modeling ability of long-range dependencies. This invention not only avoids the problems of single feature expression, reliance on large samples, and insufficient interpretability in traditional spectral index methods, but also achieves adaptive enhancement of nitrogen-sensitive bands and key regions, thereby improving the accuracy of nitrogen content prediction. Even with limited sample conditions, this invention can still achieve high-precision detection and robust prediction of nitrogen content in maize leaves. Attached Figure Description

[0041] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0042] Specific implementation method one: as follows Figure 1 As shown, the specific process of the method for detecting nitrogen content in maize leaves based on spectral symbolic aggregation operators in this embodiment is as follows: S1. Construct a regional spectral data table acquisition operator. The input to the regional spectral data table acquisition operator is a hyperspectral image (HyperImage) and an RGB image (RgbImage) of maize leaves. The operator uses the HyperImage and RgbImage to obtain hyperspectral region blocks, obtains the reflectance of maize leaves in the region corresponding to each hyperspectral region block, and stores the reflectance of maize leaves in each region along with the corresponding nitrogen content label into the regional spectral data table RegionSpectraTable. Specifically: S101. Input a hyperspectral image of maize leaves (HyperImage) and a RGB image of maize leaves (RgbImage), and obtain the width, height, and total number of bands of the HyperImage. Initialize the matrix SingleImage to zero. Among them, the matrix SingleImage is a matrix with the same height and width as the hyperspectral image of maize leaves HyperImage; S102, read the image data of each band of the hyperspectral image HyperImage in sequence, and acquire the minimum value TempMin and the maximum value TempMax in the image data of the current band, normalize the image data of the current band by using the TempMin and the TempMax, and the normalization process is specifically as follows:

[0043] wherein, is the normalized image data of the current band, is the image data of the current band, is the minimum value in the image data of the current band, and TempMax is the maximum value in the image data of the current band; S103, accumulate the normalized image data of each band to SingleImage, and update SingleImage, that is, SingleImage=updated SingleImage, and obtain the fused spectral image FusionImage by using SingleImage:

[0044] wherein, is the total number of bands of the hyperspectral image HyperImage of the corn leaf; S104, obtain the normalized vegetation index by using the near-infrared band and the red band of each pixel of HyperImage, and acquire the vegetation candidate region mask NdviMask by using the normalized vegetation index, and the process is specifically as follows: First, acquire the near-infrared band and the red band of each pixel in HyperImage, and acquire the normalized vegetation index of each pixel by using the near-infrared band and the red band of each pixel, and the process is specifically as follows:

[0045] wherein, is the normalized vegetation index of the current pixel, is the near-infrared band of the current pixel, is the red band of the current pixel; Then, set the vegetation index threshold to compare the normalized vegetation index NDVI of the current pixel with the vegetation index threshold, if then assign the current pixel to 1, and determine that the current pixel is vegetation; otherwise, assign the current pixel to 0, and determine that the current pixel is background, and finally obtain the vegetation candidate region mask NdviMask; S105, obtain a preliminary leaf mask LeafMask by using the FusionImage, the corn leaf hyperspectral image HyperImage, and the corn leaf RGB image, specifically as follows: S105-1, obtain a multi-source feature set FeatureSet by using the FusionImage, the corn leaf hyperspectral image HyperImage, and the corn leaf RGB image, specifically as follows: convert the RGB image RgbImage to HSV and Lab color spaces respectively, and extract color features and brightness features; The purpose of this step is to enhance the illumination robustness and texture difference of the leaf region; the brightness feature refers to the H channel of the HSV space, and the color feature refers to the a channel of the Lab space; use the normalized vegetation index array of each pixel in the corn leaf hyperspectral image HyperImage to form a matrix HyperImageNDVI; use a texture operator (local standard deviation filter) to process the fusion spectral image FusionImage, and extract the texture feature TextureC of the leaf region; convert the RGB image RgbImage to a grayscale image, and use a Canny operator to perform edge detection on the grayscale image of RgbImage to obtain a leaf contour structure feature map EdgeMap; use the fusion spectral image FusionImage, the texture feature TextureC, the contour structure feature map EdgeMap, and the HyperImageNDVI to form a multi-source feature set FeatureSet; S105-2, realize the fusion and compression of feature information by performing weighted superposition and normalization processing on the multi-source feature FeatureSet to obtain a composite feature map CompositeFeature; S105-3, perform Otsu threshold segmentation and morphological operations (opening and closing operations, hole filling) on the composite feature map CompositeFeature to obtain a preliminary leaf mask LeafMask; S106, obtain a final leaf mask FinalLeafMask by using the preliminary leaf mask LeafMask and the vegetation candidate region mask NdviMask, and obtain M spectral region blocks within the range of FinalLeafMask, obtain the corn leaf reflectance parameters of each band in each spectral region block, and set a nitrogen content label for the corn leaf reflectance parameters of each band in each spectral region block, specifically as follows: S106-1. Perform a logical AND operation on the preliminary leaf mask LeafMask and the vegetation candidate region mask NdviMask to remove falsely detected pixels in non-vegetation regions and obtain the final leaf mask FinalLeafMask. S106-2. Within the FinalLeafMask range, randomly select M points as region sampling locations, and extract the RegionSize with the region sampling locations as the center points. RegionSize hyperspectral region block; In this invention, RegionSize is set to 5; S106-3. Obtain the final leaf mask value of each pixel in the hyperspectral region block, use the final leaf mask value of each pixel to determine whether the current pixel is a valid pixel, and obtain the number of valid pixels in the hyperspectral region block. Use the number of valid pixels in each hyperspectral region block to obtain the corn leaf reflectance parameter of each band in each hyperspectral region block. The method of determining whether a current pixel is a valid pixel using the final leaf mask value of each pixel is as follows: If the final leaf mask value of the current pixel is 1, it means that the current pixel belongs to the leaf region and the current pixel is a valid pixel; if the final leaf mask value of the current pixel is 0, it means that the current pixel is an invalid pixel. The method of obtaining the corn leaf reflectance parameter for each band in each hyperspectral region block by utilizing the number of effective pixels in each hyperspectral region block is specifically as follows:

[0046] in, It is the first In the first hyperspectral region block Reflectance of corn leaves in each wavelength band It is the first The total number of effective pixels in each hyperspectral region block It is the first Effective pixel labels in each hyperspectral region block It is the first In the first hyperspectral region block The effective pixel in the first The original values ​​of each band, , These are hyperspectral region block labels; S106-4. Set nitrogen content labels for the corn leaf reflectance parameters of each band in each spectral region block; S107. Store the nitrogen content label and all maize leaf reflectance parameters in each spectral region block into the RegionSpectraTable. The final output of this step is a CSV file, which will be used for the training and validation of subsequent deep learning models.

[0047] S2. Construct a maize leaf spectral preprocessing module. This module utilizes RegionSpectraTable to construct a training set (TrainSet), a validation set (ValSet), and a test set (TestSet). Specifically: S201. The Savitzky-Golay filter is used to smooth and perform derivative transformation on the corn leaf reflectance parameter CornLeafR in the RegionSpectraTable to obtain the smoothed spectral data. Specifically: (1) Use a filter with a window size of 11 and a polynomial order of 2 to smooth the spectrum of CornLeafR and reduce noise; (2) Apply a first derivative transformation with a window size of 11 and a polynomial order of 3 to the smoothed CornLeafR to eliminate spectral baseline drift and enhance detail changes; In this step, each hyperspectral region block corresponds to one spectral data point; S202. Apply the standard normal variable transformation (SNV), perform mean centering and standard deviation normalization to obtain the spectral data after the standard normal variable transformation, specifically:

[0048] in, It is the first Spectral data, It is spectral data The mean, It is spectral data Standard deviation; This step further eliminates scattering effects and instrument errors; S203. Global standardization is used to process the spectral data after the standard normal variable transformation, scaling the spectral data to zero mean and unit variance to obtain preprocessed spectral data. This step ensures a balanced weight distribution of different features during model training, thereby improving numerical stability. S204. Perform a logarithmic transformation on the nitrogen content label and then standardize the logarithmically transformed nitrogen content label to obtain a preprocessed nitrogen content label. In this step, the logarithmic transformation of the nitrogen content labels can improve the skewed distribution, and the standardization process makes the nitrogen content distribution conform to Gaussian properties, thereby improving the fitting effect of the regression model. S205. The preprocessed spectral data and preprocessed nitrogen content labels are combined to form a dataset, and the dataset is divided into a training set (TrainSet), a validation set (ValSet), and a test set (TestSet) in a 7:2:1 ratio for subsequent training, tuning, and performance evaluation of the deep learning model.

[0049] Each data point in the preprocessed spectral data contains the following fields: Sample ID (SampleId), Region ID (RegionId), True value of nitrogen content (Label, mg / kg), and maize leaf reflectance parameters (Band001...BandL).

[0050] S3. Construct a symbolic aggregation module for maize leaf spectra. The input to this module is a region spectra tensor (RegionSpectraTensor), and the output is a set of symbolic aggregation vectors. Specifically: S301, Input region spectral tensor RegionSpectraTensor; in, M represents the number of regions, and L represents the total number of bands; S302. Perform embedding mapping on each row of the sequence in the RegionSpectraTensor to obtain the embedding matrix. :

[0051]

[0052] in, It is the first l Spectral input in each band, It is the sigmoid function. It is weight, It's a bias. It is the first l Embedding matrix of each band; In this step It is the first one of RegionSpectraTensor l OK; S303, Symbolize L bands as Each symbol aggregation unit uses the embedding matrix E to obtain the symbol assignment matrix. :

[0053] in, It is a matrix A The Middle k Line number l The elements of the column represent the first... kThe symbol aggregation unit for the first l The weighting of each band It is the first A symbolic prototype vector, It is a scaling factor. It is the band designation. It is the first l Embedding of each band, It is the first Embedding of each band, , It is a symbolic aggregation unit label. It is the total number of symbolic aggregation units; S304. Using symbolic allocation matrices To obtain the set of symbolic aggregated vectors, specifically: ,

[0054]

[0055] in, It is the first A symbolic aggregate vector, , It is a minimal constant that prevents the denominator from being zero. It is the first A symbolic aggregate vector, It is the feature dimension of the embedding space, representing the number of feature channels in the embedding space for each symbolic aggregation vector.

[0056] In this step, the symbolic aggregation vector set is "symbolic aggregation representation of maize leaf spectrum", which compresses redundant band information and highlights nitrogen-sensitive spectral bands.

[0057] S4. Construct a corn leaf region correlation vein modulation module. The input of the corn leaf region correlation vein modulation module is a set of symbolic aggregated vectors, and the output is a global vein aggregation feature vector. Specifically: S401, Input symbolic aggregated vector set ; S402, to Channel and spatial context modulation is performed to obtain channel attention weights:

[0058] GAP stands for Global Average Pooling, and GMP stands for Global Max Pooling. The ReLU activation function is used. and The weight matrix is ​​a learnable matrix. Channel attention weights; S403. Utilization The spatial attention weights are obtained as follows:

[0059] in, Indicates the kernel size as The one-dimensional convolution operators are AvgPool for average pooling and MaxPool for max pooling. Spatial attention weights; S404. Obtain the enhanced region feature matrix using channel attention weights and spatial attention weights:

[0060] in, It is the enhanced region feature matrix. It is an element-wise weighted operation; S405. Obtain weighted results of different regional features using the enhanced regional feature matrix. :

[0061]

[0062]

[0063]

[0064]

[0065] in, It is an activation function. , and It is a learnable linear mapping matrix. It is the dimension of K. It is the first The weighted result of regional characteristics, It is the first The weighted result of regional characteristics, It is a query vector. It is a key vector. It is a value vector; S406. Obtain the global context aggregation feature vector, specifically:

[0066]

[0067]

[0068]

[0069]

[0070] in, It is a global context aggregation feature vector. It is the first The network weight of each region For learnable weight vectors, For the first The first-class feature representation of each region It is the first The second type of mapping feature representation of each region express The region feature vector obtained after average pooling processing It is an element-wise multiplication operation. and It's weight. and It is a bias.

[0071] S5. The detection model is constructed by calling the maize leaf spectral symbolization aggregation module and the maize leaf region correlation vein modulation module. The input of the detection model is the preprocessed spectral data, and the output is the predicted nitrogen content value, specifically: S501. Input the preprocessed spectral data into the Adaptive Feature Pyramid (AFP) to obtain a feature representation that integrates multi-scale information. :

[0072] in, The kernel size is convolution, The kernel size is convolution, The kernel size is convolution, This is the preprocessed spectral data; S502, Call the corn leaf spectrum symbolization aggregation module to obtain the symbolized aggregation vector set; the input of the corn leaf spectrum symbolization aggregation module is RegionSpectraTensor= The output is a set of symbolic aggregated vectors, CharacterS; S503, Invoke the corn leaf region correlation vein modulation module; the input to the corn leaf region correlation vein modulation module is a set of symbolic aggregated vectors. =CharacterS; the output is the global feature vector. ; S504. Aggregate the global context feature vector. Input the final regressor to obtain the predicted nitrogen content:

[0073] in, , and It is the weight matrix of the fully connected layer. It is an activation function.

[0074] S6. Train, validate, and test the detection model using the training set TrainSet, validation set ValSet, and test set TestSet respectively to obtain the nitrogen content detection model Model, specifically: S601. Call the detection model, train the detection model using the training set TrainSet, monitor the training process of the detection model using the validation set ValSet, stop iterative training using an early stopping strategy, and obtain the trained detection model. In this step, the detection model is iteratively trained through forward propagation, error calculation, and backpropagation. The early stopping strategy for stopping iterative training is as follows: obtain the determination coefficient of the trained detection model in each round, and stop iterative training when the determination coefficient no longer increases; the trained detection model corresponding to the maximum value of the determination coefficient is the nitrogen content detection model. The formula for the coefficient of determination is as follows:

[0075] in, It is the sample label. It is the total number of samples. It is the first The true nitrogen content of each sample It is the first Predicted nitrogen content for each sample It is the average value of the actual nitrogen content. It is the coefficient of determination; This step monitors the training process and uses strategies such as early stopping, learning rate scheduling, and weight decay to prevent overfitting and improve generalization ability, so as to obtain a well-trained detection model corresponding to the model parameters that perform best on the validation set. S602. Test the trained detection model using the TestSet to obtain the coefficient of determination. Root mean square error With mean absolute error ,like , or The trained detection model will be output as the nitrogen content detection model; otherwise, return to S601 to retrain. The coefficient of determination Root mean square error With mean absolute error It can be obtained through the following formula:

[0076] S7. Acquire the hyperspectral image (NewHyperImage) and RGB image (NewRgbImage) of the maize leaves to be tested. Use NewHyperImage and NewRgbImage to obtain the spectral data to be predicted, ProcessedNewSpectra. Input the spectral data to be predicted, ProcessedNewSpectra, into the nitrogen content detection model to obtain the predicted nitrogen content values, PredictedValues. Use the predicted nitrogen content values, PredictedValues, to obtain the final prediction result, NitrogenContent, specifically: S701. Input the hyperspectral image of the maize leaf to be tested, NewHyperImage, and the RGB image of the maize leaf to be tested, NewRgbImage, ensuring that the input format is consistent with the training phase. S702. Call the regional spectral data table to obtain the operator processing NewHyperImage and NewRgbImage, and obtain the regional spectral data table NewSpectraTable; S703. Preprocess each spectral data in NewSpectraTable to obtain preprocessed spectral data ProcessedNewSpectra. The preprocessing of each spectral data in NewSpectraTable includes: smoothing and derivative transformation using a Savitzky-Golay filter, standard normal variable transformation (SNV) processing, and global normalization processing. S704. Input ProcessedNewSpectra into the nitrogen content detection model Model, perform forward propagation, and output the predicted nitrogen content values ​​PredictedValues. S705. Perform inverse standardization and inverse logarithmic transformation (expm1) on PredictedValues ​​to obtain the final nitrogen content value NitrogenContent and output it.

Claims

1. A method for detecting nitrogen content in maize leaves based on spectral symbolic aggregation operators, characterized in that... The specific process of the method is as follows: Acquire the hyperspectral image (NewHyperImage) and RGB image (NewRgbImage) of the maize leaf to be tested. Use NewHyperImage and NewRgbImage to obtain the spectral data to be predicted. Input the spectral data to be predicted into the nitrogen content detection model to obtain the predicted values ​​of nitrogen content (PredictedValues). Use the predicted values ​​of nitrogen content (PredictedValues) to obtain the final prediction result (NitrogenContent).

2. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 1, characterized in that: The nitrogen content detection model is obtained in the following way: S1. Construct a regional spectral data table acquisition operator. The input of the regional spectral data table acquisition operator is a hyperspectral image of maize leaves (HyperImage) and an RGB image of maize leaves (RgbImage). The regional spectral data table acquisition operator uses HyperImage and RgbImage to obtain hyperspectral region blocks, obtains the reflectance of maize leaves in the region corresponding to each hyperspectral region block, and stores the reflectance of maize leaves in each region and the corresponding nitrogen content label in the regional spectral data table (RegionSpectraTable). S2. Construct a maize leaf spectral preprocessing module. The maize leaf spectral preprocessing module uses RegionSpectraTable to construct a training set TrainSet, a validation set ValSet, and a test set TestSet. S3. Construct a symbolic aggregation module for maize leaf spectra. The input of the symbolic aggregation module for maize leaf spectra is the region spectra tensor RegionSpectraTensor, and the output is a set of symbolic aggregation vectors. S4. Construct a corn leaf region correlation vein modulation module. The input of the corn leaf region correlation vein modulation module is a set of symbolic aggregated vectors, and the output is a global vein aggregation feature vector. ; S5. Call the corn leaf spectral symbolization aggregation module and the corn leaf region correlation vein modulation module to build a detection model. The input of the detection model is the preprocessed spectral data, and the output is the predicted value of nitrogen content. S6. Use the training set TrainSet, validation set ValSet, and test set TestSet to train, validate, and test the detection model respectively to obtain the nitrogen content detection model Model.

3. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 2, characterized in that: The operator for constructing the regional spectral data table in S1 takes a hyperspectral image (HyperImage) and an RGB image (RgbImage) of maize leaves as inputs. The operator uses the HyperImage and RgbImage to obtain hyperspectral region blocks, acquires the reflectance of maize leaves in each hyperspectral region block, and stores the reflectance of maize leaves in each region along with the corresponding nitrogen content label in the RegionSpectraTable. Specifically: S101. Input a hyperspectral image of maize leaves (HyperImage) and a RGB image of maize leaves (RgbImage), and obtain the width, height, and total number of bands of the HyperImage. Initialize the matrix SingleImage to zero. Among them, the matrix SingleImage is a matrix with the same height and width as the hyperspectral image of maize leaves HyperImage; S102. Sequentially read the image data of each band of the hyperspectral image HyperImage, and simultaneously obtain the minimum value TempMin and the maximum value TempMax in the image data of the current band. Normalize the image data of the current band using TempMin and TempMax, specifically as follows: in, It is the normalized image data for the current band. This is the image data for the current band. It is the minimum value in the image data of the current band, and TempMax is the maximum value in the image data of the current band; S103. Accumulate the normalized image data of each band into SingleImage, and update SingleImage, setting SingleImage = the updated SingleImage. Use SingleImage to obtain the fused spectral image FusionImage: in, This represents the total number of bands in the hyperspectral image of maize leaves (HyperImage). S104. Obtain the normalized vegetation index (NDI) using the near-infrared and red bands of each pixel in the HyperImage, and then use the NDI to obtain the vegetation candidate region mask NdviMask. Specifically: First, the near-infrared and red bands of each pixel in the HyperImage are obtained. Then, the normalized vegetation index (NDI) of each pixel is obtained using the near-infrared and red bands. Specifically: in, It is the normalized vegetation index of the current pixel. It is the near-infrared band of the current pixel. It is the red light band of the current pixel; Then, the normalized difference vegetation index (NDVI) of the current pixel is compared with the vegetation index threshold. If If the value is 1, then the current cell is assigned a value of 1; otherwise, the current cell is assigned a value of 0, and finally the vegetation candidate region mask NdviMask is obtained. in, It is the vegetation index threshold; S105. Using FusionImage, HyperImage (a hyperspectral image of corn leaves), and RGB image of corn leaves, a preliminary leaf mask (LeafMask) is obtained, specifically as follows: S105-1. Obtain a multi-source feature set (FeatureSet) using FusionImage, HyperImage (hyperspectral image of maize leaves), and RGB image of maize leaves, specifically: The RGB image (RgbImage) is converted to HSV and Lab color spaces respectively to obtain color and brightness features. The normalized vegetation index (NDI) of each pixel in the hyperspectral image of maize leaves (HyperImage) is used to form the matrix HyperImageNDVI. The texture operator is used to process the fused spectral image FusionImage to extract the texture features TextureC of the leaf region; The RGB image RgbImage is converted into a grayscale image, and the Canny operator is used to perform edge detection on the grayscale image of RgbImage to obtain the blade contour structure feature map EdgeMap. A multi-source feature set, FeatureSet, is composed of FusionImage (fusion spectral image), TextureC (texture feature), EdgeMap (contour structure feature map), and HyperImageNDVI (hyperimage NDVI). S105-2. By performing weighted superposition and normalization on the multi-source feature set, a composite feature map (CompositeFeature) is obtained. S105-3. Perform Otsu threshold segmentation and morphological operations on the CompositeFeature to obtain the preliminary LeafMask. S106. Using the preliminary leaf mask LeafMask and the vegetation candidate area mask NdviMask, obtain the final leaf mask FinalLeafMask. Within the FinalLeafMask range, obtain M spectral region blocks, obtain the maize leaf reflectance parameters for each band in each spectral region block, and set a nitrogen content label for the maize leaf reflectance parameters for each band in each spectral region block. S107. Store the nitrogen content label and all maize leaf reflectance parameters in each spectral region block into the RegionSpectraTable.

4. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 3, characterized in that: In step S106, the preliminary leaf mask (LeafMask) and the vegetation candidate region mask (NdviMask) are used to obtain the final leaf mask (FinalLeafMask). Within the FinalLeafMask range, M spectral regions are obtained. The reflectance parameters of maize leaves in each band of each spectral region are obtained, and a nitrogen content label is set for the reflectance parameters of maize leaves in each band of each spectral region. Specifically: S106-1. Perform a logical AND operation on the preliminary leaf mask LeafMask and the vegetation candidate area mask NdviMask to obtain the final leaf mask FinalLeafMask. S106-2. Within the FinalLeafMask range, randomly select M points as region sampling locations, and extract the RegionSize with the region sampling locations as the center points. RegionSize hyperspectral region block; S106-3. Obtain the final leaf mask value of each pixel in the hyperspectral region block, use the final leaf mask value of each pixel to determine whether the current pixel is a valid pixel, and obtain the number of valid pixels in the hyperspectral region block. Use the number of valid pixels in each hyperspectral region block to obtain the corn leaf reflectance parameter of each band in each hyperspectral region block. The method of determining whether a current pixel is a valid pixel using the final leaf mask value of each pixel is as follows: If the final leaf mask value of the current pixel is 1, it means that the current pixel is a valid pixel; if the final leaf mask value of the current pixel is 0, it means that the current pixel is an invalid pixel. The method of obtaining the corn leaf reflectance parameter for each band in each hyperspectral region block by utilizing the number of effective pixels in each hyperspectral region block is specifically as follows: in, It is the first In the first hyperspectral region block Reflectance of corn leaves in each wavelength band It is the first The total number of effective pixels in each hyperspectral region block It is the first Effective pixel labels in each hyperspectral region block It is the first In the first hyperspectral region block The effective pixel in the first The original values ​​of each band, , These are hyperspectral region block labels; S106-4. Set nitrogen content labels for the corn leaf reflectance parameters of each band in each spectral region block.

5. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 4, characterized in that: The corn leaf spectral preprocessing module in S2 utilizes RegionSpectraTable to construct a training set (TrainSet), a validation set (ValSet), and a test set (TestSet). Specifically: S201. The corn leaf reflectance parameter CornLeafR in RegionSpectraTable is smoothed and transformed by a filter to obtain the smoothed spectral data. S202. Perform a standard normal transformation on the spectral data to obtain the transformed spectral data, specifically as follows: in, It is the first Spectral data, It is spectral data The mean, It is spectral data Standard deviation; S203. Global standardization is used to process the spectral data after the standard normal variable transformation, scaling the spectral data to zero mean and unit variance to obtain preprocessed spectral data. S204. Perform a logarithmic transformation on the nitrogen content label and then standardize the logarithmically transformed nitrogen content label to obtain a preprocessed nitrogen content label. S205. The preprocessed spectral data and the preprocessed nitrogen content labels are combined to form a dataset, and the dataset is divided into a training set TrainSet, a validation set ValSet, and a test set TestSet.

6. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 5, characterized in that: The S3 module for constructing symbolic aggregation of maize leaf spectra takes a region spectra tensor (RegionSpectraTensor) as input and outputs a set of symbolic aggregation vectors. Specifically: S301, Input region spectral tensor RegionSpectraTensor; S302. Perform embedding mapping on each row of the sequence in the RegionSpectraTensor to obtain the embedding matrix. : in, It is the first one of RegionSpectraTensor l OK, It is the sigmoid function. It's weight. It's a bias. It is the first l Embedding matrix of each band; S303, will L Each band is symbolized as Each symbol aggregation unit utilizes an embedding matrix E Obtain the symbolic allocation matrix ; matrix A The Middle k Line number l The elements of the column are, specifically: in, It is the first k The symbol aggregation unit for the first l The weighting of each band It is the first A symbolic prototype vector, It is a scaling factor. It is the band designation. It is the first l Embedding of each band, It is the first Embedding of each band, , It is a symbolic aggregation unit label. It is the total number of symbolic aggregation units; S304. Using symbolic allocation matrices To obtain the set of symbolic aggregated vectors, specifically: in, It is the first A symbolic aggregate vector, , It is a constant. It is the first A symbolic aggregate vector.

7. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 6, characterized in that: The corn leaf region correlation vein modulation module in S4 takes a set of symbolic aggregated vectors as input and outputs a global vein aggregation feature vector. Specifically: S401, Input symbolic aggregated vector set ; S402, to Channel and spatial context modulation is performed to obtain channel attention weights: GAP stands for Global Average Pooling, and GMP stands for Global Max Pooling. The ReLU activation function is used. and This is the weight matrix. Channel attention weights; S403. Utilization The spatial attention weights are obtained as follows: in, Indicates the kernel size as The one-dimensional convolution operators are AvgPool for average pooling and MaxPool for max pooling. Spatial attention weights; S404. Obtain the enhanced region feature matrix using channel attention weights and spatial attention weights: in, It is the enhanced region feature matrix. It is an element-wise weighted operation; S405. Obtain weighted results of different regional features using the enhanced regional feature matrix. : in, It is an activation function. , and It is a linear mapping matrix. It is the dimension of K. It is the first The weighted result of regional characteristics, It is the first The weighted result of regional characteristics, It is a query vector. It is a key vector. It is a value vector; S406. Obtain the global context aggregation feature vector, specifically: in, It is a global context aggregation feature vector. It is the first The network weight of each region For the weight vector, For the first The first-class feature representation of each region It is the first The second type of mapping feature representation of each region It is a region feature vector. It is an element-wise multiplication operation. and It's weight. and It is a bias.

8. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 7, characterized in that: In step S5, the detection model is constructed by calling the maize leaf spectral symbolization aggregation module and the maize leaf region correlation vein modulation module. The input of the detection model is the preprocessed spectral data, and the output is the predicted nitrogen content value. Specifically: S501. Input the preprocessed spectral data into the Adaptive Feature Pyramid (AFP) to obtain a feature representation that integrates multi-scale information. : in, The kernel size is convolution, The kernel size is convolution, The kernel size is convolution, This is the preprocessed spectral data; S502, Call the corn leaf spectrum symbolization aggregation module to obtain the symbolized aggregation vector set; the input of the corn leaf spectrum symbolization aggregation module is RegionSpectraTensor= The output is a set of symbolic aggregated vectors, CharacterS; S503, Invoke the corn leaf region correlation vein modulation module; the input to the corn leaf region correlation vein modulation module is a set of symbolic aggregated vectors. =CharacterS; the output is the global feature vector. ; S504. Aggregate the global context feature vector. Input the final regressor to obtain the predicted nitrogen content: in, , and It is a weight matrix. It is an activation function.

9. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 8, characterized in that: In step S6, the detection model is trained, validated, and tested using the training set TrainSet, validation set ValSet, and test set TestSet respectively to obtain the nitrogen content detection model Model. Specifically: S601. Call the detection model, train the detection model using the training set TrainSet, monitor the training process of the detection model using the validation set ValSet, stop iterative training using an early stopping strategy, and obtain the trained detection model. S602. Test the trained detection model using the TestSet to obtain the coefficient of determination. Root mean square error With mean absolute error ,like , or If the training is successful, the trained detection model will be output as the nitrogen content detection model; otherwise, return to S601 to retrain.

10. The method for detecting nitrogen content in maize leaves based on a spectral symbolic aggregation operator according to claim 9, characterized in that: The process involves obtaining the spectral data to be predicted using NewHyperImage and NewRgbImage, inputting this spectral data into the nitrogen content detection model to obtain predicted values ​​(PredictedValues), and then using these predicted values ​​to obtain the final prediction result, NitrogenContent. Specifically: S701. Input the hyperspectral image of the corn leaf to be tested, NewHyperImage, and the RGB image of the corn leaf to be tested, NewRgbImage. S702. Call the regional spectral data table to obtain the operator processing NewHyperImage and NewRgbImage, and obtain the regional spectral data table NewSpectraTable; S703. Preprocess each spectral data in NewSpectraTable to obtain preprocessed spectral data ProcessedNewSpectra. The preprocessing of each spectral data in NewSpectraTable includes: smoothing and derivative transformation using filters, standard normal variable transformation (SNV) processing, and global normalization processing. S704. Input ProcessedNewSpectra into the nitrogen content detection model Model, and output the predicted nitrogen content values ​​PredictedValues. S705. Perform inverse standardization and inverse logarithmic transformation on PredictedValues ​​to obtain the final nitrogen content value NitrogenContent and output it.

Images