A soil nutrient monitoring method and system based on multi-scale feature learning

Abstract

The application discloses a soil nutrient monitoring method and system based on multi-scale feature learning, and the method comprises the following steps: S1, acquiring multi-scale hyperspectral image data of soil to be detected, performing ROI automatic cutting, and obtaining a hyperspectral image of a pure soil window; S2, performing preprocessing on effective pixel spectra in the pure soil window image according to pixels, and generating multi-class sample spectral representation through pixel mean aggregation; S3, performing feature coding on the multi-class sample spectral representation, and mapping spectral representations of different categories to latent space features of the same dimension; S4, performing channel attention weighting processing and cross-modal interaction fusion on the latent space features, and obtaining multi-modal representation containing coupling information; and S5, inputting the multi-modal representation into a multi-gate multi-expert prediction model, and outputting a predicted value of soil nutrient content; through multi-scale feature coding, cross-modal interaction fusion and the multi-gate multi-expert prediction model, the application realizes prediction of soil nutrient content.

Description

Technical Field

[0001] This invention relates to the field of soil testing technology, and more specifically to a method and system for monitoring soil nutrients based on multi-scale feature learning. Background Technology

[0002] Current research, primarily based on laboratory or field hyperspectral data, has established quantitative inversion models for soil nutrient content, demonstrating the feasibility of hyperspectral technology in rapid monitoring of soil properties. However, most studies rely mainly on one-dimensional spectral curves and feature optimization, and have not yet formed a comprehensive characterization system combining feature selection, band optimization, and spectral response. Furthermore, the comprehensive impact of factors such as noise interference on spectral characteristics and quantitative accuracy has not been elucidated.

[0003] While reasonable spectral preprocessing and feature band selection methods (such as SG derivative, wavelet denoising, SNV / MSC, and band selection based on correlation or model importance) can improve the accuracy and robustness of soil property inversion models to some extent, most current studies often perform feature selection independently under a single preprocessing procedure, without fully exploring the impact of different preprocessing combinations on model performance. Furthermore, they do not adequately consider the differences in spectral response of weakly absorbing elements (especially P and K), which can easily lead to feature dimensional redundancy and insufficient information representativeness, thereby restricting the generalizability and reliability of the model.

[0004] Furthermore, although machine learning and deep learning-based methods (such as PLSR, RF, XGBoost, ANN, CNN, etc.) can improve the accuracy and generalization ability of soil property estimation compared with traditional empirical models, most existing works focus on single-task or single-modal modeling, with insufficient attention to the joint inversion of soil nutrients, lack of multi-task learning frameworks that integrate key hyperspectral bands, and relatively lack of systematic evaluation of the robustness and stability of models at different soil type levels and spatial scales. Summary of the Invention

[0005] The purpose of this invention is to provide a soil nutrient monitoring method and system based on multi-scale feature learning, which can predict soil nutrient content through multi-scale feature encoding, cross-modal interactive fusion and multi-gated multi-expert prediction model.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A soil nutrient monitoring method based on multi-scale feature learning includes the following steps: S1. Acquire multi-scale hyperspectral image data of the soil to be tested, perform automatic ROI cropping, and obtain a hyperspectral image of the pure soil window. S2. The effective pixel spectrum in the hyperspectral image of the pure soil window is preprocessed according to pixels, and multi-class sample spectrum representation is generated by pixel mean aggregation. S3. Perform feature encoding on the spectral representations of multiple sample classes, mapping the spectral representations of different classes to latent spatial features of the same dimension; S4. Perform channel attention weighting processing and cross-modal interaction fusion on the latent spatial features to obtain a multimodal representation containing coupled information; S5. Input the multimodal representation into a multi-gated multi-expert prediction model that includes a shared expert pool and a task-gated network, and output the predicted value of soil nutrient content.

[0007] Furthermore, in S1, the process of automatically cropping the ROI from the multi-scale hyperspectral image data of the soil to be tested specifically includes: PCA is used to reduce the spectral vectors of pixels in multi-scale hyperspectral image data to obtain principal component representations, and KMeans clustering is performed based on the principal component representations to distinguish background and soil candidate clusters. Within the soil candidate cluster, a conservative threshold strategy is used to filter out bright abnormal pixels, and connected component cleaning is performed to determine the main connected components of the soil. Perform a sliding window search within the main connected domain of the soil to find a pure ROI window that satisfies the condition that all pixels within the window are soil, and output the cropped pure soil hyperspectral image.

[0008] Furthermore, in S2, the process of preprocessing the effective pixel spectrum by pixel includes: performing one or more of the following processes on the effective pixel spectrum: original reflectance, standard normal variable transformation, Savitzky-Golay first derivative, Savitzky-Golay second derivative, wavelet denoising, and Gaussian smoothing.

[0009] Furthermore, in step S4, the method for obtaining the multimodal representation containing coupling information is specifically as follows: Obtain a global description of the latent spatial features; The global description is processed through a two-layer attention network to generate channel weights; The channel weights are multiplied one by one with the corresponding latent spatial features to obtain the weighted representation of each mode; The multi-head attention mechanism is used to learn the dependencies between the weighted representations of each modality, and a feedforward network is used for nonlinear combination to obtain a multimodal representation containing coupling information.

[0010] Furthermore, the process of obtaining the predicted values ​​of soil nutrient content in S5 is specifically as follows: The fused multimodal representation is input into a shared expert pool, wherein the shared expert pool contains multiple expert networks, each of which outputs an expert hidden representation. Gated networks are set up for the three tasks of nitrogen, phosphorus, and potassium. The weights of each expert network are calculated through the gated networks, and the hidden representations of the experts are weighted and aggregated to obtain the task-specific representations and output the predicted values ​​of soil nutrient content.

[0011] Furthermore, in S5, the training process of the multi-gated multi-expert prediction model specifically involves: introducing dynamic weight terms and adaptively adjusting the loss weight ratios of different tasks based on the sliding statistics of the loss of each task during the training process of the multi-gated multi-expert prediction model, in order to reduce multi-task negative transfer.

[0012] The present invention also provides a system for performing a soil nutrient monitoring method based on multi-scale feature learning, comprising: The input module is used to acquire multi-scale hyperspectral image data of the soil to be tested; The ROI automatic cropping module is used to automatically crop the ROI from the multi-scale hyperspectral image data of the soil to be tested, and obtain a hyperspectral image of the pure soil window. The spectral preprocessing and statistical analysis module is used to preprocess the effective pixel spectrum in the pure soil window image by pixel, and generate multi-class sample spectral representations by aggregating pixel mean values, and output correlation Excel / CSV and r curve graphs; The multimodal fusion and interaction module is used to encode the features of spectral representations of multiple types of samples, mapping spectral representations of different categories to latent spatial features of the same dimension. The latent spatial features are subjected to channel attention weighting and cross-modal interaction fusion to obtain a multimodal representation containing coupled information; The multi-task prediction module is used to input the multimodal representation into a multi-gated multi-expert prediction model that includes a shared expert pool and a task gating network, and output the predicted value of soil nutrient content. The results output and visualization module is used to output prediction results, evaluation scatter plots, processing diagrams, and log information.

[0013] Furthermore, the process by which the input module acquires multi-scale hyperspectral image data of the soil to be detected is specifically as follows: If the input module receives a single multi-scale hyperspectral image data TIF file, it treats it as a single sample. If it receives a folder, it iterates through all the multi-scale hyperspectral image data TIF files within it as batch samples. The input module extracts numbers from the filenames as sample serial numbers and aligns them with the serial number field in the laboratory truth table for use in supervised learning during the training phase and for comparison with the truth during the evaluation phase.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a soil nutrient monitoring method based on multi-scale feature learning.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a soil nutrient monitoring method based on multi-scale feature learning.

[0016] According to specific embodiments provided by the present invention, the present invention has the following technical effects compared to the prior art: This invention acquires multi-scale hyperspectral images of the soil to be tested and performs automatic ROI cropping and pure soil window extraction, effectively eliminating background noise interference and ensuring data purity. Combined with pixel mean aggregation preprocessing, it significantly improves the signal-to-noise ratio and stability of the sample spectrum. By mapping different types of spectra to a unified latent space through feature encoding, and introducing channel attention weighting and cross-modal interactive fusion mechanisms, it not only strengthens the feature weights of key spectral bands, but also achieves effective coupling and complementarity of multi-scale information, thereby greatly enhancing the expressive power of features. Through a multi-gated multi-expert prediction model that includes a shared expert pool and a task-gated network, it can adaptively mine shared features and specifically allocate them for different nutrient prediction tasks, effectively solving the problem of nonlinear complexity of hyperspectral data, and significantly improving the accuracy of soil nutrient content monitoring and the generalization robustness of the model. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] The following description, in conjunction with the accompanying drawings, further illustrates a soil nutrient monitoring method and system based on multi-scale feature learning according to the present invention; Figure 1 This is a schematic diagram of the overall process of the soil nutrient monitoring method based on multi-scale feature learning in Embodiment 1 of the present invention; Figure 2 This is a Tif sample example of the soil nutrient monitoring method based on multi-scale feature learning in Embodiment 1 of the present invention; Figure 3 is an example of soil region clipping in the soil nutrient monitoring method based on multi-scale feature learning in Embodiment 1 of the present invention; where (a) is the soil candidate region and (b) is the result after clipping the pure soil ROI; Figure 4This is an example of the spectral characteristic curve of the soil nutrient monitoring method based on multi-scale feature learning in Embodiment 1 of the present invention; Figure 5 This is a band selection example of the soil nutrient monitoring method based on multi-scale feature learning in Embodiment 1 of the present invention; Figure 6 This is an example diagram of the evaluation indicators for the soil nutrient monitoring method based on multi-scale feature learning in Embodiment 1 of the present invention; Figure 7 This is a structural diagram of the soil nutrient monitoring system based on multi-scale feature learning in Embodiment 2 of the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0020] To better understand the purpose, structure, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings.

[0021] Example 1 like Figure 1 As shown, this invention provides a soil nutrient monitoring method based on multi-scale feature learning, comprising the following steps: S1. Acquire multi-scale hyperspectral image data of the soil to be tested, perform automatic ROI cropping, and obtain a hyperspectral image of the pure soil window. In step S1, the process of automatically cropping the ROI from the multi-scale hyperspectral image data of the soil to be tested specifically includes: PCA is used to reduce the spectral vectors of pixels in multi-scale hyperspectral image data to obtain principal component representations, and KMeans clustering is performed based on the principal component representations to distinguish background and soil candidate clusters. Within the soil candidate cluster, a conservative threshold strategy is used to filter out bright abnormal pixels, and connected component cleaning is performed to determine the main connected components of the soil. Perform a sliding window search within the main connected domain of the soil to find a pure ROI window that satisfies the condition that all pixels within the window are soil, and output the cropped pure soil hyperspectral image.

[0022] If the multi-scale hyperspectral image data TIF file is a single file, it is treated as a single sample; if it is a folder, all multi-scale hyperspectral image data TIF files within it are processed as batch samples. Figure 2 As shown, numbers are extracted from filenames as sample serial numbers and aligned with the serial number field in the laboratory truth table for use in supervised learning during the training phase and for comparison with the truth during the evaluation phase.

[0023] This embodiment specifically employs automatic ROI cropping (pure soil window search).

[0024] For each hyperspectral TIF image, the pixel spectral vectors are first reduced in dimensionality to obtain principal component representations, and then clustering is performed to distinguish background and soil candidate clusters. Within the soil candidate clusters, a conservative strategy is used to further filter out bright abnormal pixels and clean connected components. Subsequently, a sliding window search is used within the main connected components of the soil to find a pure ROI window that satisfies the condition that "all pixels within the window are soil," and the hyperspectral TIF image after ROI cropping is output, as shown in Figure 3. The purpose of this step is to remove background and edge intrusion to the greatest extent possible, ensuring the stability and consistency of the model input data.

[0025] S2. The effective pixel spectrum in the pure soil window image is preprocessed according to the pixel, and multi-class sample spectral representation is generated by pixel mean aggregation; In S2, the process of preprocessing the effective pixel spectrum by pixel includes: performing one or more of the following processes on the effective pixel spectrum: original reflectance, standard normal variable transformation, Savitzky-Golay first derivative, Savitzky-Golay second derivative, wavelet denoising, and Gaussian smoothing.

[0026] This embodiment includes: for each ROI cropped sample, preprocessing the effective pixel spectrum (excluding all zero or NaN pixels) by pixel, and then taking the average value of all effective pixels to obtain the sample spectral curve, such as... Figure 4 As shown. Preprocessing includes 14 categories: raw reflectance, SNV, first-order SG derivative, second-order SG derivative, wavelet denoising, Gaussian smoothing, and combinations thereof. The system outputs a sample-level spectral summary table and band statistics table for each type of processing, and plots single-sample spectral curves and full-sample mean ± standard deviation curves to verify the preprocessing effect and data quality.

[0027] The system reads the laboratory truth table, performs band-by-band Pearson correlation analysis on the summarized spectra and N / P / K true values ​​of each type of preprocessed sample, outputs the correlation coefficient r and significance p for each band, and plots correlation curves and significance point markers, such as... Figure 5 As shown. This step provides quantitative evidence for band sensitivity and can be used for subsequent band selection or to explain the rationale for the bands of interest in the model.

[0028] S3. Perform feature encoding on the spectral representations of multiple sample classes, mapping the spectral representations of different classes to latent spatial features of the same dimension; S4. Perform channel attention weighting processing and cross-modal interaction fusion on the latent spatial features to obtain a multimodal representation containing coupled information; In step S4, the method for obtaining the multimodal representation containing coupling information is as follows: Obtain a global description of the latent spatial features; The global description is processed through a two-layer attention network to generate channel weights; The channel weights are multiplied one by one with the corresponding latent spatial features to obtain the weighted representation of each mode; The multi-head attention mechanism is used to learn the dependencies between the weighted representations of each modality, and a feedforward network is used for nonlinear combination to obtain a multimodal representation containing coupling information.

[0029] In this embodiment, the multi-head attention mechanism learning includes: channel attention (CA) and cross-modal interaction (CMI), specifically: A channel attention mechanism is introduced for each modality-coded output z: first, a global description s is obtained, then channel weights a are generated through a two-layer attention network, and finally, the weighted representation z′=z⊙a is obtained by multiplying each channel, so as to automatically highlight key channels and suppress redundant noise.

[0030] Subsequently, multi-head attention is used to learn intermodal dependencies, and a feedforward network is used to enhance the nonlinear combination capability, resulting in a multimodal representation that includes coupling information. This step differs from existing simple splicing methods, as it can explicitly capture modal complementarity and enhance the representation capability of N / P / K related structural information.

[0031] S5. Input the multimodal representation into a multi-gated multi-expert prediction model that includes a shared expert pool and a task-gated network, and output the predicted value of soil nutrient content.

[0032] In step S5, the process of introducing dynamic weights to adaptively adjust the loss contribution of each task and jointly outputting the predicted values ​​of soil nitrogen, phosphorus, and potassium content is as follows: The fused multimodal representation is input into a shared expert pool, wherein the shared expert pool contains multiple expert networks, each of which outputs an expert hidden representation. Gated networks are set up for the three tasks of nitrogen, phosphorus, and potassium. The weights of each expert network are calculated through the gated networks, and the hidden representations of the experts are weighted and aggregated to obtain the task-specific representations and output the predicted values ​​of soil nutrient content.

[0033] In S5, the training process of the multi-gated multi-expert prediction model includes: introducing dynamic weight terms, and adaptively adjusting the loss weight ratio of different tasks according to the sliding statistics of the loss of each task during the training process of the multi-gated multi-expert prediction model, so as to reduce the negative transfer of multiple tasks.

[0034] In this embodiment, the MMoE multi-task prediction of dynamic weight DW is specifically as follows: The fused multimodal representations are input into a shared expert pool, with each expert serving as the output of a hidden expert representation via a small fully connected network. For each task (N, P, K), a gating network is used to generate expert weights, which are then weighted and aggregated to obtain a task-specific representation before outputting the predicted value. To adapt to differences in task difficulty and sample differences, a dynamic weight term is introduced to adaptively adjust the task loss: during training, a dynamic weight ratio is formed based on the sliding statistics of the losses for each task, making training more balanced and reducing the risk of negative transference. This structure allows each task to select the optimal combination from the shared expert pool, simultaneously achieving the generalization ability brought by sharing and the task specificity brought by gating.

[0035] The method in this embodiment also includes: output and visualization; The system outputs the predicted N / P / K values ​​for a single sample during the inference phase; during the batch inference phase, it outputs a results table with columns including "Serial Number (File Name) - N - P - K". In evaluation mode, it can also output a scatter plot of the actual and predicted values, and R... 2 Metrics such as RMSE are used to validate model performance. Figure 6 As shown in the image. Furthermore, the system displays the running progress and step status on the interface, ensuring that users can track the current processing stage.

[0036] Example of the complete working process of this embodiment: (1) Prediction process of a single hyperspectral TIF.

[0037] When a user enters the path to a single TIF file on the interface and clicks "Start Recognition," the system performs the following steps: automatic cropping of the ROI in the original image → construction of spectral vectors and microscopic grain texture features → execution of CA and CMI → outputting N / P / K predicted values ​​via DW+MMoE in S8 → displaying N, P, and K contents in three result boxes on the interface and recording processing steps and gating weight information in the log area; simultaneously, the ROI cropping map and necessary process files for the sample can be generated in the specified output directory.

[0038] (2) Prediction process for batch samples in folders.

[0039] When the user enters a folder path containing multiple TIFs and clicks "Start Recognition," the system iterates through all TIFs in the folder, automatically cropping the ROI of the original image → constructing spectral vectors and microscopic grain texture features → performing CA and CMI → outputting N / P / K predicted values ​​through DW+MMoE → displaying the N, P, and K contents in three result boxes on the interface, and recording the processing steps and gating weight information in the log area. The system also outputs a table composed of the extracted sequence number of each filename and the predicted N / P / K values ​​to the user-specified directory, displays the overall progress bar and the current sample progress bar on the interface, and scrolls through the log area to output the current processed sample number and the current step status, thereby achieving batch automated detection.

[0040] In summary, this embodiment first performs automatic soil identification and ROI clipping on the input hyperspectral TIFF. Through PCA dimensionality reduction and KMeans clustering for soil / background separation and conservative threshold screening, a pure soil window is obtained, generating clipped TIFFs and ROI records. Secondly, the clipped spectra undergo multi-strategy preprocessing and aggregation, forming 14 types of spectral representations: reflectance, SNV, Savitzky-Golay first / second derivative, wavelet denoising, Gaussian smoothing, and their combinations. Sample-level spectral tables and band correlation analysis results are output. Finally, a multi-task prediction model is constructed, consisting of "Channel Attention (CA) + Cross-Modal Interaction (CMI) + Dynamic Weighting (DW) + Multi-Gated Multi-Expert (MMoE)". This model adaptively highlights key band information at the feature level and adjusts expert contributions at the task level through gating and dynamic weighting, jointly regressing to output N / P / K content. Compared to existing technologies, this embodiment also has the following technical advantages: (1) Automatic pure soil cropping of ROI significantly reduces background interference and improves the proportion of effective information and data consistency. Traditional methods often use manual selection, fixed threshold binarization, or simple morphological segmentation, which easily mixes white background, shadow edges, or impurity areas into the sample, resulting in spectral mean shift and decreased comparability between samples. This invention uses "PCA dimensionality reduction + KMeans clustering" to distinguish soil from background, and uses conservative percentile screening and connected component constraints within soil clusters to further remove high-brightness abnormal areas. Finally, it searches for pure window ROIs that are "all soil" within the main connected component of the soil, so that the pixels entering the modeling are all from stable soil areas, suppressing noise propagation and label regression bias from the source, reducing manual intervention and improving batch processing efficiency.

[0041] (2) A systematic output of 14 types of spectral preprocessing enhances robustness to noise, scattering, and baseline drift, and improves feature interpretability. Existing technologies often use only a single preprocessing method (such as SNV or SG filtering) or empirical selection, which is difficult to take into account different imaging conditions and soil sample differences. This invention generates 14 spectral representations in parallel, including reflectance, SNV, first / second derivative of SG, wavelet denoising, Gaussian smoothing, and combinations, and forms a sample-level summary table and band statistics; at the same time, it calculates the Pearson correlation coefficient and significance of N / P / K with each band, providing the basis for band contribution from the data level. This system can provide a comparable processing link between "scattering correction (SNV) - noise suppression (wavelet / Gaussian) - sensitive feature enhancement (SG derivative)," making subsequent modeling more stable for instrument noise, illumination changes, and sample surface differences.

[0042] (3) Introducing Channel Attention (CA) to enhance the weight of sensitive bands reduces the masking of weak signals and improves the upper limit of regression accuracy. Traditional multi-task or multi-input regression often directly splices features, which can easily lead to the average dilution of key bands due to high-dimensional channel redundancy. This invention adds channel attention at the output of each modal encoder, obtains channel weights through a global description—two-layer attention network—Sigmoid, and multiplies them with features channel by channel, structurally achieving "automatic amplification of sensitive bands / suppression of invalid bands". This mechanism can theoretically reduce the interference of redundant channels on gradients, making the model more focused on spectral channels that are highly correlated with changes in N / P / K, and improving feature utilization efficiency.

[0043] (4) Cross-modal interaction (CMI) explicitly models modal complementarity, reducing information fragmentation and task conflicts caused by simple sharing / splitting. Existing methods usually simply fuse multi-source information, making it difficult to capture the high-order coupling relationships between different modalities. This invention uses the weighted modal representation as a sequence input interaction module (such as multi-head attention / bilinear interaction ideas) to explicitly learn the association structure between "modal-modal" relationships, enabling the model to use complementary information to improve discriminative power and reduce fusion mismatch caused by modal scale differences, thereby improving the stability of multi-task prediction.

[0044] (5) Dynamic Weights (DW) + Multi-Gated Multi-Expert (MMoE) Reduces Negative Transfer in Multi-Task Learning and Improves Overall Consistency Among the Three Tasks. Existing multi-task learning methods often use hard sharing or fixed weight summation, which can easily lead to one task dominating training while the performance of other tasks degrades. This invention adopts a shared expert pool and a task-gated network, enabling the N / P / K tasks to adaptively select expert combinations based on samples; at the same time, it introduces a dynamic weight term based on loss contribution to adaptively adjust the training difficulty and convergence state of different tasks. Structurally, it achieves "shared expert layer but separable gating and variable task weights," which theoretically can suppress negative transfer and improve the efficiency and final accuracy of the three tasks converging together.

[0045] (6) Engineered output and reproducible evaluation improve application efficiency and reduce labor intensity. This invention automatically outputs ROI cropping results, visualizations, spectral summary tables, correlation analysis results, and model evaluation scatter plots to a designated directory, forming a closed-loop process from data preprocessing to modeling and evaluation. Compared to existing processes that require multiple manual processing and recording, this invention significantly reduces manual operation steps, lowers the probability of errors, and allows for the reproduction of results under the same standard process, facilitating deployment and promotion.

[0046] (7) Experimental data and evaluation methods (can be directly used for the instruction manual / comparative experiment writing). Under the same dataset and the same training / validation partitioning conditions, RMSE, MAE, and R... 2As an evaluation metric, the comparison method can be either "single preprocessing + single task regression" or "simple feature concatenation + shared backbone with multiple outputs".

[0047] This invention, after incorporating pure soil clipping of the ROI, results in more stable effective pixels in the sample; after adding CA+CMI+DW+MMoE, multi-task R... 2 It can be optimized simultaneously with RMSE, and the error distribution of the three tasks is more balanced. The experimental conditions are suggested as follows: the hyperspectral TIFF of the cropped soil sample collected by the same sensor and the N / P / K measured by the same laboratory are used as the ground truth; the Adam optimizer and early stopping strategy are used for training; the comparison items keep the same data input and the same index calculation method. Based on the structural characteristics, its improvement can be deduced from: ROI noise reduction, channel recalibration, modal interaction enhancement, and task-level dynamic gating and weight balancing.

[0048] The above effects are all based on the combined structure of the hyperspectral ROI pure soil trimming, 14 types of preprocessing and correlation analysis, and the CA+CMI+DW+MMoE multi-task architecture adopted in this invention. The key difference from the existing technology is that it achieves synergistic improvement from four aspects: data validity, feature adaptive emphasis, modal coupling modeling and task conflict suppression, thereby obtaining better results in terms of accuracy, robustness and automation efficiency.

[0049] Example 2 like Figure 7 As shown, the present invention also provides a system for implementing the soil nutrient monitoring method based on multi-scale feature learning in Example 1, comprising: The input module is used to acquire multi-scale hyperspectral image data of the soil to be tested; The process by which the input module acquires multi-scale hyperspectral image data of the soil to be detected is specifically as follows: If the input module receives a single multi-scale hyperspectral image data TIF file, it treats it as a single sample. If it receives a folder, it iterates through all the multi-scale hyperspectral image data TIF files within it as batch samples. The input module extracts numbers from the filenames as sample serial numbers and aligns them with the serial number field in the laboratory truth table for use in supervised learning during the training phase and for comparison with the truth during the evaluation phase.

[0050] This embodiment is used to receive the hyperspectral TIF file path or folder path input by the user, and parse the numbers in the file name as the sample number to achieve the alignment of the sample with the laboratory truth table.

[0051] The ROI automatic cropping module is used to automatically crop the ROI in the multi-scale hyperspectral image data of the soil to be tested, so as to obtain a hyperspectral image of pure soil window. Specifically, this embodiment is used to identify background and soil regions in the original hyperspectral image, automatically search for pure soil windows, output the cropped ROI hyperspectral TIF, and simultaneously output the bounding box view, the cropped area view, and the ROI coordinate record file.

[0052] The spectral preprocessing and statistical analysis module is used to preprocess the effective pixel spectrum in the pure soil window image by pixel, and generate multiple sample spectral representations by aggregating pixel mean values. It generates 14 types of preprocessing results for the cropped ROI hyperspectral data and outputs a sample-level spectral mean summary table, band statistics table, and spectral curve. At the same time, it performs Pearson correlation analysis on N / P / K bands and outputs correlation Excel / CSV and r curve.

[0053] The multimodal fusion and interaction module is used to encode the features of spectral representations of multiple types of samples, mapping spectral representations of different categories to latent spatial features of the same dimension. The latent spatial features are subjected to channel attention weighting and cross-modal interaction fusion to obtain a multimodal representation containing coupled information; This embodiment includes a channel attention module (CA) and a cross-modal interaction module (CMI), which are used to adaptively recalibrate the channels after each modality is encoded, and to explicitly model the complementary coupling relationship between modalities.

[0054] The multi-task prediction module is used to input the multimodal representation into a multi-gated multi-expert prediction model that includes a shared expert pool and a task gating network, and output the predicted value of soil nutrient content. This embodiment includes a shared expert pool and a task gating network, which are used to calculate expert weights for the three tasks N / P / K and then aggregate them in a weighted manner. At the same time, a dynamic weight DW is introduced to adaptively adjust the contribution of task loss and reduce the negative migration of multiple tasks.

[0055] The results output and visualization module is used to output prediction results, evaluation scatter plots, processing diagrams, and log information.

[0056] This embodiment outputs the predicted values ​​of sample N / P / K, a batch sample result table (serial numbers N, P, K), as well as a scatter plot of model evaluation, a processing diagram, and log information, which facilitates tracking of the running status and verification of results.

[0057] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a soil nutrient monitoring method based on multi-scale feature learning.

[0058] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a soil nutrient monitoring method based on multi-scale feature learning.

[0059] During system operation, the input module acquires the user's TIF or catalog and then passes the image to the ROI cropping module to generate pure soil ROIs. The cropped results can either be sent to the spectral preprocessing module for statistical analysis or to the feature construction module to generate model input. Finally, the MMoE module outputs the N / P / K prediction results. This structure differs from the "manual selection + single-task regression + simple stitching" approach in the background technology. The key difference lies in the introduction of pure soil ROI window search, channel attention recalibration, cross-modal interaction, and dynamic weighted MMoE, thus forming a system architecture that can run automatically, provide interpretable statistical output, and suppress multi-task conflicts.

[0060] The above description of the disclosed embodiments enables those skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A soil nutrient monitoring method based on multi-scale feature learning, characterized in that, Includes the following steps: S1. Acquire multi-scale hyperspectral image data of the soil to be tested, perform automatic ROI cropping, and obtain a hyperspectral image of the pure soil window. S2. The effective pixel spectrum in the hyperspectral image of the pure soil window is preprocessed according to pixels, and multi-class sample spectrum representation is generated by pixel mean aggregation. S3. Perform feature encoding on the spectral representations of multiple sample classes, mapping the spectral representations of different classes to latent spatial features of the same dimension; S4. Perform channel attention weighting processing and cross-modal interaction fusion on the latent spatial features to obtain a multimodal representation containing coupled information; S5. Input the multimodal representation into a multi-gated multi-expert prediction model that includes a shared expert pool and a task-gated network, and output the predicted value of soil nutrient content.

2. The soil nutrient monitoring method based on multi-scale feature learning according to claim 1, characterized in that, In step S1, the process of automatically cropping the ROI from the multi-scale hyperspectral image data of the soil to be tested specifically includes: PCA is used to reduce the spectral vectors of pixels in multi-scale hyperspectral image data to obtain principal component representations, and KMeans clustering is performed based on the principal component representations to distinguish background and soil candidate clusters. Within the soil candidate cluster, a conservative threshold strategy is used to filter out bright abnormal pixels, and connected component cleaning is performed to determine the main connected components of the soil. Perform a sliding window search within the main connected domain of the soil to find a pure ROI window that satisfies the condition that all pixels within the window are soil, and output the cropped pure soil hyperspectral image.

3. The soil nutrient monitoring method based on multi-scale feature learning according to claim 1, characterized in that, In S2, the process of preprocessing the effective pixel spectrum by pixel includes: performing one or more of the following processes on the effective pixel spectrum: original reflectance, standard normal variable transformation, Savitzky-Golay first derivative, Savitzky-Golay second derivative, wavelet denoising, and Gaussian smoothing.

4. The soil nutrient monitoring method based on multi-scale feature learning according to claim 1, characterized in that, In step S4, the method for obtaining the multimodal representation containing coupling information is as follows: Obtain a global description of the latent spatial features; The global description is processed through a two-layer attention network to generate channel weights; The channel weights are multiplied one by one with the corresponding latent spatial features to obtain the weighted representation of each mode; The multi-head attention mechanism is used to learn the dependencies between the weighted representations of each modality, and a feedforward network is used for nonlinear combination to obtain a multimodal representation containing coupling information.

5. The soil nutrient monitoring method based on multi-scale feature learning according to claim 1, characterized in that, The process of obtaining the predicted values ​​of soil nutrient content in S5 is as follows: The fused multimodal representation is input into a shared expert pool, wherein the shared expert pool contains multiple expert networks, each of which outputs an expert hidden representation. Gated networks are set up for the three tasks of nitrogen, phosphorus, and potassium. The weights of each expert network are calculated through the gated networks, and the hidden representations of the experts are weighted and aggregated to obtain the task-specific representations and output the predicted values ​​of soil nutrient content.

6. The soil nutrient monitoring method based on multi-scale feature learning according to claim 1, characterized in that, In S5, the training process of the multi-gated multi-expert prediction model is as follows: a dynamic weight term is introduced, and the loss weight ratio of different tasks is adaptively adjusted according to the sliding statistics of the loss of each task during the training process of the multi-gated multi-expert prediction model, so as to reduce the negative transfer of multiple tasks.

7. A soil nutrient monitoring system based on multi-scale feature learning, used to execute the soil nutrient monitoring method based on multi-scale feature learning as described in any one of claims 1-6, characterized in that, include: The input module is used to acquire multi-scale hyperspectral image data of the soil to be tested; The ROI automatic cropping module is used to automatically crop the ROI from the multi-scale hyperspectral image data of the soil to be tested, and obtain a hyperspectral image of the pure soil window. The spectral preprocessing and statistical analysis module is used to preprocess the effective pixel spectrum in the pure soil window image by pixel, and generate multi-class sample spectral representations by aggregating pixel mean values, and output correlation Excel / CSV and r curve graphs; The multimodal fusion and interaction module is used to encode the features of spectral representations of multiple types of samples, mapping spectral representations of different categories to latent spatial features of the same dimension. The latent spatial features are subjected to channel attention weighting and cross-modal interaction fusion to obtain a multimodal representation containing coupled information; The multi-task prediction module is used to input the multimodal representation into a multi-gated multi-expert prediction model that includes a shared expert pool and a task gating network, and output the predicted value of soil nutrient content. The results output and visualization module is used to output prediction results, evaluation scatter plots, processing diagrams, and log information.

8. The soil nutrient monitoring system based on multi-scale feature learning according to claim 7, characterized in that, The process by which the input module acquires multi-scale hyperspectral image data of the soil to be detected is as follows: If the input module receives a single multi-scale hyperspectral image data TIF file, it treats it as a single sample. If it receives a folder, it iterates through all the multi-scale hyperspectral image data TIF files within it as batch samples. The input module extracts numbers from the filenames as sample serial numbers and aligns them with the serial number field in the laboratory truth table for use in supervised learning during the training phase and for comparison with the truth during the evaluation phase.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the soil nutrient monitoring method based on multi-scale feature learning as described in any one of claims 1-6.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the soil nutrient monitoring method based on multi-scale feature learning as described in any one of claims 1-6.

Images