Hyperspectral modeling method for soil organic matter content based on sae-gs-tabnet

Abstract

The application discloses a soil organic matter content hyperspectral modeling method based on SAE-GS-TabNet, and particularly relates to the technical field of soil hyperspectral detection, and collects hyperspectral reflectivity of the same soil sample under natural water content, tablet water film removal and standard humidity recovery states and constructs a three-state spectral matrix; through bad wave band elimination, scattering correction, water film difference spectrum and structure difference spectrum fusion, an anti-water salt disturbance spectral matrix is obtained; the anti-water salt disturbance spectral matrix is input into a shared coding layer and a double decoding layer SAE, water film, salt frost and carbon particle interference components and organic matter response components are respectively reconstructed, and deep organic matter characteristics are obtained; then, based on correlation contribution, three-state consistency and interference residual error suppression, GS global stability search is carried out, a stable organic matter characteristic subset is screened, and the stable organic matter characteristic subset is used as a TabNet prior feature mask to train a prediction model; the application can reduce the interference of water film, salt frost and carbon particles on the spectral response of organic matter, and improve the prediction stability of complex field-to-field soil organic matter.

Description

Technical Field

[0001] This invention relates to the field of soil hyperspectral detection technology, specifically to a hyperspectral modeling method for soil organic matter content based on SAE-GS-TabNet. Background Technology

[0002] Soil organic matter content is an important indicator for evaluating the degree of fertility recovery in the topsoil after greenhouse vegetable fields are converted back to farmland. Hyperspectral imaging, with its advantages of speed and non-destructive testing, has been used to invert soil organic matter content. However, in areas where greenhouse vegetable fields have been converted back to farmland due to salinization, the soil has long been affected by high evaporation, heavy irrigation, fertilizer and salt accumulation, and residual plastic film. After conversion, the topsoil often exhibits uneven natural moisture content, localized salt blooms, and a mixture of carbon particles or dark spots of humus. Especially in the short period after irrigation, a very thin water film forms on the soil sample surface. The reflection of the water film, the strong scattering of the salt crust, and the absorption information of organic matter overlap in the visible-near-infrared band, causing false absorption peaks, abrupt changes in reflectance, and local saturation in the spectral curve, thus masking the true response of organic matter.

[0003] Existing hyperspectral modeling methods for soil organic matter typically involve smoothing, derivative transformation, principal component analysis, partial least squares, random forests, or neural network modeling directly from spectra acquired under a single state. These methods primarily rely on full-band statistical correlation or single feature selection results, making it difficult to distinguish between interference components such as water film, salt efflorescence, and carbon particles and organic matter response components. When the sample size is small, salinity is spatially uneven, and soil samples from the same plot exhibit significant differences in water content, the model is prone to misclassifying water-salt disturbance features as organic matter-sensitive features, leading to unstable feature selection, attention allocation bias, and abrupt changes in prediction results.

[0004] While existing TabNet-like models can perform feature attention learning, their input features are mostly derived from conventional preprocessed spectra. They lack interference separation mechanisms for the spectral differences of the three-state soil samples and do not combine water film difference, structural difference, and deep reconstruction residuals for organic matter feature screening. Therefore, in the complex small-scale scenario of converting saline-alkali facility vegetable fields back to farmland, how to extract stable organic matter features from hyperspectral data disturbed by water film, salt frost, and carbon particles, and establish a high-precision prediction model with resistance to water and salt disturbances, has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a hyperspectral modeling method for soil organic matter content based on SAE-GS-TabNet, in order to address the shortcomings of the prior art.

[0006] To achieve the above objectives, this invention provides the following technical solution: a hyperspectral modeling method for soil organic matter content based on SAE-GS-TabNet, comprising: S100, for the area where vegetable fields in saline-alkali facilities were converted back to farmland, the same soil sample was collected under natural moisture content, water-removing film pressing and standard rehumidification conditions, and its organic matter content was measured to construct a three-state spectral matrix X and label Y. S200, perform bad band removal and scattering correction on the three-state spectral matrix X, calculate the water film difference spectrum between the natural water content spectrum and the tablet dehydration film spectrum, and the structural difference spectrum between the standard rehumidification spectrum and the tablet dehydration film spectrum, and fuse them to obtain the water and salt disturbance spectral matrix X1; S300, input X1 into the SAE model with a shared coding layer and a dual decoding layer, where the first decoding layer reconstructs the water film, salt frost and carbon particle interference components, and the second decoding layer reconstructs the organic matter response component. Based on the reconstruction residuals of the two, the deep organic matter feature F is obtained. S400, calculate the correlation contribution value, three-state consistency value and interference residual suppression value of each feature in F with Y respectively, and generate the comprehensive fitness through GS global stability search, and screen to obtain the stable organic matter feature subset Fs; S500, Fs is used as the prior feature mask of TabNet and input into each decision step, so that TabNet only performs attention allocation in the stable organic matter feature subset, and the prediction model M is trained. S600: Perform the same processing steps from S200 to S500 sequentially on the hyperspectral data of the soil to be tested, input the prediction model M, and output the soil organic matter content.

[0007] Preferably, bad band removal includes: The natural moisture content spectrum, the tablet dehydration film spectrum, and the standard rehumidification spectrum were registered to the same band sequence, and the differences between the natural moisture content spectrum and the tablet dehydration film spectrum, as well as the differences between the standard rehumidification spectrum and the tablet dehydration film spectrum, were calculated respectively. Using the median of the neighborhood of the current band as a reference, the number of order reversals where the difference sign is opposite and exceeds the sign determination tolerance is counted to obtain the three-state order consistency. The standardized deviation of the current band relative to neighboring bands is calculated to obtain the neighborhood abruptness, and bands that simultaneously satisfy the abnormal decrease in the consistency of the three-state order and the abnormal increase in the neighborhood abruptness are removed.

[0008] Preferably, scattering correction includes: Using the tablet dehydration film spectrum in the effective three-state spectral matrix X0 as the scattering correction benchmark, the comprehensive difference between the natural water content spectrum, the standard rehumidification spectrum and the tablet dehydration film spectrum in each effective band is calculated. The predetermined proportion band with the smallest overall difference is determined as the stable band interval, and the mean and standard deviation of the natural water content spectrum, the standard rehumidification spectrum, and the tablet dehydration film spectrum are calculated respectively within the stable band interval. Based on the mean and standard deviation of the dehydration film spectrum, the slope-intercept correction was performed on the natural moisture content spectrum and the standard rehumidification spectrum to obtain the corrected three-state spectral matrix Xc.

[0009] Preferably, the construction of the water- and salt-resistant spectral matrix X1 includes: Based on the corrected three-state spectral matrix Xc, the normalized difference between the corrected spectrum of natural water content and the corrected spectrum of the dehydrated film in the tablet is calculated to obtain the water film difference spectrum. The initial difference spectrum between the standard rehumidification correction spectrum and the tablet dehydration film correction spectrum is calculated, and the structural difference spectrum is obtained by subtracting the water film component in the same direction from the non-negative projection of the initial difference spectrum onto the direction of the water film difference spectrum. Based on the amplitude of the water film differential spectrum, a disturbance suppression weight is generated, and the corrected spectrum of the tablet dewatering film, the inverse weighted result of the water film differential spectrum, and the structural differential spectrum are spliced ​​together in a fixed order to obtain the anti-water and salt disturbance spectrum matrix X1.

[0010] Preferably, the construction of the interference target vector and the organic matter response target vector includes: Local reflection anomalies were calculated based on the corrected spectrum of the tablet dehydration membrane, and structural difference anomalies were calculated based on the structural difference spectrum. The bands where both local reflection anomalies and structural difference anomalies exceed the anomaly boundary determined by the sample's own fluctuations are identified as local interference bands of salt frost or carbon particles, and salt frost and carbon particle interference components are constructed. The inverse weighted result of the salt frost and carbon particle interference components and the water film difference spectrum is concatenated to form the interference target vector TI, and the difference between the anti-water and salt disturbance spectral unit X1i and the interference target vector TI is used as the organic matter response target vector TO.

[0011] Preferably, obtaining the deep organic matter feature F includes: After standardizing the anti-water and salt disturbance spectral unit X1i, the interference target vector TI, and the organic matter response target vector TO with the same mean and standard deviation, they are input into the shared coding layer to obtain the low-dimensional hidden vector h. The interference components of water film, salt frost and carbon particles are reconstructed through the first decoding layer, and the organic matter response component is reconstructed through the second decoding layer. The cosine similarity constraint of the two reconstructed vectors is added to the training loss to reduce the repetitive expression of interference information and organic matter response information. Based on the first reconstruction residual after removing interference components from the input spectrum and the second reconstruction residual after further removing organic matter response components, the organic matter response confidence coefficient is calculated, and the organic matter response confidence coefficient is multiplied by the low-dimensional hidden vector h to obtain the deep organic matter feature F.

[0012] Preferably, the calculation of the relevant contribution value includes: The samples corresponding to the deep organic matter feature F are sorted from low to high according to the organic matter content of label Y, and divided into multiple organic matter content layers according to the number of samples. For each dimension of feature F, calculate its median in each organic matter content layer, and determine the main direction of change of that dimension of feature based on the difference between the highest and lowest content layers. The number of times this feature changes in the same direction as the main change and the number of times it fluctuates in the opposite direction between adjacent content layers are counted, and the relevant contribution value is obtained by combining the rank correlation between this feature and label Y.

[0013] Preferably, the calculation of the three-state consistency value and the interference residual suppression value includes: The single-dimensional feature in F is mapped back to the first decoding layer of S300 to obtain the local interference reconstruction vector corresponding to the feature in that dimension, and is divided into the tablet dehydration film source segment, the natural moisture source segment and the standard rehumidification source segment according to the splicing order of X1. By comparing the response differences of the three source segments, the tri-state consistency value of this feature dimension is obtained; Based on the residual after reconstruction of the second decoding layer of S300, the feature of this dimension is perturbed and replaced in the same organic matter content layer, and the interference residual suppression value is obtained when the residual decreases after replacement and the interference reconstruction component weakens synchronously.

[0014] Preferably, S400 to S500 include: The relevant contribution value, the three-state consistency value, and the interference residual suppression value are normalized respectively, and the geometric mean of the three is used as the comprehensive fitness of each feature dimension. Perform a global stable search (GS) based on the overall fitness from high to low, and use an alternating approach of adding candidate features in the forward direction and removing highly relevant redundant features in the reverse direction to obtain a stable organic matter feature subset Fs. Based on Fs, construct the prior feature mask V of TabNet, and in each decision step of TabNet, set the attention candidate values ​​of non-Fs features to the minimum value, so that the feature selection mask only allocates attention within Fs, and trains the prediction model M.

[0015] Preferably, S600 includes: The natural water content spectrum, the pressed water removal film spectrum, and the standard rewetting spectrum of the soil to be tested were processed using the effective band set, stable band interval, and water film disturbance intensity determined during the training phase to obtain the water and salt disturbance spectral unit X1test to be tested. Input X1test into the trained shared encoder layer and dual decoder layer SAE, calculate the organic matter response confidence coefficient of the soil to be tested, and obtain the deep feature Ftest of the organic matter to be tested. After constraining Ftest with the stable organic matter feature subset Fs and the prior feature mask V, the model M is input to the prediction model, and the soil organic matter content to be measured is output.

[0016] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention does not directly model the entire spectrum of a single-state soil, but rather addresses the unique interference scenario of "thin water film, salt bloom, carbon particles, and humus dark spots" coexisting in areas where saline-alkali farmland has been converted back to farmland. It constructs three-state spectra: natural water content, sheet-grown water film, and standard rewetting. Using the sheet-grown water film state as an approximate stable reference, it retains real surface interferences such as salt bloom and carbon particles while reducing the masking effect of the surface free water film. Furthermore, it utilizes the consistency of the three-state order and the degree of neighborhood abrupt change to jointly eliminate bad bands, avoiding the accidental deletion of organic matter-sensitive bands based solely on single spectral abrupt changes. Further, through the subtraction of the water film difference spectrum, structural difference spectrum, and the non-negative projection of the water film's unidirectional component, it separates water film disturbance from changes in soil particle structure after rewetting. This ensures that the input X1 is no longer a simple set of reflectance, but rather an expression resistant to water and salt disturbances, incorporating "reference reflectance, water film reverse compensation, and retention of structural differences." This directly reduces the masking effect of water and salt pseudo-absorption peaks on the organic matter response.

[0017] This invention differs from the direct feature learning methods of ordinary SAEs or TabNets. It inputs the water-salt perturbation-resistant spectrum into an SAE with a shared coding layer and dual decoding layers. The first decoding layer reconstructs the interference components of water film, salt frost, and carbon particles, while the second decoding layer reconstructs the organic matter response component. The organic matter response confidence coefficient is obtained from the reconstruction residual, forming the deep organic matter feature F. This process does not simply increase the number of network layers; rather, it forcibly distinguishes between spectral changes that can be interpreted as interference and those that can be interpreted as organic matter responses during the feature formation stage. Subsequently, this invention uses correlation contribution values, tri-state consistency values, and interference residual suppression values ​​to perform a global stability search (GS), selecting a stable organic matter feature subset Fs. Fs is then used as the prior feature mask for each decision step of the TabNet, ensuring that attention is only allocated to stable features. This avoids the model misinterpreting high reflectivity of salt frost, dark spots on carbon particles, or changes in residual water film as sensitive organic matter features, improving the stability and resilience of prediction results in complex plots with small samples. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1This is a flowchart of the hyperspectral modeling method for soil organic matter content based on SAE-GS-TabNet of the present invention.

[0020] Figure 2 This is a flowchart of the prediction model construction method of the present invention.

[0021] Figure 3 The flowchart shows the method for calculating the relevant contribution value of this invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1, please refer to Figure 1 and Figure 2 and Figure 3 As shown in this embodiment, the hyperspectral modeling method for soil organic matter content based on SAE-GS-TabNet includes: S100, for areas where saline-alkali facility vegetable fields were converted back to farmland, the same soil sample was collected under natural moisture content, pressed water removal film and standard rehumidification conditions for hyperspectral reflectance, and its organic matter content was measured to construct a three-state spectral matrix X and a label Y.

[0024] In one embodiment, the area where salinized greenhouses were converted back to farmland was selected as the sampling area. Soil samples from the 0-20cm topsoil layer were collected within each sampling unit using a five-point mixing method to obtain the i-th soil sample. During sampling, non-soil impurities such as crop stubble, gravel, and plastic film fragments were removed. Visible salt blooms, fine salt crystals, carbon particles, and dark spots of humus in the sample were retained. If isolated large salt crusts were present, they were gently broken up and mixed with the soil from that sampling point to maintain the actual spectral interference characteristics of the soil in the salinized greenhouse conversion area. At least 500g of soil was collected from each sampling point, placed in sealed bags, numbered, and stored. The sampling location, sampling time, surface moisture content, and salt bloom distribution were recorded. After the i-th soil sample was brought back to the laboratory, it was slightly broken up and mixed without chemical treatment, and then divided into a first subsample, a second subsample, a third subsample, and a chemical determination subsample. The first sample, kept in its natural moisture content at the time of collection, was placed in a black, non-reflective sample cup with a thickness of at least 10 mm. The surface was smoothed but not compacted to obtain a sample in its natural moisture content state. The second sample was placed in a sample cup of the same size, and a clean, inert, water-absorbing filter membrane was placed on its surface. A pressure of 0.3–0.8 MPa was applied using a flat plate pressing device and maintained for 30–90 seconds to allow the surface free water film of the soil sample to be adsorbed and removed by the filter membrane. After removing the filter membrane, the surface was not disturbed; only the edges were smoothed to obtain a sample in the pressed, dehydrated state. This state is used to preserve the soil particle, salt bloom, and carbon particle structure, while reducing the shading effect of the surface water film on spectral reflectance. The third sample was air-dried at room temperature in the dark until the difference between two consecutive weighings was less than 0.1%. After passing through a 2 mm sieve, deionized water was added according to the target mass moisture content, which was 10%–20%, or 50%–70% of the field water holding capacity of this type of soil. When the mass of air-dried soil is M0, the initial moisture content is w0, and the target moisture content is ws, the amount of water added ΔM is calculated using the formula ΔM=M0×(ws-w0) / (1+w0). After adding water, mix thoroughly, seal and let stand for 8–24 hours to allow the moisture to redistribute evenly in the soil sample, obtaining a standard rehydrated sample.

[0025] In this embodiment, a hyperspectral analyzer was used to collect the reflectance spectra of the samples in the three states described above. Dark current correction and standard white plate correction were performed before acquisition. The reflectance was calculated as R = (I - D) / (W - D), where I is the sample spectral intensity, D is the dark current intensity, and W is the white plate spectral intensity. During sample acquisition for the three states, the distance to the light source, probe height, integration time, scanning angle, and ambient temperature were kept consistent. At least three spectra were acquired for each state. After each acquisition, the sample cup was rotated or the measurement position was changed, and obvious abnormal spectra were removed. The average value was then taken to obtain the hyperspectral reflectance for that state.

[0026] In this study, each spectrum contains m identical bands. The natural water content spectrum, the pressed water removal film spectrum, and the standard rehumidification spectrum of the i-th soil sample are denoted as xiN, xiP, and xiR, respectively, and are arranged in a fixed order to form a three-state spectral unit Xi = [xiN, xiP, xiR]. The above operation is repeated for n soil samples to obtain a three-state spectral matrix X = {X1, X2, ..., Xn}, where the dimension of X is n × 3 × m.

[0027] Simultaneously, a chemical analysis subsample of the i-th soil sample is taken, air-dried, ground, and sieved. The soil organic matter content is then determined using either the potassium dichromate oxidation method or elemental analysis. At least two parallel measurements are set for each sample. If the relative deviation of the parallel measurements meets the preset requirements, the average value is taken as the organic matter content label yi; otherwise, the measurement is repeated. The organic matter content of the n samples forms a label vector Y = {y1, y2, ..., yn}. The natural water content, dewatering film state, and standard rehumidification state spectra corresponding to the same sample share the same label yi, thus providing a data foundation for the subsequent construction of water film difference spectra, structural difference spectra, and water-salt disturbance spectral matrices.

[0028] S200, bad band removal and scattering correction are performed on the three-state spectral matrix X, and the water film difference spectrum between the natural water content spectrum and the tablet dehydration film spectrum, and the structural difference spectrum between the standard rehumidification spectrum and the tablet dehydration film spectrum are calculated and fused to obtain the water and salt disturbance spectral matrix X1.

[0029] In one possible implementation, the three-state spectral matrix X obtained by S100 is subjected to bad band removal, scattering correction, differential spectrum calculation, and fusion processing. Assume there are n soil samples, each containing m bands. The natural water content spectrum of the i-th sample is denoted as xNi, the spectrum from the dehydration film is denoted as xPi, and the standard rehumidification spectrum is denoted as xRi; xNi(j), xPi(j), and xRi(j) represent the reflectance of the i-th sample at the j-th band lambdaj, i=1,2,…,n, j=1,2,…,m.

[0030] First, band registration is performed on xNi, xPi, and xRi. The wavelength range commonly covered by the spectra of the three states is used as the processing range. When the original wavelength points of the three states are inconsistent, linear interpolation is used to unify the spectra of the three states to the same band sequence lambda1 to lambdam.

[0031] For the i-th sample and the j-th band, calculate the difference dNPi(j) between the natural moisture content spectrum and the dehydration film spectrum, and the difference dRPi(j) between the standard rehumidification spectrum and the dehydration film spectrum: dNPi(j) = xNi(j) - xPi(j); dRPi(j) = xRi(j) - xPi(j); where dNPi(j) represents the reflectance change of the natural moisture content state relative to the dehydration film state, and dRPi(j) represents the reflectance change of the standard rehumidification state relative to the dehydration film state.

[0032] The neighborhood Omegaj is formed by two bands to the left and right of the j-th band lambdaj. When the j-th band is located at the beginning or end of the band sequence, resulting in insufficient two bands to the left and right, the actual adjacent bands are used to form Omegaj; Omegaj includes at least 3 bands. The median dNPi in Omegaj, medNPi(j), and the median dRPi in Omegaj, medRPi(j), are calculated. To avoid sign misjudgment due to noise when the difference is close to 0, a sign judgment tolerance epssi is set: epssi = 0.001 × Meanj{[xNi(j) + xPi(j) + xRi(j)] / 3}; where Meanj represents the average value of all bands of the i-th sample; epssi is the sign judgment tolerance of the i-th sample, which is 0.1% of the three-state average reflectance. This value is used to exclude differences close to small instrument fluctuations.

[0033] When abs[dNPi(j)] is greater than epssi, abs[medNPi(j)] is greater than epssi, and dNPi(j) and medNPi(j) have opposite signs, it is counted as one order reversal; when either of their absolute values ​​is not greater than epssi, it is not counted as an order reversal. dRPi(j) and medRPi(j) are judged in the same way.

[0034] The consistency of the three-state order C(j) of the j-th band is calculated by the following formula: C(j) = 1 - Nrev(j) / Nall(j); where Nrev(j) represents the total number of times the order is reversed in the two types of difference judgments of dNPi(j) and dRPi(j) for n samples; Nall(j) represents the total number of times the sign judgment is participated in; when Nall(j) = 0, let C(j) = 1. The smaller C(j) is, the easier it is for this band to disrupt the relative change relationship between the three states of natural moisture content, dehydration film of tablet, and standard rehumidification.

[0035] Further calculate the neighborhood abruptness A(j) of the j-th band. For the k-th state spectrum of the i-th sample, where k takes the values ​​N, P, or R, calculate the standardization deviation. : ;in, represents the neighborhood band set of the j-th band, represents the reflectance of the spectrum of the i-th sample in the k-th state at the j-th band; when k=N, =xNi(j); when k=P, =xPi(j); when k=R, =xRi(j); Mediant denotes taking the median for the band t in the neighborhood Ωj; MADt denotes the median absolute deviation of the reflectance of each band in the neighborhood Ωj relative to its median; is set to 0.000001 to avoid division by zero.

[0036] The median of Gki(j) of all samples under the three states is taken to obtain the neighborhood mutation degree A(j): Here, Mediani,k denotes taking the median jointly for all samples i and the three states k. A larger A(j) indicates a stronger local mutation of this band relative to the adjacent bands.

[0037] Let TC be the median of all C(j) minus 3 times the median absolute deviation of C(j), and let TA be the median of all A(j) plus 3 times the median absolute deviation of A(j): TC = Medianj{C(j)} − 3×MADj{C(j)}; TA = Medianj{A(j)} + 3×MADj{A(j)}; where Medianj denotes taking the median for all bands j; MADj denotes the median absolute deviation of the values corresponding to all bands relative to their median. The 3 times median absolute deviation determines the anomaly boundary according to the dispersion degree of the current batch of spectra, which is automatically relaxed when the noise is large and automatically tightened when the spectra are stable. When C(j) < TC and A(j) > TA, λj is determined as a bad band and eliminated. After retaining the remaining bands, the valid three-state spectrum matrix X0 is obtained, and X0 is set to contain m' valid bands.

[0038] The pressed pellet water-removed film spectrum in X0 is used as the scattering correction reference. For each valid band λj in X0, the three-state difference Q(j) is calculated: Q(j) = Mediani{abs[xNi(j)−xPi(j)]+abs[xRi(j)−xPi(j)]}; where abs denotes taking the absolute value, Q(j) denotes the comprehensive difference of the natural water-containing state and the standard rewetting state relative to the pressed pellet water-removed film state on the j-th valid band; a smaller Q(j) indicates that this band is less affected by the change of water film and rewetting structure. All Q(j) are sorted from small to large, and the 25% of the bands with the smallest values are taken as the stable band interval B.

[0039] It should be noted that the 25% value is based on the fact that this proportion can provide a sufficient number of bands for the calculation of the mean and standard deviation, while excluding most bands that are strongly disturbed by water film, salt frost and rehumidification structures.

[0040] For the i-th sample, calculate the mean value muNi and standard deviation sigmaNi of the natural water content spectrum within the stable band interval B, and calculate the mean value muPi and standard deviation sigmaPi of the tablet dehydration film spectrum within B. The correction slope aNi and intercept bNi of the natural water content spectrum are: aNi = sigmaPi / (sigmaNi + 0.000001); bNi = muPi - aNi × muNi.

[0041] The corrected natural water content spectrum xNci(j) is: xNci(j) = aNi × xNi(j) + bNi; where aNi is used to correct the difference in reflectance amplitude between the natural water content spectrum and the dehydrated film spectrum, and bNi is used to correct the overall reflectance shift.

[0042] The standard rehumidification spectrum is corrected using the same method. The mean *muRi* and standard deviation *sigmaRi* of the standard rehumidification spectrum within range B are calculated, yielding: aRi = sigmaPi / (sigmaRi + 0.000001); bRi = *muPi* - aRi × *muRi*; xRci(j) = aRi × xRi(j) + bRi. The tablet dehydration film spectrum is used as a reference without amplitude transformation and is denoted as: xPci(j) = xPi(j). The corrected three-state spectral matrix Xc is constructed from xNci, xPci, and xRci.

[0043] Based on the corrected three-state spectral matrix Xc, the water film difference spectrum DWi(j) of the i-th sample in the j-th effective band is calculated: DWi(j)=[xNci(j)-xPci(j)] / [xNci(j)+xPci(j)+0.000001]. Here, DWi(j) represents the normalized difference between the natural water content state and the dehydrated film state after tableting, characterizing the spectral perturbation caused by the surface water film; the 0.000001 in the denominator is used to avoid a denominator of 0.

[0044] Then calculate the initial differential spectrum DR0i(j) of standard rehumidification: DR0i(j) = xRci(j) - xPci(j); where DR0i(j) represents the difference in reflectance between the standard rehumidification state and the state of the dehydrated film after pressing.

[0045] To subtract components in the same direction as the water film disturbance from DR0i(j), the projection ratio alphai is calculated: alphai = max{0, Sumj[DR0i(j)×DWi(j)] / [Sumj[DWi(j)×DWi(j)]+0.000001]}; where Sumj represents the summation of all m' effective bands retained over X0; alphai represents the non-negative projection ratio of the initial standard rewetted difference spectrum in the direction of the water film difference spectrum; max{0, ·} indicates that when the projection ratio is less than 0, it is taken as 0, only subtracting the part that changes in the same direction as the water film difference spectrum, avoiding the accidental deletion of reverse structural changes. The structural difference spectrum DSi(j) is: DSi(j) = DR0i(j) - alphai×DWi(j); where DSi(j) represents the standard rewetted structural difference after subtracting the water film component in the same direction, used to retain spectral information related to soil particle structure, salt frost distribution, and organic matter response.

[0046] For each effective band lambdaj, the water film disturbance intensity E(j) is calculated as follows: E(j) = Mediani{abs[DWi(j)]} + 1.4826 × MADi{abs[DWi(j)]}; where Mediani represents the median of n samples; MADi represents the median absolute deviation of abs[DWi(j)] of n samples relative to the median; 1.4826 is used to convert the median absolute deviation to a scale close to the standard deviation, making the disturbance intensity of different bands comparable. The disturbance suppression weight wi(j) of the i-th sample in the j-th effective band is: wi(j) = E(j) / [E(j) + abs[DWi(j)] + 0.000001]; where wi(j) is greater than 0 and not greater than 1; when abs[DWi(j)] increases, wi(j) decreases, indicating that the water film disturbance in this band is strong and its impact on subsequent feature extraction needs to be reduced.

[0047] The corrected spectrum xPci of the water film removed by the tablet, the inverse weighted result -wi×DWi of the water film difference spectrum, and the structural difference spectrum DSi are spliced ​​in a fixed order to obtain the water and salt disturbance spectral unit X1i of the i-th sample: X1i=[xPci, -wi×DWi, DSi]; where -wi×DWi represents multiplying DWi(j) by wi(j) band by band and then taking the negative number, as the inverse compensation feature for water film disturbance; DSi is used to provide the structural difference features after subtracting the water film unidirectional component. The above processing is repeated for n samples to obtain the water and salt disturbance spectral matrix X1, with the dimension of X1 being n×3m'. This matrix is ​​used as the input for subsequent SAE to extract the deep features of soil organic matter that are less affected by water film, salt frost, and scattering differences.

[0048] S300, input X1 into the SAE model with a shared coding layer and a dual decoding layer, where the first decoding layer reconstructs the water film, salt frost and carbon particle interference components, and the second decoding layer reconstructs the organic matter response component. Based on the reconstruction residuals of the two, the deep organic matter features F are obtained.

[0049] In one implementation, the water-salt perturbation spectral matrix X1 obtained in S200 is input into an SAE model with a shared coding layer and a dual decoding layer. Let X1i be the water-salt perturbation spectral unit of the i-th sample in X1. X1i is obtained by splicing the tablet dehydration film correction spectrum xPci, the inverse weighted result Wi of the water film difference spectrum, and the structural difference spectrum DSi in a fixed order, i.e.: X1i = [xPci, Wi, DSi]; Wi = -wi × DWi. Here, xPci, Wi, and DSi each contain m' effective bands, the length of X1i is d, d = 3m', DWi is the water film difference spectrum of the i-th sample, and wi is the corresponding perturbation suppression weight.

[0050] First, an interference target vector TIi is constructed based on X1i for learning in the first decoding layer. For the j-th effective band, the local reflectance anomalies of the corrected spectrum of the tablet dehydration film are calculated. : ;in, This represents the reflectance of the tablet dehydration membrane-corrected spectrum of the i-th sample in the j-th effective wavelength band; Let t represent the set of neighboring bands of the j-th effective band; t and u both represent the neighborhood. The band number within; Median indicates the median; MAD indicates the median absolute deviation.

[0051] Recalculate the structural difference outliers The expression is: ;in, Let t and u represent the structure difference value of the i-th sample in the j-th effective band, where t and u both represent the neighborhood. The band number within. Salt frost usually manifests as localized high reflectance anomalies, while carbon particles usually manifest as localized low reflectance anomalies. The two will form a localized structural difference abrupt change between the standard rehumidification state and the tablet dehydration film state. Therefore, LPi(j) and LSi(j) are used simultaneously to identify salt frost and carbon particle interference.

[0052] Let TP be the median of LPi(j) for all samples and all effective bands plus three times the median absolute deviation, and let TS be the median of LSi(j) for all samples and all effective bands plus three times the median absolute deviation: TP = Mediani,j{LPi(j)} + 3 × MADi,j{LPi(j)}; TS = Mediani,j{LSi(j)} + 3 × MADi,j{LSi(j)}; where Mediani,j represents the median of all samples i and all effective band j; MADi,j represents the median absolute deviation of the corresponding data relative to the median. TP and TS are determined by the fluctuations of the samples themselves and are used to distinguish between stable absorption changes and isolated strong abrupt changes.

[0053] When LPi(j) > TP and LSi(j) > TS, let qSCi(j) = 1; otherwise, let qSCi(j) = 0. qSCi(j) indicates whether the i-th sample has local interference from salt frost or carbon particles in the j-th effective band. The salt frost and carbon particle interference components are thus constructed: In the formula, This represents the local reflection interference components of salt frost and carbon particles obtained from the corrected spectrum of the tablet dehydration membrane at the j-th effective band for the i-th sample; This represents the salt frost and carbon particle structure interference components obtained from the structural difference spectrum of the i-th sample at the j-th effective band. This represents the local interference judgment value of salt frost or carbon particles at the j-th effective band of the i-th sample. It is 1 when local interference exists and 0 otherwise. t represents the reflectance of the tablet dehydration membrane-corrected spectrum of the i-th sample in the j-th effective wavelength band; t represents the neighborhood. Band number within; This represents the structural difference spectrum value of the i-th sample in the j-th effective band.

[0054] The interference components of salt bloom and carbon particles in the corrected spectrum of the tablet dehydration film, the inverse weighted result of the water film difference spectrum, and the interference components of salt bloom and carbon particles in the structural difference spectrum are concatenated to obtain the interference target vector TIi: TIi = [PSCi, Wi, SSCi]. Then, X1i is subtracted from TIi to obtain the organic matter response target vector TOi: TOi = X1i - TIi. Here, TOi represents the candidate organic matter response component after deducting the local interference of water film, salt bloom, and carbon particles.

[0055] Subsequently, X1i, TIi, and TOi were standardized using the same scale. Using the mean MeanX(j) and standard deviation StdX(j) of each dimension of X1 in the training samples as benchmarks, we obtained: Zi(j)=[X1i(j)-MeanX(j)] / StdX(j); ZIi(j)=[TIi(j)-MeanX(j)] / StdX(j); ZOi(j)=[TOi(j)-MeanX(j)] / StdX(j).

[0056] Where Zi is the input of the SAE model, ZIi is the reconstruction target of the first decoding layer, and ZOi is the reconstruction target of the second decoding layer.

[0057] The shared coding layer compresses Zi layer by layer to obtain the latent feature hi: h1i = Relu(W1 × Zi + b1); hi = Relu(W2 × h1i + b2).

[0058] Where W1 and W2 are encoding weights, b1 and b2 are encoding biases, ReLU represents the activation function that outputs 0 when the input is less than 0 and outputs the original value when the input is not less than 0, and hi is the low-dimensional latent vector of the shared encoding layer output. The dimension r of hi is determined according to the cumulative variance contribution rate of the Zi covariance matrix in the training samples, and is taken as the smallest integer that makes the cumulative variance contribution rate not less than 95%; this value preserves the main spectral variations while compressing redundant band noise.

[0059] The first decoding layer takes hi as input and outputs the perturbation reconstruction vector Ihati; the second decoding layer takes the same hi as input and outputs the organic matter response reconstruction vector Ohati. During training, the loss function L is minimized: L=Meani{MSE(Ihati,ZIi)+MSE(Ohati,ZOi)+COS(Ihati,Ohati)}.

[0060] Where MSE(Ihati,ZIi) represents the mean square error between Ihati and ZIi, which is used to constrain the reconstruction of water film, salt frost and carbon particle interference components in the first decoding layer; MSE(Ohati,ZOi) represents the mean square error between Ohati and ZOi, which is used to constrain the reconstruction of organic matter response components in the second decoding layer. This is used to reduce the repetitive representation of the same spectral information in two decoding outputs. Among them, This represents the interference reconstruction vector of the i-th sample. This represents the organic matter response reconstruction vector of the i-th sample. (j) and (j) represents the values ​​of the two vectors in the j-th dimension, d represents the dimension of the reconstructed vector, and ε is a very small constant to prevent the denominator from being zero, which is taken as 0.000001. This formula represents the absolute value of the cosine of the angle between the two reconstructed vectors, which is used to constrain the information expressed by the two vectors to avoid overlap as much as possible.

[0061] After training, the first reconstruction residual R1i and the second reconstruction residual R2i are calculated: R1i = Zi - Ihati; R2i = R1i - Ohati. Here, R1i represents the residual after removing the reconstruction interference component from the input spectrum, and R2i represents the residual that is still unexplained after the second decoding layer reconstructs the organic matter response component. The organic matter response confidence coefficient etai is calculated based on R1i and R2i: etai = max{0, 1 - Meanj{abs[R2i(j)]} / [Meanj{abs[R1i(j)]}]}. Here, etai ranges from 0 to 1; the smaller the second reconstruction residual R2i is relative to the first reconstruction residual R1i, the more the spectral change after removing interference can be explained by the organic matter response component, and the closer etai is to 1.

[0062] Finally, multiply etai by the shared coding layer output hi to obtain the deep organic matter feature Fi of the i-th sample: Fi = etai × hi.

[0063] The above calculations are repeated for n samples to obtain the deep organic matter feature matrix F = {F1, F2, ..., Fn}. F serves as the input for subsequent feature contribution calculations and GS global stability search, used to screen stable features that are related to soil organic matter content and are less affected by water film, salt frost, and carbon particle interference.

[0064] S400 calculates the correlation contribution value, three-state consistency value and interference residual suppression value of each feature in F with Y, and generates a comprehensive fitness through GS global stability search to obtain a stable organic matter feature subset Fs.

[0065] In one possible implementation, stable feature screening is performed on the organic matter deep feature matrix F obtained in S300. Let F contain n samples and r-dimensional features, Fi(p) represent the p-th dimension organic matter deep feature of the i-th sample, Yi represent the organic matter content label of the i-th sample, and p=1,2,…,r.

[0066] In this embodiment, all samples are sorted according to Yi from low to high, and divided into K organic matter content layers. K is determined by the number of samples, K=floor(sqrt(n)), and K is 3 when less than 3 and 7 when greater than 7; floor represents rounding down. This value ensures that each layer has a sufficient number of samples, while avoiding the influence of a single sample on the median of adjacent layers due to too many layers.

[0067] For the p-th feature, its median MFl(p) in the l-th content layer is calculated, where l = 1, 2, ..., K. If MFK(p) - MF1(p) is not less than 0, the main direction of change for this feature is recorded as positive; if MFK(p) - MF1(p) is less than 0, it is recorded as negative. The difference MF(l+1)(p) - MFl(p) is calculated for adjacent content layers. When the absolute value of this difference is not greater than epsFp, it is not counted; epsFp = 0.05 × MADi{Fi(p)}, where MADi represents the median absolute deviation of the p-th feature in all samples, and 0.05 is used to exclude small random fluctuations between layers. If the difference between adjacent layers is in the same direction as the main direction of change, it is recorded as one change in the same direction; if it is opposite, it is recorded as one fluctuation in the opposite direction. Next, calculate the absolute value of the rank correlation coefficient rhop between the p-th dimension feature and Y. Divide "the number of changes in the same direction plus 1" by "the sum of the number of changes in the same direction and the number of fluctuations in the opposite direction plus 2", and then multiply by rhop to obtain the correlation contribution value RCp. This value reflects both the monotonicity of the feature with organic matter content and its resistance to fluctuations in the opposite direction.

[0068] In this embodiment, the interference components reconstructed by the first decoding layer in S300 are mapped back to the corresponding dimension of F. Specifically, the shared coding layer output hi saved in S300 is called; if only Fi is saved, it is restored according to hi=Fi / (etai+0.000001), where etai is the organic matter response confidence coefficient obtained in S300.

[0069] For the p-th feature, all dimensions of hi except the p-th dimension are set to 0 and input into the first decoding layer to obtain the local interference reconstruction vector IHi,p corresponding to this feature. Following the splicing order of X1i in S200, IHi,p is divided into the tablet dehydration film source segment, the natural moisture source segment, and the standard rehumidification source segment, corresponding to xPci, Wi, and DSi, respectively, where Wi is the inverse weighted result of the water film difference spectrum. The average of the absolute values ​​of the three source segments is calculated to obtain BPi(p), BNi(p), and BRi(p). The median of all samples is taken to obtain BP(p), BN(p), and BR(p). The tri-state consistency value TCp is obtained by subtracting the difference between the maximum and minimum values ​​of the three values ​​from 1 and then dividing by the sum of the maximum value and 0.000001; when the responses of the three source segments are close, TCp is close to 1; when a certain source segment is dominated by interference, TCp decreases.

[0070] Specifically, based on the residual R2i remaining after reconstruction from the second decoding layer of S300, the features in F are perturbed and permuted. For the p-th dimension feature, within each organic matter content layer, the Fi(p) of the sample with the highest average absolute value of R2i in that layer is replaced with the Fi(p) of the sample with the lowest average absolute value of R2i in that layer, while other dimensions remain unchanged, resulting in the permuted feature matrix Frepp. This permutation is performed only within the same organic matter content layer to eliminate residual perturbations unrelated to the label, while preserving the hierarchical relationship of organic matter content.

[0071] In this process, Frepp is restored to the shared coding layer scale according to the corresponding etai, and then input into the first and second decoding layers respectively to obtain the permuted interference reconstruction components and the remaining residuals. If the average absolute value of the remaining residuals after permutation is less than that before permutation, and the average absolute value of the interference reconstruction components is also less than that before permutation, then the average of their relative reduction magnitudes is calculated as the interference residual suppression value DRp; otherwise, DRp is set to 0. The larger this value is, the better the p-th dimension feature can suppress water film, salt frost, and carbon particle interference while maintaining the organic matter hierarchical relationship.

[0072] Finally, RCp, TCp, and DRp are normalized to 0 to 1 to obtain NRCp, NTCp, and NDRp. The overall fitness FITp of the p-th dimension feature is taken as the geometric mean of NRCp, NTCp, and NDRp; when any index approaches 0, FITp decreases synchronously, thus avoiding the reliance on a single correlation to select features. A candidate sequence is formed according to FITp from high to low, and a global stable search (GS) is performed: candidate features are added sequentially from the empty set; after each feature is added, the absolute rank correlation coefficient of each pair of features in the current feature set is calculated. If the absolute rank correlation coefficient of a pair of features is greater than the median of the absolute rank correlation coefficients of all candidate features plus one times the median absolute deviation, then the feature with the lower FITp is removed. Forward addition and reverse removal are performed alternately until the feature set no longer changes after two consecutive rounds of processing, resulting in a stable organic feature subset Fs. This subset is used as the prior feature mask input for TabNet in S500.

[0073] S500, Fs is used as the prior feature mask of TabNet and input into each decision step, so that TabNet only performs attention allocation in the stable organic matter feature subset, and the prediction model M is trained.

[0074] In one possible implementation, the stable organic matter feature subset Fs obtained by S400 is used to constrain the feature attention allocation of the TabNet model. Let the organic matter deep feature matrix F obtained by S300 contain n samples and r-dimensional features, and Fs be the feature index set obtained by S400, Fs={p1, p2, ..., ps}, where ps represents the s-th feature index in Fs, and s is the number of stable organic matter features.

[0075] First, construct a prior feature mask V based on Fs. The length of V is r. When the p-th feature belongs to Fs, let V(p) = 1; when the p-th feature does not belong to Fs, let V(p) = 0. Standardize each feature in F according to the mean and standard deviation of the training samples to obtain the standardized feature matrix Fstd. The standardization formula is: Fstdi(p) = [Fi(p) - MeanF(p)] / [StdF(p) + 0.000001].

[0076] Where Fi(p) represents the p-th dimension of the deep organic matter feature of the i-th sample, MeanF(p) represents the mean of the p-th dimension feature in the training samples, StdF(p) represents the standard deviation of the p-th dimension feature in the training samples, and 0.000001 is used to avoid the denominator being 0.

[0077] Then, Fstdi is multiplied dimension by dimension by the prior feature mask V to obtain the TabNet model input feature Fini: Fini(p) = Fstdi(p) × V(p).

[0078] Therefore, features that do not belong to Fs are set to 0 before entering TabNet, while features that belong to Fs retain their standardized values.

[0079] In each decision step of TabNet, the candidate attention vector At for the current step is generated from the output of the previous decision step, where t represents the t-th decision step. To ensure that attention is allocated only within the stable organic feature subset Fs, At is constrained dimension-wise with the prior feature mask V: when V(p) = 0, the p-th dimension attention candidate value is set to -1000000; when V(p) = 1, the p-th dimension attention candidate value is retained. Subsequently, the constrained candidate attention vector is sparsely normalized to obtain the feature selection mask Maskt for the current decision step. Since the non-Fs features have been minimized before sparse normalization, the weight of Maskt on the non-Fs features is 0, and TabNet allocates attention only within Fs.

[0080] The Maskt is multiplied dimension-wise with the Fini algorithm to obtain the weighted input features for the t-th decision step, which are then fed into the TabNet feature transformation layer to obtain the decision output Dt for that step. The Dt values ​​for all decision steps are accumulated to obtain the comprehensive decision representation Di of the sample, which is then linearly mapped to output the predicted soil organic matter content value Yhati.

[0081] During training, the organic matter content label Yi measured by S100 was used as the supervision value, and the mean squared error was used as the prediction loss to gradually approximate Yi. Simultaneously, an attention sparsity constraint was added, causing the Mask to preferentially focus on a small number of stable features within Fs. After training, the prediction model M was obtained, which included the trained TabNet parameters, the standardized mean MeanF(p), standard deviation StdF(p) of the training samples, and the prior feature mask V. This processing avoids TabNet allocating attention to interference features such as water film, salt frost, and residual carbon particles, improving the stability and resilience of soil organic matter content prediction.

[0082] S600: Perform the same processing steps from S200 to S500 sequentially on the hyperspectral data of the soil to be tested, input the prediction model M, and output the soil organic matter content.

[0083] In one possible implementation, the organic matter content of the soil to be tested is predicted using hyperspectral data. The soil to be tested comes from a saline-alkali facility vegetable field that has been converted back to farmland, similar to the training samples. Following the same data collection method in S100, the hyperspectral reflectance of the soil to be tested under natural moisture content, under a water-removed film pressing state, and under a standard rehumidification state is obtained, denoted as xNtest, xPtest, and xRtest. Here, xNtest ​​represents the natural moisture content spectrum of the soil to be tested, xPtest represents the water-removed film pressing spectrum of the soil to be tested, and xRtest represents the standard rehumidification spectrum of the soil to be tested.

[0084] Specifically, xNtest, xPtest, and xRtest undergo the same band registration and bad band processing as in S200. This involves unifying the three state spectra to the band sequence determined during the training phase and truncating them according to the valid band positions retained during training to obtain the effective three-state spectra to be tested. Here, the bad band threshold and effective band set are not recalculated; instead, the effective band set formed during training is directly used to ensure consistency between the input dimension of the tested data and the prediction model M. Based on the stable band interval B determined in S200, using the test tablet dehydration film spectrum xPtest as the scattering reference, slope-intercept correction is performed on the test natural water content spectrum xNtest ​​and the test standard rehumidification spectrum xRtest to obtain xNctest, xPctest, and xRctest, respectively. Here, xPctest = xPtest; xNctest is the corrected natural water content spectrum; and xRctest is the corrected standard rehumidification spectrum. During the calibration process, the effective band set and stable band interval B determined in the training phase are used, but the mean, standard deviation, slope and intercept are calculated from the three-state spectrum of the soil to be tested within B, thereby maintaining the consistency of the three-state spectrum within the sample to be tested.

[0085] In this embodiment, the water film difference spectrum (DWtest), the standard rewetted initial difference spectrum (DR0test), and the structural difference spectrum (DStest) of the soil under test are calculated based on xNctest, xPctest, and xRctest. The water film difference spectrum is calculated using the normalized difference between the natural moisture content correction spectrum and the pressed water removal film correction spectrum; the standard rewetted initial difference spectrum is calculated using the difference between the standard rewetted correction spectrum and the pressed water removal film correction spectrum; then, by using the non-negative projection ratio of the standard rewetted initial difference spectrum in the direction of the water film difference spectrum, the components in the same direction as the water film disturbance are subtracted to obtain the structural difference spectrum (DStest).

[0086] Subsequently, based on the water film disturbance intensity E obtained in each effective band during the training phase, the disturbance suppression weight wtest of the soil under test is calculated. The disturbance suppression weight of the j-th effective band is determined according to the ratio of E(j) to abs[DWtest(j)]; when abs[DWtest(j)] is larger, wtest(j) is smaller, so as to reduce the impact of water film disturbance in this band on subsequent predictions. xPctest, the inverse weighted result of the water film difference spectrum -wtest×DWtest, and the structural difference spectrum DStest are spliced ​​together in the same order as in the training phase to obtain the water and salt disturbance resistance spectral unit X1test of the soil under test.

[0087] Then, X1test is input into the SAE model trained on S300 with a shared encoding layer and dual decoding layers. Before input, X1 is standardized using the mean (MeanX) and standard deviation (StdX) of each dimension of X1 saved during training to obtain Ztest. The latent vector htest of the soil under test is obtained through the shared encoding layer; the interference reconstruction vector Ihattest is obtained through the first decoding layer; and the organic matter response reconstruction vector Ohattest is obtained through the second decoding layer. The first reconstruction residual R1test and the second reconstruction residual R2test are calculated based on Ztest, Ihattest, and Ohattest, and the organic matter response confidence coefficient etatest is calculated. Multiplying etatest by htest yields the deep organic matter feature Ftest of the soil under test.

[0088] Next, based on the stable organic matter feature subset Fs obtained from S400, features of the corresponding dimensions are extracted from Ftest and standardized according to the mean MeanF and standard deviation StdF of the training samples saved in S500 to obtain the stable features to be tested. According to the prior feature mask V in S500, features that do not belong to Fs are set to 0, so that the features to be tested input to the prediction model M maintain the same dimension and the same feature order as the TabNet input in the training stage.

[0089] Finally, the processed features to be tested are input into the prediction model M. In each decision step, the prediction model M only assigns attention to the features corresponding to the stable organic matter feature subset Fs. After feature transformation and linear mapping, it outputs the predicted value Yhattest of the organic matter content of the soil to be tested. Yhattest is the result of the organic matter content of the soil to be tested, and its units are consistent with the measurement units of label Y in S100.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A hyperspectral modeling method for soil organic matter content based on SAE-GS-TabNet, characterized in that, include: S100, for the area where vegetable fields in saline-alkali facilities were converted back to farmland, the same soil sample was collected under natural moisture content, water-removing film pressing and standard rehumidification conditions, and its organic matter content was measured to construct a three-state spectral matrix X and label Y. S200, perform bad band removal and scattering correction on the three-state spectral matrix X, calculate the water film difference spectrum between the natural water content spectrum and the tablet dehydration film spectrum, and the structural difference spectrum between the standard rehumidification spectrum and the tablet dehydration film spectrum, and fuse them to obtain the water and salt disturbance spectral matrix X1; S300, input X1 into the SAE model with a shared coding layer and a dual decoding layer, where the first decoding layer reconstructs the water film, salt frost and carbon particle interference components, and the second decoding layer reconstructs the organic matter response component. Based on the reconstruction residuals of the two, the deep organic matter feature F is obtained. S400, calculate the correlation contribution value, three-state consistency value and interference residual suppression value of each feature in F with Y respectively, and generate the comprehensive fitness through GS global stability search, and screen to obtain the stable organic matter feature subset Fs; S500, Fs is used as the prior feature mask of TabNet and input into each decision step, so that TabNet only performs attention allocation in the stable organic matter feature subset, and the prediction model M is trained. S600: Perform the same processing steps from S200 to S500 sequentially on the hyperspectral data of the soil to be tested, input the prediction model M, and output the soil organic matter content.

2. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 1, characterized in that, Bad band removal includes: The natural moisture content spectrum, the tablet dehydration film spectrum, and the standard rehumidification spectrum were registered to the same band sequence, and the differences between the natural moisture content spectrum and the tablet dehydration film spectrum, as well as the differences between the standard rehumidification spectrum and the tablet dehydration film spectrum, were calculated respectively. Using the median of the neighborhood of the current band as a reference, the number of order reversals where the difference sign is opposite and exceeds the sign determination tolerance is counted to obtain the three-state order consistency. The standardized deviation of the current band relative to neighboring bands is calculated to obtain the neighborhood abruptness, and bands that simultaneously satisfy the abnormal decrease in the consistency of the three-state order and the abnormal increase in the neighborhood abruptness are removed.

3. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 2, characterized in that, Scattering correction includes: Using the tablet dehydration film spectrum in the effective three-state spectral matrix X0 as the scattering correction benchmark, the comprehensive difference between the natural water content spectrum, the standard rehumidification spectrum and the tablet dehydration film spectrum in each effective band is calculated. The predetermined proportion band with the smallest overall difference is determined as the stable band interval, and the mean and standard deviation of the natural water content spectrum, the standard rehumidification spectrum, and the tablet dehydration film spectrum are calculated respectively within the stable band interval. Based on the mean and standard deviation of the dehydration film spectrum, the slope-intercept correction was performed on the natural moisture content spectrum and the standard rehumidification spectrum to obtain the corrected three-state spectral matrix Xc.

4. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 3, characterized in that, The construction of the water-salt perturbation spectral matrix X1 includes: Based on the corrected three-state spectral matrix Xc, the normalized difference between the corrected spectrum of natural water content and the corrected spectrum of the dehydrated film in the tablet is calculated to obtain the water film difference spectrum. The initial difference spectrum between the standard rehumidification correction spectrum and the tablet dehydration film correction spectrum is calculated, and the structural difference spectrum is obtained by subtracting the water film component in the same direction from the non-negative projection of the initial difference spectrum onto the direction of the water film difference spectrum. Based on the amplitude of the water film differential spectrum, a disturbance suppression weight is generated, and the corrected spectrum of the tablet dewatering film, the inverse weighted result of the water film differential spectrum, and the structural differential spectrum are spliced ​​together in a fixed order to obtain the anti-water and salt disturbance spectrum matrix X1.

5. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 4, characterized in that, The construction of the interference target vector and the organic matter response target vector includes: Local reflection anomalies were calculated based on the corrected spectrum of the tablet dehydration membrane, and structural difference anomalies were calculated based on the structural difference spectrum. The bands where both local reflection anomalies and structural difference anomalies exceed the anomaly boundary determined by the sample's own fluctuations are identified as local interference bands of salt frost or carbon particles, and salt frost and carbon particle interference components are constructed. The inverse weighted result of the salt frost and carbon particle interference components and the water film difference spectrum is concatenated to form the interference target vector TI, and the difference between the anti-water and salt disturbance spectral unit X1i and the interference target vector TI is used as the organic matter response target vector TO.

6. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 5, characterized in that, The acquisition of deep organic matter characteristics F includes: After standardizing the anti-water and salt disturbance spectral unit X1i, the interference target vector TI, and the organic matter response target vector TO with the same mean and standard deviation, they are input into the shared coding layer to obtain the low-dimensional hidden vector h. The interference components of water film, salt frost and carbon particles are reconstructed through the first decoding layer, and the organic matter response component is reconstructed through the second decoding layer. The cosine similarity constraint of the two reconstructed vectors is added to the training loss to reduce the repetitive expression of interference information and organic matter response information. Based on the first reconstruction residual after removing interference components from the input spectrum and the second reconstruction residual after further removing organic matter response components, the organic matter response confidence coefficient is calculated, and the organic matter response confidence coefficient is multiplied by the low-dimensional hidden vector h to obtain the deep organic matter feature F.

7. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 6, characterized in that, The relevant contribution value calculation includes: The samples corresponding to the deep organic matter feature F are sorted from low to high according to the organic matter content of label Y, and divided into multiple organic matter content layers according to the number of samples. For each dimension of feature F, calculate its median in each organic matter content layer, and determine the main direction of change of that dimension of feature based on the difference between the highest and lowest content layers. The number of times this feature changes in the same direction as the main change and the number of times it fluctuates in the opposite direction between adjacent content layers are counted, and the relevant contribution value is obtained by combining the rank correlation between this feature and label Y.

8. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 7, characterized in that, The calculation of the three-state consistency value and the disturbance residual suppression value includes: The single-dimensional feature in F is mapped back to the first decoding layer of S300 to obtain the local interference reconstruction vector corresponding to the feature in that dimension, and is divided into the tablet dehydration film source segment, the natural moisture source segment and the standard rehumidification source segment according to the splicing order of X1. By comparing the response differences of the three source segments, the tri-state consistency value of this feature dimension is obtained; Based on the residual after reconstruction of the second decoding layer of S300, the feature of this dimension is perturbed and replaced in the same organic matter content layer, and the interference residual suppression value is obtained when the residual decreases after replacement and the interference reconstruction component weakens synchronously.

9. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 8, characterized in that, S400 to S500 include: The relevant contribution value, the three-state consistency value, and the interference residual suppression value are normalized respectively, and the geometric mean of the three is used as the comprehensive fitness of each feature dimension. Perform a global stable search (GS) based on the overall fitness from high to low, and use an alternating approach of adding candidate features in the forward direction and removing highly relevant redundant features in the reverse direction to obtain a stable organic matter feature subset Fs. Based on Fs, construct the prior feature mask V of TabNet, and in each decision step of TabNet, set the attention candidate values ​​of non-Fs features to the minimum value, so that the feature selection mask only allocates attention within Fs, and trains the prediction model M.

10. The method for hyperspectral modeling of soil organic matter content based on SAE-GS-TabNet according to claim 9, characterized in that, The S600 includes: The natural water content spectrum, the pressed water removal film spectrum, and the standard rewetting spectrum of the soil to be tested were processed using the effective band set, stable band interval, and water film disturbance intensity determined during the training phase to obtain the water and salt disturbance spectral unit X1test to be tested. Input X1test into the trained shared encoder layer and dual decoder layer SAE, calculate the organic matter response confidence coefficient of the soil to be tested, and obtain the deep feature Ftest of the organic matter to be tested. After constraining Ftest with the stable organic matter feature subset Fs and the prior feature mask V, the model M is input to the prediction model, and the soil organic matter content to be measured is output.

Images