Soil type prediction model training method, mapping method and system based on multi-modal feature fusion and spatial neighborhood constraint

Abstract

The application discloses a soil type prediction model training method, mapping method and system based on multi-modal feature fusion and spatial neighborhood constraint, and the training method comprises the following steps: constructing a multi-source heterogeneous digital soil mapping data sample set, wherein the data sample set comprises continuous environmental variables, discrete environmental variables and multi-source remote sensing images; extracting feature quantities of the data sample set and performing multi-modal adaptive feature fusion, adaptively determining fusion weights by using a cross attention mechanism, and obtaining joint feature quantities; constructing a composite loss function comprising a spatial neighborhood smoothing regularization term and a classification cross-entropy loss; and performing joint optimization training on a preset deep learning model according to the joint feature quantities and the composite loss function until a preset requirement is met, so as to determine a soil type prediction model. According to the embodiment of the application, the accuracy and spatial continuity of the mapping results output by the model and the generalization performance of the model can be effectively improved, and the application can be widely applied to the technical field of digital soil mapping.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital soil mapping, in particular to a soil type prediction model training method, mapping method and system based on multi-modal feature fusion and spatial neighborhood constraint. BACKGROUND

[0002] Soil type mapping is based on the physical, chemical and biological properties of soil, and comprehensively considers the influence of environmental factors such as topography, climate, parent material, vegetation and human activities, and realizes the fine expression and visualization mapping of regional soil types through spatial distribution characteristic analysis. The traditional mapping method mainly relies on field profile investigation, expert experience interpretation and remote sensing image visual interpretation, and has limitations such as long time consumption, high cost and strong subjectivity. In the areas with complex terrain or high heterogeneity, the boundary demarcation is inaccurate, and the mapping accuracy is difficult to meet the needs of modern fine agriculture, ecological environment protection, land resource planning and sustainable development. In order to solve the above problems, digital soil mapping (DSM) technology emerges as the times require. Through building a quantitative relationship model between soil and environment and combining with spatial inference algorithm, automatic soil type prediction and comprehensive mapping are realized, which has obvious advantages in improving mapping accuracy and ensuring objectivity of the results.

[0003] In recent years, with the development of high-performance computing and deep learning technology, deep learning, as an important branch of machine learning, has gradually become a research hotspot in the field of DSM. Among them, the deep learning method represented by convolutional neural network (CNN) is widely used in spatial prediction of soil types due to its excellent multi-scale spatial structure perception ability. Convolutional neural network can automatically extract rich multi-scale spatial features from remote sensing images and spatial variables through end-to-end feature learning mechanism, and improve the classification accuracy and spatial continuity of mapping results. However, the application of CNN model in DSM has problems such as insufficient multi-modal feature fusion, insufficient spatial context modeling ability, insufficient cross-regional generalization and adaptation ability, which leads to low expression accuracy of mapping results output by the model, low spatial continuity and poor generalization performance of the model. SUMMARY

[0004] Therefore, in order to solve one of the above problems, the purpose of the embodiments of the present application is to provide a soil type prediction model training method, mapping method and system based on multi-modal feature fusion and spatial neighborhood constraint, which can effectively improve the expression accuracy and spatial continuity of the mapping results output by the model and the generalization performance of the model.

[0005] In one aspect, the embodiments of the present application provide a soil type prediction model training method based on multi-modal feature fusion and spatial neighborhood constraint, comprising:

[0006] constructing a multi-source heterogeneous digital soil mapping data sample set, the data sample set comprising continuous environmental variables, discrete environmental variables and multi-source remote sensing images;

[0007] extracting feature quantities of the data sample set;

[0008] performing multi-modal adaptive feature fusion on the feature quantities of the data sample set, adaptively determining fusion weights by using a cross-attention mechanism to obtain joint feature quantities;

[0009] constructing a composite loss function comprising a spatial neighborhood smoothing regular term and a classification cross-entropy loss; performing joint optimization training on a preset deep learning model according to the joint feature quantities and the composite loss function until a preset requirement is met to determine a soil type prediction model.

[0010] Specifically, the constructing a multi-source heterogeneous digital soil mapping data sample set comprises:

[0011] obtaining a multi-source heterogeneous environmental variable data set;

[0012] performing data format standardization, spatial reference system unification, spatial registration, spatial resolution unification, missing value filling, outlier removal, data cropping and rasterization on the environmental variable data set to form a digital soil mapping data sample set that is uniform in spatial scale and consistent in semantic expression.

[0013] Specifically, the feature quantities of the continuous environmental variables are extracted by the following method:

[0014] performing batch normalization processing on the continuous environmental variables to obtain a standardized feature vector; the standardized feature vector comprises terrain factors, soil characteristic factors and meteorological factors;

[0015] performing feature splicing on the standardized feature vector to form the feature quantities of the continuous environmental variables.

[0016] Specifically, the feature quantities of the discrete environmental variables are extracted by the following method:

[0017] performing one-hot encoding on the discrete environmental variables to obtain an encoded data set; the discrete environmental variables comprise land use types, parent material types and soil texture types;

[0018] mapping the encoded data set to a continuous high-dimensional semantic feature space by embedding to form a high-dimensional semantic feature set, and determining the feature quantities of the discrete environmental variables according to the high-dimensional semantic feature set.

[0019] Specifically, the feature quantities of the multi-source remote sensing images are extracted by the following method:

[0020] radiation correction, atmospheric correction and normalization processing are performed on the multi-source remote sensing images to obtain preprocessed images;

[0021] Deep feature extraction is performed on the preprocessed images by using a convolutional neural network to obtain joint spatial and spectral features;

[0022] The feature quantity of the multi-source remote sensing images is determined according to the joint spatial and spectral features.

[0023] Specifically, the multi-modal adaptive feature fusion is performed on the feature quantity of the data sample set, the fusion weight is adaptively determined by using a cross-attention mechanism, and a joint feature quantity is obtained, including:

[0024] The feature quantity of the multi-source remote sensing images is taken as a query vector;

[0025] The feature quantities of the continuous and discrete environmental variables are spliced, and the spliced result is input into a feature transformation module to generate a key vector and a numerical vector, respectively;

[0026] Based on the query vector, the key vector and the numerical vector, the weight between the query vector and the key vector is adaptively calculated by using a cross-attention mechanism to obtain a fused joint feature quantity.

[0027] On the other hand, the embodiment of the present application provides a soil type mapping method based on multi-modal feature adaptive fusion and spatial neighborhood constraint, including:

[0028] Obtaining the to-be-predicted data of all spatial units in a research area;

[0029] Using the soil type prediction model trained according to the above method to predict each spatial unit to obtain the corresponding soil type prediction result;

[0030] According to the prediction results of adjacent spatial units, the type prediction result of each spatial unit is spatially post-processed and integrated to obtain a soil type mapping result which is spatially continuous and has a smooth boundary.

[0031] Specifically, according to the prediction results of adjacent spatial units, the type prediction result of each spatial unit is spatially post-processed and integrated to obtain a soil type mapping result which is spatially continuous and has a smooth boundary, including:

[0032] Using the majority voting method, the soil type prediction results of each spatial unit and its adjacent spatial units are analyzed to determine the final soil type prediction result of each spatial unit;

[0033] The final soil type prediction result of each space unit is subjected to spatial filtering processing to obtain a soil type mapping result which is spatially continuous and has a smooth boundary.

[0034] In another aspect, the embodiment of the present application also provides a digital soil mapping application system, comprising:

[0035] at least one processor;

[0036] at least one memory for storing at least one program;

[0037] When the at least one program is executed by the at least one processor, the at least one processor implements the method as described above.

[0038] In another aspect, the embodiment of the present application also provides a computer readable storage medium, wherein a processor executable program is stored, and the processor executable program is used to execute the method as described above when executed by a processor. The embodiment of the present application has the following beneficial effects:

[0039] The embodiment of the present application provides a soil type prediction model training method, mapping method and system based on multi-modal feature fusion and spatial neighborhood constraint. On the one hand, the model training method constructs a multi-source heterogeneous data sample set, extracts feature quantities of various data types of data from the data sample set, and performs multi-modal adaptive feature fusion on the obtained various types of feature quantities to obtain joint feature quantities for determining initial parameters of a preset model. By designing a multi-modal feature fusion mechanism, the connection between the features of each modality is analyzed, and the problems of large scale difference, information redundancy and insufficient feature fusion existing in the model during multi-modal data fusion are solved, which can effectively improve the accuracy of the mapping result expressed by the model output. On the other hand, the model training method determines a composite loss function for model training based on a spatial neighborhood smoothing regularization term and a classification cross-entropy loss. The cross-entropy loss function can measure the difference between the model prediction and the real soil type label, guide the model to improve the multi-class discrimination ability, and the spatial regularization term is used to constrain the spatial smoothness of the prediction result, suppress excessive fluctuations or local discontinuity, and enhance the geographical consistency and interpretability of the model output, thereby improving the spatial continuity of the mapping result expressed by the model output. Further, the composite loss function is used as a penalty function for model training, which has an adaptive weight adjustment function and can improve the generalization performance and application versatility of the model in different ecological regions and multi-scale scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a step flowchart of a soil type prediction model training method based on multi-modal feature fusion and spatial neighborhood constraint provided by the embodiment of the present application;

[0041] Figure 2 is a step flowchart of another soil type prediction model training method provided by an embodiment of the present application;

[0042] Figure 3 is a step flowchart of a soil type mapping method based on multi-modal feature fusion and spatial neighborhood constraint provided by an embodiment of the present application;

[0043] Figure 4 is a result comparison chart of a soil type mapping method based on multi-modal feature fusion and spatial neighborhood constraint provided by an embodiment of the present application;

[0044] Figure 5 is a structural block diagram of a digital soil mapping application system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] The present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of description and explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0046] Several terms involved in the present application are explained as follows:

[0047] SCORPAN model (soil forming factor model): a classic digital soil mapping theoretical framework. The model combines soil properties with existing soil data (S, Soil), climate conditions (C, Climate), biological activities (O, Organisms), terrain relief (R, Relief), parent material characteristics (P, Parent material), soil formation time (A, Age), and spatial location (N, Space) and other environmental factors to establish a quantitative relationship by mathematical or statistical methods, to clearly reveal the spatial distribution of soil types and properties, and is widely used in precision agriculture, environmental monitoring and land resource management fields.

[0048] Multi-modal feature fusion: refers to the fusion of heterogeneous data from multiple different modalities (such as remote sensing images, terrain data, climate variables, soil properties, etc.) into a unified feature representation, in order to utilize the information complementary relationship between modalities to improve the performance and stability of the prediction model.

[0049] Spatial adjacency matrix: a matrix describing the adjacency relationship between spatial units, usually represented in 0-1 form, with the element value in the matrix being 1 indicating that the corresponding spatial unit has an adjacency relationship in space, and the value being 0 indicating no adjacency relationship. Spatial adjacency matrix is widely used to express spatial topological structure information, facilitating spatial smoothing, interpolation or spatial statistical analysis.

[0050] Spatial smoothing regularizer: A constraint condition usually added to the loss function of the model, which explicitly constrains the smoothness and continuity of the model's output in the spatial range by minimizing the difference of the prediction results of the samples within the spatial neighborhood, avoiding obvious spatial fragmentation or local abnormal fluctuations in the prediction results.

[0051] Batch Normalization: A feature standardization method in neural network training, which can effectively alleviate the problems of gradient vanishing and gradient explosion by normalizing the mean and variance of each batch of features, accelerate the training process of the model, and improve the generalization performance of the model.

[0052] Embedding mechanism: refers to mapping high-dimensional sparse discrete feature representation (such as One-Hot encoding of categorical features) to low-dimensional continuous and dense vector space to better capture the potential semantic association between features and improve the quality of feature representation and the prediction performance of the model.

[0053] As shown in Figure 1 The embodiment of the present application provides a soil type prediction model training method based on multi-modal feature fusion and spatial neighborhood constraint, which includes steps S100-S400 as shown below.

[0054] S100: Construct a multi-source heterogeneous digital soil mapping data sample set, which includes continuous environmental variables, discrete environmental variables and multi-source remote sensing images.

[0055] According to the soil forming factor theory and SCORPAN model, multi-source heterogeneous environmental variables are collected, and unified data format normalization processing, unified spatial reference system conversion, geometric correction and spatial registration processing, spatial resolution standardization, missing value filling (spatial interpolation method) and abnormal value removal processing are performed on the environmental variables. Finally, a high-quality digital soil mapping data sample set with unified spatial scale and consistent semantic expression is formed, providing a stable and reliable data basis for subsequent refined feature extraction and multi-modal fusion.

[0056] S200: Extract the feature quantity of the data sample set.

[0057] The feature quantities of the continuous environmental variables, discrete environmental variables and multi-source remote sensing images in the data sample set are extracted respectively to determine the continuous environmental feature input, discrete environmental feature input and remote sensing feature input for subsequent multi-modal fusion, and determine the data basis for subsequent multi-modal feature fusion.

[0058] S300: Multi-modal adaptive feature fusion is performed on the feature quantity of the data sample set, and the cross-attention mechanism is used to adaptively determine the fusion weight to obtain the joint feature quantity.

[0059] According to the cross attention mechanism, the feature quantities of the continuous environment variables, the feature quantities of the discrete environment variables, and the feature quantities of the multi-source remote sensing images obtained in step S200 are subjected to multi-modal adaptive feature fusion, (the remote sensing feature input is taken as a query vector (Query), and the continuous and discrete environment feature inputs determine a key vector (Key) and a value vector (Value)), and a joint feature quantity for performing soil type prediction and mapping tasks is determined according to the fusion result, which can effectively enhance the fusion and expression ability of the soil type prediction model to different source data features.

[0060] S400: Construct a composite loss function including a spatial neighborhood smoothing regular term and a classification cross-entropy loss; and perform joint optimization training on a preset deep learning model according to the joint feature quantity and the composite loss function until a preset requirement is met, to determine the soil type prediction model.

[0061] Based on the geographical spatial position relationship of the sample points in the data sample set of step S100, and in combination with the multi-modal joint embedding feature expression obtained in step S300, a spatial adjacency matrix is constructed to quantify the geographical proximity between the samples. On this basis, a spatial neighborhood smoothing regular term is introduced, and the difference between the soil type prediction probability distributions of adjacent samples is designed as a penalty function to impose a spatial continuity constraint on the model output result. This regular mechanism can effectively suppress local prediction noise and speckle phenomenon, enhance the geometric consistency and boundary stability of the soil type spatial distribution, and improve the geometric consistency and boundary stability of the soil type distribution, thereby enhancing the coherence and geological rationality of the digital soil mapping result in the spatial dimension. Further, based on the constructed spatial neighborhood structure, a composite loss function including a classification cross-entropy loss and a spatial smoothing regular term is introduced, and the model is jointly trained through an Adam optimization algorithm to obtain a soil type prediction model with high classification accuracy and spatial coherence. Among them, the classification loss adopts a cross-entropy loss to measure the difference between the model prediction and the real soil type label, and to improve the multi-class discrimination ability of the model; the spatial regular term suppresses local transition fluctuations and spatial discontinuity by constraining the spatial variation of the prediction result, and enhances the spatial consistency and interpretability of the model output.

[0062] The following is a further introduction to the process of the soil type prediction model training method based on multi-modal feature fusion and spatial neighborhood constraint:

[0063] Specifically, in step S100, a multi-source heterogeneous digital soil mapping data sample set is constructed, including:

[0064] S110: Obtain a multi-source heterogeneous environment variable data set.

[0065] According to the soil forming factor theory and the SCORPAN model, multi-source and heterogeneous environmental variable data sets are obtained by collecting environmental variables such as terrain factors (such as slope, slope direction, curvature), remote sensing image data, climate factors (such as precipitation, air temperature), soil profile properties (such as texture, organic matter content, etc.), and land cover / land use types.

[0066] S120: The environmental variable data set is subjected to data format standardization, spatial reference system unification, spatial registration, spatial resolution unification, missing value filling, outlier removal, data cropping and rasterization to form a digital soil mapping data sample set with unified spatial scale and consistent semantic expression.

[0067] The environmental variable data set obtained above is subjected to unified data preprocessing, and the data preprocessing means include but are not limited to data format standardization processing, unified spatial reference system conversion, geometric correction and spatial registration processing, spatial resolution standardization, missing value filling and outlier removal processing, etc. Finally, a high-quality digital soil mapping basic data set with unified spatial scale and consistent semantic expression is formed, and a multi-modal high-quality soil mapping data set with spatial consistency is obtained, which is used for subsequent multi-modal feature fusion and soil type prediction.

[0068] Optionally, the data preprocessing includes the following processing means and order:

[0069] (1) Data format standardization: The multi-source and heterogeneous DSM environmental variables in the multi-source and heterogeneous environmental variable data sample set, such as GeoTIFF, Shapefile, NetCDF, etc., are uniformly converted to GeoTIFF format to ensure subsequent compatible reading and processing.

[0070] (2) Unified coordinate system (projection conversion): All spatial data after format standardization are uniformly converted to the same coordinate reference system (optional coordinate reference systems include GC2000) to ensure the consistency of geographical location.

[0071] (3) Spatial calibration (registration): Further, the different source data after unifying the coordinate system are subjected to spatial alignment processing to eliminate the spatial offset problem between images / rasters / vectors. Common methods include affine transformation or registration algorithm based on control points.

[0072] (4) Resolution standardization (resampling): The data with different spatial resolutions after spatial calibration are resampled to a unified 30-meter resolution spatial scale to ensure the consistency of input data dimensions.

[0073] (5) Missing value filling and outlier processing: The missing or abnormal data in the environmental variable data after the foregoing processing are subjected to interpolation, filling or rejection processing to improve data quality.

[0074] (6) Data cropping and rasterization: Finally, all data after the treatment of outliers is cropped according to the range of the study area, and necessary vector data is converted into raster format to obtain a digital soil mapping data sample set with spatial consistency and multi-modal high quality.

[0075] In some embodiments, in step S200, the characteristic quantity of the continuous environmental variable is extracted by the following steps S211-S212:

[0076] S211: Batch normalization is used for the continuous environmental variable to obtain a standardized feature vector; the standardized feature vector includes terrain factors, soil characteristic factors and meteorological factors.

[0077] The batch normalization method is used to uniformly process continuous environmental variables, including slope, slope direction, curvature, terrain humidity index, precipitation, annual average temperature, evapotranspiration and other types of continuous environmental variables. High-order feature expressions of various types of continuous environmental variables are obtained through a neural network structure of a nonlinear activation function, and the standardized feature vector is obtained after integration, which provides the basis for variable splicing to provide the characteristic quantity of the continuous environmental variable.

[0078] Specifically, the process of batch normalization processing of terrain features in the continuous environmental variable is as follows:

[0079] 1.1 Extracting original terrain parameter values: extracting terrain derived parameters such as slope direction, slope, slope length, planar curvature, profile curvature, east bias index, north bias index, east degree, north degree, ridge / river Euclidean distance, terrain position index, terrain humidity index, and terrain ruggedness index to obtain original terrain parameter values.

[0080] 1.2 Batch normalization (Batch Normalization) is used to eliminate the dimensional difference of the characteristic quantity to obtain standardized terrain feature variables The mathematical expression is shown in equation (1):

[0081]

[0082] where x terrain is the original terrain parameter value, μ terrain and σ terrain 2 are the mean and variance of the terrain feature batch, and ε = 10 -5 is a numerical stability constant.

[0083] Then, a learnable weight matrix W terrain and a bias term b terrain terrain

[0084]

[0085] terrain terrain terrain

[0086]

[0087] 2.1 Extracting original soil feature values: soil-forming parent material data, extracting sandy clay, sandy clay, dark clay, clay, and composite type soil-forming parent material composition parameters as soil classification features.

[0088] 2.2 Batch normalization (Batch Normalization) is used to eliminate feature dimension differences to obtain standardized soil feature vectors

[0089]

[0090] soil soil soil 2 -5

[0091] soil soil soil

[0092]

[0093] soil soil ​​​​​​​​​​​​​​​​​​​​​​is a bias term, soil is a bias term, is a normalized soil feature vector.

[0094] Specifically, the process of batch normalization for meteorological feature variables in continuous environmental variables is as follows:

[0095] 3.1 Extracting original meteorological feature values: according to the integrated annual average rainfall (unit: mm), solar radiation flux (W / m 2 ), multi-year average temperature (℃), diurnal / night surface temperature mean (℃), water vapor pressure (kPa), wind speed (m / s) and evapotranspiration (mm / day) and other continuous climate feature variables, as original climate feature values.

[0096] 3.2 Adopting batch normalization (Batch Normalization) to eliminate feature dimension difference, obtaining normalized meteorological feature vector The mathematical expression is shown in equation (5):

[0097]

[0098] where x climax represents the original climate feature value, μ climax and σ climax 2 are the mean and variance of the meteorological feature batch, respectively, and ε = 10 -5 is a numerical stability constant.

[0099] Then, through the learnable parameter matrix W climax and the bias term b climax , the normalized meteorological feature vector is linearly transformed, and a nonlinear activation function (ReLU) is introduced to enhance the model expression ability, obtaining high-dimensional climate features (meteorological factors) Z climax , as shown in equation (6):

[0100]

[0101] where Z climax is a high-dimensional meteorological feature vector, W climax is a learnable weight matrix, b climax is a bias term, is a normalized meteorological feature vector.

[0102] S212: Feature splicing is performed on the normalized feature vector to form the feature quantity of the continuous environmental variable.

[0103] The terrain factor, the soil characteristic factor and the meteorological factor obtained are spliced to form a unified high-dimensional numerical feature vector to form the characteristic quantity Z of the continuous environmental variable continuous .

[0104] Specifically, the process of forming the characteristic quantity Z of the continuous environmental variable is shown in formula (7): continuous

[0105] Z continuous = contact(Z terrain + Z soil + Z climax ) (7)

[0106] Wherein, contact() represents splicing multiple feature vectors in the channel dimension, used for integrating multi-source environmental variables, that is, connecting the terrain factor Z terrain , the soil characteristic factor Z soil and the meteorological factor Z climax along the channel dimension into a unified high-dimensional feature vector Z continuous (that is, the characteristic quantity of the continuous environmental variable), which is used for subsequent multi-modal fusion and soil type prediction tasks.

[0107] In some embodiments, in step S200, the characteristic quantity of the discrete environmental variable is extracted by the following steps S221-S222:

[0108] S221: One-hot encoding is performed on the discrete environmental variable to obtain an encoded data set; the discrete environmental variable includes land use type, parent material type and soil texture type.

[0109] First, determine the input data: including paddy field, watered land, dry land, orchard, tea garden, rubber garden, tree forest land, bamboo forest land, mangrove forest land, forest swamp, shrub forest land, bare land and other land type data; second, express whether it belongs to a certain specific (that is, each feature value is 0 or 1, wherein 0 represents nonexistence and 1 represents existence) by binary coding, to obtain an encoded data set. This One-Hot coding method ensures that the model can recognize the independent existence or absence of each land type, providing a discrete basis for subsequent classification feature processing.

[0110] S222: The encoded data set is mapped to a continuous high-dimensional semantic feature space by embedding; a high-dimensional semantic feature set is formed, and the characteristic quantity of the discrete environmental variable is determined according to the high-dimensional semantic feature set.

[0111] First, the One-Hot encoded data set is mapped to a continuous high-dimensional vector space through an Embedding layer to capture the semantic association between land types in the data set. As shown in formula (8): ​

[0112] E i =Embedding(W embed ,x i ) (8)

[0113] where x i represents the One-Hot encoding feature of the i-th land cover and land use type, W embed is the embedding matrix parameter, and E i represents the high-dimensional continuous feature vector after embedding.

[0114] Secondly, the category features after Embedding are mapped through the nonlinear activation function ReLU to further extract high-dimensional semantic features. As shown in equation (9):

[0115] Z class =ReLU(W class E i +b class ) (9)

[0116] In the equation, Z class is the type feature after nonlinear activation, and W class and b class are the weight and bias parameters to be learned, respectively.

[0117] Through the above processing, the type features are effectively embedded in the high-dimensional space, and the model can more deeply mine the subtle differences and potential correlations between land use types, providing high-quality feature expression support for subsequent accurate classification and prediction of soil types.

[0118] In some embodiments, in step S200, the feature quantities of the multi-source remote sensing image are extracted through the following steps S231-S233:

[0119] S231: Perform radiation correction, atmospheric correction, and normalization processing on the multi-source remote sensing image to obtain a preprocessed image.

[0120] Radiation correction, atmospheric correction, and normalization processing are performed on the obtained multi-source remote sensing image to ensure the consistency and effectiveness of the image data, serving as the data basis for subsequent deep feature extraction.

[0121] S232: Use a convolutional neural network to perform deep feature extraction on the preprocessed image to obtain joint features of space and spectrum.

[0122] The processed remote sensing image data is input into the CNN neural network, and deep spatial features are extracted through multi-layer convolution operations to obtain joint features of space and spectrum. The specific convolution operation calculation process is shown in equation (10):

[0123]

[0124] wherein, indicate the feature map of the l-1th layer, and and are the convolution kernel and bias term of the lth layer respectively, e is the convolution operation, and ReLU is a nonlinear activation function; the convolution network is set to 5 layers, each layer uses multiple convolution kernels with a size of 5x5, and the number of channels increases by 32 layer by layer to enhance the extraction ability of multi-scale features.

[0125] S233: determining the feature quantity of the multi-source remote sensing image according to the joint feature of space and spectrum.

[0126] The joint feature of space and spectrum extracted by the depth is reduced in dimension through a pooling layer (Pooling), and the pooling method is Max Pooling with a window size of 2x2 and a step size of 2. The output after pooling is the final remote sensing image depth feature, and the feature quantity of the multi-source remote sensing image is finally obtained The calculation process is shown in formula (11):

[0127]

[0128] Specifically, in step S300, the process of performing multi-modal adaptive feature fusion on the feature quantity of the data sample set based on the cross-attention mechanism to obtain the joint feature quantity includes steps S310-S330 as follows:

[0129] S310: taking the feature quantity of the multi-source remote sensing image as a query vector;

[0130] The feature quantity of the multi-source remote sensing image is input into the fusion network as a query vector Q in the cross-attention mechanism. The query vector is mainly used to describe the input data, and the obtained query vector includes the feature quantity and the learnable linear projection matrix W Q The specific calculation formula is shown in formula (12):

[0131]

[0132] S320: splicing the feature quantities of the continuous environmental variables and the discrete environmental variables, and inputting the splicing result into a feature transformation module to generate a key vector and a numerical vector, respectively.

[0133] According to the feature quantity Z contimous of the continuous environmental variables and the feature quantity Z class of the discrete environmental variables, splicing is performed, and then input into the feature transformation module, and the corresponding learnable linear projection matrix W K and W V, to obtain the key vector K and the value vector V, and the specific calculation process is shown in equation (13) and equation (14):

[0134] Key vector: K = concat(Z contimous ,Z class )W K (13)

[0135] Value vector: V = concat(Z contimous ,Z class )W V (14)

[0136] S330: Based on the query vector, the key vector and the value vector, the weight between the query vector and the key vector is adaptively calculated through the cross attention mechanism, and the fused joint feature quantity is obtained.

[0137] First, cross attention calculation is performed according to the query vector, the key vector and the value vector, and the specific calculation process is shown in equation (15):

[0138]

[0139] Secondly, through the cross attention mechanism, the dependency relationship between different modal features is obtained, and the modal importance weight is automatically learned, so as to obtain the unified multi-modal joint feature embedding with physical meaning, and determine the joint feature quantity Z fused . As shown in equation (16):

[0140] Z fused = Attention(Q,K,V) (16)

[0141] Specifically, in step S400, the process of determining the spatial neighborhood smoothing regular term is as follows:

[0142] First, define the spatial adjacency matrix Based on the spatial coordinate information of the samples in the digital soil mapping data sample set, the K nearest neighbor (KNN) algorithm is used to construct, as shown in equation (17):

[0143]

[0144] Where N k (i) represents the k spatial nearest neighbor set of sample point i, and N is the total number of samples.

[0145] Secondly, the calculation process of the smoothing regular definition is as follows:

[0146] Let a soil type prediction probability matrix output by the soil type prediction model, wherein N is a sample number, C is a soil type category number, and L is defined as a spatial consistency (spatial smoothing) regular term spatial , and a mathematical expression thereof is shown in equation (18):

[0147]

[0148] wherein ε = {(i,j) | A ij = 1} represents an adjacency matrix set, |ε| is a total number of edges, is a type prediction result of a sample point, is a type prediction result of a sample point in an adjacency matrix.

[0149] By explicitly minimizing the difference between the prediction results of adjacent samples, the model is guided to generate consistent output within the spatial neighborhood, thereby enhancing the spatial continuity and smoothness of the prediction results. In the embodiments of the present application, L spatial is incorporated into the total loss function of the model, prompting the model to explicitly consider spatial dependence, resulting in continuous and stable prediction output, effectively suppressing the spatial fragmentation problem of the prediction results, and improving the geographical rationality of the model results.

[0150] To achieve the above spatial constraint target, a composite objective function is further constructed by combining the classification loss term and the spatial smoothing regular term, and the calculation process is as follows:

[0151] First, the cross-entropy loss function is defined, and the specific calculation process is shown in equation (19)

[0152]

[0153] wherein Y i,c represents a One-Hot label of the true category of sample i, represents the prediction probability of the model that sample i belongs to category c.

[0154] Secondly, the composite loss function is defined: combining the classification loss and the spatial smoothing regular term to form the final composite loss function (optimization objective function). As shown in equation (20):

[0155] L = L cls + λL spatial (20)

[0156] Wherein, λ is an adjustable balance coefficient, used to balance the classification accuracy and spatial continuity of land types. The model adopts the Adam optimizer to obtain a soil type prediction model with accurate classification and smooth and reasonable spatial expression by optimizing the classification accuracy and spatial continuity constraint at the same time. The soil type prediction model obtained based on the composite loss function of the fusion classification loss term and the spatial neighborhood smoothing regular term is used to predict the soil type of each grid cell in the study area one by one in the inference stage, and a complete digital soil type map is generated. The map can not only accurately reflect the spatial distribution of different soil types and the transition relationship of the boundaries, but also effectively alleviate the common problems of patch fragmentation and boundary oscillation, so that the mapping result is significantly improved in both classification accuracy and spatial expression continuity.

[0157] Implementing the embodiments of the present application includes the following beneficial effects:

[0158] The embodiments of the present application provide a soil type prediction model training method, mapping method and system based on multi-modal feature fusion and spatial neighborhood constraint. On the one hand, by constructing a multi-source heterogeneous data sample set, extracting the feature quantities corresponding to various data (such as spectrum, terrain, climate, soil survey, etc.), and using a multi-modal adaptive feature fusion mechanism to jointly model them, a joint feature expression is generated for initializing the model parameters. The feature fusion mechanism can analyze the internal correlation between different modalities, effectively address the problems of large scale difference, information redundancy and insufficient fusion in the fusion process of multi-modal data, and thus significantly improve the accuracy and expressiveness of the model prediction result in mapping expression; on the other hand, a composite loss function is constructed based on the spatial neighborhood smoothing regular term and the classification cross-entropy loss term, which is used to optimize the model training process. The classification cross-entropy loss is used to measure the difference between the model prediction and the real soil type label, and to improve the multi-class discrimination ability of the model; the spatial regular term effectively suppresses the excessive fluctuation and local discontinuity phenomenon by constraining the spatial smoothness of the prediction result, and enhances the geographical consistency and interpretability of the output result, thereby improving the coherence of the mapping result in the spatial dimension. In addition, the composite loss function has an adaptive weight adjustment mechanism, which can automatically balance the classification accuracy and spatial continuity according to different ecological regions and multi-scale scenarios, thereby significantly improving the generalization ability and application universality of the model.

[0159] As shown in Figure 2 , the embodiments of the present application also provide another soil type prediction model training method, which comprises:

[0160] S1: Construct a digital soil mapping data set

[0161] Collecting multi-source heterogeneous geographic data and unifying format, coordinate and resolution, constructing high-quality digital soil mapping data set with high spatial consistency. Specifically, by collecting and processing multi-source remote sensing images of the study area, digital elevation model (DEM) derived slope, aspect, curvature, terrain humidity index, and annual average temperature, annual precipitation, annual potential evaporation, etc. Climate variables, combined with land use type and soil texture classification information, unify format and coordinate system and standardize resolution, construct high-quality multi-source digital soil mapping data set.

[0162] S2: processing continuous environmental features

[0163] Based on the continuous variables in the data set of step S1, high-dimensional continuous environmental features are extracted by batch normalization and neural network, and the effective modeling of the complex nonlinear relationship between environmental variables and soil types is realized.

[0164] S3: processing discrete environmental features

[0165] Based on the discrete variables in the data set of step S1, One-hot encoding is performed and mapped to high-dimensional dense space to extract category semantic features to enhance the model's expression ability for semantic association of classification features.

[0166] S4: processing multi-source remote sensing image features

[0167] The multi-source remote sensing images of step S1 are corrected and normalized for pretreatment, and CNN is used to extract high-dimensional visual features coupled with surface coverage and spatial patterns as input for subsequent fusion of remote sensing image modalities.

[0168] S5: cross-modal adaptive feature fusion

[0169] The multi-modal features extracted in steps S2 to S4 are input into the fusion network to guide the cross-modal attention mechanism with remote sensing features, adaptively model the contribution of each factor to soil spatial differentiation, and generate joint feature representation.

[0170] S6: construct spatial neighborhood smoothing constraint mechanism

[0171] Based on the spatial relationship of step S1 and the multi-modal features of step S5, a spatial adjacency matrix is constructed and a smoothing regularization term is introduced to penalize the difference between adjacent position prediction results, and to constrain the spatial consistency and boundary stability of soil type identification.

[0172] S7: design model joint optimization training strategy

[0173] On the basis of step S6, a fusion composite loss function is constructed, the classification accuracy and the spatial continuity are balanced through a weight coefficient, an Adam optimizer is used for joint iterative training, and a soil type prediction model is obtained to predict and generate a digital soil map.

[0174] The method embodiment has the following beneficial effects:

[0175] 1. The problem of insufficient adaptability of multi-modal feature fusion is solved:

[0176] In view of the fact that traditional machine learning methods and convolutional neural networks (CNN) generally only use a simple splicing strategy when processing multi-modal data such as remote sensing images, terrain, climate, and soil parent material, resulting in problems such as large scale difference, serious information redundancy, and insufficient fusion depth, the present application proposes a multi-modal feature adaptive fusion method based on a cross-attention mechanism. The method takes remote sensing images as the leading modality, explicitly models the semantic association relationship between different modalities, and dynamically adjusts the weight of each modality feature through the cross-attention mechanism, thereby realizing deep information complementation and efficient fusion between multi-source environmental variables.

[0177] 2. The problem of insufficient spatial context structure modeling capability is solved:

[0178] In view of the fact that existing methods ignore the continuity and spatial neighborhood relationship of the spatial distribution of soil types, which easily causes the key problems of fragmented prediction results and incoherent boundary transition, the present application innovatively constructs a spatial smoothing regularization constraint mechanism based on a spatial adjacency matrix. The mechanism explicitly models the context structure relationship between adjacent spatial units, and significantly improves the continuity of the spatial distribution of soil types, the smoothness of boundary transition, and the geographical rationality of the mapping results by introducing spatial neighborhood constraints.

[0179] 3. The problem of insufficient cross-regional generalization capability and adaptability is solved:

[0180] In view of the fact that existing soil mapping models often have problems such as decreased prediction accuracy and insufficient generalization capability when facing cross-ecological zones and large-scale spatial heterogeneity environments, the present application proposes a composite loss function optimization strategy that fuses classification loss and spatial smoothing regularization terms, which realizes adaptive generalization of model performance under cross-regional conditions by dynamically adjusting the loss weight. The method significantly improves the robustness of the model in multiple ecological zones and multiple topographic scenarios, and meets the application requirements of large-scale digital soil survey and fine soil mapping.

[0181] As shown in Figure 3 The present application embodiment also provides a soil type mapping method based on multi-modal feature adaptive fusion and spatial neighborhood constraints, which includes steps S500-S700 as shown below:

[0182] S500: Obtain the to-be-predicted data of all spatial units in the study area.

[0183] Obtain the environmental variable data of each spatial unit in the study area as the basis of the to-be-predicted data.

[0184] S600: Use the soil type prediction model trained according to the above model training method to predict each spatial unit to obtain the corresponding soil type prediction result.

[0185] Use the digital soil prediction model trained according to the above training method to predict the soil type of each spatial unit in the study area one by one to obtain the corresponding soil type probability matrix of each spatial unit And determine the initial class C i :

[0186]

[0187] S700: Spatially post-process and integrate the type prediction result of each spatial unit according to the prediction result of the adjacent spatial unit to obtain a soil type mapping result that is spatially continuous and has a smooth boundary.

[0188] According to the prediction result of each spatial unit and its adjacent spatial unit, the type prediction of the sample point can consider the prediction result of the adjacent space, so that the spatial continuity of the type prediction of the sample point is improved and the boundary smoothness is enhanced. After the type prediction result of each spatial unit after processing is integrated, a soil type mapping result that is spatially continuous and has a smooth boundary is obtained.

[0189] Specifically, in step S700, the type prediction result of each spatial unit is spatially post-processed and integrated according to the prediction result of the adjacent spatial unit to obtain a soil type mapping result that is spatially continuous and has a smooth boundary, including steps S710-S720 as follows:

[0190] S710: Use the majority voting method to analyze the soil type prediction result of each spatial unit and its adjacent spatial unit to determine the final soil type prediction result of each spatial unit.

[0191] Define the spatial neighborhood N(i) of each spatial unit, and use the majority voting method to determine the final type prediction C' of each unit i , which is formally defined as:

[0192]

[0193] In the formula, δ(C m ,k) is the Kronecker function, which is 1 when the class C m of the neighborhood unit m is k, otherwise it is 0;

[0194] S720: Spatial filtering is applied to the final type prediction results of each spatial unit to obtain spatially continuous and smooth-boundary soil type mapping results.

[0195] Spatial filtering is applied to the type prediction results of each spatial unit to further remove isolated and anomalous categories, resulting in a final soil type mapping result that is spatially continuous, has smooth boundary transitions, and is geographically reasonable.

[0196] To verify the effectiveness of the soil type mapping method based on multimodal feature fusion and spatial neighborhood constraints provided in this invention, a typical region was selected as a test area for digital soil mapping experiments. The test area is a hilly-plain transition zone with significant topographic relief and diverse and complex soil types, making it representative of the region. Figure 4 This is a comparison chart of the results of a soil type mapping method based on multimodal feature fusion and spatial neighborhood constraints provided by an embodiment of the present invention. In this embodiment, the mainstream Random Forest (RF) model is selected as the baseline method, and a comparative experiment is conducted under the same training samples and category labeling system. Figure 4 The left side A part and Figure 4 Part B on the right side of the image shows the soil type mapping results of the method of this invention and the RF model in the study area. The results show that this method exhibits better spatial coherence and boundary smoothness in areas with significant spatial heterogeneity, such as hill edges, river valley transition zones, and complex cultivated land boundaries. The predicted results are highly consistent with the remote sensing background distribution. (Local magnified area) Figure 4 Figures (a) to (c) are magnified local areas of the mapping results based on the method of the embodiments of the present invention. Figure 4 Figures (d) to (f) in the figure show magnified local areas of the mapping results based on the RF model. Further, it can be seen that the method can effectively suppress the "salt and pepper" noise misclassification patches commonly found in traditional RF models, reduce small-scale class jump phenomenon, avoid soil prediction image fragmentation, and improve the interpretability of image structure and the naturalness of spatial expression.

[0197] From the quantitative evaluation results (see Tables 1 and 2), the model of the present application achieved an overall accuracy (OA) of 0.9248 and a Kappa coefficient of 0.8678 in this area, which is significantly better than the OA (0.7264) and Kappa (0.6537) of the random forest model. Among the 69 soil codes, the present method achieved higher user accuracy (UA), producer accuracy (PA) and F1 score in most categories, especially in typical soil types (such as category 1, category 10, category 35, category 49, category 57, etc.), the F1 score improved significantly, showing stronger class recognition ability and generalization robustness. In summary, the present method significantly enhances the discriminant performance and spatial consistency expression of the model by fusing multi-modal environmental factors and introducing a spatial structure constraint mechanism, which is suitable for large-scale, high-precision automatic digital soil mapping tasks and has good popularization value.

[0198] Table 1 Comparison of overall mapping accuracy of the present method and random forest model in the test area

[0199]

[0200]

[0201] Table 2 Comparison of prediction accuracy of each soil type (present method vs. random forest)

[0202]

[0203]

[0204] As Figure 5 shown, the present embodiment also provides a digital soil mapping application system, comprising:

[0205] at least one processor;

[0206] at least one memory for storing at least one program;

[0207] When the at least one program is executed by the at least one processor, the at least one processor implements the model training method or the soil type mapping method as described above.

[0208] On the other hand, the present embodiment also provides a computer-readable storage medium, wherein a processor-executable program is stored, and the processor-executable program is used to execute the method as described above when executed by a processor.

[0209] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. The memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include remote memories that are remotely arranged relative to the processor, and the remote memories can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0210] It can be seen that the contents in the method embodiments are all applicable to the system embodiments, the system embodiments specifically implement the same functions as the above drawing method embodiments, and achieve the same beneficial effects as the above drawing method embodiments.

[0211] In addition, the embodiments of the present application further disclose a computer program product or a computer program, which is stored in a computer readable storage medium. The processor of the computer device can read the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the above method.

[0212] The embodiments of the present application also provide a computer readable storage medium storing a program executable by a processor, and the program executable by the processor is used to implement the above method when executed by the processor. Similarly, the contents in the method embodiments are all applicable to the storage medium embodiments, the storage medium embodiments specifically implement the same functions as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.

[0213] It is to be understood that all or some of the steps, systems, etc. in the methods disclosed above can be performed by software, firmware, hardware, or any suitable combination thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a micro-processing unit, as hardware, or as an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to those of ordinary skill in the art, computer storage media includes all computer-readable media in which data, such as computer readable instructions, data structures, program modules or other data, is tangibly embodied. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

Claims

1. A soil type prediction model training method based on multi-modal feature fusion and spatial neighborhood constraint, characterized in that, The method comprises the following steps: constructing a multi-source heterogeneous digital soil mapping data sample set, the data sample set comprising continuous environmental variables, discrete environmental variables and multi-source remote sensing images; extracting feature quantities of the data sample set; performing multi-modal adaptive feature fusion on the feature quantities of the data sample set, adaptively determining fusion weights by using a cross-attention mechanism, and obtaining joint feature quantities; constructing a composite loss function comprising a spatial neighborhood smoothing regularization term and a classification cross-entropy loss; and performing joint optimization training on a preset deep learning model according to the joint feature quantities and the composite loss function until a preset requirement is met, to determine a soil type prediction model.

2. The method of claim 1, wherein, The method of constructing a multi-source heterogeneous digital soil mapping data sample set comprises the following steps: obtaining a multi-source heterogeneous environmental variable data set; performing data format standardization, spatial reference system unification, spatial registration, spatial resolution unification, missing value filling, abnormal value removal, data cropping and rasterization on the environmental variable data set, to form a digital soil mapping data sample set that is uniform in spatial scale and consistent in semantic expression.

3. The method of claim 1, wherein, The feature quantities of the continuous environmental variables are extracted in the following manner: performing batch normalization processing on the continuous environmental variables to obtain standardized feature vectors; the standardized feature vectors comprise terrain factors, soil characteristic factors and meteorological factors; performing feature splicing on the standardized feature vectors to form the feature quantities of the continuous environmental variables.

4. The method of claim 1, wherein, The feature quantities of the discrete environmental variables are extracted in the following manner: performing one-hot encoding on the discrete environmental variables to obtain an encoded data set; the discrete environmental variables comprise land use types, parent material types and soil texture types; embedding the encoded data set into a continuous high-dimensional semantic feature space to generate a high-dimensional semantic feature set, and determining the feature quantities of the discrete environmental variables according to the high-dimensional semantic feature set.

5. The method of claim 1, wherein, The feature quantities of the multi-source remote sensing images are extracted in the following manner: performing radiation correction, atmospheric correction and normalization processing on the multi-source remote sensing images to obtain preprocessed images; performing deep feature extraction on the preprocessed images by using a convolutional neural network, to obtain joint features that fuse spatial and spectral information; determining the feature quantities of the multi-source remote sensing images according to the joint features that fuse spatial and spectral information.

6. The method of claim 1, wherein, The method of performing multi-modal adaptive feature fusion on the feature quantities of the data sample set, adaptively determining fusion weights by using a cross-attention mechanism, and obtaining joint feature quantities comprises the following steps: taking the feature quantities of the multi-source remote sensing images as query vectors; splicing the feature quantities of the continuous environmental variables and the discrete environmental variables, and inputting the splicing result into a feature transformation module to respectively map key vectors and numerical vectors; adaptively calculating the weights between the query vectors and the key vectors by using a cross-attention mechanism based on the query vectors, the key vectors and the numerical vectors, to obtain fused joint feature quantities.

7. A soil type mapping method based on multi-modal feature adaptive fusion and spatial neighborhood constraint, characterized in that, The method comprises the following steps: obtaining all spatial units in a research area for which data are to be predicted; The soil type prediction model obtained by training according to any one of claims 1-6 is used to predict each spatial unit to obtain a corresponding soil type prediction result; The soil type prediction result of each spatial unit is spatially post-processed and integrated according to the prediction results of adjacent spatial units to generate a soil type mapping result that is spatially continuous and has smooth boundaries.

8. The method of claim 7, wherein, The soil type prediction result of each spatial unit is spatially post-processed and integrated according to the prediction results of adjacent spatial units to generate a soil type mapping result that is spatially continuous and has smooth boundaries, including: The soil type prediction results of each spatial unit and its adjacent spatial units are analyzed by using the majority voting method to determine the final soil type prediction result of each spatial unit; The final soil type prediction result of each spatial unit is spatially filtered to obtain a soil type mapping result that is spatially continuous and has smooth boundaries.

9. A digital soil mapping application system characterized in that, It includes: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-8.

10. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The program executable by the processor when executed by the processor is used to execute the method of any one of claims 1-8.

Images