A multi-modal feature alignment fusion method and system for soil multi-source data

Abstract

The present application relates to the technical field of soil information acquisition and intelligent analysis, and particularly relates to a multi-modal feature alignment fusion method and system for soil multi-source data. The method obtains multi-source soil information, standardizes and scales different modal data, projects different modal soil data into a unified feature space through feature mapping, introduces multi-modal feature alignment constraints to reduce semantic differences, adaptively weights and fuses each modal feature based on modal importance modeling, introduces robustness constraints in the fusion process to suppress the influence of abnormal modal and noise data, and generates a comprehensive soil feature vector. The present application realizes the collaborative expression of multi-source soil data in a unified feature space, improves the accuracy, stability and applicability of multi-modal soil information fusion, and can be applied to soil nutrient inversion and digital soil mapping.

Description

Technical Field

[0001] This invention relates to the field of soil information acquisition and intelligent analysis technology, and in particular to a multimodal feature alignment and fusion method and system for multi-source soil data. Background Technology

[0002] With the continuous development of remote sensing technology, ground sensor technology, and rapid soil testing technology, the means of acquiring soil information are becoming increasingly diversified. Current methods for acquiring soil information mainly include laboratory soil testing, remote sensing observation, and ground and near-ground sensor measurements. To improve the accuracy of soil property analysis, existing technologies have attempted to combine multiple soil data sources. Although multi-source data fusion has improved prediction accuracy to some extent, existing methods mainly focus on simple data layering and model input layer stitching, failing to fully address the significant differences in spatial resolution, temporal scale, data dimensionality, and physical meaning among different data sources. Based on the shortcomings of existing technologies, it is necessary to propose a multimodal feature alignment and fusion method for multi-source soil data. Through a systematic data alignment and fusion processing mechanism, this method can improve the accuracy, stability, and applicability of joint modeling of multi-source soil data. Summary of the Invention

[0003] The purpose of this invention is to address the problem that differences in data type, spatial scale, temporal resolution, and physical semantics among multi-source soil data make it difficult to effectively align and fuse different modalities, resulting in insufficient stability and applicability of the fusion results. This invention provides a multi-modal feature alignment and fusion method and system for multi-source soil data. By standardizing multi-source heterogeneous soil data, performing feature mapping alignment and fusion calculations, it achieves collaborative expression of different modalities of soil data in a unified feature space, thereby improving the accuracy and robustness of multi-source soil data fusion analysis.

[0004] To achieve the above objectives, this invention provides a multimodal feature alignment and fusion method for multi-source soil data, comprising the following steps:

[0005] S1, Multi-source soil data acquisition and unified input: Multi-source soil data of the area to be tested is acquired through various data acquisition methods such as remote sensing platform, ground and near-ground sensors, gamma spectrum detector and laboratory testing, and the multi-source soil data is input into the data processing system through a unified data interface, so as to realize the centralized management of different types of soil data from the source and provide a complete data foundation for subsequent multimodal processing;

[0006] S2, Standardization and scaling of multi-source soil data: After the input of multi-source soil data is completed, the multi-source soil data is standardized and scaled to eliminate the inconsistency in numerical values ​​of different modal soil data, thereby establishing a unified data expression basis for subsequent cross-modal feature processing.

[0007] S3, Construction of Spatial Consistency of Multi-Source Soil Data: On the basis of completing the scale unification processing, further spatial consistency processing is carried out on the soil data to address the differences in spatial resolution, sampling method and spatial coverage of different modal soil data. This enables soil data from different sources to establish a correspondence under a unified spatial reference framework, ensuring the spatial alignment of multimodal data in subsequent feature processing.

[0008] S4, Multimodal Soil Feature Mapping and Alignment: Based on the construction of spatial consistency, feature extraction and mapping processing are performed on multi-source soil data to convert soil data of different modalities and dimensions into feature representations in a unified feature space, thereby achieving alignment of soil data of different modalities at the feature level and creating conditions for the collaborative utilization of multimodal information;

[0009] S5, Adaptive weighted modeling of multimodal feature importance: After completing the multimodal feature alignment, considering the differences in the contribution of different modal soil features to the expression of soil information in different regions and application scenarios, the importance of each modal soil feature is modeled to form weight parameters that reflect the relative contribution of each modality, providing a basis for subsequent fusion processing;

[0010] S6, Multimodal Soil Feature Fusion and Robustness Processing: Based on the importance modeling results, multimodal aligned features are fused to generate a comprehensive soil feature representation. A robust processing mechanism is introduced during the fusion process to reduce the impact of abnormal modal data on the fusion results, thereby improving the stability and reliability of the fused features.

[0011] S7, Output and storage of fusion results: After completing the fusion and robustness processing, the comprehensive soil characteristic data is output and stored in the data storage unit to support subsequent applications, including soil property analysis, model prediction and result comparison.

[0012] Furthermore, the multi-source soil data in S1 includes multispectral remote sensing data, gamma spectral data, two-dimensional digital image data, soil moisture data, laboratory soil testing data, and vegetation index data calculated based on remote sensing data.

[0013] Furthermore, S2 addresses the differences in dimensions, value ranges, and statistical distributions of soil data from different sources by performing scale unification and standardization on the soil data of each modality. The processing relationship is expressed as follows:

[0014] ;

[0015] in, Indicates the first The original feature vector of soil-like data, , Let represent the minimum and maximum values ​​of the feature vector, respectively. This represents the feature vector of soil data after scale unification;

[0016] The scale unification process is used to eliminate the differences in numerical ranges between different modal soil data, providing a unified input condition for subsequent feature alignment.

[0017] Furthermore, S3 addresses the inconsistencies in spatial resolution and sampling density between different modal soil data by performing spatial alignment and distribution correction on the standardized soil data. The correction relationship is expressed as follows:

[0018] ;

[0019] in, Indicates the first Spatial alignment function for soil-like data, Represents the target space grid. This represents the feature vector of the soil data after spatial alignment.

[0020] The spatial alignment is used to map point-scale soil data to a unified spatial reference system to ensure the correspondence between different modal data in spatial dimensions.

[0021] Furthermore, in step S4, the multi-source soil data undergoing spatial alignment processing are feature-mapped, projecting soil data of different modalities onto a unified feature space. The mapping relationship is expressed as follows:

[0022] ;

[0023] in, Indicates that for the first The feature mapping function set for soil-like data, This represents the aligned feature vector after mapping;

[0024] Semantic feature alignment is achieved by constraining the consistency of the distribution of features from different modalities in a unified feature space. The constraint relationship is expressed as follows:

[0025] ;

[0026] in, A measure of the distributional differences among multimodal features;

[0027] The semantic alignment is used to reduce the expression bias of different modal features in a unified feature space.

[0028] Furthermore, to address the issue of varying information contribution levels of different modal soil data in different regions and application scenarios in S5, adaptive weights are introduced for each modal feature. The weight calculation relationship is expressed as follows:

[0029] ;

[0030] in, Indicates the first Importance scoring parameters for soil characteristics This represents the corresponding adaptive fusion weights. Indicates the number of modes participating in the fusion;

[0031] The adaptive weights are used to dynamically adjust the contribution ratio of different modal soil data in the fusion process.

[0032] Furthermore, S6, based on adaptive weights, performs fusion calculations on the multimodal alignment features to generate a comprehensive soil feature vector, the fusion relationship of which is expressed as follows:

[0033] ;

[0034] Simultaneously, robust constraints are introduced into the fusion results to suppress the interference of anomalous modal features on the fusion results. The constraint relationship is expressed as follows:

[0035] ;

[0036] in, Represents the robust constraint function;

[0037] The robust constraints are used to reduce the impact of noisy data on the fusion results.

[0038] Furthermore, S7 outputs the comprehensive soil characteristic data after robust fusion processing and stores it in the data storage unit for subsequent soil attribute analysis, prediction modeling, and result comparison and backtracking.

[0039] On the other hand, the present invention also provides a multimodal feature alignment and fusion system for multi-source soil data, comprising:

[0040] Multi-source data acquisition module: used to acquire and access multi-source soil data;

[0041] Data preprocessing module: Connected to the multi-source data acquisition module, it is used to perform standardized preprocessing on multi-source soil data;

[0042] Feature alignment module: Connected to the data preprocessing module, it is used for feature mapping and alignment of soil data of different modalities;

[0043] Fusion processing module: Connected to the feature alignment module, it is used to perform fusion calculations on the aligned multimodal features;

[0044] Consistency Constraint Module: Connected to the fusion processing module, it is used to perform consistency constraint processing on the fused features;

[0045] Data storage and output module: Connected to the consistency constraint module, it is used to store and output the fusion results.

[0046] The beneficial effects of this invention are:

[0047] By constructing a multimodal feature alignment and fusion processing flow for multi-source soil data, this invention performs unified standardization, spatial consistency processing, feature alignment, and fusion on soil data from different sources, scales, and data formats. This achieves collaborative expression of multi-source soil data within a unified feature space, overcoming the problems of direct fusion of multi-source soil data and insufficient stability of fusion results in existing technologies. By introducing multimodal feature importance modeling and a robust fusion mechanism, the fusion weights can be dynamically adjusted according to the actual information contribution of different modal data, reducing the impact of abnormal data and noise on the fusion results and improving the reliability and generalization ability of the fusion results. The method of this invention has a clear structure and strong scalability, and can flexibly adapt to various types and numbers of soil data sources, providing stable and efficient technical support for applications such as soil property analysis, digital soil mapping, and precision agricultural management. Attached Figure Description

[0048] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0049] Figure 1 This is a schematic diagram of the multimodal feature alignment and fusion method and system for multi-source soil data according to the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0051]

Example 1

[0052] like Figure 1As shown, this invention discloses a multimodal feature alignment and fusion method for multi-source soil data, comprising the following steps:

[0053] S1. First, acquire multispectral remote sensing image data covering the study area, and calculate the corresponding vegetation index data based on the remote sensing images; acquire gamma spectrum data corresponding to the study area through ground detection equipment; acquire soil moisture data through soil moisture sensors; acquire soil physicochemical test data through field sampling and laboratory analysis; and acquire two-dimensional digital image data corresponding to the spatial area.

[0054] The above multi-source soil data are input into the data processing system according to a unified spatial and temporal identifier to form a multi-source soil dataset:

[0055] ;

[0056] in, This represents multispectral remote sensing and vegetation index characteristic data. This represents the characteristic data of the gamma spectrum. Represents two-dimensional digital image feature data. This indicates soil moisture data. This indicates laboratory soil testing data;

[0057] S2, for continuous data such as multispectral reflectance, laboratory test values, and soil moisture parameters, standardization is performed on each modal data separately, and the processing relationship is expressed as follows:

[0058] ;

[0059] in, and These represent the mean and standard deviation of the corresponding modal data, respectively.

[0060] Through the above processing, the data on multispectral reflectance, laboratory test indicators (including soil organic matter, total nitrogen, available phosphorus, available potassium) and soil moisture content are kept consistent on the numerical scale.

[0061] S3, addressing the characteristics of multispectral image data being planar and laboratory test data being point-based, introduces a unified spatial grid to perform spatial mapping on various data types. The processing relationship is expressed as follows:

[0062] ;

[0063] in, Represents a unified spatial reference grid;

[0064] Through spatial mapping processing, laboratory testing point data, gamma spectrum observation data, etc. are mapped to corresponding spatial grid cells to realize the correspondence of multi-source soil data in spatial dimensions;

[0065] S4. After completing the spatial consistency processing, feature extraction is performed on the soil data of different modalities, and corresponding feature vectors are constructed.

[0066] Construction of multispectral remote sensing feature vectors: Vegetation indices are calculated based on multispectral band data, and the calculation relationship is expressed as follows:

[0067] ;

[0068] Multispectral reflectance is combined with vegetation indices to form a multispectral feature vector:

[0069] ;

[0070] Construction of Feature Vectors for Laboratory Soil Testing: Characterizing the Soil Physicochemical Indicators Obtained from Laboratory Testing.

[0071] ;

[0072] in, Indicates the soil organic matter content. , , These represent soil nitrogen, phosphorus, and potassium levels, respectively.

[0073] Gamma Spectrum Feature Vector Construction: Gamma spectral data are counted and statistically analyzed within a preset energy range to form a spectral feature vector.

[0074] ;

[0075] Soil moisture feature vector construction: Representing soil moisture content and its statistical characteristics as follows:

[0076] ;

[0077] S5, the above-mentioned feature vectors of different modalities are mapped to a unified feature space, and the mapping relationship is expressed as follows:

[0078] ;

[0079] in, This represents the original characteristics of the input soil. Apply feature mapping function The resulting soil characteristics are represented;

[0080] And based on the importance of each modal feature to the comprehensive soil properties, the corresponding weights are calculated:

[0081] ;

[0082] Based on the modal weights The multimodal soil sub-feature vectors, after feature mapping and alignment, are fused to generate a comprehensive soil feature vector characterizing the overall soil properties of the study area. The fusion relationship is expressed as follows:

[0083] ;

[0084] in, Indicates the first Sub-feature vectors of soil-like data after feature mapping and alignment. This represents the fusion weight of the corresponding modal soil features, and satisfies:

[0085] ;

[0086] Furthermore, the comprehensive soil feature vector can be expressed as:

[0087] ;

[0088] in, Represents the comprehensive soil feature vector. The dimension of the comprehensive soil feature vector;

[0089] The structure of each modal soil sub-feature vector in the unified feature space is represented as follows:

[0090] ;

[0091] in, Indicates the first Dimensions of modal soil characteristics;

[0092] The comprehensive soil feature vector includes, but is not limited to, the following feature components:

[0093]

[0094] Each feature component has undergone the aforementioned standardization, spatial consistency, and feature mapping and alignment processes.

[0095] S7. The integrated soil feature vector is output as the final fusion result and stored according to spatial units for subsequent soil attribute inversion and digital soil mapping.

[0096]

Example 2

[0097] In a typical embodiment of the present invention, this embodiment discloses a multimodal feature alignment and fusion system for multi-source soil data, comprising:

[0098] Data acquisition module:

[0099] Raw soil data was collected using drones equipped with portable gamma spectrometers and optical multispectral cameras, as well as manually operated high-precision soil moisture sensors; simultaneously, standardized chemical composition data were obtained by combining laboratory soil sample analysis. Specifically, this included:

[0100] Gamma spectral data ,in, For spatial coordinates, Energy channels; multispectral image data ,in, Spectral bands; soil moisture data ,in, Time dimension; laboratory test data ,in, This refers to the concentration of soil nutrients and other elements.

[0101] Data preprocessing module:

[0102] The collected raw data is preprocessed, including:

[0103] Gamma spectral data denoising and background correction:

[0104] ;

[0105] in, For the background spectrum, The corrected energy spectrum data;

[0106] Multispectral image radiometric calibration:

[0107] Original digital quantity Converted to ground reflectance :

[0108] ;

[0109] in, Pixel In the band Reflectance of ground features after calibration; Pixel In the band The original digital quantity; Band The gain coefficient is obtained through the calibration plate; Band The offset is used to correct the sensor's zero bias;

[0110] Considering atmospheric correction, it can be further expressed as:

[0111] ;

[0112] in, Pixel In the band Atmospherically corrected reflectance; : Measured ground radiance; Solar irradiance; : Solar zenith angle; Atmospheric scattering and absorption correction term;

[0113] Geometric correction of multispectral images:

[0114] Original cell coordinates Mapping to geographic coordinates :

[0115] ;

[0116] in, : Original image pixel coordinates (in pixels); : Corresponding geographic coordinates (such as latitude and longitude, projected coordinates, in meters); Translation parameters are used to correct for origin offset. Scale / scaling factor, used to correct cell size in the X and Y directions; Rotation and shearing coefficients are used to correct for tilt in drone footage.

[0117] Imputation of missing values ​​and time series smoothing of soil moisture data:

[0118] ;

[0119] in, Original soil moisture value; Smoothed soil moisture value; : Sliding window size.

[0120] Laboratory test data were standardized to obtain a unified numerical range:

[0121] ;

[0122] in, Raw values ​​from laboratory testing; Standardized values; Soil elements;

[0123] Feature extraction and modality mapping module:

[0124] For gamma-ray spectral data, convolutional neural networks (CNNs) are used to extract local spectral features. ;

[0125] For multispectral image data, a CNN+attention mechanism is used to extract spatial-spectral features. ;

[0126] Numerical feature vectors were extracted from soil moisture data and laboratory data using a multilayer perceptron (MLP). ;

[0127] in, : Feature vectors for each modality; Convolutional neural networks, attention-enhancing networks, and multilayer perceptrons;

[0128] Different modal features are aligned to a shared feature space through a modality mapping layer:

[0129] ;

[0130] in, For the aligned feature representation, These are learnable mapping parameters.

[0131] This module can map different modal data (images, energy spectrum, soil moisture, laboratory data) to a unified feature space, providing a consistent representation for multimodal fusion and improving the comparability and relevance of cross-modal information;

[0132] Multimodal fusion and soil feature vector generation module:

[0133] Fuse the aligned multimodal features:

[0134] ;

[0135] The final multidimensional feature vector of soil is generated using fused features. :

[0136] ;

[0137] in, : Fuse feature vectors; : Fusion layer parameters; The final soil multidimensional feature vector includes information on nutrients, heavy metals, microbial activity, and soil moisture.

[0138] The fused feature vectors can be used for soil quality assessment, precision fertilization decisions, pollution monitoring, and other intelligent agricultural applications. This module further encodes the fused features through a fully connected network, making the soil information highly condensed and easy for downstream tasks to use.

[0139] System Application and Verification:

[0140] The system was deployed in an experimental farmland, and data was collected by drone flight, which was then verified in conjunction with laboratory soil samples.

[0141] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention can have various modifications and variations; any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal feature alignment and fusion method for multi-source soil data, characterized in that, Includes the following steps: S1, acquire multi-source soil data of the area to be tested, and input the multi-source soil data into the data processing system through a unified data interface; S2, In view of the differences in the dimensions, value range and statistical distribution of soil data from different sources, the multi-source soil data is standardized and scaled. S3 addresses the differences in spatial resolution, sampling method, and spatial coverage of different modal soil data by constructing spatial consistency for the standardized soil data, enabling the soil data of each modality to establish a corresponding relationship under a unified spatial reference framework; S4 extracts features from multi-source soil data that have completed spatial consistency construction, and projects soil data of different modalities to a unified feature space through feature mapping to achieve multi-modal feature alignment; S5. Based on the contribution of different modal soil features to the expression of soil information, the importance of each modal feature is modeled, and corresponding adaptive weight parameters are generated. S6. Based on the adaptive weights, the aligned multimodal soil features are fused, and robust constraints are introduced during the fusion process to suppress the influence of abnormal modes, generating a comprehensive soil feature vector. S7. Output and store the comprehensive soil feature vector for subsequent soil property analysis and prediction modeling.

2. The method for multimodal feature alignment and fusion of multi-source soil data according to claim 1, characterized in that, The multi-source soil data includes multispectral remote sensing data, gamma spectral data, two-dimensional digital image data, soil moisture data, laboratory soil testing data, and vegetation index data calculated based on remote sensing images.

3. The multimodal feature alignment and fusion method for multi-source soil data according to claim 1, characterized in that, The multi-source soil data standardization and scale unification processing in step S2 includes normalization and standardization processing of soil data of different modalities, and the processing relationship is expressed as follows: ； in, Indicates the first The original feature vector of modal soil data, , These represent the minimum and maximum values ​​of the feature vector, respectively.

4. The method for aligning and fusing multimodal features of multi-source soil data according to claim 1, characterized in that, The spatial consistency construction in step S3 maps soil data of different modalities to a unified spatial reference frame through a spatial mapping function. The mapping relationship is expressed as follows: ； in, Indicates the first The spatial mapping function corresponding to modal soil data, Represents the target space grid. This represents the feature vector of the soil data after spatial consistency processing. The spatial mapping function is used to uniformly map point-scale soil data to the target spatial grid to ensure the spatial alignment of soil data of different modes.

5. The multimodal feature alignment and fusion method for multi-source soil data according to claim 1, characterized in that, The multimodal soil feature mapping in step S4 projects different modal soil data into a unified feature space through a feature mapping function. The mapping relationship is expressed as follows: ； in, Indicates that for the first The feature mapping function set for modal soil data, This represents the aligned feature vector after mapping; Semantic feature alignment is achieved by constraining the distribution differences of different modal features in a unified feature space. The constraint relationship is expressed as follows: ； in, A function representing the distributional difference measure between different modal features; By minimizing the distribution difference metric, semantic consistency of soil features of different modalities in a unified feature space is achieved.

6. The multimodal feature alignment and fusion method for multi-source soil data according to claim 1, characterized in that, In step S5, adaptive weights are introduced for each modal feature based on its information contribution level. The weight calculation relationship is expressed as follows: ； in, Indicates the first Importance scoring parameters for modal soil characteristics This represents the adaptive fusion weights for corresponding modal soil features. Let represent the number of modes participating in the fusion, and satisfy: ； The adaptive weights are used to dynamically adjust the contribution ratio of different modal soil characteristics in the fusion process according to different regions.

7. The multimodal feature alignment and fusion method for multi-source soil data according to claim 1, characterized in that, In step S6, the multimodal alignment features are fused based on adaptive weights to generate a comprehensive soil feature vector, the fusion relationship of which is expressed as follows: ； in, Represents the comprehensive soil feature vector. Indicates the first Alignment of feature vectors of modal soil features in a unified feature space; Meanwhile, robustness constraints are introduced into the fusion results, and the constraint relationship is expressed as follows: ； in, This represents a robustness constraint function used to suppress the influence of anomalous modal features on the fusion results, thereby improving the stability and reliability of the integrated soil feature vector.

8. A multimodal feature alignment and fusion system for multi-source soil data, characterized in that, include: The multi-source data acquisition module is used to acquire and access multi-source soil data; The data preprocessing module is connected to the multi-source data acquisition module and is used to standardize and scale-unify multi-source soil data. A spatial consistency construction module, connected to the data preprocessing module, is used to perform spatial consistency processing on soil data of different modalities. The feature alignment module, connected to the spatial consistency construction module, is used to extract, map, and align features of soil data of different modalities. The fusion processing module, connected to the feature alignment module, is used to fuse multimodal features based on adaptive weights and generate a comprehensive soil feature vector. The data storage and output module is connected to the fusion processing module and is used to store and output the fusion results.

Images