Agricultural multi-modal data processing method and device, electronic equipment and storage medium

Abstract

The invention provides an agricultural multi-modal data processing method and device, electronic equipment and a storage medium, and relates to the technical field of agricultural data processing, and the method comprises the steps: carrying out the data preprocessing of an aperture radar image, an optical image and geographic information data based on auxiliary geographic data, obtaining a target aperture radar image, a target optical image and target geographic information data; determining crop structure features corresponding to the target crop based on the target aperture radar image and the target optical image; determining crop physiological features corresponding to the target crop based on the target optical image; determining soil physical and chemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data; and determining multi-modal fusion feature data corresponding to the target crop based on the crop structure features, the crop physiological features and the soil physicochemical features. According to the method, the problem of insufficient characteristic heterogeneity processing in the prior art is solved, the internal association of the multi-modal data is effectively mined, and the fine management requirement of precision agriculture can be met.

Description

Technical Field

[0001] This invention relates to the field of agricultural data processing technology, and in particular to an agricultural multimodal data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the accelerated development of precision agriculture and smart agriculture, the demand for data-driven decision-making in agricultural production is becoming increasingly urgent. Agricultural systems are inherently highly complex, involving the dynamic interaction of multiple factors such as crop growth, soil conditions, and meteorological environment. This makes data from a single source insufficient to support the needs of refined production management. Therefore, multimodal agricultural data, encompassing satellite remote sensing (such as microwave scattering data and optical spectral reflectance data from Synthetic Aperture Radar (SAR)), ground sampling, and IoT sensor monitoring, has become the core carrier for current research and application. However, technological development in this field still faces significant bottlenecks: on the one hand, the diversification of data sources leads to significant heterogeneity in multimodal data, with large differences in the physical meaning and structure of different types of data; on the other hand, the spatiotemporal correlation of agricultural factors, such as the dynamic coupling between crop growth cycles and soil moisture changes, is difficult to capture effectively. These problems collectively make it difficult for existing processing methods to achieve efficient integration and accurate analysis of multimodal data, becoming a key obstacle restricting the application of agricultural big data in precision management.

[0003] Currently, agricultural data processing mainly relies on single-modal analysis or simple fusion methods. Single-modal analysis extracts vegetation indices from optical images or retrieves soil moisture from SAR images, neglecting the coupling relationship between crops and soil. Simple fusion methods directly stitch together multimodal features and input them into classification models without considering the physical differences between multimodal data. These methods fail to fully leverage the value of multimodal data in practical applications, resulting in insufficient handling of multimodal feature heterogeneity, an inability to effectively uncover the intrinsic correlations within multimodal data, and an inability to meet the refined management needs of precision agriculture. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for processing agricultural multimodal data. It addresses the shortcomings of existing technologies that fail to fully leverage the value of multimodal data, resulting in insufficient handling of multimodal feature heterogeneity, inability to effectively uncover the intrinsic correlations within multimodal data, and inability to meet the refined management needs of precision agriculture. The technical solution of this invention maximizes the value of multimodal data through preprocessing, solves the problem of insufficient handling of feature heterogeneity in existing technologies, increases data accuracy by incorporating crop structure, physiological, and soil physicochemical characteristics, effectively uncovers the intrinsic correlations within multimodal data, and solves the problem of incomplete features in traditional methods, thereby meeting the refined management needs of precision agriculture.

[0005] This invention provides a method for processing multimodal agricultural data, applied to a local node, comprising the following steps.

[0006] Acquire synthetic aperture radar imagery, optical imagery, geographic information data, and auxiliary geographic data corresponding to the target crop; Based on auxiliary geographic data, the aperture radar image, the optical image, and the geographic information data are preprocessed to obtain the target aperture radar image, the target optical image, and the target geographic information data. Based on the target aperture radar image and the target optical image, determine the crop structural features corresponding to the target crop; Based on the target optical image, determine the crop physiological characteristics corresponding to the target crop; The soil physicochemical characteristics corresponding to the target crop are determined based on the target aperture radar image and the target geographic information data. Based on the crop structural characteristics, crop physiological characteristics, and soil physicochemical characteristics, the multimodal fusion feature data corresponding to the target crop is determined.

[0007] According to the present invention, an agricultural multimodal data processing method includes data preprocessing including denoising, correction, alignment, coordinate normalization, and numerical normalization. The data preprocessing based on auxiliary geographic data of the aperture radar image, the optical image, and the geographic information data to obtain target aperture radar image, target optical image, and target geographic information data includes: The aperture radar image is denoised to obtain the denoised aperture radar image corresponding to the aperture radar image. The actual reflectance is retrieved based on the optical image, and the optical image is then corrected based on the actual reflectance to obtain the corrected optical image corresponding to the optical image. The denoised aperture radar image and the corrected optical image are aligned based on the auxiliary geographic data to obtain the target aperture radar image and the target optical image. The geographic information data is corrected to obtain corrected geographic information data; The corrected geographic information data is subjected to coordinate normalization and numerical normalization to obtain the target geographic information data.

[0008] According to the present invention, an agricultural multimodal data processing method is provided, wherein the crop structural features include the entropy, anisotropy, and normalized vegetation index corresponding to the target crop; The step of determining the crop structure features corresponding to the target crop based on the target aperture radar image and the target optical image includes: Obtain the polarization scattering matrix corresponding to the target aperture radar image; The coherence matrix and covariance matrix are determined based on the polarization scattering matrix; Eigenvalue decomposition is performed on the coherence matrix and the covariance matrix to obtain the target eigenvalues; The entropy and anisotropy are determined based on the target feature values; The normalized vegetation index is determined based on the actual reflectance of the target optical image in the near-infrared and infrared bands, respectively.

[0009] According to the present invention, an agricultural multimodal data processing method is provided, wherein the crop physiological characteristics include the red edge slope and normalized water index corresponding to the target crop; The step of determining the crop physiological characteristics corresponding to the target crop based on the target optical image includes: The red edge slope is determined based on the actual reflectance of the target optical image in the hyperspectral band. The normalized moisture index is determined based on the actual reflectance of the target optical image in the short infrared band.

[0010] According to the agricultural multimodal data processing method provided by the present invention, the soil physicochemical characteristics include soil moisture characteristics, normalized organic matter content and normalized pH corresponding to the target crop; The determination of the soil physicochemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data includes: The soil moisture characteristics are determined based on the target aperture radar imagery; The normalized organic matter content and the normalized pH are determined based on the target geographic information data.

[0011] According to the present invention, an agricultural multimodal data processing method is provided, wherein determining the multimodal fusion feature data corresponding to the target crop based on the crop structural features, the crop physiological features, and the soil physicochemical features includes: The first cross-modal fusion feature is determined based on the crop structural features and the crop physiological features; The second cross-modal fusion feature is determined based on the normalized water index in the soil physicochemical characteristics and crop physiological characteristics. Time-enhanced features were determined based on the normalized vegetation index in the crop structural characteristics. Spatial enhancement features are determined based on soil moisture characteristics among the aforementioned soil physicochemical characteristics; The third cross-modal fusion feature is determined based on the temporal enhancement feature and the spatial enhancement feature; The multimodal fusion feature data is determined based on the first cross-modal fusion feature, the second cross-modal fusion feature, and the third cross-modal fusion feature.

[0012] According to the present invention, an agricultural multimodal data processing method further includes: The multimodal fusion feature data is input into the crop classification model to obtain the crop type, growth status, and soil fertility level output by the crop classification model; the crop classification model is trained by the server based on the sample multimodal feature data and distributed to the local node.

[0013] The present invention also provides an agricultural multimodal data processing device, applied to a local node, comprising the following modules: The acquisition module is used to acquire synthetic aperture radar images, optical images, geographic information data, and auxiliary geographic data corresponding to the target crop. The preprocessing module is used to perform data preprocessing on the aperture radar image, the optical image, and the geographic information data based on auxiliary geographic data to obtain the target aperture radar image, the target optical image, and the target geographic information data. A structural feature module is used to determine the crop structural features corresponding to the target crop based on the target aperture radar image and the target optical image; A physiological feature module is used to determine the crop physiological features corresponding to the target crop based on the target optical image; A soil feature module is used to determine the soil physicochemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data. The fusion module is used to determine the multimodal fusion feature data corresponding to the target crop based on the crop structural features, the crop physiological features, and the soil physicochemical features.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the agricultural multimodal data processing method described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the agricultural multimodal data processing method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the agricultural multimodal data processing method as described above.

[0017] This invention provides an agricultural multimodal data processing method, apparatus, electronic device, and storage medium. It acquires synthetic aperture radar (SAR) images, optical images, geographic information data, and auxiliary geographic data corresponding to a target crop. Based on the auxiliary geographic data, it preprocesses the SAR images, optical images, and geographic information data to obtain target SAR images, target optical images, and target geographic information data. Based on the target SAR images and target optical images, it determines the crop structural features corresponding to the target crop. Based on the target optical images, it determines the crop physiological features corresponding to the target crop. Based on the target SAR images and target geographic information data, it determines the soil physicochemical features corresponding to the target crop. Based on the crop structural features, crop physiological features, and soil physicochemical features, it determines the multimodal fusion feature data corresponding to the target crop. This invention's technical solution maximizes the value of multimodal data through preprocessing, solving the problem of insufficient feature heterogeneity processing in existing technologies. By increasing data accuracy through crop structure, physiological, and soil physicochemical features, it effectively uncovers the inherent correlations in multimodal data, solving the problem of incomplete features in traditional methods, thereby meeting the refined management needs of precision agriculture. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the agricultural multimodal data processing method provided by the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of the agricultural multimodal data processing device provided by the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] To address the aforementioned problems in the prior art, this invention provides an agricultural multimodal data processing method applied to a local node. Figure 1 This is a flowchart illustrating the agricultural multimodal data processing method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps 110 to 160.

[0024] Step 110: Acquire synthetic aperture radar imagery, optical imagery, geographic information data, and auxiliary geographic data corresponding to the target crop.

[0025] Specifically, it can acquire Synthetic Aperture Radar Imaging (SAR) images, optical images, geographic information data, and auxiliary geographic data of the target crop.

[0026] SAR imagery can be acquired via satellite remote sensing, such as using Sentinel-1 (C-band) or ALOS-2 (L-band) satellites, through active microwave imaging technology. The acquisition frequency of SAR imagery is based on the crop growth cycle (e.g., wheat every 3-5 days), covering key growth stages (jointing, grain filling, etc.). The SAR imagery format is Geographic Tagged Image File Format (GeoTIFF), containing geographic coordinate information. SAR imagery parameters include resolution of 5 to 30 meters, polarization (vertical transmission / vertical reception / vertical transmission / horizontal reception), and backscattering coefficient.

[0027] Optical imagery can be acquired via satellite remote sensing, such as using Landsat-8 / 9 (multispectral), Sentinel-2 (multispectral), and HJ-1A (hyperspectral) satellites to passively receive solar radiation reflected from the Earth's surface. Acquisition conditions include clear skies without clouds (to avoid spectral distortion caused by cloud cover). The image format is GeoTIFF. Multiple bands can be selected during acquisition, for example, 4 to 12 bands (including infrared, near-infrared, and short-infrared bands), resulting in an image resolution of 10 to 30 meters. Hyperspectral images can be acquired in the 100+ band (e.g., 400-1000 nanometers), with a resolution of 30 to 50 meters.

[0028] Geographic information data can include soil sampling and meteorological data. Soil sampling can, for example, use a grid-based method (e.g., a 500m × 500 grid) to collect topsoil samples from 0 to 20 cm depth, and then measure organic matter, pH, and other parameters in the laboratory. Meteorological data can be collected from regional weather stations or from drones equipped with sensors, including temperature, precipitation, and humidity. Soil data can be in comma-separated values ​​(CSV) format (containing latitude and longitude of sampling points and attribute values) or vector format (shp files, indicating the spatial distribution of sampling points); meteorological data can be in time-series CSV (containing timestamps and temperature / precipitation values) or raster data (spatial distribution after meteorological interpolation).

[0029] Supporting geographic data can include basic geographic data and accuracy requirements. Basic geographic data: Administrative division and land parcel boundary data obtained from the National Geographic Information Public Service Platform. Accuracy requirements: The coordinate system must be unified as WGS84 (a globally accepted benchmark). Supporting geographic data can be in the format of shapefile vector files (land parcel boundaries, administrative divisions) and digital elevation models (DEMs), including GeoTIFF, reflecting terrain slope.

[0030] Step 120: Perform data preprocessing on the aperture radar image, the optical image, and the geographic information data based on auxiliary geographic data to obtain the target aperture radar image, the target optical image, and the target geographic information data.

[0031] Specifically, aperture radar images, optical images, and geographic information data can be preprocessed based on auxiliary geographic data to obtain target aperture radar images corresponding to aperture radar images, target optical images corresponding to optical images, and target geographic information data corresponding to geographic information data.

[0032] In one embodiment, the data preprocessing includes denoising, correction, alignment, coordinate normalization, and numerical normalization. The data preprocessing based on auxiliary geographic data of the aperture radar image, the optical image, and the geographic information data to obtain target aperture radar image, target optical image, and target geographic information data includes: The aperture radar image is denoised to obtain the denoised aperture radar image corresponding to the aperture radar image. The actual reflectance is retrieved based on the optical image, and the optical image is then corrected based on the actual reflectance to obtain the corrected optical image corresponding to the optical image. The denoised aperture radar image and the corrected optical image are aligned based on the auxiliary geographic data to obtain the target aperture radar image and the target optical image. The geographic information data is corrected to obtain corrected geographic information data; The corrected geographic information data is subjected to coordinate normalization and numerical normalization to obtain the target geographic information data.

[0033] Specifically, denoising can be performed on aperture radar images to obtain denoised aperture radar images. In SAR images, multiplicative noise, caused by the random characteristics of microwave scattering and manifested as image speckles, is addressed using Least Error Squares (LEE) adaptive filtering based on local statistical characteristics to extract interference structure features. This preserves edge details (such as crop row boundaries) while smoothing the speckle noise. (Denoising Aperture Radar Image) It can be expressed by the following formula: in, This represents the average value of a 3x3 serial port. ,express Window variance Indicates the noise variance. Estimation through crop-free areas This represents aperture radar imagery.

[0034] Furthermore, atmospheric scattering / absorption (such as blue light attenuation caused by aerosols) and topographic shading lead to reflectivity distortion, making it unsuitable for direct use in physiological characteristic calculations. Actual reflectivity can be retrieved from optical images; for example, atmospheric correction can be performed using the FLAASH (Fast Line-of-sight Atmospheric Analysis of Spectral Hypercubes, a physical model-based atmospheric correction algorithm) algorithm to retrieve actual reflectivity. It can be expressed by the following formula: in, This indicates the image radiance corresponding to the optical image. This indicates the solar irradiance at the top of the atmosphere corresponding to the optical image. This indicates the solar zenith angle corresponding to the optical image. This represents the atmospheric transmittance corresponding to the optical image.

[0035] Then, the optical image can be corrected based on the actual reflectivity to obtain the corrected optical image.

[0036] After obtaining the denoised aperture radar image and the corrected optical image, the denoised aperture radar image and the corrected optical image can be aligned based on auxiliary geographic data to obtain the target aperture radar image and the target optical image.

[0037] For example, to address the issue of mismatched features of the same land parcel caused by spatial resolution or coordinate deviation between SAR and optical images, a spatial registration method using Scale-invariant Feature Transform (SIFT) feature matching is employed. SIFT feature points (such as road intersections and land parcel corners) are extracted from the optical image, and corresponding points are found in the SAR image. The SAR image is then mapped to the optical image coordinate system through affine transformation, achieving pixel-level alignment (error ≤ 1 pixel).

[0038] In addition, geographic information data can be corrected, for example, by C-correction, which is suitable for mountainous data, to obtain corrected geographic information data. In corrected geographic information data, the coordinate systems (e.g., North X City 54, West X City 80) and numerical ranges (e.g., soil organic matter 1% to 5%, temperature 10 to 35 degrees Celsius) of different data sources are inconsistent, making direct fusion impossible. Therefore, coordinate normalization and numerical normalization are performed on the corrected geographic information data to obtain the target geographic information data. For example, coordinate normalization can be performed by using a projection transformation tool (e.g., ArcGIS) to convert all data to WGS84 latitude and longitude coordinates. Numerical normalization uses min-max normalization for soil properties and meteorological data. , This represents the result of numerical normalization. Indicates the current data value. This represents the maximum value in the set of data. This represents the minimum value in the set of data, mapped to... scope.

[0039] Agricultural data is characterized by "multi-source heterogeneity," but the raw data contains noise and spatial misalignment, directly affecting the accuracy of feature extraction. In the above embodiment, denoising, correction, and standardization are used to spatially align the multimodal data, ensure format compatibility, and eliminate noise, providing high-quality input for subsequent feature extraction.

[0040] Step 130: Determine the crop structure features corresponding to the target crop based on the target aperture radar image and the target optical image.

[0041] Specifically, because SAR imagery is sensitive to crop geometry (microwave scattering intensity is positively correlated with plant height and density), and the red band reflectivity of optical imagery is negatively correlated with leaf density, the crop structural characteristics corresponding to the target crop can be determined based on the target SAR imagery and the target optical imagery. These crop structural characteristics reflect the physical attributes of crop plant type and density.

[0042] In one embodiment, the crop structural features include the entropy, anisotropy, and normalized vegetation index corresponding to the target crop; The step of determining the crop structure features corresponding to the target crop based on the target aperture radar image and the target optical image includes: Obtain the polarization scattering matrix corresponding to the target aperture radar image; The coherence matrix and covariance matrix are determined based on the polarization scattering matrix; Eigenvalue decomposition is performed on the coherence matrix and the covariance matrix to obtain the target eigenvalues; The entropy and anisotropy are determined based on the target feature values; The normalized vegetation index is determined based on the actual reflectance of the target optical image in the near-infrared and infrared bands, respectively.

[0043] Specifically, the crop structural features include the entropy corresponding to the target crop. Anisotropy and Normalized Difference Vegetation Index .

[0044] First, the polarization scattering matrix corresponding to the target aperture radar image can be obtained. ,in, , , , These represent the intensity of the scattered echo signal under horizontal-horizontal, horizontal-vertical, vertical-horizontal, and vertical-vertical polarization modes, respectively.

[0045] Furthermore, the coherence matrix and covariance matrix can be calculated based on the polarization scattering matrix. Then, eigenvalue decomposition can be performed on the coherence matrix and covariance matrix to obtain the target eigenvalues. , , ,in, .

[0046] Furthermore, entropy and anisotropy can be determined based on target feature values. For example, entropy can be extracted using Cloude-Pottier polarization decomposition. and anisotropy ,entropy Reflects the complexity of the scattering mechanism (the higher the crop density, the more complex the scattering). Larger); anisotropy Reflects the consistency of scattering direction (the more uniform the plant shape, the better). The smaller). and It can be expressed by the following formula: in, Indicates the first The proportion of each eigenvalue in the sum of all eigenvalues, and satisfying the following conditions: .

[0047] From the perspective of crop structure characterization The value range is [0,1], and its core meaning is to reflect the complexity of the crop scattering mechanism. When the crop is in the early stage of growth, the plants are short and the density is low, the SAR signal is mainly scattered with the soil surface, and the scattering mechanism is relatively simple. At this time, the entropy is... The values ​​are relatively small; as crops grow, plant density gradually increases, and the number of stems and leaves increases, the SAR signal will undergo multiple scattering and volume scattering processes within the crop canopy, making the scattering mechanism more complex. The value increases accordingly. Therefore, through The size and trend of changes can effectively determine the crop growth stage and density, providing basic information on crop structure for subsequent agricultural production decisions.

[0048] The normalized vegetation index (NDI) can also be determined based on the actual reflectance of the target optical image in the near-infrared and infrared bands, respectively. It can be expressed by the following formula: in, This represents the actual reflectance of the target's optical image in the near-infrared band (wavelength range of 700-1100 nanometers). This indicates the actual reflectivity of the target optical image in the infrared band (wavelength range of 620 to 670 nanometers).

[0049] In the above embodiments, the entropy, anisotropy, and normalized vegetation index corresponding to the target crop are determined by using target aperture radar imagery and target optical imagery, which can accurately express the structural characteristics of the target crop.

[0050] Step 140: Determine the crop physiological characteristics corresponding to the target crop based on the target optical image.

[0051] Specifically, since the near-infrared reflectance of optical images is positively correlated with leaf water content, and the red-edge slope of hyperspectral images is positively correlated with chlorophyll content, the physiological characteristics of the target crop can be determined based on the target optical images. These physiological characteristics reflect indicators of photosynthesis and water stress.

[0052] In one embodiment, the crop physiological characteristics include the red edge slope and normalized water index corresponding to the target crop; The step of determining the crop physiological characteristics corresponding to the target crop based on the target optical image includes: The red edge slope is determined based on the actual reflectance of the target optical image in the hyperspectral band. The normalized moisture index is determined based on the actual reflectance of the target optical image in the short infrared band.

[0053] Specifically, the red edge slope can be determined based on the actual reflectance of the target optical image in the hyperspectral band. It can be expressed by the following formula: in, Indicates wavelength range, Represents the continuous reflectivity of the red-edge band. It can reflect the chlorophyll content.

[0054] Furthermore, the normalized water content index can be determined based on the actual reflectance of the target optical image in the short infrared band. It can reflect the water content of leaves, normalized water index It can be expressed by the following formula: in, This indicates the actual reflectivity of the target's optical image in the short infrared band.

[0055] In the above embodiments, the red edge slope and normalized moisture index are determined based on the target optical image, which can accurately reflect the chlorophyll content and leaf moisture content of the crop.

[0056] Step 150: Determine the soil physicochemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data.

[0057] Specifically, since the target geographic information data directly measures the organic matter content, and the backscattering coefficient of SAR imagery is positively correlated with soil moisture (moisture affects the dielectric constant), the soil physicochemical characteristics corresponding to the target crop can be determined based on the target aperture radar imagery and target geographic information data. These soil physicochemical characteristics reflect the properties of soil organic matter and pH value.

[0058] In one embodiment, the soil physicochemical characteristics include soil moisture characteristics, normalized organic matter content, and normalized pH corresponding to the target crop; The determination of the soil physicochemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data includes: The soil moisture characteristics are determined based on the target aperture radar imagery; The normalized organic matter content and the normalized pH are determined based on the target geographic information data.

[0059] Specifically, soil moisture characteristics can be determined based on target aperture radar imagery. These characteristics can be represented by the denoised backscattering coefficient, which can be expressed by the following formula: in, This represents the backscattering coefficient after denoising.

[0060] Normalized organic matter content and normalized pH can also be determined based on target geographic information data. The above process can be represented by the following formula: in, Indicates normalized organic matter content. Indicates organic matter content. This represents the minimum organic matter content. This indicates the maximum organic matter content. Indicates normalized pH. Indicates pH level. Indicates the minimum pH value. This indicates the maximum acidity or alkalinity.

[0061] In the above embodiments, the soil moisture characteristics, normalized organic matter content and normalized pH of the target crop are determined based on the target SAR image and target geographic information data, which can accurately express the soil physicochemical characteristics of the target crop.

[0062] Step 160: Determine the multimodal fusion feature data corresponding to the target crop based on the crop structural features, the crop physiological features, and the soil physicochemical features.

[0063] Specifically, crop structural characteristics, crop physiological characteristics, and soil physicochemical characteristics can be fused to obtain multimodal fused feature data corresponding to the target crop.

[0064] In one embodiment, determining the multimodal fusion feature data corresponding to the target crop based on the crop structural features, the crop physiological features, and the soil physicochemical features includes: The first cross-modal fusion feature is determined based on the crop structural features and the crop physiological features; The second cross-modal fusion feature is determined based on the normalized water index in the soil physicochemical characteristics and crop physiological characteristics. Time-enhanced features were determined based on the normalized vegetation index in the crop structural characteristics. Spatial enhancement features are determined based on soil moisture characteristics among the aforementioned soil physicochemical characteristics; The third cross-modal fusion feature is determined based on the temporal enhancement feature and the spatial enhancement feature; The multimodal fusion feature data is determined based on the first cross-modal fusion feature, the second cross-modal fusion feature, and the third cross-modal fusion feature.

[0065] Specifically, the first cross-modal fusion feature can be determined based on the red-edge slope in crop structural and physiological characteristics. First, the first sub-fusion feature can be determined; for example, the first sub-fusion feature could be the normalized water index. Weighted fusion with red edge slope, first sub-fusion feature It can be expressed by the following formula: in, This represents the first weight, for example, it could be 0.6. This indicates the second weight, which can be determined, for example, by field trial data, as 0.4.

[0066] Further first cross-modal fusion features It can be expressed by the following formula: in, express and The attention weight can be represented by the following formula: in, Indicates transpose. This represents the natural exponential function.

[0067] First cross-modal fusion feature A total of 64 dimensions, including entropy. Anisotropy , Core features such as red edge slope reflect the relationship between crop morphology and physiological state.

[0068] Furthermore, the second cross-modal fusion feature can be determined based on the normalized water index in soil physicochemical characteristics and crop physiological characteristics. For example, the local node will... , , Concatenate into feature vectors The second cross-modal fusion feature was obtained after Z-score standardization. Second cross-modal fusion features It can be expressed by the following formula: in, Represents the local characteristic mean. This represents the standard deviation. Furthermore, the server can aggregate the second cross-modal fusion features from multiple local nodes according to data volume weights.

[0069] Second cross-modal fusion feature A total of 128 dimensions, including , , and These factors reflect the impact of soil conditions on crop growth.

[0070] Because crop growth is seasonal (e.g., the temporal variation of wheat jointing-grain filling), and soil moisture fluctuates periodically due to the influence of precipitation, it is necessary to determine temporal and spatial enhancement characteristics, both of which reflect the crop growth cycle and the trend of soil moisture changes.

[0071] Temporal enhancement features can be determined based on the normalized difference in vegetation index (NDVI) in crop structural characteristics. It can be expressed by the following formula: in, express The total number of samples in the sequence, i.e., the number used to fit a linear trend. Total number of data collections This represents the normalized vegetation index. It can reflect the crop growth rate. When the value is greater than zero, crop growth... Crops decline when the value is less than zero.

[0072] Based on soil moisture characteristics in soil physicochemical properties, spatial enhancement features can be determined, which can improve the analysis of soil moisture characteristics. Calculate the local variance of a 3x3 window. This local variance is the spatial augmentation feature, which can be expressed by the following formula: in, This represents the horizontal coordinate index of neighboring pixels within the window. This represents the vertical coordinate index of the neighboring pixels within the window. Represents the horizontal coordinate of a spatial pixel. Represents the vertical coordinate of a spatial pixel. This represents the average soil moisture characteristic of all cells in a 3x3 window. It can reflect the spatial heterogeneity of humidity; the higher the value, the greater the difference between irrigated and non-irrigated areas.

[0073] Furthermore, the third cross-modal fusion feature can be determined based on temporal and spatial enhancement features. It can be expressed by the following formula: in, This represents the key bias parameter, used to adjust the baseline offset of the spatiotemporal fusion features and optimize the nonlinear mapping effect of feature fusion. and These represent the third and fourth weights, respectively. This represents the Sigmoid activation function, which controls the importance of features.

[0074] Third cross-modal fusion features A total of 64 dimensions, including and These reflect the growth cycle and spatial heterogeneity.

[0075] Finally, multimodal fusion feature data can be determined based on the first, second, and third cross-modal fusion features. This multimodal fusion feature data is formed through cross-modal feature concatenation and dimensionality compression, and can be represented by the following formula: in, This represents multimodal fusion feature data.

[0076] Alternatively, principal component analysis can be used to select a subset of principal components that retain 95% of the information from the 256-dimensional features of the multimodal fused feature data. Eliminate redundancy and reduce computational complexity.

[0077] In the above embodiments, the accuracy and reliability of data are further improved by enhancing temporal and spatial features. After local fusion of multimodal features according to different crop properties, the first cross-modal fusion feature, the second cross-modal fusion feature, and the third cross-modal fusion feature are obtained. Then, feature splicing is performed to obtain the final multimodal fusion feature data, thus completing the accurate processing of agricultural data.

[0078] This invention provides an agricultural multimodal data processing method that acquires synthetic aperture radar (SAR) images, optical images, geographic information data, and auxiliary geographic data corresponding to a target crop. Based on the auxiliary geographic data, the SAR images, optical images, and geographic information data are preprocessed to obtain target SAR images, target optical images, and target geographic information data. Based on the target SAR images and target optical images, the crop structural features corresponding to the target crop are determined. Based on the target optical images, the crop physiological features corresponding to the target crop are determined. Based on the target SAR images and target geographic information data, the soil physiochemical features corresponding to the target crop are determined. Based on the crop structural features, crop physiological features, and soil physiochemical features, the multimodal fusion feature data corresponding to the target crop is determined. This invention's technical solution maximizes the value of multimodal data through preprocessing, solving the problem of insufficient feature heterogeneity processing in existing technologies. By increasing data accuracy through crop structure, physiological, and soil physiochemical features, it effectively uncovers the inherent correlations in multimodal data, solving the problem of incomplete features in traditional methods, thereby meeting the refined management needs of precision agriculture.

[0079] In one embodiment, the method further includes: The multimodal fusion feature data is input into the crop classification model to obtain the crop type, growth status, and soil fertility level output by the crop classification model; the crop classification model is trained by the server based on the sample multimodal feature data and distributed to the local node.

[0080] Specifically, the server can train a crop classification model based on sample multimodal feature data and distribute the crop classification model to each local node. The local nodes can input multimodal fusion feature data into the crop classification model to obtain the crop type, growth status, and soil fertility level output by the crop classification model.

[0081] Classification objectives: Crop type (wheat, corn, rice, etc.), growth status (healthy / mildly stressed / severely stressed), soil fertility level (high / medium / low). The crop classification model can be an improved Transformer model, and its structure can be as follows: (1) Input layer: Receives 256-dimensional multimodal fusion feature data; (2) Encoding layer: 6-layer self-attention mechanism (Multi-Head Attention) to capture long-distance dependencies between features (such as the association between soil organic matter and crop NDVI). (3) Output layer: 3 parallel classification heads (crop type, growth status, soil fertility level), which output the probability distributions corresponding to crop type, growth status, and soil fertility level respectively.

[0082] For example, in crop classification model judgment, the model outputs three probabilities. (wheat), (corn), (Rice), select the category corresponding to the maximum value, and it must meet the probability threshold: like wheat corn rice and wheat The output crop type is wheat; like corn wheat rice and corn The output crop type is corn; like rice wheat corn and rice The output crop type is rice.

[0083] The probability threshold can be predetermined. For example, it can be optimized using a validation set (containing 5000 labeled samples covering all growth stages of three crops): when the threshold is 0.5, the overall accuracy is 88%, but the misclassification rate is high (corn is confused with rice); when the threshold is 0.6, the overall accuracy is 92%. The score reached 0.91 (wheat 0.93, corn 0.90, rice 0.92), and the misclassification rate dropped below 5%; threshold At that time, the accuracy improvement was not significant, but the recall rate decreased (some marginal samples were missed), so the probability threshold can be determined to be 0.6.

[0084] Growth status can be determined based on a combination of feature thresholds: healthy: and ; Mild stress: or ; Severe stress: or .

[0085] The combination of feature thresholds can be predetermined, for example, based on 300 field-measured samples: Healthy sample: Leaf relative water content > 75%, chlorophyll content > 45 SPAD, corresponding to (Vigorous growth) (well hydrated); Mild stress samples: leaf relative water content 60%-75%, chlorophyll content 35-45 SPAD, corresponding to or ; Severe stress samples: leaf relative water content <60%, chlorophyll content <35 SPAD, corresponding to (Growth stagnation) or (Severe water shortage).

[0086] In judging soil fertility levels, it can be based on Threshold division: High fertility: ; Medium fertility: ; Low fertility: .

[0087] Threshold classification can refer to the soil fertility grading standards in the "Technical Specifications for the Second National Soil Survey": Highly fertile soil: ,correspond (Due to the sample) , ; Medium-fertility soil: ,correspond hour (Adjusted to 0.3 to match actual fertility distribution). Low-fertility soil: ,correspond .

[0088] Optionally, corresponding charts can be determined based on crop type, growth status, and soil fertility level. Crop type distribution map: GeoTIFF raster format, with pixel values ​​indicating crop type, and a vector boundary file (shp format) attached. The accuracy report includes overall accuracy and Kappa coefficient. Growth status warning map: Graded rendering color map, green indicates healthy areas, yellow indicates mild stress (25%), and red indicates severe stress (10%), with the coordinates and area of ​​the stress center marked. Soil fertility level map: Graded rendering map, dark blue indicates high fertility (20%), light blue indicates medium fertility (50%), and gray indicates low fertility (30%), with sampling point verification marks attached.

[0089] Optionally, it can also produce decision-making suggestions.

[0090] Decision rules triggered by feature thresholds based on classification results: (1) Areas with severe stress and low fertility: Triggering conditions: (Severe water shortage) and (Low organic matter); Recommendation: Prioritize irrigation and increase the application of organic fertilizer.

[0091] (2) Mild stress + moderate fertility area: Triggering conditions: and ; Recommendation: Moderate irrigation + application of nitrogen fertilizer.

[0092] The above applications, based on measured data and classification thresholds set by industry standards, improve the accuracy of crop type identification, control the error in determining growth status and soil fertility level, and significantly enhance the interpretability of the results.

[0093] In the above embodiments, a federated learning architecture is innovatively introduced into the classification application of agricultural data. Through the fusion of local node features and the aggregation of server models, cross-regional knowledge sharing is achieved while protecting farmers' privacy and data security.

[0094] The agricultural multimodal data processing device provided by the present invention will be described below. The agricultural multimodal data processing device described below and the agricultural multimodal data processing method described above can be referred to in correspondence.

[0095] Figure 2 This is a schematic diagram of the structure of the agricultural multimodal data processing device provided by the present invention, as shown below. Figure 2 As shown, the agricultural multimodal data processing device 200 is applied to a local node and includes the following modules: The acquisition module 210 is used to acquire synthetic aperture radar images, optical images, geographic information data and auxiliary geographic data corresponding to the target crop; Preprocessing module 220 is used to perform data preprocessing on the aperture radar image, the optical image and the geographic information data based on auxiliary geographic data to obtain target aperture radar image, target optical image and target geographic information data; The structural feature module 230 is used to determine the crop structural features corresponding to the target crop based on the target aperture radar image and the target optical image; Physiological feature module 240 is used to determine the crop physiological features corresponding to the target crop based on the target optical image; Soil feature module 250 is used to determine the soil physicochemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data; The fusion module 260 is used to determine the multimodal fusion feature data corresponding to the target crop based on the crop structural features, the crop physiological features and the soil physicochemical features.

[0096] In one embodiment, the data preprocessing includes denoising, correction, alignment, coordinate normalization, and numerical normalization; the preprocessing module 220 is specifically used for: The aperture radar image is denoised to obtain the denoised aperture radar image corresponding to the aperture radar image. The actual reflectance is retrieved based on the optical image, and the optical image is then corrected based on the actual reflectance to obtain the corrected optical image corresponding to the optical image. The denoised aperture radar image and the corrected optical image are aligned based on the auxiliary geographic data to obtain the target aperture radar image and the target optical image. The geographic information data is corrected to obtain corrected geographic information data; The corrected geographic information data is subjected to coordinate normalization and numerical normalization to obtain the target geographic information data.

[0097] In one embodiment, the crop structural features include the entropy, anisotropy, and normalized vegetation index corresponding to the target crop; the structural feature module 230 is specifically used for: Obtain the polarization scattering matrix corresponding to the target aperture radar image; The coherence matrix and covariance matrix are determined based on the polarization scattering matrix; Eigenvalue decomposition is performed on the coherence matrix and the covariance matrix to obtain the target eigenvalues; The entropy and anisotropy are determined based on the target feature values; The normalized vegetation index is determined based on the actual reflectance of the target optical image in the near-infrared and infrared bands, respectively.

[0098] In one embodiment, the crop physiological characteristics include the red edge slope and normalized water index corresponding to the target crop; the physiological characteristic module 240 is specifically used for: The red edge slope is determined based on the actual reflectance of the target optical image in the hyperspectral band. The normalized moisture index is determined based on the actual reflectance of the target optical image in the short infrared band.

[0099] In one embodiment, the soil physicochemical characteristics include soil moisture characteristics, normalized organic matter content, and normalized pH corresponding to the target crop; the soil characteristic module 250 is specifically used for: The soil moisture characteristics are determined based on the target aperture radar imagery; The normalized organic matter content and the normalized pH are determined based on the target geographic information data.

[0100] In one embodiment, the fusion module 260 is specifically used for: The first cross-modal fusion feature is determined based on the crop structural features and the crop physiological features; The second cross-modal fusion feature is determined based on the normalized water index in the soil physicochemical characteristics and crop physiological characteristics. Time-enhanced features were determined based on the normalized vegetation index in the crop structural characteristics. Spatial enhancement features are determined based on soil moisture characteristics among the aforementioned soil physicochemical characteristics; The third cross-modal fusion feature is determined based on the temporal enhancement feature and the spatial enhancement feature; The multimodal fusion feature data is determined based on the first cross-modal fusion feature, the second cross-modal fusion feature, and the third cross-modal fusion feature.

[0101] In one embodiment, the agricultural multimodal data processing device further includes a classification module, which is specifically used for: The multimodal fusion feature data is input into the crop classification model to obtain the crop type, growth status, and soil fertility level output by the crop classification model; the crop classification model is trained by the server based on the sample multimodal feature data and distributed to the local node.

[0102] This invention provides an agricultural multimodal data processing device that acquires synthetic aperture radar (SAR) images, optical images, geographic information data, and auxiliary geographic data corresponding to a target crop. Based on the auxiliary geographic data, it preprocesses the SAR images, optical images, and geographic information data to obtain target SAR images, target optical images, and target geographic information data. Based on the target SAR images and target optical images, it determines the crop structural features corresponding to the target crop. Based on the target optical images, it determines the crop physiological features corresponding to the target crop. Based on the target SAR images and target geographic information data, it determines the soil physicochemical features corresponding to the target crop. Based on the crop structural features, crop physiological features, and soil physicochemical features, it determines the multimodal fusion feature data corresponding to the target crop. This invention's technical solution maximizes the value of multimodal data through preprocessing, solving the problem of insufficient feature heterogeneity processing in existing technologies. By increasing data accuracy through crop structure, physiological, and soil physicochemical features, it effectively uncovers the inherent correlations in multimodal data, solving the problem of incomplete features in traditional methods, thereby meeting the refined management needs of precision agriculture.

[0103] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute an agricultural multimodal data processing method, which includes: Acquire synthetic aperture radar imagery, optical imagery, geographic information data, and auxiliary geographic data corresponding to the target crop; Based on auxiliary geographic data, the aperture radar image, the optical image, and the geographic information data are preprocessed to obtain the target aperture radar image, the target optical image, and the target geographic information data. Based on the target aperture radar image and the target optical image, determine the crop structural features corresponding to the target crop; Based on the target optical image, determine the crop physiological characteristics corresponding to the target crop; The soil physicochemical characteristics corresponding to the target crop are determined based on the target aperture radar image and the target geographic information data. Based on the crop structural characteristics, crop physiological characteristics, and soil physicochemical characteristics, the multimodal fusion feature data corresponding to the target crop is determined.

[0104] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the agricultural multimodal data processing method provided by the above methods, the method comprising: Acquire synthetic aperture radar imagery, optical imagery, geographic information data, and auxiliary geographic data corresponding to the target crop; Based on auxiliary geographic data, the aperture radar image, the optical image, and the geographic information data are preprocessed to obtain the target aperture radar image, the target optical image, and the target geographic information data. Based on the target aperture radar image and the target optical image, determine the crop structural features corresponding to the target crop; Based on the target optical image, determine the crop physiological characteristics corresponding to the target crop; The soil physicochemical characteristics corresponding to the target crop are determined based on the target aperture radar image and the target geographic information data. Based on the crop structural characteristics, crop physiological characteristics, and soil physicochemical characteristics, the multimodal fusion feature data corresponding to the target crop is determined.

[0106] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the agricultural multimodal data processing method provided by the methods described above, the method comprising: Acquire synthetic aperture radar imagery, optical imagery, geographic information data, and auxiliary geographic data corresponding to the target crop; Based on auxiliary geographic data, the aperture radar image, the optical image, and the geographic information data are preprocessed to obtain the target aperture radar image, the target optical image, and the target geographic information data. Based on the target aperture radar image and the target optical image, determine the crop structural features corresponding to the target crop; Based on the target optical image, determine the crop physiological characteristics corresponding to the target crop; The soil physicochemical characteristics corresponding to the target crop are determined based on the target aperture radar image and the target geographic information data. Based on the crop structural characteristics, crop physiological characteristics, and soil physicochemical characteristics, the multimodal fusion feature data corresponding to the target crop is determined.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing multimodal agricultural data, characterized in that, Applied to local nodes, including: Acquire synthetic aperture radar imagery, optical imagery, geographic information data, and auxiliary geographic data corresponding to the target crop; Based on auxiliary geographic data, the aperture radar image, the optical image, and the geographic information data are preprocessed to obtain the target aperture radar image, the target optical image, and the target geographic information data. Based on the target aperture radar image and the target optical image, determine the crop structural features corresponding to the target crop; Based on the target optical image, determine the crop physiological characteristics corresponding to the target crop; The soil physicochemical characteristics corresponding to the target crop are determined based on the target aperture radar image and the target geographic information data. Based on the crop structural characteristics, crop physiological characteristics, and soil physicochemical characteristics, the multimodal fusion feature data corresponding to the target crop is determined.

2. The agricultural multimodal data processing method according to claim 1, characterized in that, The data preprocessing includes denoising, correction, alignment, coordinate normalization, and numerical normalization. The data preprocessing based on auxiliary geographic data of the aperture radar image, the optical image, and the geographic information data to obtain target aperture radar image, target optical image, and target geographic information data includes: The aperture radar image is denoised to obtain the denoised aperture radar image corresponding to the aperture radar image. The actual reflectance is retrieved based on the optical image, and the optical image is then corrected based on the actual reflectance to obtain the corrected optical image corresponding to the optical image. The denoised aperture radar image and the corrected optical image are aligned based on the auxiliary geographic data to obtain the target aperture radar image and the target optical image. The geographic information data is corrected to obtain corrected geographic information data; The corrected geographic information data is subjected to coordinate normalization and numerical normalization to obtain the target geographic information data.

3. The agricultural multimodal data processing method according to claim 1, characterized in that, The crop structural features include the entropy, anisotropy, and normalized vegetation index corresponding to the target crop. The step of determining the crop structure features corresponding to the target crop based on the target aperture radar image and the target optical image includes: Obtain the polarization scattering matrix corresponding to the target aperture radar image; The coherence matrix and covariance matrix are determined based on the polarization scattering matrix; Eigenvalue decomposition is performed on the coherence matrix and the covariance matrix to obtain the target eigenvalues; The entropy and anisotropy are determined based on the target feature values; The normalized vegetation index is determined based on the actual reflectance of the target optical image in the near-infrared and infrared bands, respectively.

4. The agricultural multimodal data processing method according to claim 1, characterized in that, The crop physiological characteristics include the red edge slope and normalized water index corresponding to the target crop; The step of determining the crop physiological characteristics corresponding to the target crop based on the target optical image includes: The red edge slope is determined based on the actual reflectance of the target optical image in the hyperspectral band. The normalized moisture index is determined based on the actual reflectance of the target optical image in the short infrared band.

5. The agricultural multimodal data processing method according to claim 1, characterized in that, The soil physicochemical characteristics include soil moisture characteristics, normalized organic matter content, and normalized pH corresponding to the target crop; The determination of the soil physicochemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data includes: The soil moisture characteristics are determined based on the target aperture radar imagery; The normalized organic matter content and the normalized pH are determined based on the target geographic information data.

6. The agricultural multimodal data processing method according to any one of claims 1 to 5, characterized in that, The determination of multimodal fusion feature data corresponding to the target crop based on the crop structural features, the crop physiological features, and the soil physicochemical features includes: The first cross-modal fusion feature is determined based on the crop structural features and the crop physiological features; The second cross-modal fusion feature is determined based on the normalized water index in the soil physicochemical characteristics and crop physiological characteristics. Time-enhanced features were determined based on the normalized vegetation index in the crop structural characteristics. Spatial enhancement features are determined based on soil moisture characteristics among the aforementioned soil physicochemical characteristics; The third cross-modal fusion feature is determined based on the temporal enhancement feature and the spatial enhancement feature; The multimodal fusion feature data is determined based on the first cross-modal fusion feature, the second cross-modal fusion feature, and the third cross-modal fusion feature.

7. The agricultural multimodal data processing method according to claim 6, characterized in that, The method further includes: The multimodal fusion feature data is input into the crop classification model to obtain the crop type, growth status, and soil fertility level output by the crop classification model; the crop classification model is trained by the server based on the sample multimodal feature data and distributed to the local node.

8. An agricultural multimodal data processing device, characterized in that, Applied to local nodes, including: The acquisition module is used to acquire synthetic aperture radar images, optical images, geographic information data, and auxiliary geographic data corresponding to the target crop. The preprocessing module is used to perform data preprocessing on the aperture radar image, the optical image, and the geographic information data based on auxiliary geographic data to obtain the target aperture radar image, the target optical image, and the target geographic information data. A structural feature module is used to determine the crop structural features corresponding to the target crop based on the target aperture radar image and the target optical image; A physiological feature module is used to determine the crop physiological features corresponding to the target crop based on the target optical image; A soil feature module is used to determine the soil physicochemical characteristics corresponding to the target crop based on the target aperture radar image and the target geographic information data. The fusion module is used to determine the multimodal fusion feature data corresponding to the target crop based on the crop structural features, the crop physiological features, and the soil physicochemical features.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the agricultural multimodal data processing method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the agricultural multimodal data processing method as described in any one of claims 1 to 7.

Images