AI-supported system for site-specific recommendations on cultivation and fertilization based on multimodal data integration

Abstract

A system (100) for AI-supported, site-specific recommendations on plants and fertilizers based on multimodal data integration, wherein the system (100) comprises: a variety of IoT-based sensors (1) configured to measure one or more soil and environmental parameters in real time, including NPK values, pH values, soil temperature and soil moisture; a data acquisition unit (2) configured to collect, preprocess and transmit sensor data in real time from the multitude of sensors (1); a feature extraction module (3) configured to extract features from multimodal data received by the data acquisition unit (2), wherein the multimodal data includes at least soil data, plant or field data and environmental or weather data; a feature selection module (4) configured to select an optimal subset of features from the extracted features; a processing unit (5) comprising: a Graph Convolutional FPMax model (GCFPMax) (51) configured to generate real-time recommendations for crop cultivation, and a Recurrent FPMax model (RFPMax) (52) configured to generate real-time fertilizer recommendations; and a communication interface (53) configured to transmit crop and fertilizer recommendations to user devices or display units; wherein the system (100) is configured to continuously update crop and fertilizer recommendations in real time based on incoming multimodal data and changing field conditions.

Description

INVENTION AREA

[0001] The present invention relates to the field of precision agriculture and artificial intelligence (Cl). In particular, the invention relates to an AI-supported system for site-specific recommendations on plants and fertilizers using multimodal data integration from soil, plant, environmental and weather data acquired via IoT-based sensors and external data sources. BACKGROUND OF THE INVENTION

[0002] The subject matter discussed in the "Background" section should not be considered prior art solely because it is mentioned therein. Likewise, a problem mentioned in or related to the subject matter of the "Background" section should not be considered prior art. The subject matter in the "Background" section merely presents various approaches, which could themselves be inventions.

[0003] Conventional agricultural practices often rely on uniform recommendations for cultivation and fertilization that apply to an entire field or even several fields within a region. Such generalized recommendations do not account for spatial variations in nutrient levels, pH, moisture, temperature, and soil microclimatic conditions. As a result, farmers frequently face challenges such as: • Over- or under-fertilization; • Reduction in crop yield and quality; • Increased input costs and environmental pollution; and • Inability to adapt recommendations in real time to changing field conditions.

[0004] Existing decision support systems for agriculture typically utilize only limited data sources or rely on static soil analysis reports. Many systems neither utilize multimodal data (soil sensors, crop images, weather data, historical harvest and fertilizer records, etc.) nor employ advanced AI architectures capable of simultaneously modeling spatial relationships between field locations and temporal developments in crop and soil conditions.

[0005] Furthermore, most available tools provide recommendations at the field level rather than at the site-specific level (e.g., grid- or zone-based recommendations within a field). Therefore, there is a need for a unified AI-enabled system that... • Integrates multimodal real-time and historical data; • extracted and selected discriminatory features from the data; • Special AI models are used to provide site-specific recommendations for plants and fertilizers; and • the recommendations and warnings are presented to farmers in an intuitive way via mobile or web applications.

[0006] The present invention overcomes these limitations.

[0007] The use of any examples or illustrative phrases (e.g., "as") relating to specific embodiments serves only to better illustrate the invention and does not constitute a limitation of the otherwise claimed scope of the invention. No wording in the description shall be construed as referring to an unclaimed element that is essential for carrying out the invention.

[0008] The information disclosed above in this "Background" section is provided solely for a better understanding of the background of the invention and may therefore contain information that is not part of the prior art already known to a person skilled in the art in this country. SUMMARY

[0009] Before describing the systems and methods presented here, it should be noted that this application is not limited to the specific systems and methods described, as there may be several possible embodiments not expressly presented in this disclosure. It should also be noted that the terminology used in the description serves only to describe the specific versions or embodiments and is not intended to limit the scope of this application.

[0010] In one aspect, the invention provides a system (100) for AI-supported, site-specific recommendations for plants and fertilizers based on multimodal data integration. The system (100) comprises: • A variety of IoT-based sensors (1) configured to measure soil and environmental parameters such as NPK values, pH values, soil moisture and soil temperature in real time at different spatial locations within a field. • A data acquisition unit (2) configured to collect, preprocess and transmit sensor data in real time from the multitude of sensors (1) and optionally integrate external data sources such as weather station data, satellite images or historical operational records. • A feature extraction module (3) configured to extract features from multimodal data received by the data acquisition unit (2), where the multimodal data may include soil data, crop or field data, environmental data, weather data and historical data. • A feature selection module (4) configured to select an optimal subset of features from the extracted features using advanced methods such as an Ant Lion Fuzzy Principal Component Analyzer (ALFPCA). • A processing unit (5) comprising the following: • a Graph Convolutional FPMax Model (GCFPMax) (51) configured to generate site-specific recommendations for crop cultivation in real time; and • a Recurrent FPMax Model (RFPMax) (52) configured to generate site-specific fertilizer recommendations in real time. • A communication interface (53) configured to transmit recommendations on plants and fertilizers, notifications and warnings to user devices or display units such as mobile phones, tablets, web dashboards or local terminals.

[0011] The system (100) is configured to continuously update recommendations in real time based on incoming multimodal data and changing field conditions. In some embodiments, data processing and AI models are provided on a cloud platform, enabling scalability and access from multiple regions. BRIEF DESCRIPTION OF THE DRAWING

[0012] To clarify various aspects of some embodiments of the present invention, a more detailed description of the invention is given with reference to specific embodiments thereof, which are illustrated in the accompanying drawings. It is understood that these drawings represent only illustrative embodiments of the invention and are therefore not to be considered as limiting its scope. The invention is described and explained with additional specificity and detail using the accompanying drawings.

[0013] To make the advantages of the present invention easily understandable, a detailed description of the invention is given below in conjunction with the accompanying drawings, which, however, are not to be understood as limiting the scope of the invention to the accompanying drawings in which: Fig. Figure 1 shows a block diagram representation of a AI-capable system (100) for site-specific recommendations on plants and fertilizers based on multimodal data integration. DETAILED DESCRIPTION

[0014] The present invention relates to an AI-enabled system (100) for site-specific recommendations on plants and fertilizers based on multimodal data integration.

[0015] Fig. shows a detailed block diagram representation of the AI-enabled system (100) for site-specific recommendations on plants and fertilizers based on multimodal data integration.

[0016] Although the present disclosure has been described with the aim of providing an AI-capable system for site-specific recommendations on plants and fertilizers based on multimodal data integration, it should be noted that this is merely to illustrate the invention by way of example and to highlight other purposes or functions for which the described structures or configurations could be used and which fall within the scope of the present disclosure.

[0017] The present invention will now be described in detail with reference to exemplary embodiments. These embodiments serve to improve understanding of the invention and are not intended to limit the scope of the claims. System overview

[0018] With reference to Fig.1 The invention provides a system (100) for AI-supported, site-specific recommendations for plants and fertilizers based on multimodal data integration. The system (100) is used in one or more agricultural fields and comprises: • a variety of IoT-based sensors (1); • One data acquisition unit (2); • a feature extraction module (3); • a feature selection module (4); • a processing unit (5) with GCFPMax (51) and RFPMax (52); and • A communication interface (53) and a notification module.

[0019] The system (100) can be implemented as a distributed architecture, with sensors deployed in the field, edge devices at the farm level, and a backend server or cloud platform on which the AI ​​models and data analyses are run. IoT-based sensors (1)

[0020] The multitude of IoT-based sensors (1) are deployed at various spatial positions within the agricultural field. In one embodiment, the field is divided into a grid or a series of zones, and each grid point or zone is equipped with one or more sensors (1). The sensors (1) may include, among other things: • Soil nutrient sensors for measuring NPK content; • Soil pH sensors; • Soil moisture sensors; • Soil temperature sensors; • Environmental sensors for ambient temperature, humidity, light intensity, etc.

[0021] In some embodiments, the sensors (1) also include leaf or crown sensors, optical sensors or multispectral cameras for detecting indicators of crop health, such as NDVI or color-based stress indices.

[0022] Each sensor node can include a low-power microcontroller and a wireless communication module that supports protocols such as Bluetooth, Wi-Fi, LoRaWAN, or other IoT-based networks. Power can be supplied via batteries, solar panels, or other suitable sources.

[0023] The sensors (1) measure the relevant parameters regularly or continuously and transmit the data together with location and timestamp information to the data acquisition unit (2). Data acquisition unit (2)

[0024] The data acquisition unit (2) serves as a gateway or hub for data acquisition. It can be implemented as follows: • a local Edge device installed on the farm; • an internet-connected gateway device; or • a cloud-based service that receives data from multiple gateways.

[0025] The data acquisition unit (2) is configured to: • Receiving data streams from multiple sensors (1); • Perform preprocessing, such as noise filtering, outlier removal, sensor calibration, handling missing values, timestamp alignment, and unit normalization; • Integration of external data sources, such as: • Real-time or forecast weather data (temperature, humidity, precipitation, wind); • Satellite or drone images; • Historical yield records and fertilizer application logs; and • Transfer the processed and integrated data to the feature extraction module (3) in real time or at defined intervals.

[0026] The data acquisition unit (2) can communicate with the backend via wired or wireless connections using standard network protocols and secure channels. Feature extraction module (3)

[0027] The feature extraction module (3) receives the multimodal data output by the data acquisition unit (2). The multimodal data may include the following: • Time series data from soil and environmental sensors (1); • Spatial data representing the location of each sensor or grid cell; • Harvest images or vegetation indices; • Historical and real-time weather data; and • Previous fertilization and irrigation records.

[0028] To obtain informative and diverse features, the feature extraction module (3) can apply one or more of the following techniques: • Frequency pattern analysis The feature extraction module (3) can apply a Fourier transform (e.g., discrete Fourier transform) to time series data such as soil moisture or temperature. This allows the system (100) to identify periodic patterns such as irrigation cycles, daily temperature cycles, or recurring stress patterns. Frequency domain features such as dominant frequencies, spectral energy, and bandwidth can be extracted. • Entropy pattern analysis The feature extraction module (3) can apply a discrete cosine transform (DCT) or related transformations to quantify entropy, variability, or irregularities in the sensor signals. Entropy-based features detect fluctuations due to irregular irrigation, irregular rainfall, or nutrient variations and are useful for distinguishing stable from unstable zones. • S-Transform-based time-frequency analysis The feature extraction module (3) can use the S-transformation or similar time-frequency methods to observe how the spectral content changes over time. This is crucial for capturing temporally localized phenomena such as sudden rainfall, rapid soil moisture loss, heat waves, or temporary nutrient changes. • Convolutional trait extraction For spatial data and images, the feature extraction module (3) can use convolutional operations, similar to those in convolutional neural networks (CNNs), to extract features relating to soil texture patterns, plant stand structure, leaf coloration, and disease signatures. These convolutional components can be applied to images or spatial maps of soil properties.

[0029] The output of the feature extraction module (3) is a high-dimensional feature set that represents temporal behavior, spatial relationships, and spectral properties of the multimodal data together. Feature selection module (4)

[0030] The feature selection module (4) receives the high-dimensional feature set from the feature extraction module (3) and selects an optimal subset of features to reduce the dimensionality and improve model performance.

[0031] In one embodiment, the feature selection module (4) uses an Ant Lion Fuzzy Principal Component Analyzer (ALFPCA) that combines fuzzy clustering, Ant Lion Optimization (ALO) and Principal Component Analysis (PCA): • Fuzzy clustering The feature selection module (4) divides the feature space into fuzzy clusters, where each feature can belong to multiple clusters with varying degrees of membership. This soft clustering offers flexibility in processing heterogeneous agricultural data and reduces sensitivity to noise. • Antlion Optimization (ALO) Within and between the clusters, the trait selection module (4) uses ALO, a population-based metaheuristic inspired by the hunting behavior of antlions, to prioritize traits that offer high discriminatory power with respect to crop yield, nutrient status and other target variables. • Principal Component Analysis (PCA) After selecting the most promising features using ALO, the feature selection module (4) applies PCA to further reduce redundancies and retain the principal components with maximum variance. The resulting reduced feature set is compact yet highly informative.

[0032] The selected features are forwarded to the processing unit (5) for model inference. Processing unit (5)

[0033] The processing unit (5) is the AI ​​core of the system (100) and can be deployed on a server, a cloud platform, or a high-performance edge device. It includes: • A graph convolutional FPMax model (GCFPMax) (51) for recommendations on crop cultivation; and • a recurrent FPMax model (RFPMax) (52) for fertilizer recommendations. Graph Convolutional FPMax model (GCFPMax) (51)

[0034] The GCFPMax (51) was developed to utilize spatial relationships between different locations in the field. In one embodiment: • Graph construction The field is represented as a graph, where the nodes correspond to the sensor locations or grid cells, and the edges encode spatial proximity or similarity (e.g., based on distance, soil type, or topographic features). An adjacency matrix defines which nodes are connected. • Graph Convolutional Network (GCN) The features selected by the feature selection module (4) are assigned to each node. A GCN propagates information along the edges so that the representation of each node is updated based on information from its neighbors. This captures local spatial dependencies and smooths out noise. • Frequent Pattern Mining (FPMax) The GCFPMax (51) uses FPMax, a Frequent Pattern Mining technique, to identify frequent combinations of traits and conditions at the node level (e.g., “certain nutrient levels with certain moisture regimes consistently lead to good yields”). • Creation of recommendations for cultivation By combining GCN-derived node embeddings with common patterns identified by FPMax, GCFPMax (51) generates site-specific recommendations for cultivation, such as recommended plant varieties, crop rotation strategies, or the suitability of certain plants for each field zone. Recurrent FPMax model (RFPMax) (52)

[0035] The RFPMax (52) focuses on temporal dynamics and fertilizer requirements. In one embodiment: • Recurrent neural network (RNN) Time series data (e.g., moisture, nutrient content, weather) for each zone are processed using a RNN (such as LSTM or GRU). The RNN captures sequential dependencies such as plant growth stages, seasonal variations, and residual effects of previous fertilizer applications. • Frequent Pattern Mining (FPMax) Historical records of fertilizer applications and the resulting soil and plant responses are analyzed using FPMax to identify frequently occurring patterns (e.g., typical nutrient consumption rates after a specific fertilizer dose under certain conditions). • Preparation of fertilizer recommendations The RFPMax (52) synthesizes the temporal representation of the RNN with the patterns discovered by FPMax to generate precise, context-specific fertilizer recommendations, including fertilizer type, quantity, timing and site-specific application rates. Communication interface (53) and notification module

[0036] The communication interface (53) connects the processing unit (5) with farmers and other stakeholders. It may include the following: • APIs for mobile and web applications; • Local display terminals installed in the office of the agricultural operation; • SMS or messaging integrations.

[0037] The recommendations generated by GCFPMax (51) and RFPMax (52) are transmitted via the communication interface (53) and displayed as follows: • Location-specific recommendation maps; • Tabular recommendations per zone; • Warnings and notifications.

[0038] A notification module (which can be implemented inside or next to the communication interface (53)) is configured to: • To warn users when significant changes are detected, such as rapid loss of soil moisture, sudden pH changes, or forecast heavy rainfall; • To notify users of a predicted nutrient deficiency or the risk of over-fertilization; • To provide timely recommendations on fertilizer use, irrigation planning, or plant management. Cloud-based implementation

[0039] In some embodiments, the system (100) is implemented on a cloud platform, wherein: • Sensor data from multiple farms can be securely uploaded; • the feature extraction module (3), the feature selection module (4) and the processing unit (5) are executed on scalable computing resources; • Farmers from different regions can access recommendations via mobile or web applications; • Models can be continuously retrained or updated using aggregated anonymized data to improve performance over time. Example of a use case

[0040] Imagine an agricultural field divided into 40 grid cells, each equipped with sensors (1) that measure NPK, pH, moisture, and temperature. The data acquisition unit (2) collects and processes the sensor data every 30 minutes, integrating weather forecasts and historical records of fertilizer application.

[0041] The feature extraction module (3) generates a variety of temporal and spatial features using frequency, entropy, S-transformation, and convolutional analysis. The feature selection module (4) applies ALFPCA to select a compact and informative subset of features.

[0042] The GCFPMax (51) generates a diagram in which nodes represent the grid cells and edges represent the neighboring cells. GCN layers disseminate information and, together with FPMax, generate site-specific recommendations regarding which plants are best suited, taking into account soil fertility and water availability in each grid cell.

[0043] The RFPMax (52) processes time-series data for each grid cell, recording the plant development stage and seasonal effects. It generates fertilization recommendations for each grid cell. and for the dosage for the coming week. The communication interface (53) sends these recommendations, along with warnings about zones where nutrient deficiency is imminent, to the farmer's mobile application. ADVANTAGES OF THE INVENTION

[0044] The present invention offers several advantages over existing systems: • Site-specific recommendations with high spatial resolution instead of uniform recommendations at the field level. • Multimodal data integration that combines soil, environmental, crop and weather data for more accurate decisions. • Use of advanced AI models (GCFPMax and RFPMax) that evaluate spatial and temporal patterns together with frequent pattern mining. • Real-time and continuous updating of recommendations based on current field conditions. • Reduction of fertilizer waste and environmental impact through optimized dosage and timing. • Improved crop yields, quality and resource efficiency for farmers. • Scalable cloud-based deployment, suitable for a large number of farms and regions.

[0045] The figure and the preceding description provide examples of embodiments. Those skilled in the art will recognize that one or more of the described elements can be combined to form a single functional element. Alternatively, certain elements can be divided into several functional elements. Elements from one embodiment can be added to another embodiment. For example, the sequence of the processes described here can be changed and is not limited to the manner described here. Furthermore, the actions of a block diagram need not be implemented in the sequence shown, nor does it necessarily have to be executed all actions. In addition, those actions that are not dependent on other actions can be executed in parallel with the other actions. The scope of embodiments is by no means limited by these specific examples.

[0046] Although the embodiments of the invention have been described in language relating to structural features and / or methods, it should be noted that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of embodiments of the invention.

Claims

[1] A system (100) for AI-supported, site-specific recommendations on plants and fertilizers based on multimodal data integration, wherein the system (100) comprises: a variety of IoT-based sensors (1) configured to measure one or more soil and environmental parameters in real time, including NPK values, pH values, soil temperature and soil moisture; a data acquisition unit (2) configured to collect, preprocess and transmit sensor data in real time from the multitude of sensors (1); a feature extraction module (3) configured to extract features from multimodal data received by the data acquisition unit (2), wherein the multimodal data includes at least soil data, plant or field data and environmental or weather data; a feature selection module (4) configured to select an optimal subset of features from the extracted features; a processing unit (5) comprising: a Graph Convolutional FPMax model (GCFPMax) (51) configured to generate real-time recommendations for crop cultivation, and a Recurrent FPMax model (RFPMax) (52) configured to generate real-time fertilizer recommendations; and a communication interface (53) configured to transmit crop and fertilizer recommendations to user devices or display units; wherein the system (100) is configured to continuously update crop and fertilizer recommendations in real time based on incoming multimodal data and changing field conditions. [2] System (100) according to claim 1, wherein the plurality of IoT-based sensors (1) is configured such that they: Measurement of NPK, pH, moisture, and soil temperature at multiple locations within a field to enable site-specific recommendations; and Transmission of real-time data using wireless communication protocols, including at least one of the following: Bluetooth, Wi-Fi or IoT-based networks. [3] System (100) according to claim 1, wherein the feature extraction module (3) is configured to apply, individually or in combination, the following: Frequency pattern analysis using a Fourier transform to identify periodic patterns in agricultural data; an entropy pattern analysis using a discrete cosine transform (DCT) to quantify the variability and irregularity in the data; an S-transform-based time-frequency analysis to capture temporally localized changes in environmental conditions; and Convolution operations to extract spatial features from image and sensor inputs, including patterns relating to soil texture, plant structure, or leaf health; thereby generating a diverse and representative set of features from the multimodal data. [4] System (100) according to one of the preceding claims, wherein the feature selection module (4) uses an Ant Lion Fuzzy Principal Component Analyzer (ALFPCA) to identify relevant features, wherein the ALFPCA is configured to: to partition the feature space into fuzzy clusters with membership degrees assigned via a membership matrix; to perform an antlion optimization (ALO) to prioritize traits with higher distinctiveness; and Applying principal component analysis (PCA) to the selected features to reduce dimensionality while maintaining maximum variance; thereby providing the processing unit (5) with an optimized subset of features. [5] System (100) according to one of the preceding claims, wherein the processing unit (5) is configured such that: The Graph Convolutional FPMax Model (GCFPMax) (51) integrates a Graph Convolutional Network with Frequent Pattern Mining (FPMax) to model spatial and relational dependencies between field locations and to generate site-specific recommendations for cultivation; and the Recurrent FPMax Model (RFPMax) (52) integrates a recurrent neural network with Frequent Pattern Mining (FPMax) to capture temporal variations in agricultural data and generate site-specific fertilizer recommendations based on plant growth stage, seasonal patterns and historical use. [6] System (100) according to any one of the preceding claims, wherein: the communication interface (53) is configured to provide users with recommendations for cultivation and fertilization as well as real-time alerts via mobile or web applications; a notification module is configured to inform users of significant changes in soil or environmental conditions, a predicted nutrient deficiency, or the risk of over-fertilization; and Real-time data processing is performed on a cloud platform to enable scalable deployment and access to location-specific recommendations across multiple fields and regions.

Images