A Smart Irrigation Decision-Making Method for Cherries Based on Multimodal and Spatiotemporal Prediction

Abstract

This invention relates to the field of intelligent irrigation technology, and particularly to an intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction. The method includes acquiring a growth status dataset containing image data and real-time information; preprocessing the dataset to extract multi-scale features; extracting growth status and morphological information through a visual model; using a prediction model to analyze temporal features via a temporal flow network and learn plot correlations via a spatial flow network, fusing features to generate regional irrigation amount and soil moisture prediction information; projecting visual, sensor, and spatiotemporal label data into a decision model based on a pre-trained large model, and combining this with knowledge base retrieval to output the optimal water and fertilizer ratio. This invention effectively solves the problems of traditional irrigation methods, such as difficulty in developing plans for different growth stages and states of cherries and the reliance on experience leading to uneven water and fertilizer supply.

Description

Technical Field

[0001] This invention relates to the field of intelligent irrigation technology, and in particular to an intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction. Background Technology

[0002] Currently, global and domestic resources are scarce, and traditional irrigation methods have low water and fertilizer utilization rates. Therefore, intelligent irrigation technology is widely used in cherry production to achieve efficient irrigation through precise control. Existing technologies for cherry irrigation often use integrated water and fertilizer equipment and intelligent irrigation valves to mix water and fertilizer in the correct proportions and distribute flow according to volume. However, in cherry cultivation management, it is difficult to identify different growth stages and conditions of cherries and provide targeted irrigation plans. Furthermore, precise water and fertilizer management based on real-time environmental conditions is necessary. Relying on experience to design irrigation plans cannot guarantee sufficient water and fertilizer for cherries in all planting areas, which is detrimental to cherry growth.

[0003] Therefore, it is necessary to propose a smart irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction to solve the above problems. Summary of the Invention

[0004] The purpose of this invention is to provide a smart irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction, in order to solve the problem that in cherry planting and management, it is difficult to identify different growth stages and states of cherries during irrigation, so as to provide targeted irrigation solutions. Furthermore, it is necessary to carry out precise water and fertilizer management based on real-time environmental conditions. Relying on experience to design irrigation solutions makes it difficult to ensure that cherries in each planting area can be provided with sufficient water and fertilizer, which is detrimental to cherry growth.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A smart irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction, the method comprising:

[0007] A cherry growth status dataset is obtained, the dataset is preprocessed to obtain visual information, and feature extraction is performed on the visual information to obtain multi-scale features. The dataset includes image data and real-time information.

[0008] By using a visual model to extract features from multi-scale features, we can obtain information on the growth status and morphology of cherries.

[0009] Based on growth status information and morphological information, the temporal characteristics of historical climate fluctuations and soil moisture changes are analyzed through the temporal flow network in the prediction model to obtain temporal flow characteristics. The spatial flow network in the model is used to learn the spatial correlation between different plots in the orchard to obtain spatial flow characteristics. The temporal flow characteristics and spatial flow characteristics are fused together and combined with the regression network to generate prediction information, wherein the prediction information includes regional irrigation amount and soil moisture prediction information.

[0010] Visual data, sensor data, and spatiotemporal labeled data are projected to obtain multiple mapping data. The mapping data are then input into a preset decision model for analysis and calculation to obtain the optimal water and fertilizer ratio scheme. The visual data includes growth status information and morphological information, the sensor data includes growth environment information, and the spatiotemporal labeled data includes time series prediction information of agricultural events and environmental changes. The decision model is a water and fertilizer intelligent agent decision model used to output irrigation strategies, and the construction of the decision model is based on a pre-trained large model.

[0011] Preferably, the preprocessing of the dataset to obtain visual information specifically includes:

[0012] Random rotation was used during image data processing to obtain cherry images taken from different angles;

[0013] The image data is divided into multiple small blocks, and histogram equalization is performed on each block, limiting the degree of contrast enhancement. The transformation function for histogram equalization is:

[0014] ,

[0015] Where T is the transformation function. It is the input pixel value. This represents the output pixel value, where k is the grayscale index and L is the number of grayscale levels. Here, j represents the grayscale level, j is the pixel value, and N is the total pixel value.

[0016] Image inpainting or generative adversarial networks are used to fill in the missing parts;

[0017] During segmentation, a masked region convolutional neural network is used. The masked region convolutional neural network adds a branch for predicting the target mask to the fast region convolutional neural network target detection framework.

[0018] Preferably, the image data is a 2D image, and the real-time information includes the combined count of buds, flowers and fruits at all cherry vegetation stages, as well as the total number of cherries collected at harvest. The real-time information is obtained by manually recording reference branches throughout the growing season.

[0019] Preferably, the step of extracting multi-scale features from visual information specifically includes the following steps:

[0020] Based on the color of cherries at different growth stages, the color of cherries in the image data is divided by histogram to obtain a color histogram.

[0021] Based on the morphological parameters of cherries, morphological data is obtained by identifying the morphology of cherries in image data through image processing, and growth curves are constructed based on the morphological data.

[0022] By combining color histograms with growth curves, multi-scale features are obtained through analysis.

[0023] Preferably, the visual model includes a detection head, a bidirectional fusion neck, and a backbone network. The feature extraction process using the visual model for multi-scale features specifically includes the following steps:

[0024] The visual model extracts multi-scale features through a backbone network;

[0025] Based on multi-scale features, the bidirectional fusion neck fuses high-level semantic information and low-level spatial details in multi-scale features through bidirectional information flow to generate a feature map with contextual information.

[0026] Based on the feature map, the detection head performs bounding box regression and category prediction using the features fused in the feature map to obtain growth state information and morphological information.

[0027] Preferably, data preprocessing is performed before obtaining temporal and spatial features. For temporal feature data, the units of meteorological data and soil moisture are unified by normalization, historical data of the past week are extracted using sliding window technology, and seasonal features are introduced by date coding. For spatial feature data, orchard areas are divided according to soil fertility, and missing data are filled in by spatial interpolation.

[0028] Preferably, the fusion of temporal and spatial flow features, combined with a regression network to generate prediction information, specifically includes the following steps:

[0029] The features of time flow and spatial flow are fused through a cross-attention layer, and cross-dimensional relationships are established while retaining information from their respective domains.

[0030] The fused and correlated features are then used for deep pattern learning through a converter module, and combined with a regression network to generate predictive information.

[0031] Preferably, the method further includes:

[0032] The decision-making model is equipped with a knowledge base containing information on the cherry's growth cycle, optimal growth conditions, soil type adaptability, and regional climate characteristics. The knowledge base is retrieved through a retrieval enhancement generation method, which incorporates real-time data.

[0033] Preferably, the enhanced retrieval generation method retrieves information from the knowledge base, specifically including the following steps:

[0034] Load unstructured data from a local document and convert it into text using a loader;

[0035] Use a text splitter to divide long text into multiple text blocks;

[0036] Vectorize the text blocks;

[0037] Based on the retrieved text blocks and combined with a pre-set prompt template, prompts for the decision-making model are generated, wherein the prompt template is used to integrate relevant knowledge with the user query;

[0038] The generated prompts are input into the decision model, which then uses the relevant knowledge in the prompts to perform reasoning and calculations, and finally outputs the irrigation plan for the corresponding text block.

[0039] The technical effects and advantages of the present invention in the above technical solution are as follows:

[0040] This invention acquires a cherry growth status dataset and preprocesses it to extract multi-scale features. It then combines this with a visual model to accurately obtain growth status and morphological information. Next, it utilizes a predictive model's temporal flow network to analyze historical climate and soil moisture temporal features, and a spatial flow network to learn the spatial correlation of orchard plots. By fusing spatiotemporal features, it generates regional irrigation amount and soil moisture prediction information. Finally, through a decision model based on a pre-trained large model combined with enhanced retrieval from an external knowledge base, it outputs the optimal water and fertilizer ratio scheme. This effectively solves the problems of traditional irrigation methods, such as difficulty in developing schemes tailored to different growth stages and states of cherries and the reliance on experience leading to uneven water and fertilizer supply. Furthermore, the lightweight design of the decision model has the potential for deployment on edge devices and embedded sensors, making it highly adaptable. It provides precise and efficient irrigation decision support for cherry cultivation, helping the industry reduce costs and increase efficiency. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of a smart irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction according to the present invention.

[0042] Figure 2 This is a diagram illustrating the data processing in the data preprocessing section of this invention.

[0043] Figure 3This is a color display diagram of the fruit at different growth stages according to the present invention.

[0044] Figure 4 This is a schematic diagram of the overall structure of the DEIM model of the present invention.

[0045] Figure 5 This is a comparison chart of the detection results of the actual labels and the DEIM model in this invention.

[0046] Figure 6 This is a schematic diagram of the LSTM-Transformer hybrid architecture of the present invention.

[0047] Figure 7 This is a schematic diagram of the dual-stream spatiotemporal attention network method of the present invention.

[0048] Figure 8 This is a schematic diagram of the multimodal knowledge graph of the present invention.

[0049] Figure 9 This is a schematic diagram showing the comparison of data before and after normalization in this invention.

[0050] Figure 10 This is a schematic diagram of the water and fertilizer intelligent agent decision-making model of the present invention.

[0051] Figure 11 This is a schematic diagram of the DoRA structure of the present invention.

[0052] Figure 12 This is a schematic diagram of the Qlora fine-tuning structure of the present invention.

[0053] Figure 13 This is a schematic diagram of the RAG process structure of the present invention.

[0054] Figure 14 This is a flowchart illustrating the overall workflow of the intelligent agent of this invention. Detailed Implementation

[0055] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] This invention provides Figure 1 The method shown is a smart irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction. The method includes:

[0057] Step S1: Obtain cherry growth status dataset, preprocess the dataset to obtain visual information, and extract features from the visual information to obtain multi-scale features. The dataset includes image data and real-time information.

[0058] Step S2: Use a visual model to extract features from multi-scale features to obtain information on the growth status and morphology of cherries.

[0059] Step S3: Based on growth status information and morphological information, analyze the temporal characteristics of historical climate fluctuations and soil moisture changes through the temporal flow network in the prediction model to obtain temporal flow characteristics. Learn the spatial correlation between different plots in the orchard through the spatial flow network in the model to obtain spatial flow characteristics. Integrate the temporal flow characteristics and spatial flow characteristics and combine them with the regression network to generate prediction information. The prediction information includes regional irrigation amount and soil moisture prediction information.

[0060] Step S4: Project the visual data, sensor data, and spatiotemporal label data respectively to obtain multiple mapping data. Input the mapping data into a preset decision model for analysis and calculation to obtain the optimal water and fertilizer ratio scheme. The visual data includes growth status information and morphological information, the sensor data includes growth environment information, and the spatiotemporal label data includes time series prediction information data of agricultural events and environmental changes. The decision model is a water and fertilizer intelligent agent decision model for outputting irrigation strategies, and the construction of the decision model is based on a pre-trained large model.

[0061] This invention's dataset covers five major cherry varieties (Red Lantern, Early Red, Summit, Pioneer, and Rainier) in core production areas such as Hanyuan and Wenchuan, encompassing the entire growth stage, including budding, flowering, fruit expansion, color change, and ripening. It includes high-resolution images under different lighting conditions, shading scenarios, and pest and disease status. To optimize the dataset and reduce manual data collection costs, data preprocessing and enhancement operations were performed.

[0062] In one embodiment of the present invention, preprocessing the dataset to obtain visual information specifically includes:

[0063] Random rotation was used during image data processing to obtain cherry images taken from different angles;

[0064] The image data is divided into multiple small blocks, and histogram equalization is performed on each block separately, while limiting the degree of contrast enhancement. The transformation function for histogram equalization is:

[0065] ,

[0066] Where T is the transformation function. It is the input pixel value. This represents the output pixel value, where k is the grayscale index and L is the number of grayscale levels. Here, j represents the grayscale level, j is the pixel value, and N is the total pixel value.

[0067] Image inpainting or generative adversarial networks are used to fill in the missing parts;

[0068] During segmentation, a masked region convolutional neural network (MRNN) is used. This MNN adds a branch for predicting the target mask to the fast region convolutional neural network (GRN) object detection framework. Its loss function... This is due to multitasking losses.

[0069] The image data is 2D images. The real-time information includes the combined count of buds, flowers and fruits at all stages of cherry vegetation, as well as the total number of cherries collected at harvest. The real-time information is obtained by manually recording reference branches throughout the growing season.

[0070] like Figure 2 The partial data processing illustration shown illustrates that, in this embodiment of the invention, random rotation is used when processing image data to obtain cherry images taken from different angles, thereby simulating cherry images taken from different angles and increasing diversity.

[0071] To address the occlusion problem in images, image inpainting or generative adversarial networks can be used to fill in missing parts, thereby enhancing model robustness and improving generalization ability.

[0072] The segmentation process can not only detect the bounding box of the target, but also perform pixel-level segmentation of the target, enabling real-time monitoring and automated management of cherry growth status.

[0073] This dataset originates from Cherry Set, a cherry farm in Jiuxiang Town, Hanyuan County, Ya'an, and Fruits Dataset. Cherry Set contains a series of 2D images covering three cherry trees at more than 10 different time points, from dormancy in March to flowering and growth, until harvest in July 2024. In addition to image data, Cherry Set also provides manually recorded ground-based information obtained from reference branches throughout the growing season, including comprehensive bud, flower, and fruit counts for all vegetation stages, as well as the total number of cherries collected at harvest. This dataset is specifically designed for the growth status of cherry trees, with both images and ground-based information closely focused on the cherry growth process. Compared to other general agricultural image datasets, it is more targeted and better meets the specific task of identifying different growth stages of cherries. Simultaneously, data was collected and labeled offline at the cherry farm in Jiuxiang Town, Hanyuan County, Ya'an, using different cherry varieties to enhance the model's generalization ability. The Fruits Dataset contains a variety of fruit types. This invention selects a dataset related to cherries from it, manually labels it using labelimg, and uses it as a test set.

[0074] In one embodiment of the present invention, the step of extracting features from visual information to obtain multi-scale features specifically includes the following steps:

[0075] Based on the color of cherries at different growth stages, the color of cherries in the image data is divided by histogram to obtain a color histogram.

[0076] Based on the morphological parameters of cherries, morphological data is obtained by identifying the morphology of cherries in image data through image processing, and growth curves are constructed based on the morphological data.

[0077] By combining color histograms with growth curves, multi-scale features are obtained through analysis.

[0078] like Figure 3 The color display of the fruit at different growth stages shown in this embodiment of the invention indicates that, in the young fruit stage, the color histogram of the image may show a higher peak value in the channel corresponding to cyan; while as the fruit matures, the peak value of the red or yellow channel will gradually increase; this quantified color information provides solid data support for accurately determining the growth stage of cherries.

[0079] From the young fruit stage to maturity, cherries continuously expand. Morphological parameters, especially fruit diameter measurements, provide a clear picture of these size changes. In practice, image processing technology is used to identify the outline of the cherry fruit, allowing for precise diameter measurement. Obtaining fruit diameter data provides a clear understanding of the cherry's growth process, enabling the construction of growth curves that visually represent its growth trend over time.

[0080] Combining color histograms and morphological parameters—two manual feature extraction methods—can provide rich and comprehensive information for cherry growth status detection, significantly improving the accuracy and reliability of the detection.

[0081] In one embodiment of the present invention, the visual model includes a detection head, a bidirectional fusion neck, and a backbone network.

[0082] In this embodiment of the invention, the visual model is a DEIM-based visual model. The DEIM model was chosen as the core framework for cherry ripeness detection primarily because of its outstanding advantages in feature extraction and small object recognition. This model adopts a highly modular design, providing five pre-configured versions: nano(n), small(s), medium(m), large(l), and super-large(x). After comparative analysis, we selected the DEIM(s) version as the basic framework for this study, as it achieves the optimal balance between model complexity and detection performance.

[0083] like Figure 4The overall structure of the DEIM model is shown below. The model runs on an NVIDIA RTX 3090 GPU and was developed using the PyTorch 2.1.0 + cu12 framework. The experimental results are shown in Table 1 below:

[0084] Table 1 Experimental Results:

[0085]

[0086] After 200 rounds of training, the model demonstrated excellent performance in cherry maturity detection. According to the COCO metric, the model achieved an overall average precision of 59.7%, indicating that it can accurately identify the location and size of cherries under different IoU thresholds. Under loose matching conditions, the mAP@50 reached 86.7%, demonstrating the model's reliable detection of most cherries; under strict localization conditions, the mAP@75 was 63.2%, further validating the model's ability in bounding box accuracy. In terms of target scale, the model achieved APs of 79.8% and 58.5% for large and medium targets, respectively, demonstrating strong detection capabilities and accurately identifying the location and state of ripe cherries in the orchard. Regarding recall, the overall AR@100 reached 66.0%, and the AR@50 under loose matching was as high as 92.4%, ensuring effective detection of most targets and providing comprehensive data support for maturity assessment.

[0087] To enhance the model's detection capabilities in real-world orchard scenarios, we employed various data augmentation techniques during training, including image flipping, scaling, and adjustments to brightness and contrast. These measures improved the model's adaptability to different lighting conditions, shooting angles, and fruit postures, making it more robust on images acquired by the sensor. In real-world sensor acquisition scenarios, the model demonstrated high accuracy in identifying medium to large targets, ensuring rapid screening of ripe cherries and yield assessment, and exhibiting strong robustness. Its ability to adapt to various lighting conditions and shooting angles makes it suitable for image acquisition in different orchard environments. Overall, the model exhibited high accuracy, high recall, and good robustness in cherry ripeness detection. The data augmentation and other training strategies further enhanced its detection capabilities, providing a stable and reliable technical guarantee for ripe cherry identification, yield assessment, and orchard management in practical sensor applications.

[0088] To more intuitively see the differences and consistency between the real labels and the model detection results, we conducted a visualization analysis. Figure 5 The actual label shown Figure 5 (a) Detection results of DEIM model Figure 5Figure (b) shows a comparative analysis of the detection results between the real labels and the proposed DEIM model. Part (a) shows the manually labeled real labels, with fruits and buds defined by bounding boxes as a baseline reference. Part (b) shows the detection results generated by the DEIM model under the same conditions. The comparison results show that the DEIM model can successfully identify most fruit targets and maintain a high degree of consistency with the real labels. Especially when the fruit is partially obscured by leaves, there are significant changes in lighting, or the background is complex, the model still demonstrates strong detection capabilities. The bounding boxes generated by the DEIM model closely match the real labels in terms of position and scale, reflecting the robustness and reliability of the detection process. These comparative results verify the effectiveness of the DEIM model in a real orchard environment and demonstrate its application potential in automated cherry monitoring tasks.

[0089] Feature extraction processing of multi-scale features using a visual model includes the following steps:

[0090] Visual models extract multi-scale features through a backbone network;

[0091] Based on multi-scale features, the bidirectional fusion neck fuses high-level semantic information and low-level spatial details in multi-scale features through bidirectional information flow to generate feature maps with contextual information.

[0092] Based on the feature map, the detection head performs bounding box regression and category prediction using the features fused in the feature map to obtain growth state information and morphological information;

[0093] In this embodiment of the invention, the backbone network is HGNetv2, which consists of a Stem module, multiple HG-Blocks, and multiple convolutional modules to achieve multi-scale hierarchical feature extraction. These multi-scale features are then input into a neck network that combines a Feature Pyramid Network (FPN) and a Path Aggregation Network (PANet). This neck network deeply fuses high-level semantic information with low-level spatial details through bidirectional information flow, generating a feature map rich in contextual information. Finally, the detection head uses these fused features for bounding box regression and class prediction. The detection head incorporates two key training strategies for performance optimization: a dense one-to-one matching strategy optimizes traditional Hungarian matching, assigning an appropriate prediction head to each object to avoid ignoring small objects or minority classes; and a loss function sensitive to matching quality is introduced, which can dynamically adjust the weights of suboptimal samples, further enhancing the model's robustness in complex backgrounds.

[0094] To further enhance feature fusion capabilities, we replaced the original model's neck structure based on FPN and PANet with an innovative and improved FTPN architecture. This module fuses the MS-DF module with the Transformer encoder to achieve more accurate multi-scale feature fusion. The MS-DF module employs a multi-scale diffusion mechanism to improve the interaction efficiency between different scales. The Transformer encoder is used to enhance the global context modeling capability of deep semantic features, thereby further improving detection performance. Considering the model's real-time requirements, the Transformer encoder is only applied to deep feature maps to avoid excessive computational overhead on shallow high-resolution features. This design is particularly crucial in the cherry maturity detection task: cherry maturity mainly depends on subtle color and texture differences, and fruits of different sizes and shooting distances exhibit complex multi-scale features. The improved FTPN neck network can more effectively fuse key information from different scales and enhance the perception of subtle differences through global context enhancement, thereby improving the model's accuracy and robustness in recognizing cherry maturity in complex natural environments. This model achieves a good balance between accuracy and computational cost, requiring only 11.51 million parameters and 30.43 GFLOPs. This lightweight design demonstrates the potential for deployment on edge devices and embedded sensors, enabling real-time online monitoring in agricultural environments.

[0095] In one embodiment of the present invention, data preprocessing is performed before obtaining temporal and spatial flow features. For temporal feature data, the dimensions of meteorological data and soil moisture are unified by normalization method, historical data of the past week are extracted by sliding window technology, and seasonal features are introduced by date encoding. For spatial feature data, orchard areas are divided according to soil fertility, and missing data are filled by spatial interpolation method.

[0096] In embodiments of the present invention, by introducing, for example Figure 6 The LSTM-Transformer hybrid architecture shown combines, for example, Figure 7 The dual-stream spatiotemporal attention network method shown breaks through the technical bottleneck of single-modal prediction and constructs a full-link technical system of temporal feature extraction, spatial semantic perception, and causal reasoning prediction. Through the bidirectional collaborative modeling and dynamic optimization mechanism of spatiotemporal features in this improved architecture, the prediction accuracy and real-time response capability are effectively enhanced, providing a better technical solution for precision irrigation management.

[0097] The extended module introduces multi-scale modeling to separate short-term fluctuations from long-term trends, simulates the impact of different irrigation strategies on crop growth, and integrates multimodal data to enhance prediction accuracy, ultimately constructing a spatiotemporal prediction model that adapts to dynamic climate and soil conditions.

[0098] Embedding such in the model architecture Figure 8 The multimodal knowledge graph shown integrates structured knowledge from crop physiology, soil science, and irrigation agronomy to enhance the agronomical validity of the prediction results.

[0099] Based on ontology, a knowledge graph covering crop growth cycle, soil nutrient cycle, water use efficiency and other fields is constructed. The causal relationship between meteorological data, soil parameters and irrigation prediction is encoded into a semantic network, and the implicit association between nodes is mined using graph neural network.

[0100] Through the semantic association matching module, the real-time predicted irrigation amount, soil moisture and other parameters are dynamically verified with the agronomic constraints in the knowledge graph (such as the critical period of crop water requirement and soil infiltration threshold). The knowledge distillation mechanism is used to transform expert experience into interpretable rule constraints to ensure that the prediction results conform to the physiological adaptation law of crops.

[0101] Meanwhile, by combining a causal reasoning framework to extrapolate the potential impact of extreme weather or abnormal irrigation strategies, and by dynamically correcting prediction biases through knowledge graph-driven feedback loops, an irrigation prediction system that combines data-driven accuracy with agronomical logic completeness is ultimately formed.

[0102] The system has been validated in major production areas such as Hanyuan in Ya'an and Luding in Sichuan. It has achieved a 93% accuracy rate in predicting the ripening period of cherry fruits (with a warning 5-7 days in advance) and an 89.7%-91.4% accuracy rate in early identification of anthracnose, which is 16-24 percentage points higher than traditional methods. By linking with an agricultural Internet of Things platform, it dynamically optimizes agricultural operations (such as triggering calcium fertilizer spraying based on rainfall warnings), reducing the fruit cracking rate from 12.3% to 4.1% and increasing water and fertilizer utilization by 21%.

[0103] In one embodiment of the present invention, temporal flow features and spatial flow features are fused and combined with a regression network to generate prediction information, specifically including the following steps:

[0104] The features of time flow and spatial flow are fused through a cross-attention layer, and cross-dimensional relationships are established while retaining information from their respective domains.

[0105] The fused and correlated features are then used for deep pattern learning through a converter module, and combined with a regression network to generate predictive information.

[0106] In this embodiment, the LSTM module is responsible for parsing long-term dependencies in the time series. Its gating mechanism filters key information through input gates, forget gates, and output gates, transforming the time series patterns of historical meteorological and soil data into hidden state sequences, providing dynamic input for subsequent spatiotemporal analysis. The Transformer module captures global spatiotemporal dependencies through a self-attention mechanism. Multi-head self-attention learns the correlation between climate, soil moisture, and irrigation demand from multiple dimensions, and positional encoding strengthens the temporal sequence information. The feedforward neural network further generates irrigation prediction values ​​with nonlinear mappings. In the system integration stage, the time series features output by the LSTM are processed by the Transformer and then output as predictions of soil moisture and irrigation volume for each region in the next few days through a regression network.

[0107] In one embodiment of the present invention, visual data, sensor data and spatiotemporal tag data are projected to obtain multiple mapping data. The mapping data are then input into a preset decision model for analysis and calculation to obtain the optimal water and fertilizer ratio scheme.

[0108] In this embodiment of the invention, OpenCV functions are used to eliminate edge distortion caused by wide-angle lenses for visual data, and the Retinex algorithm is used to eliminate the effects of backlighting or shadows.

[0109] For sensor data, time-series coding is used to capture the changing patterns of water and fertilizer parameters over time. Furthermore, different indicators collected by soil sensors have completely different dimensions and ranges, as shown in Table 2 below:

[0110] Table 2. Normalized parameters specific to agriculture:

[0111]

[0112] This invention employs agricultural-specific normalized parameters and a multi-sensor synchronization strategy to achieve unified time-based data capture and scientific normalization, resulting in the following: Figure 9 The data shown is a comparison of the data before and after normalization.

[0113] For spatiotemporal labeled data, the methods for constructing data features are described from two dimensions: time labels and historical events. Time labels: Generating time labels with agricultural guidance significance, including seasons and diurnal cycles. Seasonal labels help determine crop growth needs under different climatic cycles; diurnal labels can be used to analyze the different water and fertilizer requirements of crops under varying light conditions. Time interval event frequency statistics: Accurately calculating the time interval since the last irrigation and fertilization provides a basis for determining the timing of the next irrigation and fertilization. By continuously tracking and recording irrigation and fertilization events, combined with the current time, accurate time interval data is obtained.

[0114] Historical Events: Statistics on past irrigation frequency. Analysis of irrigation and fertilization frequency patterns provides data support for optimizing water and fertilizer management strategies. Historical comparative analysis compares current soil moisture and other parameters with those of the same period last year to analyze annual soil environmental changes, providing a reference for developing annual water and fertilizer management plans.

[0115] Visual feature projection: Process the input features and output the growth status of cherries in the image. Maintain independent scaling parameters (γ) and translation parameters (β) for each growth stage. Load stage-specific water and fertilizer threshold templates and perform feature space correction for feature combinations that do not conform to agronomic common sense.

[0116] Sensor feature projection: Real-time calculation of core agronomic parameters such as nitrogen-phosphorus ratio and potassium saturation, mapped to [0,1] based on the NP ratio of 2.5-3.5 in the cherry orchard.

[0117] Time-based feature projection: LSTM networks process time series of agricultural events such as irrigation, fertilization, and pesticide application, establish a rainfall-evaporation balance model, correct the reference value of historical irrigation records, and automatically enhance the feature contribution of recent temperature data under high temperature warning conditions.

[0118] In this embodiment of the invention, the decision model adopts a water and fertilizer decision model based on the InternLM pre-trained large model. This model is based on the Transformer architecture and adopts a Decoder-Only structure. It supports processing complete growing season data (approximately 15k sensor readings + agricultural records). In understanding cherry growth cycle descriptions, soil characteristic literature, etc., it can accurately extract key information for irrigation decisions.

[0119] like Figure 10 As shown, this invention provides a water and fertilizer intelligent agent decision-making model. In actual fine-tuning, our team's base model is InternLM-20B. We collected a large amount of cherry cultivation-related knowledge, including planting techniques and management suggestions, from agricultural literature, research papers, online agricultural forums, and social media platforms such as Reddit. The text data totaled approximately 7,800 entries, which underwent word segmentation, stop word removal, and normalization to enable the model to better capture key information in the agricultural field. The fine-tuning method adopted a hybrid parameter fine-tuning approach: a LoRA+DoRA combination configuration and progressive learning of agricultural knowledge. The DoRA structure is as follows: Figure 11 As shown, the Qlora fine-tuning structure is as follows: Figure 12As shown, the core idea of ​​LoRA is to freeze the weights of the pre-trained model and inject trainable parameters through low-rank matrix factorization to adapt to downstream tasks. By training two low-rank matrices, the linear layers in the large language model are frozen, and a new trainable linear bypass containing two consecutive low dimensions is added. DoRA improves upon LoRA by decomposing the pre-trained weights into magnitude and direction components. This approach allows for more flexible updates to the direction component, approaching complete fine-tuning. The combination of LoRA and DoRA can leverage the advantages of both and be flexibly adjusted in different scenarios. For example, in scenarios with small datasets and memory-sensitive models, LoRA can be used primarily to reduce parameter updates and memory usage; in scenarios requiring higher accuracy and with sufficient computational resources, DoRA can be used to fine-tune the weight direction components to improve model performance.

[0120] The finely tuned large model establishes a complex mapping relationship between environmental data, cherry growth stages, and the optimal water-fertilizer ratio. Based on this data, the large model uses sophisticated algorithms to analyze and calculate, providing the most suitable water-fertilizer ratio under the current conditions.

[0121] In one embodiment of the present invention, the method further includes:

[0122] The decision-making model is equipped with a knowledge base containing information on the cherry's growth cycle, optimal growing conditions, soil type adaptability, and regional climate characteristics. The knowledge base is retrieved through a retrieval-enhanced generation method, which incorporates real-time data.

[0123] The enhanced retrieval generation method retrieves information from the knowledge base, specifically including the following steps:

[0124] Load unstructured data from a local document and convert it into text using a loader;

[0125] Use a text splitter to divide long text into multiple text blocks;

[0126] Vectorize the text blocks;

[0127] Based on the retrieved text blocks and combined with pre-defined prompt templates, prompts for the decision-making model are generated. The prompt templates are used to integrate relevant knowledge with user queries.

[0128] The generated prompts are input into the decision model, which then uses the relevant knowledge in the prompts to perform reasoning and calculations, and finally outputs the irrigation plan for the corresponding text block.

[0129] In embodiments of the present invention, such as Figure 13The RAG flowchart shown demonstrates how the system can dynamically acquire the latest botanical and environmental data from the knowledge base, significantly improving the model's interpretability and decision-making capabilities. This not only enhances the accuracy of irrigation decisions but also allows the system to adapt to more diverse growing environments, enabling customized irrigation decisions for different situations.

[0130] like Figure 14 The diagram shown illustrates the overall workflow of the intelligent agent. This agent enables end-to-end conversion from multi-source heterogeneous data to agronomic compliance instructions, transforming the language understanding of large models into actionable irrigation decisions, truly serving the core needs of the cherry industry to reduce costs and increase efficiency.

[0131] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A smart irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction, characterized in that, The method includes: A cherry growth status dataset is obtained, the dataset is preprocessed to obtain visual information, and feature extraction is performed on the visual information to obtain multi-scale features. The dataset includes image data and real-time information. By using a visual model to extract features from multi-scale features, we can obtain information on the growth status and morphology of cherries. Based on growth status information and morphological information, the temporal characteristics of historical climate fluctuations and soil moisture changes are analyzed through the temporal flow network in the prediction model to obtain temporal flow characteristics. The spatial flow network in the model is used to learn the spatial correlation between different plots in the orchard to obtain spatial flow characteristics. The temporal flow characteristics and spatial flow characteristics are fused together and combined with the regression network to generate prediction information, wherein the prediction information includes regional irrigation amount and soil moisture prediction information. Visual data, sensor data, and spatiotemporal label data are projected to obtain multiple mapping data. The mapping data are then input into a preset decision model for analysis and calculation to obtain the optimal water and fertilizer ratio scheme. The visual data includes growth status information and morphological information, the sensor data includes growth environment information, and the spatiotemporal label data includes time series prediction information data of agricultural events and environmental changes. The decision model is a water and fertilizer intelligent agent decision model used to output irrigation strategies, and the construction of the decision model is based on a pre-trained large model. The process of extracting multi-scale features from visual information specifically includes the following steps: Based on the color of cherries at different growth stages, the color of cherries in the image data is divided by histogram to obtain a color histogram. Based on the morphological parameters of cherries, morphological data is obtained by identifying the morphology of cherries in image data through image processing, and growth curves are constructed based on the morphological data. By combining color histograms with growth curves, multi-scale features are obtained through analysis. The visual model includes a detection head, a bidirectional fusion neck, and a backbone network. The feature extraction process using the visual model for multi-scale features specifically includes the following steps: The visual model extracts multi-scale features through a backbone network; Based on multi-scale features, the bidirectional fusion neck fuses high-level semantic information and low-level spatial details in multi-scale features through bidirectional information flow to generate a feature map with contextual information. Based on the feature map, the detection head performs bounding box regression and category prediction using the features fused in the feature map to obtain growth state information and morphological information.

2. The intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction according to claim 1, characterized in that, The process of preprocessing the dataset to obtain visual information specifically includes: Random rotation was used during image data processing to obtain cherry images taken from different angles; The image data is divided into multiple small blocks, and histogram equalization is performed on each block, limiting the degree of contrast enhancement. The transformation function for histogram equalization is: ， Where T is the transformation function. It is the input pixel value. This represents the output pixel value, where k is the grayscale index and L is the number of grayscale levels. Here, j represents the grayscale level, j is the pixel value, and N is the total pixel value. Image inpainting or generative adversarial networks are used to fill in the missing parts; During segmentation, a masked region convolutional neural network is used. The masked region convolutional neural network adds a branch for predicting the target mask to the fast region convolutional neural network target detection framework.

3. The intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction according to claim 1, characterized in that: The image data is a 2D image, and the real-time information includes the combined count of buds, flowers and fruits at all cherry vegetation stages, as well as the total number of cherries collected at harvest. The real-time information is obtained by manually recording reference branches throughout the growing season.

4. The intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction according to claim 1, characterized in that: Before obtaining temporal and spatial features, data preprocessing is performed. For temporal feature data, the units of meteorological data and soil moisture are unified by normalization, historical data from the past week are extracted using sliding window technology, and seasonal features are introduced by date coding. For spatial feature data, orchard areas are divided according to soil fertility, and missing data are filled in by spatial interpolation.

5. The intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction according to claim 1, characterized in that, The process of fusing temporal and spatial features and combining them with a regression network to generate prediction information specifically includes the following steps: The features of time flow and spatial flow are fused through a cross-attention layer, and cross-dimensional relationships are established while retaining information from their respective domains. The fused and correlated features are then used for deep pattern learning through a converter module, and combined with a regression network to generate predictive information.

6. The intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction according to claim 1, characterized in that, The method further includes: The decision-making model is equipped with a knowledge base containing information on the cherry's growth cycle, optimal growth conditions, soil type adaptability, and regional climate characteristics. The knowledge base is retrieved through a retrieval enhancement generation method, which incorporates real-time data.

7. The intelligent irrigation decision-making method for cherries based on multimodal and spatiotemporal prediction according to claim 6, characterized in that, The enhanced retrieval generation method retrieves information from the knowledge base, specifically including the following steps: Load unstructured data from a local document and convert it into text using a loader; Use a text splitter to divide long text into multiple text blocks; Vectorize the text blocks; Based on the retrieved text blocks and combined with a pre-set prompt template, prompts for the decision-making model are generated, wherein the prompt template is used to integrate relevant knowledge with the user query; The generated prompts are input into the decision model, which then uses the relevant knowledge in the prompts to perform reasoning and calculations, and finally outputs the irrigation plan for the corresponding text block.

Images