System and method for agricultural yield forecasting and optimization using artificial intelligence

Abstract

The present invention describes an integrated agricultural data analysis system using artificial intelligence to optimize greenhouse crop management. The system includes image capture devices, an AI module for data analysis, including yield estimation and maturity prediction, cloud-based storage, an interactive user interface, and a logistics management module. The associated method has two phases: data preparation and training, followed by real-time analysis and prediction, thereby enabling precise planning and logistics management of resources and transportation, thereby promoting sustainable and economical agriculture.

Description

[0001] Description

[0002] Title of the invention: System and method for forecasting and optimizing agricultural yield using Artificial Intelligence

[0003] technical field

[0004] This invention falls within the field of precision agriculture and, in particular, offers a system designed to address the challenges of managing and optimizing greenhouse crops such as tomatoes. This artificial intelligence-based system is designed to accurately analyze captured images of greenhouse crops, thus providing precise monitoring and reliable yield estimation.

[0005] Context

[0006] Fruit and vegetable producers face significant challenges in accurately estimating crop yields. This uncertainty leads to various practical and economic problems, including a lack of visibility that complicates harvest forecasting, negatively impacting stock management by distributors and supermarkets. Furthermore, this uncertainty creates inequalities in negotiations, putting small producers at a disadvantage compared to buyers who may underestimate yields to lower the purchase price. Logistically, uncertainty about harvest volume affects planning, from the size of harvesting teams to the organization of packing facilities, creating bottlenecks that cause delays and inefficiencies throughout the value chain.

[0007] State of the Art

[0008] Current artificial intelligence (AI) technologies in agriculture include advanced computer vision tools used to monitor and analyze agricultural environments. Companies like Verdant Robotics and Dilepix are leveraging these technologies to improve crop and livestock management. However, these systems don't always address all the specific challenges faced by producers, especially regarding reliable yield estimates. This gap is particularly problematic in export contexts, where, for countries like Morocco, accurate estimates are crucial to ensuring a steady and reliable supply to international markets, such as Europe. Patent CN115631366A illustrates the use of an enhanced CenterNet model to predict fruit ripeness, specifically for tomatoes, in real time on an onboard camera-equipped platform.While this method improves the accuracy of maturity prediction, it requires significant computing power and may be limited by the capabilities of embedded devices, necessitating specific adaptations to operate effectively in low-connectivity environments. Furthermore, US patent 20230409910(A1) describes a system that uses AI to analyze climate and environmental data to optimize agricultural practices, improving productivity through precise resource allocation. However, its complexity and reliance on extensive environmental data may pose significant barriers for smallholder farmers, potentially increasing implementation costs and complexity. Patent W02020081901A2 introduces an advanced monitoring system that uses sensors to collect and analyze images to monitor crop health.This system, while accurate, can be expensive and complex, making it less accessible for small and medium-sized farms. Its reliance on highly specialized sensors can also increase maintenance and operating costs.

[0009] Our invention offers an innovative, AI-powered solution for accurate and reliable crop yield estimation, improving logistics and negotiations, and reducing food waste. This invention has the potential to transform crop management and optimize agricultural supply chains, providing a sustainable response to the current challenges of modern agriculture.

[0010] Description of the invention

[0011] This invention relates to an innovative system and method for processing and analyzing agricultural data, using advanced artificial intelligence technology to optimize greenhouse crop management and improve precision in farming. The core of the system is an artificial intelligence module configured to process video input signals from a capture device. This module uses deep learning image recognition algorithms to identify specific biometric attributes of plants, such as the size and color of floral elements.

[0012] The system also offers an image analysis process that uses contextual segmentation to classify these elements based on multiple criteria such as texture, pigmentation, and morphological geometry. This allows for a detailed and accurate assessment of expected crop yields through statistical extrapolation of the processed data, incorporating predictive models that adjust forecasts according to seasonal environmental variations.

[0013] Also included is an image capture device specially adapted to the variable lighting conditions of greenhouses, equipped with a stabilization mechanism to minimize artifacts in captured videos.

[0014] The system also includes a maturity prediction module that uses convolutional neural networks to classify the maturity stages of fruits and vegetables in order to optimize harvesting and minimize waste. For accurate classification, this module analyzes aspects such as the color and size of the produce.

[0015] The system's user interface offers an interactive visualization of analysis results and yield forecasts, with advanced customization options tailored to the specific needs of users in the agricultural sector. This interface is supported by a process for integrating environmental data into the data processing system, enabling dynamic adjustments to yield forecasts.

[0016] Finally, the invention proposes a logistics optimization method that uses estimated yield data to efficiently orchestrate harvest planning, resource management, and transportation coordination. This method aims to optimize greenhouse crop management through precise, real-time analysis, comprising two main phases: a preparation and training phase, and an analysis and prediction phase. This method can be integrated into a comprehensive enterprise management system to maximize overall effectiveness and efficiency.

[0017] Thus, the invention offers an advanced technological solution to meet the challenges of modern agriculture, increasing the accuracy of yield predictions and optimizing agricultural management and logistics processes. Advantages

[0018] The goal of this invention is to substantially improve greenhouse crop management by integrating cutting-edge artificial intelligence technologies, thereby increasing the accuracy and efficiency of agricultural monitoring. It enables precise, real-time analysis of visual data captured by advanced devices to reliably estimate crop yields. This detailed analysis is essential for adapting agricultural practices to fluctuating environmental conditions, ensuring continuous optimization of resources such as water and nutrients. The innovative aspect of this method lies in its ability to process complex visual data to identify specific crop characteristics, such as plant size, color, and health.This translates into better management of the growth cycle and a significant reduction in food waste thanks to more accurate maturity predictions. As a result, farmers can plan harvest times more effectively, reduce operating costs, and minimize environmental impact.

[0019] Brief description of the drawings

[0020] Figure 1 illustrates the components and data flow of the system proposed by the invention.

[0021] Figure 2 represents the process of the Data Preparation and Training step relating to the method proposed by the invention.

[0022] Figure 3 represents the process of the analysis and prediction phase relating to the method proposed by the invention.

[0023] Figure 4 illustrates the annotation process used in the method of the invention, where each relevant object in the image is framed by a bounding box.

[0024] Detailed description

[0025] In the following development, the technical solution proposed by this invention will be described in full and precise detail, with reference to the attached Figures 1 to 4. It should be noted that the embodiments described constitute only a selection of the various possibilities offered by this invention. The modalities presented are based on the principles of the invention, and it is understood that any other variant developed within the same framework, and not requiring inventive effort, is also covered by the scope of protection of this invention. The present invention describes an integrated agricultural data processing and analysis system (10) based on artificial intelligence, specifically designed to optimize greenhouse crop management. As illustrated in Figure 1, this system comprises several interconnected components that work together to provide real-time and accurate analyses of crop condition and predict future yield.The system begins with an image capture device (1), positioned in specific locations to capture detailed images of the crops in the greenhouse (2). These images are essential for analysis, providing a visual basis upon which the data processing and analysis module (5) operates. This module is configured to process video input signals (8), leveraging advanced deep learning image recognition algorithms to identify various precise biometric attributes such as crop size and color. It is also capable of recognizing and counting stems, thus facilitating the assessment of fruit load per stem. This is particularly useful for crops like tomatoes, where an accurate estimate of fruit load per stem can significantly influence farm management decisions and harvest planning.The captured videos (3) are temporarily stored on local media or transferred to a cloud environment (4), facilitating access and subsequent processing in a more robust infrastructure. In the cloud, the data is accessible for various subsequent analyses and for integration into seasonal predictive models, allowing the extrapolation of historical and current data to predict future crop yields. The processing module (5) applies contextual segmentation to distinguish and classify floral elements in the images, using hierarchical classification based on features such as texture, pigmentation, and morphological geometry.

[0026] Furthermore, the system incorporates a yield estimation algorithm that statistically extrapolates the processed data to forecast the expected harvest volume over a given period, using models adjusted for seasonal environmental variations. This forecast is crucial for accurate harvest planning and optimized resource management.

[0027] In parallel, a maturity prediction module deploys a convolutional neural network to accurately classify the maturity stages of fruits and vegetables by analyzing aspects such as the color and caliber of the products, which allows for rational harvest planning and contributes to the reduction of food waste.

[0028] The image capture device (1) is optimized to adapt to the various lighting conditions of the greenhouse (2), thanks to an adaptive image calibration algorithm and a stabilization mechanism that significantly reduces artifacts in the captured videos.

[0029] An intuitive user interface (6) allows operators to interact with the system, providing interactive visualization of image analysis results and yield forecasts. This interface includes customizable features that adapt to the specific preferences of users in the agricultural sector.

[0030] A logistics optimization and resource management module (7) leverages projected yield data to effectively orchestrate crop planning, resource management, and transportation coordination. This module offers strategies to align farming operations with maturity forecasts, optimizes the allocation of resources such as water and nutrients, and oversees transportation logistics, integrating all these functions into a business management system designed to maximize efficiency and productivity.

[0031] Finally, this system is designed to incorporate user feedback and observations, enabling continuous improvement of the AI ​​algorithms used. The data processing module (5) is configured to provide adaptive feedback based on analysis results and feedback via the interface (6), allowing for optimization of the AI ​​algorithms. Furthermore, the AI ​​module performs a multi-layered predictive analysis, using multi-level multivariate regression and incorporating real-time growth variables and agronomic indicators for a personalized and accurate yield estimation.

[0032] This adaptability ensures that the system remains at the forefront of technology and responds effectively to changes in growing conditions.

[0033] According to another aspect of the present invention, a method has been developed to optimize greenhouse crop management through precise, real-time analysis based on artificial intelligence. Referring to Figures 1 to 4, this method comprises two main phases, each of which includes steps as described below: a. Data preparation and AI model training phase (Figure 2)

[0034] - Collection (21): The first phase of this method consists of exhaustively collecting visual data from several farms. Images are captured continuously using the image capture device (1) located inside the greenhouse (2). This data is essential for training and ensuring the accuracy of the artificial intelligence model used. This is the most time-consuming step; the more data we collect, the better the model we obtain.

[0035] - Annotation (22): After collection, the images undergo an annotation step (22), or data labeling, in which each relevant element, such as fruit, leaves, or signs of potential disease, is identified and marked. This process involves using bounding boxes, as illustrated in Figure 4, to precisely locate each object in the image. Open-source software such as labellmg can be used for this purpose. The coordinates of these boxes are normalized and saved in YOLO-formatted text files, bearing the same name as the corresponding image, to ensure data consistency regardless of the dimensions or resolution of the captured images.

[0036] The bounding boxes recorded in the txt file are calculated according to this rule: xywhi ndeX 'W'H 'W' W>

[0037] Index: For each annotated object, we only have one object, meaning the index would be 0. x, y: the coordinates of the object's center. w, h: the height and width of the selected object.

[0038] We divided by W and H to normalize xywh between 0 and 1.

[0039] - Augmentation (23): To enhance the model's robustness to environmental variations, data augmentation techniques are employed, including vertical flipping (VerticalFIip), horizontal flipping (HorizontaIFlip), transposition (Transpose), random rotation, center cropping (CenterCrop), and the application of Gaussian blur. These methods aim to enrich the dataset, increasing the model's ability to generalize from various real-world agricultural scenarios. The dataset is then segmented into three distinct parts: training, validation, and testing, thus facilitating the continuous evaluation and improvement of the model.

[0040] - Training the AI ​​model (24): A crucial step in this method involves training an object detection model based on the YOLOv8 (You Only Look Once) architecture, an open-source solution renowned for its speed and real-time efficiency. The training process comprises several cycles where the model learns to accurately identify and classify the various plant attributes from annotated images. An early stopping mechanism is integrated to prevent overfitting, and techniques such as dropout, a regularization technique that randomly deactivates certain neurons during training, are also used to promote optimal generalization.

[0041] - Model Validation and Optimization (25): The model's performance is evaluated using standardized metrics such as intersection over union (loU) and mean accuracy (AP). These measures allow us to quantify the accuracy of the model's predictions relative to the actual annotations. The model is adjusted based on this feedback to maximize its accuracy and reliability. Intersection over union (loU): This metric gives us the overlap ratio relative to the total area, providing a good estimate of the proximity of the predicted bounding box to the original bounding box.

[0042] Area of ​​Overlap LoU = - -

[0043] Area of ​​Union Average Precision (AP): The AP allows us to evaluate the performance of our model; it measures the accuracy of detection using both precision and recall.

[0044] True Positives

[0045] Precision (P)) =

[0046] True Positives + False Positives, True Positives

[0047] Recall (R) ) = — - - — — - - - -

[0048] True Positives + False Negatives

[0049] Average Accuracy (AP): b. Analysis and prediction phase (figure 3)

[0050] In the analysis phase of our method, the AI ​​model, once trained and validated, is deployed in a real greenhouse environment to monitor and analyze crops in real time. The system analyzes the captured video streams (31), identifying and classifying each crop element using the detection model.

[0051] - Detection Stage (32): The model analyzes the video images, detecting tomatoes with high accuracy. It uses advanced architecture to ensure fast and precise identification of plant characteristics. The accuracy level reaches up to 94%.

[0052] - Detail Tracking (33): Each identified product, such as tomatoes, is tracked using a tracker like ByteTrack. This algorithm uses Kalman Filtering and the Hungarian Algorithm to estimate and track the identities and bounding boxes of multiple objects in video sequences. By combining advanced mathematical methods, ByteTrack enables continuous and accurate tracking of objects across video images, facilitating constant analysis and detailed classification of growth stages.

[0053] Kalman filtering is used to predict the future state of an object based on its previous states. It is a recursive algorithm that estimates the position and velocity of an object.

[0054] The prediction step forecasts the next state, and the update step refines this prediction based on new measurements. The formulas for the prediction and update steps of the Kalman filter are as follows:

[0055] Prediction:

[0056] Xkjk-1 — F 'kXk-ljk-1 + B 'kllk Update : xk\k = fc|fc-1 + ( z k ~ Hk.Xk\kl)

[0057] Pk\k = U — ^kHk)Pk\kl

[0058] Hungarian Algorithm: After predicting the state of each object, ByteTrack uses the Hungarian Algorithm to associate detections with existing tracks. This algorithm solves the assignment problem, which consists of finding the most cost-effective way to assign n objects to n tasks. In the context of ByteTrack, it assigns detection boxes to existing tracks to minimize the total cost, which is often the Euclidean distance between predicted and detected positions.

[0059] Knowing that:

[0060] - Region of interest (34): We now define a region of interest that will be used to count, classify, and measure the size of products, such as tomatoes, that pass through it. This region is configured based on analyses of detected objects, and each product is assigned a unique identifier as it passes through this region, allowing for precise tracking and detailed statistics.

[0061] - Counting (35): To calculate the calibration of each product, we begin by segmenting the products (e.g., tomatoes) from the background using bounding boxes. This segmentation is achieved by isolating the regions delimited by bounding boxes drawn around each detected product. Next, we use thresholding techniques to enhance contrast by isolating the product contours. These contours enable accurate detection and provide clear boundaries for further analysis. Once contour detection is complete, we proceed to quantify the surface area of ​​each product based on its shape characteristics. This may involve the use of geometric calculations or advanced computer vision algorithms tailored to the specific contours of the products (e.g., tomatoes).

[0062] - Color Classification (36): To classify the color of products (e.g., tomatoes), we first convert the color space of our images to the HSV (Hue, Saturation, Value) color space. This conversion allows us to better define the color range for each category. Next, we determine the HSV range corresponding to each color category (...). Once the color ranges are defined, we classify each tomato according to its HSV values, assigning it to the appropriate color category. Simultaneously, the ripeness stages of the products are determined using a convolutional neural network that analyzes these classified images to precisely identify their ripeness.

[0063] Calibration (37): This step involves measuring the size of the tomatoes after they have been graded. It uses computer vision algorithms to analyze the size and shape of each tomato detected. Based on the precisely defined contours from the counting step, the system evaluates the dimensions of each fruit to determine its size.

[0064] This analysis helps ensure that the harvested tomatoes meet the size standards required for different markets and uses, thus optimizing agricultural production results in terms of quality and meeting commercial criteria.

[0065] After classification and grading, estimated yield data, including tomato size and maturity, is used to precisely coordinate harvest planning and logistics. This includes organizing the necessary resources and planning transportation, ensuring maximum efficiency throughout the production and distribution chain. This approach optimizes resources and minimizes the time between harvest and distribution, thus contributing to more sustainable and economical agriculture.

[0066] Finally, this method is designed to dynamically integrate user feedback, continuously improving AI algorithms through adaptive feedback. Based on analyses and interactions via the user interface (6), this approach refines the algorithms for increased performance. Furthermore, it deploys a multi-layered prediction strategy that incorporates multivariate regression, adapting yield predictions to the actual morphological variations of crops observed in real time, thus ensuring personalized and accurate yield estimates.

[0067] Applications

[0068] Initially designed for tomato crops, the technology of this invention demonstrates remarkable adaptability, potentially extending its application to other crops such as blueberries, raspberries, peppers, melons, and blackberries. This versatility underscores the solution's ability to meet a wide range of needs in the agricultural sector.

Claims

Demands [Claims 1] An integrated agricultural data processing and analysis system (10) configured to optimize greenhouse crop management and yield, comprising: - an image capture device (1) for acquiring images in a greenhouse (2); - an AI module for data processing and analysis (5) using deep learning algorithms, comprising: - an image analysis module designed to identify biometric attributes of crops, including stem recognition, average fruit load per stem, and fruit size and color. - a yield estimation module - a maturity prediction module - a cloud-based data storage module (4), for processing and subsequent access to the captured video data (3) - a user interface (6) for generating an interactive visual representation of the analysis results and yield forecasts; and - a logistics management module (7) to integrate yield forecasts into harvest planning and resource management. characterized in that the yield estimation module uses an algorithm designed to perform a statistical extrapolation of the processed data in order to predict the harvest volume over a defined period, by integrating predictive models adjusted according to seasonal environmental variations. [Claims 2] A system according to claim 1, wherein the image capture device (1) is configured to automatically adapt to different lighting conditions in the greenhouse (2), incorporating an image calibration algorithm to adjust image quality according to variations in greenhouse lighting, and a stabilization mechanism to minimize artifacts in the captured videos. A system (100) according to claim 3, characterized in that the computer vision algorithms used are specifically designed for object detection, image segmentation, and classification, thus enabling accurate plant identification and counting. (140), while ensuring constant and reliable analysis by applying perspective correction and normalization techniques to the captured images. [Claims 3] System according to claim 1, wherein the maturity prediction module uses a convolutional neural network to classify the maturity stages of fruits and vegetables, by analyzing aspects such as the color and caliber of the products. [Claims 4] System according to claim 1, wherein the user interface (6) includes advanced personalization features to adapt the display to the specific preferences of users according to the agricultural domain. [Claims 5] A method for optimizing the management and yield of greenhouse crops, comprising two main phases: - a first phase of data preparation and training, comprising the steps: a. Collection (21): capturing images of crops via an image capture device (1). System (100) according to any one of the preceding claims, characterized in that it further comprises means for calculating and interpreting the vegetation index, thereby enabling the assessment of plant physiology and the identification of signs of abiotic stress or nutritional deficiencies. b. Data annotation (22): identifying and labeling relevant elements such as fruits and leaves or signs of potential diseases, using a process that involves the use of bounding boxes to precisely locate each object in the image. c.Data augmentation (23): Apply augmentation techniques to simulate various environmental conditions, such as vertical and horizontal flipping, transposition, random rotation, centered cropping, and Gaussian blurring. The dataset is then segmented into three distinct parts: training, validation, and testing; d. AI model training (24): Use of plant feature learning processes and architectures such as YOLOv8, including. Several learning cycles are used to identify and classify plant attributes. This step incorporates an early stopping mechanism to prevent overfitting and uses techniques such as dropout to improve model generalization. e. Model Validation and Optimization (25): Evaluation and adjustment of the model based on standardized metrics such as intersection over union (loU) and mean accuracy (AP). - a second phase of analysis and prediction, comprising the following steps: f. Deployment: Once trained and validated, the AI ​​model is deployed in a real-world greenhouse environment to monitor and analyze crops in real time. g. Detection (32): Analyze the captured video streams (31) to identify and classify each crop element using the trained detection model. h. Detail Tracking (33): Each detected object, such as tomatoes, is tracked using a tracking algorithm that employs a filtering technique and a technique to estimate and track identities and bounding boxes in the video sequences. i. Region of Interest (34): Define an area to count, classify, and measure the size of products, such as tomatoes, with a unique identifier assigned to each product passing through this area. j. Counting (35): We segment the products from the background using bounding boxes and thresholding techniques to isolate the edges.Once the outlines are defined, we quantify the surface area of ​​each product using geometric calculations or advanced computer vision algorithms. k. Color Classification (36): Convert the images to the HSV (Hue, Saturation, Value) color space, define the color ranges for each category, and then classify the products according to their HSV values ​​to determine their stage of maturity. l. Calibration (37): Measure the dimensions of the products using computer vision algorithms to assess their size. This ensures their compliance with market standards. m. Use of estimated yield data, including product size and maturity, for precise harvest planning and controlled logistics management, enabling the user to optimize resource use and minimize delays between harvest and distribution. [Claims 6] Method according to claim 5, characterized in that in the annotation step (b), the coordinates of the bounding boxes of figure 4 are normalized and saved in YOLO format text files, bearing the same name as the corresponding image, to ensure data uniformity regardless of the dimensions or resolution of the captured images. [Claims 7] Method according to claim 5, wherein the tracker used in the detail tracking step (g) is a ByteTrack-type algorithm that uses Kalman Filtering and the Hungarian Algorithm to estimate and track identities and bounding boxes in video sequences. [Claims 8] Method according to claim 5, characterized in that it integrates adaptive feedback based on the results of the analysis and interactions with the user interface (6), allowing continuous optimization of artificial intelligence algorithms. [Claims 9] Method according to claim 5, characterized in that it implements a multilayer predictive analysis strategy using multilevel multivariate regression, which integrates real-time growth variables and agronomic performance indicators to provide a personalized and accurate estimate of crop yield.

Images