Prawn culture feeding decision-making system based on multi-modal information fusion and AI intelligent agent

Abstract

The invention discloses a prawn culture feeding decision-making system based on multi-modal information fusion and an AI intelligent agent, and relates to the technical field of intelligent aquaculture and artificial intelligence, and the system is composed of an automatic feed platform device, a camera acquisition and visual analysis module, a prawn weight estimation module, and a residual feed identification and description module. The system is composed of a breeding production data analysis and feeding amount prediction module, a feeding decision intelligent agent module and a man-machine interaction and execution module. According to the system, low-stress and high-precision automatic monitoring and precise intelligent feeding decision can be deployed in a real commercial factory prawn culture workshop; prawn body weight estimation, residual feed sensing and daily feeding amount intelligent decision making can be realized under complex, high-density and turbid water conditions, the feed utilization rate can be improved, water quality pollution and labor cost can be reduced, and prawn culture is promoted to be transformed from experience driving to data and intelligent driving.

Description

Technical Field

[0001] This invention relates to the fields of smart aquaculture and artificial intelligence technology, and more specifically, to a shrimp farming feeding decision system based on multimodal information fusion and AI agents. Background Technology

[0002] Litopenaeus vannamei is one of the most important aquaculture species globally. Factory-style high-density farming offers significant advantages in terms of stable supply, land conservation, and increased output per unit of water. However, in closed or semi-closed factory-style farming systems, the shrimp's growth environment is complex, the water turbidity is high, and the stocking density per pond is large. Traditional feeding management methods, which rely on manual experience, have the following prominent problems: 1. Delayed acquisition of weight information: In actual production, shrimp weight is usually obtained through periodic sampling, weighing, and manual estimation. This method has limited frequency, carries stress risks, and the data lacks timeliness and representativeness, making it difficult to accurately reflect the overall population weight level on a given day or at a given moment; 2. Reliance on manual observation for monitoring leftover feed: Common feeding platform management involves farmers raising the feeding platform out of the water at fixed times, visually judging whether there is leftover feed and how much, and adjusting the feeding amount for the next meal or the following day based on experience. This method is highly subjective and greatly influenced by human experience and effort; 3. Daily feeding decisions are mainly based on experience: In many factory-style shrimp farms, daily feeding amounts are usually roughly estimated using a combination of "weight estimation + shrimp load estimation + feed conversion ratio experience table," supplemented by recent feed surplus data for correction. Due to the lack of systematic modeling and analysis using historical production and environmental data, the above experience-based methods are difficult to adapt to differences in conditions such as different workshops, water environments, and seedling batches, easily leading to overfeeding or underfeeding, affecting feed utilization, water quality stability, and shrimp health; 4. Insufficient data utilization and lack of closed-loop management: Factory-style aquaculture often has a certain level of data recording, such as daily data on water temperature, dissolved oxygen, pH, ammonia nitrogen, nitrite, and feeding amounts, but these are mostly stored in scattered tabular form, without forming a systematic data analysis and intelligent decision-making capability. At the same time, there is a lack of automated correlation and feedback between feeding execution results and subsequent growth and water quality changes, failing to form an intelligent closed loop of "feeding-response-adjustment."

[0003] In summary, current technologies lack a comprehensive solution for intelligent management of shrimp factory farming that can simultaneously achieve non-contact estimation of shrimp weight, automatic identification of leftover feed, multi-dimensional production data modeling, and intelligent and precise feeding decisions without significantly altering existing production practices and minimizing shrimp stress. Summary of the Invention

[0004] To address the aforementioned issues, the present invention aims to provide a shrimp farming feeding decision system based on multimodal information fusion and AI agents, with the goal of achieving low-stress, precise, and intelligent feeding for the industrialized farming of Litopenaeus vannamei.

[0005] To achieve the aforementioned technical objectives, this application provides a shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agents, comprising: Automatic feeding platform device, used to carry feed / shrimp in aquaculture ponds and achieve autonomous lifting and lowering; The camera acquisition and visual analysis module is used to acquire images of shrimp and leftover feed when the feeding platform is raised to a preset height above the water, and to perform image analysis based on segmentation, classification and large visual models; The shrimp weight estimation module is used to build a non-contact weight estimation model based on the projected area and visual length of the shrimp image. The leftover material identification and description module is used to determine whether there is leftover material on the material platform, and to generate a qualitative description using a large visual model when there is leftover material. The aquaculture production data analysis and feeding prediction module is used to build a feeding prediction model based on multi-dimensional production data from multiple workshops and covering more than 10,000 cubic meters of water. The feeding decision-making intelligent agent module is used to integrate multi-source information such as data-driven prediction results, empirical formulas, visual analysis results, and notes from farmers, and output structured feeding suggestions under the inference of a large language model. The human-computer interaction and execution module receives real-time aquaculture data and user notes through a web interface or application programming interface, visualizes decision suggestions, and compares and provides feedback with the actual feeding execution, thereby realizing dynamic adjustment of feeding strategies.

[0006] Preferably, the automated material station specifically includes: The feeding platform structure is constructed of corrosion-resistant metal materials, and a feeding platform mesh tray with edge baffles is installed on it to support feed and shrimp. A DC geared motor is connected to the material platform via a mechanical structure and driven by a control unit to enable the material platform to rise and fall in the water. The limiting device is located at the end of the lifting stroke of the material platform to prevent structural damage; The industrial camera unit is installed above the material table and is equipped with a high-pixel imaging sensor and dynamic wide mode. The lifting and acquisition control unit is used to control the lifting of the material platform and trigger image acquisition; as well as, The AI ​​interaction module integrates voice interaction and visual analysis. It uses a voice recognition model to parse commands, a generative artificial intelligence visual model to analyze images, and a voice synthesis model to provide feedback, thereby achieving human-computer interaction.

[0007] Preferably, the visual analysis module and the weight estimation module are used to perform shrimp target detection and mask segmentation on the acquired feed platform image using a deep learning instance segmentation model, distinguishing at least two types of postures: "side shrimp" and "dorsal shrimp". The projected area of ​​the shrimp is calculated based on the segmentation mask, and the full visual length of the shrimp is extracted from the mask using skeleton thinning and principal axis extraction algorithms. A machine learning-based regression model is used, with the total visual length and projected area as input features, to achieve non-contact estimation of shrimp weight. The estimation results are then used by subsequent modules. An image classification model is used to classify the material platform image into two categories: "with leftover material" and "without leftover material". When the image is identified as "with leftover material", a large visual model is called to analyze the image and output a natural language description including the location of the leftover material, the size of the area, the state of accumulation, the relative severity, and the details of the aquatic environment.

[0008] Preferably, the aquaculture production data analysis module is used to process missing values, remove outliers, and verify consistency of the acquired daily production records to obtain valid daily production records. It also employs a normalization method to scale the input features and target variables, dividing the data into training, validation, and test sets for model training and evaluation. The records include date, system ID, water temperature, dissolved oxygen, pH, daily water exchange volume, daily water exchange rate, ammonia nitrogen, nitrite, age of the animals, body weight, shrimp load, water volume, survival rate, average daily weight gain, number of daily feedings, and daily feed amount.

[0009] Preferably, the feeding amount prediction module includes: The model comparison unit is used to train and evaluate various regression models and time series models. The performance evaluation unit uses error analysis metrics to compare model performance. The feature correlation analysis unit calculates the correlation between input features and daily feeding amount, and identifies a subset of highly correlated features. The model selection and optimization unit selects the best-performing ensemble tree model based on the evaluation results and uses a metaheuristic optimization algorithm to search for hyperparameters. as well as, The core model unit uses the Bayesian optimization algorithm to iteratively optimize the hyperparameters of the ensemble tree model and construct the core model for feeding prediction.

[0010] Preferably, the feeding decision-making agent module includes: The data layer uses a database to store historical production records, model prediction results, and feeding execution results, and is configured with a knowledge base consisting of aquaculture technical specifications and empirical rules. The model layer integrates a core feeding prediction model optimized by Bayesian methods and an empirical feeding formula based on pond stock, water volume and shrimp weight, forming a dual prediction mechanism. The decision-making level calls upon a large-scale language model, taking the prediction results of the core model, the prediction results of empirical formulas, historical fragments, entries from the aquaculture knowledge base, and natural language notes from the visual analysis module and users as inputs, to analyze the differences and rationality of the dual prediction results, and to generate feeding suggestions by combining multimodal information. as well as, The report generation unit organizes the results output by the decision-making level into a structured decision report, including model predictions, empirical predictions, final recommended feed amount, reasons for adjustment, and operational recommendations.

[0011] Preferably, the human-computer interaction and execution module includes: Application programming interface (API) is used to receive real-time water quality indicators, biological parameters, and user notes from the workshop and send them to the prediction and decision-making module. The web front-end interface is used to display environmental and production data, model prediction results, residual material detection status, semantic descriptions generated by the visual large model, and structured feeding decision reports. as well as, The interactive feedback unit is used to record the adoption of intelligent suggestions by farmers and the actual amount of feeding, providing data support for subsequent model iterations.

[0012] Preferably, the shrimp precision intelligent feeding decision-making method executed by this system includes the following steps: Water quality parameters, farming status parameters, and biological parameters were collected from the factory-scale shrimp farming workshop on the same day, and the weight data in the biological parameters was estimated using the weight output by the aforementioned visual analysis module. The parameters are normalized, and the feature vector is input into the Bayesian-optimized core prediction model to obtain the first predicted value Q_ml; The number of shrimp in the aquaculture pond is estimated based on body weight, shrimp load, and water volume, and the second predicted value Q_rule is calculated by combining the preset feeding coefficient per unit body weight. Store Q_ml and Q_rule together with the historical records into the data layer as candidate feeding quantities for the feeding decision-making agent to call; Based on the first predicted value Q_ml, the second predicted value Q_rule, and the corresponding historical records, and receiving the natural language annotation R input by the user; If R is empty, then the recommended daily feeding amount and a brief description will be generated based on Q_ml; If R is not empty, then Q_ml, Q_rule, historical record fragments, entries related to R in the aquaculture knowledge base, and descriptions of leftover feed and environment from the visual analysis module are input into the large language model to obtain a comprehensive reasoning result that includes difference analysis, risk assessment, and adjustment logic. Extract the final recommended feeding amount Q_final and its floating range from the comprehensive reasoning results, and compare and display them with Q_ml and Q_rule to form a structured decision report; Record whether the user adopts Q_final and the actual amount of data fed to it, for use in system closed-loop optimization.

[0013] The present invention discloses the following technical effects: This invention achieves closed-loop intelligent management of "perception-analysis-decision-feedback" without increasing shrimp stress or changing traditional production habits. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the system architecture described in this invention; Figure 2 This is a schematic diagram of the automated material platform and its image acquisition structure described in this invention. Figure 3 This is a flowchart illustrating the multimodal voice interaction and visual feedback process of the automatic material station described in this invention. Figure 4 This is a schematic diagram of the visual analysis module described in this invention; Figure 5 This is a performance evaluation of the optimized regression model described in this invention and a generalization validation plot of Bayesian-LightGBM, wherein (a) is a Taylor plot summarizing the statistical performance of the optimized model; and (b) is a comparison between the optimal optimized model (Bayesian-LightGBM), the best unoptimized baseline (Bagging), and the original base model (LightGBM). (c) Validation of Bayesian-LightGBM generalization on an independent working dataset not used for training. The left panel shows the trend consistency between the true and predicted values, while the right panel shows a scatter plot with linear regression fitting; Figure 6This is a schematic diagram of the feeding decision-making intelligent agent module described in this invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0017] like Figures 1-6 As shown, this invention provides a shrimp farming feeding decision-making system based on multimodal information fusion and AI agents, specifically including the following: At the perception level, this invention automates the traditional manually operated feeding platform. When the platform rises to a set height above the water, an industrial camera captures images of shrimp and leftover feed. Based on instance segmentation and morphological feature extraction algorithms, the shrimp outline is automatically detected, the projected area and visual length are calculated, and the data are input into a lightweight ensemble regression model to achieve non-contact estimation of shrimp weight. Simultaneously, a classification model is used to identify "leftover feed / no leftover feed." When leftover feed is present, a large-scale visual model is invoked to generate a qualitative description of the distribution, accumulation degree, and environmental characteristics of the leftover feed.

[0018] At the data analysis level, this invention constructs a daily feeding prediction model based on multidimensional production data from a factory workshop covering over 10,400 m³ of water. Using 16 parameters as inputs, including water quality indicators, age of the shrimp, body weight (integrating visual estimation and manual data), shrimp carrying capacity, water volume, survival rate, average daily weight gain, and number of daily feedings, various machine learning and time series models are compared. Furthermore, Bayesian optimization and other methods are used to screen and tune the tree model ensemble, resulting in a Bayesian-LightGBM feeding prediction model with an R² of 96.5% and a MAPE of 6.93%. Its generalization performance (R² = 0.95) is validated on an independent factory periodic dataset.

[0019] At the decision-making level, this invention constructs an intelligent decision-making body for feeding amount, forming a dual prediction mechanism by combining the Bayesian-LightGBM model with empirical formulas based on pond volume, water volume, and shrimp weight, and introducing a large language model (LLM) as the "decision-making unit." When the user provides natural language annotations, the LLM integrates the dual prediction results, historical records, aquaculture knowledge base, and the leftover feed and environmental descriptions output by image recognition and the large visual model to evaluate and analyze the differences in feeding results, generating a structured decision report that includes quantitative feeding suggestions, adjustment ranges, and explanations of reasons; when no annotations are provided, the Bayesian-LightGBM prediction results are used by default.

[0020] Example: Figure 1 As shown, this invention discloses a shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agents, including: a data acquisition module, a visual analysis module, a farming production data analysis module, a feeding amount prediction module, a feeding decision-making intelligent agent module, and a human-computer interaction and execution module, which together form a closed loop of "perception-analysis-decision-feedback". Data Acquisition Module: The data acquisition module includes an automated feeding platform and a factory-style aquaculture production data acquisition unit. The automated feeding platform is controlled by a DC geared motor to raise and lower itself. When the platform reaches a preset height above the water, an industrial camera is triggered to capture video or images of the feeding platform area. The factory-style aquaculture production data acquisition unit is responsible for collecting daily production data such as water temperature, dissolved oxygen, pH, ammonia nitrogen, nitrite, as well as water exchange volume, age, weight, and shrimp stocking density.

[0021] Visual Analysis Module: The visual analysis module receives image data collected by the automated feeding station, performs shrimp detection and segmentation, morphological feature extraction, weight estimation, and "leftover feed / no leftover feed" identification on the images, and calls the large visual model to generate a semantic description of the leftover feed and the environment when "leftover feed" is detected.

[0022] Aquaculture production data analysis module: This module performs data cleaning, feature construction and standardization on daily production records from multiple workshops and multiple aquaculture cycles to form a training dataset containing at least 16 key aquaculture parameters, and integrates the weight estimation results output by the visual analysis module with the manual records.

[0023] Feeding amount prediction module: Based on the above training dataset, the feeding amount prediction module builds and optimizes multiple machine learning and time series models, selects and further fine-tunes the LightGBM model using the Bayesian optimization algorithm as the core feeding prediction model, and outputs the predicted value of daily feeding amount.

[0024] Feeding Decision Agent Module: This module uses a dual prediction mechanism consisting of built-in empirical rules and a Bayesian-LightGBM prediction model. It also calls a large language model to perform comprehensive reasoning on the dual prediction results, aquaculture knowledge base, historical records, and user natural language notes, and outputs structured feeding decision suggestions.

[0025] Human-computer interaction and execution module: This module receives real-time aquaculture data and user notes through a web interface or application programming interface, visualizes the decision suggestions generated by the system, and records the actual feeding behavior and results, providing feedback data for model iteration and rule adjustment.

[0026] Through the collaborative work of the above modules, the system in this embodiment can achieve a complete closed loop of automated feed platform monitoring, shrimp weight estimation, leftover feed identification, multi-source data modeling, and precise intelligent feeding decision-making in a factory farming environment.

[0027] Design of automated material handling systems and image acquisition: This section provides a more detailed explanation of the structure and operation of the automated material station (e.g., Figure 2 (as shown) Feeding platform support structure: The feeding platform support structure is constructed using perforated stainless steel angle steel, upon which a circular feeding tray with a diameter of approximately 30cm and an edge height of approximately 5cm is mounted to support the feed and shrimp. Of course, the shape (e.g., square) and size of the feeding platform can be adjusted according to the actual size of the aquaculture pond.

[0028] DC geared motor and control: In this embodiment, a motor with a torque of 100 kgf·cm is preferably connected to the feeding platform and driven by a multi-channel relay controlled by a serial port to realize the raising and lowering of the feeding platform in the water. This structure allows for easy configuration of the feeding platform's raising and lowering time, speed, and frequency through software programs to adapt to the needs of different ponds and aquaculture stages.

[0029] Limit switch protection mechanism: At least one limit switch is installed at the upper or lower end of the lifting stroke of the material platform to cut off the power supply to the motor when the material platform reaches the limit position, so as to prevent the motor from running dry and the mechanical structure from being damaged, thereby improving the safety and stability of the system operation.

[0030] Industrial camera configuration: An industrial camera is mounted on a fixed bracket above the material platform. In this embodiment, an industrial camera with at least 5 megapixels, a focal length of approximately 4.3mm, and a frame rate of approximately 30 frames per second is preferably used, and it is equipped with a dynamic wide mode to adapt to the complex lighting conditions in the workshop. Of course, with the improvement of image algorithms, other resolutions (such as 2 megapixels or 8 megapixels) can also be applied to this system as long as they can meet the clarity requirements for shrimp outline extraction.

[0031] Lifting and Acquisition Control Unit: The lifting and acquisition control unit is used to control the feeding platform to automatically rise to a set height above the water at a preset time or interval, and automatically trigger image or video acquisition upon reaching the set height. After completing the preset acquisition time (e.g., approximately 10 seconds), the feeding platform is then lowered back into the water to restore the shrimp's normal feeding environment.

[0032] An AI interaction module integrating speech recognition, speech synthesis, and a large-scale visual model: In this embodiment, the Doubao series of large-scale models (including streaming speech recognition, visual understanding, and speech synthesis models) are preferably used to achieve a complete closed-loop interaction process of voice-driven, visual analysis, and voice feedback. When the user issues a voice command, the system parses the user's command through the streaming speech recognition model. For control commands (such as 'rise'), the system directly drives the relay; for analysis commands, the system triggers continuous video frame acquisition, calls the large-scale visual model to generate environmental semantic descriptions, and finally provides feedback through speech synthesis.

[0033] Furthermore, this section provides a more detailed explanation of the internal structure and functions of the visual analysis module.

[0034] Instance segmentation submodule: This submodule prioritizes the YOLOv8-seg series instance segmentation models to perform shrimp target detection and mask segmentation on the acquired feed platform images, distinguishing at least two poses: "lateral shrimp" and "dorsal shrimp." The instance segmentation results provide accurate object boundary information for subsequent morphological feature extraction. It should be noted that this invention is not limited to this; other segmentation networks such as MaskR-CNN, YOLOv9, v10, or others of similar type are also applicable to this system.

[0035] Morphological Feature Extraction Submodule: This submodule calculates the projected area of ​​the shrimp based on a segmentation mask and extracts the shrimp's visual full length from the mask using skeleton refinement and principal axis extraction algorithms. Using the dimensions of known objects in the feed platform as a calibration benchmark, it converts pixel values ​​to actual length and area, obtaining morphological features with physical scale significance.

[0036] Weight estimation submodule: This submodule employs a lightweight gradient boosting tree regression model with visual total length and projected area as input features. It is trained on a dataset of 843 shrimp samples to construct a non-contact estimation model within the weight range of 0.1–25g. The estimation results can be used by the aquaculture production data analysis module and the feeding decision-making agent module to enrich the weight information in daily production records.

[0037] Leftover Material Recognition Submodule: This submodule prioritizes using the YOLOv8 classification model to perform binary classification of the feed platform image as "leftover material present / no leftover material present". This model is trained on 2476 labeled images and achieves a precision, recall, and F1 score of 1.0 on the validation set, reliably distinguishing the presence of visible leftover material and providing direct basis for adjusting the feeding strategy. It should also be noted that this invention is not limited to this; other similar segmentation networks such as Mask R-CNN, YOLOv9, v10, or others are also applicable to this system.

[0038] Leftover Material Semantic Description Submodule: When the leftover material identification submodule determines that "leftover material exists," it inputs typical frame images into the visual large-scale model for analysis, and outputs a natural language description including the location of the leftover material, the size of the area, whether it is in a state of accumulation, the relative severity, and environmental details such as water color and turbidity. This description can be presented intuitively in the decision report, helping aquaculture personnel quickly understand the on-site situation.

[0039] Furthermore, this section provides a more detailed explanation of the data sources and processing flow for the aquaculture production data analysis module: Data Source and Time Range: The data in this example comes from a factory-style shrimp farm, spanning multiple aquaculture cycles, with a total water volume of no less than 10,400 m³, covering daily records from February 2024 to August 2025. This dataset encompasses production data from different farms, water volumes, and batches of shrimp.

[0040] Data Field Composition: Each record contains multidimensional aquaculture environment and biological characteristics. In this embodiment, the following 17 typical parameters were selected as model inputs: date, system ID, water temperature (°C), dissolved oxygen (mg / L), pH, daily water exchange volume (m³), daily water exchange rate (%), ammonia nitrogen (mg / L), nitrite (mg / L), age at rearing (days), body weight (g), shrimp loading (kg / m³), water volume (m³), survival rate (%), average daily weight gain (g), number of daily feedings, and daily feed amount (kg). Except for the daily feed amount, the remaining 16 items are used as model input features. It should be noted that in practical applications, the input features can be tailored or expanded according to the sensor configuration; for example, only water temperature, dissolved oxygen, and pH can be used as environmental features.

[0041] Data cleaning and quality control: Missing values ​​were handled, outliers were removed, and consistency checks were performed on the above daily records to ensure that all fields were within physically reasonable limits, guaranteeing the continuity of the time series and the reliability of the values. After cleaning, more than 2,000 valid daily production records were obtained.

[0042] Feature standardization and dataset partitioning: Normalization methods (such as MinMaxScaler) are used to scale the input features and target variables respectively. Then, the samples are divided into training, validation and test sets in a ratio of approximately 64%:16%:20%, providing a foundation for subsequent multi-model training and evaluation.

[0043] This section details the model building, comparison, and optimization process for the feeding amount prediction module (e.g., Figure 5 (As shown).

[0044] Model Comparison Unit: This unit trains and evaluates no fewer than 36 regression models, including classic machine learning models, ensemble tree models, recurrent neural network variants, Kolmogorov-Arnold Networks variants, and Transformers. In addition, at least six time series models, including RandomForest, XGBoost, LSTM, GRU, TCN, and Transformer, are constructed for performance comparison and methodology validation.

[0045] Performance Evaluation Unit: The performance evaluation unit uses metrics such as mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²) to compare the performance of all the above models on independent test sets, and can present the results visually to assist in model selection.

[0046] Feature Correlation Analysis Unit: This unit calculates the Pearson correlation coefficient between each input feature and the daily feed amount, identifies variables with a high absolute correlation to the daily feed amount, such as age, body weight, shrimp load, average daily weight gain, pH, etc., constructs a comparative model based on these highly correlated features, and analyzes the impact of feature simplification on the model's predictive performance.

[0047] Model Selection and Optimization Unit: Based on performance evaluation results, four high-performing ensemble tree models—Bagging, XGBoost, CatBoost, and LightGBM—were selected. For each model, four metaheuristic optimization algorithms were used for hyperparameter search: Arithmetic Optimization (AOA), Particle Swarm Optimization (PSO), Genetic Algorithm (GA), and Runge-Kutta Optimization (RUN). Furthermore, several weighted average fusion and Stacking fusion models were constructed to further explore their performance potential.

[0048] The Bayesian-LightGBM core model unit: Based on multi-model comparison and pre-optimization, the LightGBM model is searched for key hyperparameters (such as tree depth, number of leaves, learning rate, etc.) using the Bayesian optimization method. Through multiple rounds of iterative optimization on the training and validation sets, the prediction accuracy is finally achieved on the test set with R² of approximately 96.5%, MAPE of approximately 6.93%, MSE of approximately 164 kg², RMSE of approximately 12.8 kg, and MAE of approximately 7.15 kg.

[0049] Meanwhile, validation on an independent workshop cycle dataset (sample size no less than 130 days) yielded an R² of approximately 0.95 and a MAPE of approximately 8.3%. Based on this, the model was selected as the core feeding prediction model for the system, used for subsequent feeding decisions. However, this does not preclude the possibility of using other gradient boosting decision tree (GBDT) models when the amount of data or computing resources changes.

[0050] This section further explains the layered architecture and workflow of the feeding decision-making agent module, as shown in the appendix. Figure 6 As shown: Data Layer: The data layer uses a cloud-based document database (such as Google Cloud Firestore, MongoDB Atlas, or Alibaba Cloud Table Storage) to store historical production records, model prediction results, and feeding execution results. It is also configured with a knowledge base consisting of shrimp farming technical specifications, experience rules, and standard operating procedures to provide data and knowledge support for upper-level models and decision-making.

[0051] Model layer: The model layer integrates the Bayesian-optimized LightGBM feeding prediction model, as well as the empirical feeding formula that calculates the number of shrimp based on the stock volume, water volume and shrimp weight, and combines it with the preset feeding coefficients divided by age and weight stage, forming a "dual prediction mechanism" based on measured data and empirical rules.

[0052] Decision-making layer: The decision-making layer invokes a large-scale language model, taking Bayesian-LightGBM prediction results, empirical formula prediction results, some historical data fragments, entries related to user notes in the aquaculture knowledge base, and descriptions of leftover feed and the environment from the visual analysis module as input. When the user provides notes, the system comprehensively analyzes the differences and rationality of the dual prediction results, and generates feeding suggestions by combining multimodal information such as leftover feed status, shrimp vitality, and water conditions; when the user does not provide notes, the system defaults to outputting the Bayesian-LightGBM prediction results as the recommended daily feeding amount.

[0053] Report Generation Unit: The report generation unit organizes the results output by the decision-making level into a structured decision report, which includes at least: a summary of the current production status, model predictions, empirical predictions, the final recommended feeding amount, the difference from the previous feeding amount, the reasons for adjustment, and operational suggestions, to help farmers understand and trace the decision-making logic.

[0054] This section provides a detailed explanation of the composition of the human-computer interaction and execution modules and their role in the closed loop: RESTful Application Programming Interface: This interface is developed based on Flask or an equivalent web framework. It is used to receive real-time water quality indicators, biological parameters (including visually estimated body weight), age information, and user-inputted notes from the workshop, and send them to the feeding amount prediction module and the feeding decision-making agent module for processing.

[0055] Web Front-End Interface: The web front-end interface is used to display historical and current environmental and production data curves, model prediction results, leftover feed detection status, semantic descriptions generated by the visual large model, and structured feeding decision reports output by the large language model, enabling aquaculture managers to intuitively understand the system's judgments and suggestions.

[0056] Interactive Feedback Unit: This unit records the adoption of intelligent suggestions by aquaculture personnel, the actual daily feeding amount, and subsequent aquaculture effects (such as shrimp growth, water quality changes, etc.), providing a data foundation for subsequent model retraining, parameter fine-tuning, or empirical rule adjustment, thereby continuously optimizing system performance.

[0057] The step-by-step process of the shrimp daily feeding prediction method in the system designed in this invention will be further described in detail.

[0058] Parameter Acquisition: Data on daily water temperature, dissolved oxygen, pH, daily water exchange volume, daily water exchange rate, ammonia nitrogen, nitrite, age at birth, body weight, shrimp stocking density, water volume, survival rate, average daily weight gain, and number of meals fed daily are collected from the factory-scale shrimp farming facility. For body weight, some or all of the estimated weight output from the visual analysis module can be used. It should be noted that although this embodiment lists 16 preferred parameters, in practical applications, the system does not require all parameters to be input. The model has the ability to handle missing features or make predictions based on feature subsets. Users can flexibly select all or some of the key indicators from the above parameters (e.g., only water temperature, dissolved oxygen, body weight, and age at birth) as model input, depending on the sensor configuration or data availability at the farming site.

[0059] Normalization and Model Input: The collected effective parameters are normalized according to the same standardization rules as in the model training phase. The feature vector is then input into the Bayesian-LightGBM core model to obtain the first predicted value Q_ml.

[0060] Empirical formula calculation: Based on the shrimp weight, shrimp load, and water volume, estimate the number of shrimp N in the aquaculture pond, and combine it with the preset unit weight feeding coefficient F divided by age and weight stage to calculate the second predicted value Q_rule.

[0061] Data storage and candidate output: Store Q_ml and Q_rule together with some historical records of the day or recent days into the data layer, and use them as candidate feeding quantities for the feeding decision-making agent to call.

[0062] Based on the system designed in this invention, the specific process of the intelligent shrimp feeding decision-making method is given as follows: Get the predicted value and notes: Get the aforementioned Q_ml, Q_rule and the corresponding historical records, and receive the natural language notes R entered by the user on the same day (R is empty if there are no notes).

[0063] Decision without comments: If R is empty, Q_ml will be directly output as the recommended daily feeding amount, and a brief description including the current main indicators, the value of Q_ml and the comparison with the previous day will be generated.

[0064] Comprehensive reasoning with notes: If R is not empty, Q_ml, Q_rule, some historical records, R-related entries in the aquaculture knowledge base, and leftover feed and environmental descriptions from the visual analysis module are input into the large language model, which outputs a comprehensive reasoning result that includes difference analysis, risk assessment, and adjustment logic.

[0065] Final Recommendations and Report Generation: Extract the final recommended feed amount Q_final and its fluctuation range from the comprehensive reasoning results, compare and display it with Q_ml and Q_rule, and provide the adjustment range and textual explanation of the reasons, forming a structured decision report returned to the user.

[0066] Results Recording: Records whether the user adopts Q_final and the actual amount of feed executed, providing a data source for subsequent iterative optimization of the model and rules, and enabling continuous improvement.

[0067] The following results were achieved through the design of this invention: 1. Low stress and compatible with existing processes: This invention modifies the mechanical and electrical control of the traditional feeding platform and collects images in a short time after the feeding platform rises out of the water. It does not require long-term changes to aquaculture environmental factors such as lighting and underwater structure, and does not increase the stress on shrimp caused by frequent harvesting or excessive disturbance. It is easy to implement in existing factory workshops.

[0068] 2. Non-contact weight estimation: This invention utilizes instance segmentation and morphological feature extraction to construct a regression model based on the shrimp's projected area and visual full length, enabling non-contact estimation of weight within the range of 0.1–25g. This reduces the labor intensity of manual sampling and weighing errors, providing more real-time weight information for precise feeding.

[0069] 3. Automatic identification and semantic description of leftover feed: This invention uses a classification model to identify "leftover feed / no leftover feed" in the feed platform image. When there is leftover feed, the large visual model is invoked to describe the distribution and accumulation of leftover feed, as well as environmental characteristics such as water color and turbidity, in natural language. This helps aquaculture personnel to quickly understand the on-site situation and make decisions based on system suggestions.

[0070] 5. High-precision daily feed prediction: Based on multi-dimensional production records covering multiple workshops and breeding cycles, this invention uses multi-model comparison and feature correlation analysis, and selects Bayesian-LightGBM as the core model. It achieves high R² and low MAPE performance indicators on the test set and independent workshop cycle dataset, indicating that the model has strong predictive ability and certain generalization ability.

[0071] 6. Dual Prediction and Intelligent Decision-Making Mechanism: This invention retains both empirical formula calculation path and machine learning prediction path. It uses a large language model to comprehensively analyze and interpret the differences between these two prediction results and on-site notes, thereby improving the credibility and interpretability of decision-making. At the same time, it avoids the adverse effects on production caused by the bias that may occur with a single model.

[0072] 7. Intelligent feeding management forming a closed loop: This invention records the actual feeding amount and subsequent production effect through human-computer interaction and execution modules, providing data support for subsequent model retraining, parameter adjustment and revision of empirical rules, thereby forming a closed loop of "prediction-execution-feedback-optimization" and continuously improving the intelligence level of the system.

[0073] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0074] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0075] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agents, characterized in that, include: Automatic feeding platform device, used to carry feed / shrimp in aquaculture ponds and achieve autonomous lifting and lowering; The camera acquisition and visual analysis module is used to acquire images of shrimp and leftover feed when the feeding platform is raised to a preset height above the water, and to perform image analysis based on segmentation, classification and large visual models; The shrimp weight estimation module is used to build a non-contact weight estimation model based on the projected area and visual length of the shrimp image. The leftover material identification and description module is used to determine whether there is leftover material on the material platform, and to generate a qualitative description using a large visual model when there is leftover material. The aquaculture production data analysis and feeding prediction module is used to build a feeding prediction model based on multi-dimensional production data from multiple workshops and covering more than 10,000 cubic meters of water. The feeding decision-making intelligent agent module is used to integrate multi-source information such as data-driven prediction results, empirical formulas, visual analysis results, and notes from farmers, and output structured feeding suggestions under the inference of a large language model. The human-computer interaction and execution module receives real-time aquaculture data and user notes through a web interface or application programming interface, visualizes decision suggestions, and compares and provides feedback with the actual feeding execution, thereby realizing dynamic adjustment of feeding strategies.

2. The shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agent according to claim 1, characterized in that: The automated material station specifically includes: The feeding platform structure is constructed of corrosion-resistant metal materials, and a feeding platform mesh tray with edge baffles is installed on it to support feed and shrimp. A DC geared motor is connected to the material platform via a mechanical structure and driven by a control unit to enable the material platform to rise and fall in the water. The limiting device is located at the end of the lifting stroke of the material platform to prevent structural damage; The industrial camera unit is installed above the material table and is equipped with a high-pixel imaging sensor and dynamic wide mode. The lifting and acquisition control unit is used to control the lifting of the material platform and trigger image acquisition; as well as, The AI ​​interaction module integrates voice interaction and visual analysis. It uses a voice recognition model to parse commands, a generative artificial intelligence visual model to analyze images, and a voice synthesis model to provide feedback, thereby achieving human-computer interaction.

3. The shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agent according to claim 2, characterized in that: The visual analysis module and weight estimation module are used to perform shrimp target detection and mask segmentation on the collected feed platform images using a deep learning instance segmentation model, distinguishing at least two types of postures: "side shrimp" and "dorsal shrimp". The projected area of ​​the shrimp is calculated based on the segmentation mask, and the full visual length of the shrimp is extracted from the mask using skeleton thinning and principal axis extraction algorithms. A machine learning-based regression model is used, with the total visual length and projected area as input features, to achieve non-contact estimation of shrimp weight. The estimation results are then used by subsequent modules. An image classification model is used to perform binary classification of the material tray image into "with remaining material / without remaining material"; When the image is identified as having "leftover material", a large visual model is invoked to analyze the image and output a natural language description including the location of the leftover material, the size of the area, the state of accumulation, the relative severity, and details of the aquatic environment.

4. The shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agent according to claim 3, characterized in that: The aquaculture production data analysis module is used to process the acquired daily production records, handle missing values, remove outliers, and verify consistency to obtain valid daily production records. It also uses a normalization method to scale the input features and target variables, and divides the data into training, validation, and test sets for model training and evaluation. The records include date, system ID, water temperature, dissolved oxygen, pH, daily water exchange volume, daily water exchange rate, ammonia nitrogen, nitrite, age of the animals, body weight, shrimp load, water volume, survival rate, average daily weight gain, number of meals fed per day, and daily feed amount.

5. The shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agent according to claim 4, characterized in that: The feeding amount prediction module includes: The model comparison unit is used to train and evaluate various regression models and time series models. The performance evaluation unit uses error analysis metrics to compare model performance. The feature correlation analysis unit calculates the correlation between input features and daily feeding amount, and identifies a subset of highly correlated features. The model selection and optimization unit selects the best-performing ensemble tree model based on the evaluation results and uses a metaheuristic optimization algorithm to search for hyperparameters. as well as, The core model unit uses the Bayesian optimization algorithm to iteratively optimize the hyperparameters of the ensemble tree model and construct the core model for feeding prediction.

6. The shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agent according to claim 5, characterized in that: The feeding decision-making intelligent agent module includes: The data layer uses a database to store historical production records, model prediction results, and feeding execution results, and is configured with a knowledge base consisting of aquaculture technical specifications and empirical rules. The model layer integrates a core feeding prediction model optimized by Bayesian methods and an empirical feeding formula based on pond stock, water volume and shrimp weight, forming a dual prediction mechanism. The decision-making level calls upon a large-scale language model, taking the prediction results of the core model, the prediction results of empirical formulas, historical fragments, entries from the aquaculture knowledge base, and natural language notes from the visual analysis module and users as inputs, to analyze the differences and rationality of the dual prediction results, and to generate feeding suggestions by combining multimodal information. as well as, The report generation unit organizes the results output by the decision-making level into a structured decision report, including model predictions, empirical predictions, final recommended feed amount, reasons for adjustment, and operational recommendations.

7. The shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agent according to claim 6, characterized in that: The human-computer interaction and execution module includes: Application programming interface (API) is used to receive real-time water quality indicators, biological parameters, and user notes from the workshop and send them to the prediction and decision-making module. The web front-end interface is used to display environmental and production data, model prediction results, residual material detection status, semantic descriptions generated by the visual large model, and structured feeding decision reports. as well as, The interactive feedback unit is used to record the adoption of intelligent suggestions by farmers and the actual amount of feeding, providing data support for subsequent model iterations.

8. The shrimp farming feeding decision-making system based on multimodal information fusion and AI intelligent agent according to claim 7, characterized in that: The precise intelligent feeding decision-making method for shrimp implemented through this system includes the following steps: Water quality parameters, farming status parameters, and biological parameters were collected from the factory-scale shrimp farming workshop on the same day, and the weight data in the biological parameters was estimated using the weight output by the aforementioned visual analysis module. The parameters are normalized, and the feature vector is input into the Bayesian-optimized core prediction model to obtain the first prediction value Q_ml; The number of shrimp in the aquaculture pond is estimated based on body weight, shrimp load, and water volume, and the second predicted value Q_rule is calculated by combining the preset feeding coefficient per unit body weight. Store Q_ml and Q_rule together with the historical records into the data layer as candidate feeding quantities for the feeding decision-making agent to call; Based on the first predicted value Q_ml, the second predicted value Q_rule, and the corresponding historical records, and receiving the natural language annotation R input by the user; If R is empty, then the recommended daily feeding amount and a brief description will be generated based on Q_ml; If R is not empty, then Q_ml, Q_rule, historical record fragments, entries related to R in the aquaculture knowledge base, and descriptions of leftover feed and environment from the visual analysis module are input into the large language model to obtain a comprehensive reasoning result that includes difference analysis, risk assessment, and adjustment logic. Extract the final recommended feeding amount Q_final and its floating range from the comprehensive reasoning results, and compare and display them with Q_ml and Q_rule to form a structured decision report; Record whether the user adopts Q_final and the actual amount of data fed to it, for use in system closed-loop optimization.

Images