Agricultural multi-source sensing knowledge identification and integrated decision system based on sugarcane_expert_v1 large model

Abstract

This invention relates to an integrated decision-making system for agricultural multi-source sensing, identification, and monitoring based on the Sugarcane_Expert_v1 large-scale model, and relates to agricultural intelligent detection technology. This system achieves deep integration of AI recognition and business management, supports multiple data input modes, provides an excellent interactive experience, and has good engineering scalability. Through self-developed fine-tuning of the large-scale model Sugarcane_Expert_v1:1.5B, the system can not only "see" pests and diseases, but also "explain" the causes and "provide" suggestions, greatly enhancing the value of decision support and thus endowing the system with intelligent decision-making capabilities.

Description

Technical Field

[0001] This invention relates to agricultural intelligent detection technology, and specifically to an integrated decision-making system for agricultural multi-source sensing, identification, and monitoring based on the Sugarcane_Expert_v1 large model. Background Technology

[0002] In current agricultural production, the identification and control of pests and diseases mainly rely on manual inspections and judgments based on experience. This method has many drawbacks: the identification results are highly subjective, inefficient, unable to cover large areas of farmland, unable to provide timely warnings, and lack process recording. Although the widespread use of mobile devices and cameras has made image / video data collection convenient, data collection alone is insufficient to form an effective management loop. Existing solutions often scatter functions such as "identification—result display—record retention—knowledge base management—business management" across different tools or platforms, resulting in information silos.

[0003] Meanwhile, frontline growers and managers urgently need a way to quickly obtain professional answers in natural language, such as asking about the diagnostic criteria for specific pests and diseases, the meaning of environmental indicators, or specific prevention and control recommendations. However, general-purpose language models often lack vertical domain expertise, resulting in inaccurate and inconsistent answers that may not align with users' actual production practices.

[0004] In addition, existing systems generally have the following shortcomings: 1. Capability fragmentation: The separation of AI model inference services from backend business systems (such as databases and file management) makes it difficult to uniformly archive and trace the recognition results.

[0005] 2. Limited input scenarios: Typically only supports image uploads, with insufficient support for video streams and real-time camera input, or complex engineering implementation.

[0006] 3. Poor interactive experience: When processing video or live streams, users lack visual feedback on the processing progress, resulting in a poor experience during the waiting process.

[0007] 4. Lack of management functions: There is a lack of unified management and statistical analysis functions for core business data such as pest and disease knowledge base, identification history, and procurement inventory. Summary of the Invention

[0008] To address the aforementioned issues, this invention provides an integrated decision-making system for agricultural multi-source sensing, identification, and monitoring based on the Sugarcane_Expert_v1 large model. This system achieves deep integration of AI identification and business management by constructing a front-end and back-end separated, multi-service collaborative architecture, supporting multiple data input modes, providing an excellent interactive experience, and exhibiting good engineering scalability.

[0009] This invention provides an integrated decision-making system for agricultural multi-source sensing, identification, and monitoring based on the Sugarcane_Expert_v1 large model, comprising: a front-end service layer, a back-end service layer, and a data storage layer; the front-end service layer includes a front-end management terminal, the back-end service layer includes AI inference services and business back-end services, and the data storage layer includes file storage services and a database; The front-end management terminal provides a user interface and accepts multi-mode data input. The AI ​​inference service takes the output data from the front-end management terminal as input, returns inference results to the front-end management terminal, and outputs inference results to the file storage service. The business back-end service connects to the front-end management terminal via a back-end API, and works collaboratively with the AI ​​inference service through the file service and the back-end API. The file storage service stores the raw data output by the business back-end service and the inference results output by the AI ​​inference service. The database stores the structured data output by the business back-end service.

[0010] The above-mentioned raw data is the unstructured data output by the business backend service. During the research process, the inventors found the following technical problems in the existing technology: 1. The most important and critical technical problem: There is a lack of a system that can deeply integrate AI pest and disease identification capabilities with agricultural production and management. Existing solutions either only provide identification functions or have independent management functions, failing to form a complete closed loop from "data collection → intelligent identification → result visualization → knowledge Q&A → business management", resulting in the identification results being unable to effectively guide production and management decisions, reducing overall efficiency and value. 2. Secondary problems: (1) Insufficient support for multimodal input: It cannot flexibly adapt to various data input sources such as pictures, videos, and real-time cameras, limiting the application scenarios of the system. (2) Interactive experience needs improvement: When processing time-consuming tasks (such as video analysis), there is a lack of a real-time progress feedback mechanism, affecting the user experience. (3) Lack of intelligent decision support: There is a lack of intelligent Q&A modules based on domain knowledge, which cannot provide users with practical and operable diagnostic suggestions and decision-making basis. (4) Poor system scalability: It is difficult to update the content of models, crop categories, knowledge bases, etc., making it difficult to adapt to the ever-changing needs of agricultural production. Therefore, the inventors propose the above-mentioned integrated decision-making system for multi-source sensing, identification, and monitoring in agriculture. This system can achieve deep integration of AI identification and business management, supports multiple data input modes, provides an excellent interactive experience, and has good engineering scalability.

[0011] In one embodiment, the reasoning result includes decision suggestions and / or detection results; The AI ​​inference service includes a visual algorithm module and a language dialogue module. The visual algorithm module is used to output detection results and includes a YOLOv11 model. The language dialogue module is used to output decision suggestions and includes a Sugarcane_Expert_v1 large model. The YOLOv11 model is used as input for the output data of the front-end management terminal. The front-end management terminal calls the API of the Sugarcane_Expert_v1 large model and combines it with the output data of the YOLOv11 model to form decision suggestions.

[0012] The aforementioned visual algorithm module is used to identify and label disease information, while the language dialogue module provides decision-making suggestions by combining the disease information output by the visual algorithm module. In practical application, the aforementioned integrated agricultural multi-source sensing, identification, monitoring, and decision-making system uploads disease samples to the front-end management terminal, which are then detected by the YOLOv11 model, outputting disease labels and confidence levels. The front-end management terminal then calls the API of the Sugarcane_Expert_v1 large model to combine the disease labels and confidence levels to output decision-making suggestions. Through the Sugarcane_Expert_v1 large model, this system not only "sees" pests and diseases but also "explains" the causes and "gives" suggestions, greatly enhancing the value of decision support and thus endowing the system with intelligent decision-making capabilities.

[0013] In one embodiment, the vision algorithm module uses Ultralytics YOLO + OpenCV + Socket.IO, and the framework of the vision algorithm module is a Python Web framework.

[0014] In one embodiment, the framework of the vision algorithm module is Flask or FastAPI.

[0015] In one embodiment, the method for constructing the Sugarcane_Expert_v1 large model includes the following steps: constructing an instruction fine-tuning dataset, performing efficient LoRA / QLoRA parameter fine-tuning on the LLM architecture in a GPU environment to form the Sugarcane_Expert_v1 large model, and deploying the Sugarcane_Expert_v1 large model locally, providing an HTTP interface.

[0016] In one embodiment, the deployment is implemented via Ollama, vLLM, or Text Generation Inference.

[0017] In one embodiment, the front-end management terminal adopts the Vite + TypeScript + TDesign technology stack, and the front-end framework of the front-end management terminal is Vue3, React or Angular.

[0018] In one embodiment, the front-end management terminal adopts the Vue3 + Vite + TypeScript + TDesign technology stack.

[0019] In one embodiment, the business backend service adopts the Spring Boot + MyBatis-Plus + MySQL technology stack.

[0020] In one embodiment, the file storage service is a local file storage service, a MinIO object storage service, or an Alibaba Cloud OSS object storage service; The database is either a MySQL database or a PostgreSQL database.

[0021] When the database mentioned above is a PostgreSQL database, MySQL in the business backend service technology stack can be used in a compatible manner.

[0022] This invention also provides an analysis and decision-making method for agriculture, comprising the following steps: collecting multi-mode data, inputting the multi-mode data into the front-end service layer of the integrated agricultural multi-source sensing, identification, and monitoring decision-making system, analyzing the data, generating decision suggestions and / or detection results, and displaying the decision suggestions and / or detection results through the front-end service layer and interacting with the user.

[0023] In one embodiment, the multi-mode data includes at least two of the following: image data, video data, or real-time camera data.

[0024] This invention also provides a method for the prevention and control of pests and diseases, which uses the aforementioned integrated decision-making system for agricultural multi-source sensing, identification, and monitoring or the aforementioned analysis and decision-making method to generate decision suggestions and / or detection results, and takes prevention and control measures based on the decision suggestions and / or detection results.

[0025] Compared with the prior art, the present invention has the following beneficial effects: The integrated decision-making system for agricultural multi-source sensing, identification, and monitoring based on the Sugarcane_Expert_v1 large model of this invention can achieve deep integration of AI recognition and business management, support multiple data input modes, provide an excellent interactive experience, and have good engineering scalability. Through the self-developed Sugarcane_Expert_v1 large model, the system can not only "see" pests and diseases, but also "explain" the causes and "give" suggestions, greatly enhancing the value of auxiliary decision-making and thus endowing the system with intelligent decision-making capabilities. Attached Figure Description

[0026] Figure 1 This is a diagram showing the overall architecture of the decision-making system in Example 1; Figure 2 This is a time sequence diagram of image recognition in the decision-making system of Example 1; Figure 3 This is a flowchart of the video recognition processing of the decision-making system in Example 1; Figure 4 This is a schematic diagram of the data model / table structure of the decision system in Example 1; Figure 5 This is a schematic diagram of the monitoring data summary page of the decision system in Example 1; Figure 6 This is a schematic diagram of the overview instrument page of the decision system in Example 1; Figure 7 , Figure 8 , Figure 9 These are schematic diagrams showing the detection results after the system processes image data, video data, and real-time camera data, respectively. Figure 10 This is a schematic diagram of the pest and disease knowledge base page of the decision-making system in Example 1; Figure 11 , Figure 12 , Figure 13 These are schematic diagrams illustrating the detection and recording data of image data, video data, and real-time camera data from the decision-making system in Example 1. Figure 14 , Figure 15 , Figure 16 These are schematic diagrams of the agricultural input procurement data page, inventory management data page, and user list page of the decision-making system in Example 1. Figure 17 This is a schematic diagram of the reasoning results displayed to the user through the front-end service layer in the decision-making system of Example 1. Detailed Implementation

[0027] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0029] Unless otherwise specified, all reagents, materials, and equipment used in this embodiment are commercially available; unless otherwise specified, all test methods are conventional test methods in this field.

[0030] definition: YOLOv11: The latest iteration of a real-time object detection algorithm based on Ultralytics YOLO.

[0031] LoRA / QLoRA: A technique for efficient fine-tuning of parameters in large models.

[0032] Sugarcane_Expert_v1 Large Model: A large language model based on LLM architecture, designed for professional papers in the agricultural field, with some parameters fine-tuned using LoRA, and has 1.5 billion parameters.

[0033] Ollama: An open-source framework for running large language models locally.

[0034] WebSocket: A protocol for full-duplex communication over a single TCP connection, used to implement real-time progress tracking.

[0035] Example 1 An integrated decision-making system for agricultural multi-source sensing, identification, and monitoring based on the Sugarcane_Expert_v1 large model.

[0036] I. System Overall Architecture.

[0037] The system consists of the following core components: a front-end service layer, a back-end service layer, and a data storage layer. The front-end service layer includes a front-end management terminal, the back-end service layer includes AI inference services and business back-end services, and the data storage layer includes file storage services and a database.

[0038] 1. Front-end Management Interface: Utilizing a Vite + TypeScript + TDesign technology stack, the front-end management interface can be structured using Vue3, React, or Angular. In this embodiment, Vue3 is used. The front-end management interface provides user login, a visual dashboard, pest and disease database management, multi-mode detection entry points (images / videos / cameras), identification record query, and procurement and inventory management interfaces. The front-end calls the back-end API via HTTP and accesses the AI ​​inference service's streaming interface through a proxy, while simultaneously receiving processing progress via Socket.IO.

[0039] 2. Business Backend Services: Utilizing Spring Boot + MyBatis-Plus + MySQL, it provides unified REST API and file service capabilities. Core functionalities include: user permission management, pest and disease database CRUD operations, identification record management, greenhouse environmental information management, procurement and inventory ledger management, file upload / download, and URL generation. The backend can selectively forward AI inference requests and automatically store them in the database.

[0040] 3. AI Inference Service: This includes a visual algorithm module and a language dialogue module. The visual algorithm module uses Ultralytics YOLO + OpenCV + Socket.IO, and its framework is a Python web framework, which can use Flask or FastAPI. In this embodiment, Flask is used. The language dialogue module includes the Sugarcane_Expert_v1 large model. The aforementioned visual algorithm module is used to identify and label disease information and output detection results. The language dialogue module provides decision suggestions by combining the disease information output by the visual algorithm module.

[0041] In practical application, this integrated decision-making system involves the front-end management end uploading disease samples, which are then detected by the YOLOv11 model, outputting disease labels and confidence scores. The front-end management end then calls the API of the Sugarcane_Expert_v1 large model to combine the disease labels and confidence scores to output decision suggestions. The AI ​​inference service provides a model weight list, image prediction, video stream prediction, real-time camera prediction, and a stop interface. When processing video / camera tasks, progress percentages are pushed in real time via WebSocket.

[0042] 4. Data and File Storage: This includes file storage services and a database, which can be a MySQL or PostgreSQL database. In this embodiment, a MySQL database is used. The MySQL database stores structured data (users, pests and diseases, records, ledgers, etc.), while the file service stores raw images / videos (i.e., unstructured data) and AI-generated result files (annotated images, transcoded videos), and provides a unified access URL.

[0043] The overall architecture diagram of the decision-making system in this embodiment is as follows: Figure 1 As shown.

[0044] II. System Workflow.

[0045] 2.1 Integrated process of multi-source sensing and recognition: The core of the system lies in its unified processing capability for "multi-source sensing" data. Whether it is a static image, a dynamic video file, or a real-time camera stream, the system treats it as "sensing data" and initiates a unified "recognition-monitoring-decision" chain.

[0046] Image recognition: Front-end uploads image to back-end file service -> Front-end calls AI service image prediction interface (passing in image URL, crop type, model weights, etc.) -> AI service downloads image, runs YOLOv11 inference, and generates labeled image -> AI service uploads the result image back to back-end file service -> AI service returns results (status, result image URL, label list, confidence score, time taken) to front-end -> Front-end displays results and records them in recognition record table.

[0047] Video recognition: Front-end uploads or provides video URL -> Front-end requests AI service through streaming interface -> AI service reads video frame by frame, infers frame by frame, packages labeled frames into multipart stream and returns preview in real time -> AI service calculates and pushes transcoding progress -> AI service outputs final video file and uploads to back-end -> AI service writes video recognition record.

[0048] Camera recognition: Front-end initiates a request -> AI service opens the local camera, performs frame-by-frame inference, and returns a real-time video stream -> AI service pushes status / progress -> Front-end can call the stop interface to interrupt -> AI service saves the result video and uploads and writes it to the camera recognition record.

[0049] Figure 2 , 3 Figures 4 and 5 are respectively a sequence diagram for image recognition, a flowchart for video recognition processing, and a schematic diagram of the data model / table structure of the decision-making system; Figure 7 , Figure 8 , Figure 9These are schematic diagrams showing the detection results after the system processes image data, video data, and real-time camera data, respectively. Figure 11 , Figure 12 , Figure 13 These are schematic diagrams of the detection and recording data of image data, video data, and real-time camera data of the decision system in Example 1.

[0050] 2.2 Sugarcane_Expert_v1 Large Model-Driven Intelligent Decision Center

[0051] The most core innovation of this system lies in using the self-developed Sugarcane_Expert_v1 large model as the "decision center" of the entire system, rather than a simple auxiliary module. It not only receives recognition results, but also integrates multi-source perception data and business data to provide users with in-depth decision support.

[0052] Data Preparation: Based on professional papers in the agricultural field, the system's pest and disease database, prevention and control manuals, environmental indicator descriptions, and other materials, a fine-tuning dataset of instructions was constructed. The sample format is "instruction + context input + standard response". In this embodiment, the professional journals used are: *China Sugar Crops*, *Southern Agricultural Journal*, and *Guangxi Agricultural Sciences*. The industry standards and prevention and control manuals used are: *Technical Regulations for Green Prevention and Control of Sugarcane Pests and Diseases*, *Chinese Journal of Sugarcane Pathology*, *Prediction and Forecasting of Crop Pests and Diseases*, and *Guiding Opinions on the Prevention and Control of Major Sugarcane Pests and Diseases in Guangxi*.

[0053] Fine-tuning process: In a GPU environment, tools such as LLaMA Factory are used to fine-tune the LoRA / QLoRA parameters of the large language model of the LLM architecture according to the above instructions, forming a special question-answering model for agricultural scenarios.

[0054] Deployment and Invocation: Deploy the model locally via Ollam and provide an HTTP interface (e.g., http: / / localhost:11434 / api / chat).

[0055] Application Integration: The front-end encapsulates the streaming dialogue interaction with the model in the environment details page and global dialog box. When a user asks a question, the system can submit the current pest and disease monitoring results (crop, tag, confidence level), environmental sensor data, and relevant pest and disease database entries as contextual prompts to the model. The model then generates diagnostic suggestions, environmental interpretations, and operational prompts, achieving an integrated presentation of "detection results + knowledge explanation + operational suggestions".

[0056] Figure 17 This is a diagram illustrating the reasoning results displayed to users through the front-end service layer of the decision-making system.

[0057] 2.2 Core Algorithm and Key Engineering Implementation Points Multi-crop adaptation: The AI ​​service provides a weighted file list interface to support pest and disease monitoring models for various crops such as corn, rice, and wheat.

[0058] Tag mapping and normalization: The built-in "crop type -> tag set" mapping table converts the index output by YOLO into specific pest and disease names, and marks the "tag + confidence" combination on the result image.

[0059] Confidence level visualization: The original confidence level of 0 to 1 is mapped to the range of 90% to 99.99%, making it easier for non-experts to understand.

[0060] YOLOv11 optimizations: During the training phase, strategies such as transfer learning, mixed precision, and automatic batch sizing are adopted; during the inference phase, images are scaled to a fixed size and half precision is used, and video / camera inference is performed frame by frame.

[0061] Video processing: OpenCV is used to write the intermediate format, and then FFmpeg is used to transcode it into MP4 to ensure browser compatibility.

[0062] III. Advantages and technical effects of this system.

[0063] 1. Achieved deep integration of AI recognition and business management: Through the "file service + record entry" mechanism, the recognition results are seamlessly integrated into the production management process, forming a complete closed loop that is traceable and analyzable.

[0064] 2. Supports multiple data input modes: Images, videos, and real-time camera input methods are all integrated into the system, meeting the needs of various work scenarios from static inspection to dynamic monitoring.

[0065] 3. Provides an excellent interactive experience: Real-time push of processing progress via WebSocket significantly improves usability and user satisfaction in video and camera recognition scenarios.

[0066] 4. The system is endowed with intelligent decision-making capabilities: Through the self-developed Sugarcane_Expert_v1 large model, the system can not only "see" pests and diseases, but also "explain" the causes and "give" suggestions, which greatly enhances the value of auxiliary decision-making.

[0067] 5. Excellent engineering scalability: Model weights, crop categories, and knowledge bases can all be updated and expanded independently, and the system architecture allows for easy addition of new functional modules in the future (such as early warning rules, multi-tenant support, mobile terminals, etc.).

[0068] Example 2 An analytical decision-making method for agriculture.

[0069] The analysis and decision-making method includes the following steps: using multi-mode data, such as at least two of image data, video data, or real-time camera data, inputting the multi-mode data into the front-end service layer of the integrated decision-making system for agricultural multi-source sensing, identification, and monitoring in Example 1, analyzing it, forming decision suggestions and / or detection results, displaying the above decision suggestions and / or detection results through the front-end service layer, and interacting with the user.

[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. An integrated decision-making system for agricultural multi-source sensing, identification, and monitoring based on the Sugarcane_Expert_v1 large-scale model, characterized in that, include: The system comprises a front-end service layer, a back-end service layer, and a data storage layer; the front-end service layer includes a front-end management terminal, the back-end service layer includes AI inference services and business back-end services, and the data storage layer includes file storage services and a database. The front-end management terminal provides a user interface and accepts multi-mode data input. The AI ​​inference service takes the output data from the front-end management terminal as input, returns inference results to the front-end management terminal, and outputs inference results to the file storage service. The business back-end service connects to the front-end management terminal via a back-end API, and works collaboratively with the AI ​​inference service through the file service and the back-end API. The file storage service stores the raw data output by the business back-end service and the inference results output by the AI ​​inference service. The database stores the structured data output by the business back-end service.

2. The integrated decision-making system for agricultural multi-source sensing, identification, and monitoring according to claim 1, characterized in that, The reasoning results include decision suggestions and / or detection results; The AI ​​inference service includes a visual algorithm module and a language dialogue module. The visual algorithm module is used to output detection results and includes a YOLOv11 model. The language dialogue module is used to output decision suggestions and includes a Sugarcane_Expert_v1 large model. The YOLOv11 model is used as input for the output data of the front-end management terminal. The front-end management terminal calls the API of the Sugarcane_Expert_v1 large model and combines it with the output data of the YOLOv11 model to form decision suggestions.

3. The integrated decision-making system for agricultural multi-source sensing, identification, and monitoring according to claim 2, characterized in that, The vision algorithm module uses Ultralytics YOLO + OpenCV + Socket.IO, and its framework is a Python web framework.

4. The integrated decision-making system for agricultural multi-source sensing, identification, and monitoring according to claim 2, characterized in that, The method for constructing the Sugarcane_Expert_v1 large model includes the following steps: constructing an instruction fine-tuning dataset; efficiently fine-tuning the LoRA / QLoRA parameters of the LLM architecture in a GPU environment to form the Sugarcane_Expert_v1 large model; and deploying the Sugarcane_Expert_v1 large model locally, providing an HTTP interface.

5. The integrated decision-making system for agricultural multi-source sensing, identification, and monitoring according to claim 1, characterized in that, The front-end management terminal adopts the Vite + TypeScript + TDesign technology stack, and the front-end framework of the front-end management terminal is Vue3, React or Angular.

6. The integrated decision-making system for agricultural multi-source sensing, identification, and monitoring according to claim 1, characterized in that, The business backend service adopts the Spring Boot + MyBatis-Plus + MySQL technology stack.

7. The integrated decision-making system for agricultural multi-source sensing, identification, and monitoring according to claim 1, characterized in that, The file storage service is a local file storage service, a MinIO object storage service, or an Alibaba Cloud OSS object storage service. The database is either a MySQL database or a PostgreSQL database.

8. An analytical decision-making method for agriculture, characterized in that, Includes the following steps: Collect multi-mode data, input the multi-mode data into the front-end service layer of the integrated decision-making system for agricultural multi-source sensing, identification and monitoring as described in any one of claims 1-7, analyze the data, form inference results, and display the inference results and interact with the user through the front-end service layer.

9. The analysis and decision-making method according to claim 8, characterized in that, The multi-mode data includes at least two of the following: image data, video data, or real-time camera data.

10. A method for controlling pests and diseases, characterized in that, The decision-making system for integrated agricultural multi-source sensing, identification, and monitoring, as described in any one of claims 1-7, or the analysis and decision-making method as described in any one of claims 8-9, is used to generate decision recommendations and / or detection results, and prevention and control measures are taken based on the decision recommendations and / or detection results.

Images