Low-cost fruit and vegetable detection system and method based on cloud-local collaborative architecture

Abstract

The invention provides a low-cost fruit and vegetable detection system and a low-cost fruit and vegetable detection method based on a cloud-local collaborative architecture. The low-cost fruit and vegetable detection method comprises the steps of using the cloud-local collaborative architecture as a core, deploying or calling an existing super-large-scale pre-training model at a cloud end, pre-training by using mass multi-modal pre-training data of a large model, and performing strong feature extraction and reasoning capability. Through prompt word engineering, tasks such as advanced feature extraction and complex mode recognition of fruit and vegetable images are completed. The user can directly call the cloud model without self-training or fine tuning; a lightweight model is deployed on edge equipment, and optimization design is carried out for a specific fruit and vegetable quality detection scene. The local lightweight model completes specific tasks such as pest and disease damage detection and defect detection; a local lightweight model adopts a LoRA fine tuning technology, and a small number of parameters are finely tuned only for a specific fruit and vegetable quality detection task. Based on the technical scheme of the invention, stable performance is ensured, and the large model training and deployment cost is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the cross - field of artificial intelligence and agricultural information technology, and particularly to a low - cost fruit and vegetable detection system and method based on a cloud - local collaborative architecture. Background Art

[0002] As the core technical node of the modern agricultural whole - chain quality control system, the application scenarios of fruit and vegetable quality detection have extended from real - time orchard grading to diversified fields such as dynamic warehouse monitoring and logistics quality traceability. However, traditional detection technologies have gradually exposed systematic defects in industrial applications: First, the manual visual inspection method has significant subjective errors. Affected by factors such as experience differences, visual fatigue, and environmental lighting of inspectors, the repeatability of quality determination is reduced, especially in the evaluation of phenotypic characteristics such as fruit color and defect recognition, cognitive biases are likely to occur. Second, the destructive physical and chemical detection method needs to obtain core parameters through invasive operations such as slicing sampling and component extraction. This technical path not only destroys the integrity of the sample, resulting in the loss of the commercial value of the detection object, but also prolongs the detection cycle due to the series operation of multiple processes, forming a bottleneck in the timeliness of the processing flow; that is, the traditional detection system is overly dependent on the professional qualities of operators. The complex instrument calibration process and multi - parameter analysis requirements lead to significant standardization obstacles in technology promotion. These methods are difficult to meet the technical requirements of modern smart agriculture for non - destructive, high - throughput, and whole - chain detection.

[0003] Currently, the AI large - model technology built based on a parameter scale of hundreds of billions and multi - modal pre - training data is profoundly reconstructing the technical paradigm of agricultural intelligent transformation. Such models show strong representation capabilities for complex agricultural scenarios by integrating multi - source perception data such as visible light, near - infrared spectra, and acoustic features. It is worth noting that the mainstream technical route generally adopts the full - parameter fine - tuning strategy to achieve domain adaptation. A typical example is the "Xiong Xiaonong" agricultural large - model, which performs full - parameter fine - tuning based on the DeepSeek - R1 - 671B architecture (with 671 billion parameters).

[0004] However, this technical path faces significant computational intensity bottlenecks: Full - parameter fine - tuning needs to complete the full - dimensional optimization of the parameter space on a distributed computing cluster, and a single training task consumes more than a thousand GPU - hours of computing resources. Industry research shows that although this mode can improve the accuracy of the model in tasks such as fruit disease recognition, the hardware investment cost is equivalent to 76% of the average annual digital budget of a medium - sized agricultural enterprise, forming a key obstacle to the popularization of technology.

[0005] This high-threshold technical path has put small and medium-sized agricultural enterprises (with IT budgets generally less than 30% of the industry's digital transformation costs), testing institutions, and supermarkets in a dilemma: The basic prompt engineering solution is limited by the particularity of agricultural scenarios (such as the morphological diversity of pests and diseases, and the continuous change characteristics of fruit maturity), while the local full-parameter fine-tuning solution requires hardware investment equivalent to more than 40% of the average annual IT budget, severely restricting the process of technology popularization. Summary of the Invention

[0006] In view of the problems of high cloud computing power costs and insufficient local model accuracy in the above-mentioned existing technologies, this application proposes a low-cost fruit and vegetable detection system and method based on a cloud-local collaborative architecture. It adopts a cloud-local collaborative architecture for an intelligent solution for fruit and vegetable quality detection. Its core technical means lies in combining a mature large model in the cloud with a lightweight inference model at the local edge to achieve an efficient and low-cost detection system. The cloud large model is responsible for handling general tasks. Through prompt engineering, it makes full use of its powerful pre-training ability and parameter scale; while the locally deployed lightweight inference model is optimized for specific scenarios and only requires a small amount of computing resources to complete the inference task. To further reduce costs, the present invention adopts an efficient fine-tuning strategy, only performing LoRA fine-tuning on the local model, avoiding full-parameter adjustment of the entire large model. This design significantly reduces training time and hardware requirements, enabling small and medium-sized agricultural enterprises, testing institutions, and supermarkets to achieve customized applications with relatively low investment. Through the collaborative work of the cloud and the local, while maintaining high performance, this solution promotes the popularization and development of agricultural large model technology, providing an innovative path for intelligent transformation in resource-constrained environments. The present invention is applicable to application scenarios that reduce the training cost of large models through prompt engineering and LoRA fine-tuning technology while ensuring inference quality.

[0007] In one embodiment, a low-cost fruit and vegetable detection system based on a cloud-local collaborative architecture of the present invention includes a data acquisition module; a cloud large model module and a local lightweight inference model module; the cloud large model module includes a preprocessing and general task module;

[0008] The data acquisition module obtains user requirements;

[0009] The cloud large model completes the extraction of high-level features and complex pattern recognition of fruit and vegetable images through prompt engineering, and transmits the processing results to the local; the cloud large model module deploys or invokes an existing ultra-large-scale pre-trained model, utilizing the pre-training of the large model's massive multi-modal pre-training data and its powerful feature extraction and inference capabilities; the commercial ultra-large-scale pre-trained model can be the DeepSeek-R1-671B architecture; the multi-modal pre-training data includes but is not limited to images and texts;

[0010] The general tasks performed by the general task module include, but are not limited to, appearance analysis, image understanding, and object detection;

[0011] The local lightweight inference model receives the feature data output by the cloud, performs specific inference tasks, and returns the task results. The specific inference tasks can be fruit appearance analysis, leaf pest and disease appearance analysis; pest and disease classification, defect detection;

[0012] The local lightweight inference model is a lightweight model deployed on edge devices, which is optimized for specific fruit and vegetable quality detection scenarios; users can fine-tune it according to specific application requirements; the edge devices can be local servers and / or mobile devices; the specific application requirements can be different fruit and vegetable types or detection standards, and fruit detection report generation.

[0013] In one embodiment, the local lightweight inference model adopts parameter-efficient fine-tuning techniques such as LoRA (Low-Rank Adaptation), and only adjusts some parameters of the local lightweight model.

[0014] In one embodiment, the prompt engineering of the cloud large model can be replaced by the LoRA (Low-Rank Adaptation) fine-tuning technique of local parameters.

[0015] In one embodiment, the calling order of the cloud-local architecture is to call the local large model first, and then call the cloud large model, or call the cloud large model first, and then call the local large model.

[0016] In one embodiment, the LoRA fine-tuning of the local lightweight inference model can be replaced by full-parameter fine-tuning.

[0017] In one embodiment, a low-cost fruit and vegetable detection method based on the cloud-local collaborative architecture of the system includes the following steps: 1) Obtain user requirements and input the requirements into the cloud large model optimized by prompt engineering; 2) The cloud large model performs general task execution and task preprocessing; 3) The result of the large model is input into the lightweight model of the local edge device fine-tuned by LoRA for specific task processing, 4) The local lightweight model gives the final feedback; the specific task processing can be fruit and vegetable disease identification, detection report generation.

[0018] The above technical features can be combined in various suitable ways or replaced by equivalent technical features as long as the purpose of the present invention can be achieved.

[0019] A low-cost fruit and vegetable detection system and method based on the cloud-local collaborative architecture provided by the present invention has at least the following beneficial effects compared with the prior art:

[0020] The present invention reduces the data exchange volume between the cloud and the local area, improves the system response speed. By leveraging the advantages of edge computing, it reduces the dependence on network bandwidth and ensures the stable operation of the system in an environment with poor network conditions. The present invention has low resource requirements, low training costs, and fast inference speed, achieving efficient resource allocation; only a small number of parameters are fine-tuned for specific fruit and vegetable quality detection tasks. While ensuring the performance of fruit and vegetable quality detection and analysis tasks, it reduces the large model training and hardware deployment costs, meeting the customized needs of small and medium-sized agricultural enterprises, testing institutions, and supermarkets for fruit and vegetable detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In the following, the present invention will be described in more detail based on embodiments and with reference to the drawings. Among them:

[0022] Figure 1 Shows the system flow schematic diagram of the present invention;

[0023] Figure 2 Shows an image of an apple with leaf spot disease for a comparative experiment;

[0024] Table 1 shows the training cost with DeepSeek-R1-671B as the large model;

[0025] Table 2 shows the deployment cost with DeepSeek-R1-671B as the large model;

[0026] Table 3 shows the test results of fruit and vegetable pest and defect detection tasks;

[0027] Table 4 shows the prompts for generating detection reports input by traditional fine-tuning of large models and the method of the present invention;

[0028] Table 5 shows the generated content obtained by traditional fine-tuning of large models and the method of the present invention for generating detection reports. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The following will further illustrate the present invention in conjunction with the drawings.

[0030] The present invention provides a low-cost fruit and vegetable detection system based on a cloud-local collaborative architecture. The execution process of the system is as Figure 1 shown. The user inputs the requirements into the cloud large model optimized by prompt engineering for general task execution and task preprocessing. The results of the large model are then input into the local edge device lightweight model fine-tuned by LoRA for specific task processing such as fruit and vegetable disease identification and detection report generation, and the final feedback is given by the local lightweight model. The present invention utilizes a cloud-local collaborative architecture, where the cloud accesses a mature large model, and the local edge side deploys a lightweight inference model or has a local lightweight model execute inference. The cloud uses prompt engineering for preprocessing and general tasks.

[0031] The present invention adopts a cloud-local collaborative architecture, which includes a cloud large model. Existing commercial ultra-large-scale pre-trained models (such as the DeepSeek-R1-671B architecture with 67.1 billion parameters) are deployed or called in the cloud. The pre-training of the large model uses massive multi-modal pre-training data (such as images, texts, etc.) and has powerful feature extraction and reasoning capabilities. Through prompt engineering, tasks such as high-level feature extraction and complex pattern recognition of fruit and vegetable images are completed to ensure task performance. Users can directly call the cloud model without the need for self-training or fine-tuning. It also includes a local lightweight inference model that deploys a lightweight model on edge devices (such as local servers, mobile devices), which is optimized for specific fruit and vegetable quality detection scenarios. This model has low resource requirements, low training costs, and fast inference speed, and is suitable for running in environments with limited computing resources. Users can fine-tune it according to specific application requirements (such as different fruit and vegetable types or detection standards). Compared with full-parameter fine-tuning of the large model, it effectively reduces training costs and hardware deployment costs.

[0032] It has an efficient fine-tuning strategy. The present invention adopts parameter-efficient fine-tuning technologies such as LoRA (Low-Rank Adaptation), which only adjusts some parameters of the local lightweight model without changing all the weights of the cloud large model. LoRA adapts to new tasks while maintaining the basic capabilities of the model by introducing a small number of trainable parameters (such as low-rank matrices). Compared with traditional full-parameter fine-tuning, the number of training parameters is reduced by 95%-98%, and the training costs and time are significantly reduced.

[0033] It has a collaborative working mechanism: The cloud large model is responsible for preprocessing and general tasks, such as performing high-level feature extraction or complex pattern analysis on fruit and vegetable images, and transmitting the processing results to the local. The local lightweight model receives the feature data output by the cloud and performs specific inference tasks, such as pest and disease classification, defect detection, etc. Through fine-tuning, the local model can accurately adapt to specific scenario requirements, ensuring the real-time and accuracy of detection.

[0034] System integration and optimization: By optimizing the data transmission protocol and model compression technology, the amount of data exchange between the cloud and the local is reduced, and the system response speed is improved. Taking advantage of edge computing, the dependence on network bandwidth is reduced to ensure that the system can still run stably in an environment with poor network conditions.

[0035] The training efficiency of this system has been significantly improved: only the lightweight inference model at the edge needs to be fine-tuned, and the training duration is reduced by 90%-95% compared with the full-parameter fine-tuning of traditional large models. The number of LoRA (Low-Rank Adaptation) fine-tuning parameters is reduced by 95%-98%; the hardware cost is greatly reduced: only the lightweight inference model needs to be deployed locally, and only 1 GPU or a few high-end CPUs are required. Compared with the multi-GPU or GPU cluster of large models, the hardware cost is reduced by about 100 times; the inference ability is comparable to that of large models with full-parameter fine-tuning: the cloud-local collaborative architecture ensures the high efficiency and accuracy of the system performance, providing a feasible solution for the customized application of large models in resource-constrained environments such as edge devices.

[0036] Specifically, in one embodiment, the present invention includes a data acquisition module; a cloud large model module and a local lightweight inference model module; the cloud large model module includes a preprocessing and general task module;

[0037] The data acquisition module obtains user requirements;

[0038] The cloud large model completes the extraction of high-level features and complex pattern recognition of fruit and vegetable images through prompt engineering, and transmits the processing results to the local; the cloud large model module deploys or invokes existing ultra-large-scale pre-trained models, and utilizes the pre-training of the massive multi-modal pre-training data of the large model and its powerful feature extraction and inference capabilities; the commercial ultra-large-scale pre-trained model can be the DeepSeek-R1-671B architecture; the multi-modal pre-training data includes but is not limited to images and texts;

[0039] The general tasks executed by the general task module include but are not limited to appearance analysis, image understanding, and object detection;

[0040] The local lightweight inference model receives the feature data output by the cloud, executes specific inference tasks and returns the task results. The specific inference tasks can be fruit appearance analysis, leaf pest and disease appearance analysis; pest and disease classification, defect detection;

[0041] The local lightweight inference model is a lightweight model deployed on edge devices, which is optimized for specific fruit and vegetable quality detection scenarios; users can fine-tune it according to specific application requirements; the edge devices can be local servers and / or mobile devices; the specific application requirements can be different fruit and vegetable types or detection standards, and the generation of fruit detection reports. The processing result of the large model is the general task execution / task preprocessing result. For appearance analysis / image understanding tasks, the output of the large model is a descriptive text of the image details.

[0042] While reducing costs, the performance of the present invention still approaches the level of traditional large models. The following takes two tasks, namely the detection of fruit and vegetable pests, diseases and defects, and the generation of fruit and vegetable detection reports, as examples.

[0043] For the task of detecting fruit and vegetable pests, diseases and defects, a publicly available dataset for intelligent identification of crop diseases is used for training and testing. This dataset contains 31,718 training images and 4,540 test images, with a total of 61 classifications. It covers a variety of fruit and vegetable species (such as apples, tomatoes, potatoes, etc.) and their common pests, diseases and defects (such as leaf spot, fruit rot, mechanical damage, etc.), and is suitable for training and evaluating fruit and vegetable quality detection models. The test results are shown in Table 1.

[0044] Table 1

[0045]

[0046] The method of the present invention only has a 0.6% decrease in image recognition accuracy, and its performance is comparable to that of traditional large models. This benefits from the combination of the advanced feature extraction ability of the cloud large model and the scene adaptability of the local model, ensuring high-precision fruit and vegetable quality detection.

[0047] In the generation of fruit and vegetable detection reports, to visually display the differences in report quality, an image of an apple with leaf spot is selected from the Internet as Figure 2 , and the traditional fine-tuned large model and the method of the present invention are respectively used to generate detection reports. The input prompt words are shown in Table 2, and the comparison of the generated content is shown in Table 3.

[0048] Table 2

[0049]

[0050] Table 3

[0051]

[0052]

[0053] It can be seen that the quality of the reports of the large model with the cloud-local collaborative architecture proposed by the method of the present invention is comparable to that of the traditional LoRA fine-tuned large model, completely encompassing the key content and correctly extracting the main features of the leaves.

[0054] In one embodiment, the local lightweight inference model adopts parameter-efficient fine-tuning techniques such as LoRA (Low-Rank Adaptation), and only adjusts some parameters of the local lightweight model.

[0055] In one embodiment, the prompt engineering of the cloud large model can be replaced by the LoRA (Low-Rank Adaptation) fine-tuning technique of local parameters.

[0056] In one embodiment, the calling order of the cloud-local architecture is to call the local large model first and then the cloud large model, or to call the cloud large model first and then the local large model.

[0057] In one embodiment, the LoRA fine-tuning of the local lightweight inference model can be replaced by full-parameter fine-tuning.

[0058] Specifically, the low-cost fruit and vegetable detection method based on the cloud-local collaborative architecture of any of the above-mentioned systems includes the following steps: 1) Obtain user requirements and input the requirements into the cloud large model optimized by prompt engineering; 2) The cloud large model performs general task execution and task preprocessing; 3) The result of the large model is input into the local edge device lightweight model fine-tuned by LoRA for specific task processing; 4) The local lightweight model gives the final feedback; the specific task processing can be fruit and vegetable disease identification and detection report generation.

[0059] The present invention significantly reduces the cost of fruit and vegetable quality detection by optimizing the training method and hardware requirements, specifically in terms of training cost and deployment cost. Taking DeepSeek-R1-671B as an example, the comparison of training costs is shown in Table 4;

[0060] Table 4

[0061]

[0062] It can be seen that compared with the traditional full-parameter fine-tuning, the training time of this method is shortened from about 2160 - 4320 hours to about 6 hours, with a reduction of 99.7% - 99.9%. The training cost is reduced from 3 million - 5 million US dollars to 300 US dollars, with a reduction of more than 99.99%. Taking DeepSeek-R1-671B as an example, the comparison of deployment costs is shown in Table 5;

[0063] Table 5

[0064]

[0065] It can be seen that the hardware cost is reduced from 5 million - 10 million yuan per year to 1000 yuan, with a reduction of 99%. The low resource requirements of the local lightweight model make it more suitable for promotion and application by small and medium-sized enterprises. The significant optimization of the present invention in terms of training cost and hardware cost greatly reduces the technical threshold of the large model for fruit and vegetable quality detection, providing an economically feasible solution for small and medium-sized agricultural enterprises, testing institutions, and supermarkets.

[0066] In summary, the present invention performs excellently in tasks such as disease detection and report generation, with little difference compared to traditional large models. At the same time, various costs have been significantly reduced, greatly improving the cost performance and facilitating the customization of fruit and vegetable detection large models by small and medium-sized agricultural enterprises, testing institutions, and supermarkets.

[0067] Although the present invention has been described herein with reference to particular embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that numerous modifications may be made to the exemplary embodiments, and other arrangements may be devised, without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the different dependent claims and the features described herein may be combined in ways different from those described in the original claims. It should also be understood that the features described in connection with separate embodiments may be used in other described embodiments.

Claims

1. A low-cost fruit and vegetable detection system based on a cloud-local collaborative architecture, characterized in that, It includes a data acquisition module, a cloud large model module, and a local lightweight inference model module; the cloud large model module includes a preprocessing and general task module; The cloud large model extracts high-level features and performs complex pattern recognition on fruit and vegetable images through prompt engineering, and transmits the processing results to the local; the cloud large model module deploys or invokes existing ultra-large-scale pre-trained models, leveraging the pre-training of the large model's massive multi-modal pre-trained data and its powerful feature extraction and inference capabilities; the commercial ultra-large-scale pre-trained models include, but are not limited to, the DeepSeek-R1-671B architecture; the multi-modal pre-trained data includes, but is not limited to, images and text; The general tasks performed by the general task module include, but are not limited to, appearance analysis, image understanding, and object detection; The local lightweight inference model receives the feature data output by the cloud and performs specific inference tasks and returns the task results. The specific inference tasks include, but are not limited to, fruit appearance analysis, leaf pest and disease appearance analysis; pest and disease classification, defect detection; The local lightweight inference model is a lightweight model deployed on edge devices, optimized for specific fruit and vegetable quality detection scenarios; users can fine-tune it according to specific application requirements; the edge devices include, but are not limited to, local servers and / or mobile devices; the specific application requirements include, but are not limited to, different fruit and vegetable types or detection standards, and fruit detection report generation.

2. The low-cost fruit and vegetable detection system based on a cloud-local collaborative architecture according to claim 1, wherein The local lightweight inference model uses local parameter efficient fine-tuning technology to adjust only some of the parameters of the local lightweight model.

3. The low-cost fruit and vegetable detection system based on the cloud-local collaborative architecture according to claim 1, characterized in that The prompt engineering of the cloud large model can be replaced by local parameter fine-tuning technology.

4. The low-cost fruit and vegetable detection system based on a cloud-local collaborative architecture according to claim 1, characterized in that, The calling order of the cloud-local architecture is to call the local large model first and then the cloud large model, or to call the cloud large model first and then the local large model.

5. The low-cost fruit and vegetable detection system based on a cloud-local collaborative architecture according to claim 1, characterized in that, The local fine-tuning of the local lightweight inference model can be replaced by full parameter fine-tuning.

6. A low-cost fruit and vegetable detection method based on a cloud-local collaborative architecture of the system according to any one of claims 1-5, characterized in that, It includes the following steps: 1) Obtain user requirements and input the requirements into the cloud large model optimized by prompt engineering; 2) The cloud large model performs general task execution and task preprocessing; 3) The results of the large model are then input into the local edge device lightweight model that has been locally fine-tuned for specific task processing, 4) and the local lightweight model gives the final feedback; the specific task processing can be fruit and vegetable disease identification, detection report generation.

Images