Agricultural product quality detection optimization method and system based on big data

Abstract

The invention discloses an agricultural product quality detection optimization method and system based on big data, and the method comprises the following steps: carrying out the preprocessing of agricultural product data, obtaining the preprocessing data, and collecting the agricultural product data through a sensor; agricultural product quality features of the preprocessed data are extracted, and the quality prediction model is updated through the agricultural product quality features; a workflow engine technology is adopted to construct an agricultural product quality detection workflow model, and work automatic scheduling is carried out through the agricultural product quality detection workflow model. Intelligent and automatic quality detection of the agricultural products is realized, the detection efficiency and accuracy are remarkably improved, and powerful support is provided for guaranteeing the quality safety of the agricultural products.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural product detection, and in particular relates to an agricultural product quality detection optimization method and system based on big data. Background Art

[0002] In the process of agricultural product quality inspection, data quality control is a key link. Due to the complexity and variability of the growth environment of agricultural products, the data collected by various sensors may have deviations or outliers, resulting in data distortion and affecting the accuracy of subsequent quality predictions. In addition, the data formats of agricultural products from different batches and different origins may be inconsistent, and improper data standardization will also cause data quality problems.

[0003] As time goes by, agricultural product varieties and planting methods will change, and the original quality prediction model may gradually become ineffective. The input features of the model can no longer well reflect the true quality of agricultural products, resulting in increasing prediction deviations. How to select appropriate features and dynamically adjust model parameters so that it can adapt to the ever-changing agricultural production environment is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a method and system for optimizing agricultural product quality detection based on big data to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above object, the present invention provides an agricultural product quality detection optimization method based on big data, comprising the following steps:

[0006] Preprocessing the agricultural product data to obtain preprocessed data, wherein the agricultural product data is collected through a sensor;

[0007] Extracting agricultural product quality features from the preprocessed data, and updating a quality prediction model based on the agricultural product quality features;

[0008] The workflow engine technology is adopted to construct an agricultural product quality detection workflow model, and the work is automatically scheduled through the agricultural product quality detection workflow model.

[0009] Preferably, the preprocessing of agricultural product data includes:

[0010] Removing obvious outliers from the agricultural product data and smoothing noise data to obtain preliminary purified data;

[0011] The box plot method is used to automatically identify and remove abnormal data in the preliminary cleansed data to obtain high-quality agricultural product data samples;

[0012] Define unified specifications and interface standards for the high-quality agricultural product data samples and perform data format conversion to obtain preprocessed data.

[0013] Preferably, extracting the agricultural product quality characteristics of the preprocessing data includes:

[0014] A deep learning model is constructed based on a convolutional neural network to automatically learn and extract high-level semantic features from the preprocessed data; the weights of the semantic features are dynamically adjusted in combination with an attention mechanism to find out a feature subset that plays a key role in predicting the quality of agricultural products.

[0015] Preferably, the update quality prediction model includes:

[0016] An incremental learning strategy is used to dynamically update the quality prediction model; when the prediction error exceeds a preset threshold, the agricultural product quality characteristics are used to optimize and adjust the model parameters.

[0017] The present invention also discloses a system for optimizing agricultural product quality detection based on big data, comprising:

[0018] A preprocessing module, used for preprocessing agricultural product data to obtain preprocessed data, wherein the agricultural product data is collected by sensors;

[0019] An updating module, used for extracting the agricultural product quality characteristics of the preprocessed data, and updating the quality prediction model according to the agricultural product quality characteristics;

[0020] The scheduling module is used to construct an agricultural product quality detection workflow model using workflow engine technology, and automatically schedule work through the agricultural product quality detection workflow model.

[0021] Preferably, the preprocessing module comprises:

[0022] A first preprocessing unit, used to remove obvious outliers in the agricultural product data and smooth noise data to obtain preliminary purified data;

[0023] The second preprocessing unit automatically identifies and removes abnormal data in the preliminary cleansed data through the box plot method to obtain high-quality agricultural product data samples;

[0024] The third preprocessing unit defines unified specifications and interface standards for the high-quality agricultural product data samples and performs data format conversion to obtain preprocessed data.

[0025] Preferably, the update module includes:

[0026] A feature extraction unit, used to construct a deep learning model based on a convolutional neural network, and automatically learn and extract high-level semantic features from the preprocessed data;

[0027] A weighting unit, used to dynamically adjust the weight of the semantic features in combination with an attention mechanism to find out the features that play a key role in predicting the quality of agricultural products;

[0028] The updating unit is used to update the quality prediction model through the key features.

[0029] Preferably, the updating unit comprises:

[0030] A dynamic updating unit, used for dynamically updating the quality prediction model by adopting an incremental learning strategy;

[0031] The optimization and adjustment unit is used to optimize and adjust the model parameters using the agricultural product quality characteristics when the prediction error exceeds a preset threshold.

[0032] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0033] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0034] Compared with the prior art, the present invention has the following advantages and technical effects:

[0035] The present invention discloses an intelligent detection optimization method and system for agricultural product quality. In view of the difficulties in data processing and analysis during the detection of agricultural product quality, the present invention first preprocesses the raw data collected by the sensor to obtain preprocessed data samples. The present invention constructs a deep learning model to automatically learn and extract high-level semantic features directly from the raw data, and combines the attention mechanism to find the key feature subset, thereby improving the accuracy and generalization ability of quality prediction. Finally, in order to adapt to the ever-changing agricultural production environment, the present invention adopts an incremental learning strategy to dynamically update the quality prediction model. The detection process is globally optimized through the workflow engine technology to achieve seamless connection from sample collection to quality judgment. The present invention realizes the intelligence and automation of agricultural product quality detection, significantly improves the detection efficiency and accuracy, and provides strong support for ensuring the quality and safety of agricultural products. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0037] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] Embodiment 1

[0041] like Figure 1 As shown, this embodiment provides an agricultural product quality detection optimization method based on big data, including the following steps:

[0042] Preprocessing the agricultural product data to obtain preprocessed data, wherein the agricultural product data is collected through a sensor;

[0043] Extract agricultural product quality features from preprocessed data and update the quality prediction model based on the agricultural product quality features;

[0044] The workflow engine technology is used to construct a workflow model for agricultural product quality inspection, and work is automatically scheduled through the agricultural product quality inspection workflow model.

[0045] Furthermore, the preprocessing of agricultural product data includes:

[0046] 1. Remove obvious outliers in agricultural product data and smooth out noise data to obtain preliminary purified data;

[0047] Specifically, the raw data collected by multiple sensors during the agricultural product quality inspection process are obtained, and the corresponding data preprocessing methods are used to process the raw data characteristics of different sensors. According to the agricultural product quality inspection index system and threshold setting, it is judged whether there are obvious outliers in each raw data. If there are outliers, they are removed to obtain the first purified agricultural product data set. For the agricultural product data set after the first purification, the data smoothing algorithm is used to smooth the noise data. The smoothing algorithm can select Kalman filtering, moving average, etc., and the agricultural product data set for the second purification is obtained through smoothing.

[0048] In this embodiment, taking apple as an example, multiple sensors such as color, hardness, sugar content, acidity, etc. can be used for data collection. The color sensor can collect color information on the surface of the apple, such as the red value, green value, and blue value of a red apple. Assume that the RGB value of an apple collected is (230, 50, 30), which indicates that the apple is bright red. The hardness sensor can measure the hardness of the apple, and the unit can be Newton or kilogram force. For example, the hardness of an apple is measured to be 8 Newtons, indicating that the apple is relatively hard and suitable for long-term storage. The sugar content sensor can measure the sweetness of the apple, in degrees Brix (°Bx). For example, the sugar content of an apple is 15°Bx, indicating that the apple is very sweet. The acidity sensor can measure the acidity of the apple, in pH value. The pH value of an apple is 5, indicating that the acidity of the apple is moderate. These raw data may have outliers and need to be preprocessed. For example, if the hardness measurement value of an apple is negative, the data is obviously wrong and needs to be eliminated. The data set after the first purification may still have noise and needs to be smoothed. For example, the moving average method can be used to smooth the sugar content data. If the sugar content of an apple is measured at 14°Bx, 15°Bx, and 16°Bx for three consecutive days, the average value of the three-day sugar content, 15°Bx, can be used as the smoothed sugar content value, which can effectively remove random fluctuations in the measurement process.

[0049] 2. Automatically identify and remove abnormal data in the preliminary cleansing data through the box plot method to obtain high-quality agricultural product data samples;

[0050] Specifically, for the preprocessed agricultural product data set, an outlier detection algorithm based on statistical principles is used for analysis, such as the 3σ criterion or the box plot method. According to the selected outlier detection algorithm, the corresponding threshold parameters are set to determine whether the data point is an outlier. Each data point in the agricultural product data set is traversed, and its outlier score is calculated by the outlier detection algorithm. If the outlier score of the data point exceeds the preset threshold, the data point is determined to be a potential outlier and marked. All data points marked as potential outliers are eliminated to obtain a subset of agricultural product data after outliers are eliminated. The quality of the agricultural product data subset after outliers are eliminated is evaluated to ensure the high quality of the data sample, and finally a high-quality agricultural product data set that can be used for subsequent analysis is obtained.

[0051] In this embodiment, if the sweetness data of a certain apple is missing, the missing value can be filled with the average value of the sweetness of all apples. Assuming that the size data of a certain apple is recorded twice and the values ​​are the same, one of the duplicate data needs to be removed. Outlier detection is another important link in data preprocessing. Commonly used outlier detection algorithms include the 3σ criterion and the box plot method. The 3σ criterion states that if a data point deviates from the mean by more than 3 times the standard deviation, the data point is considered an outlier. The box plot method identifies outliers through quartiles and interquartile ranges. For example, assuming that the average weight of a batch of apples is 200 grams and the standard deviation is 20 grams, according to the 3σ criterion, apples weighing less than 140 grams or more than 260 grams will be marked as potential outliers. Using the box plot method, if the weight of an apple is greater than the upper quartile plus 1.5 times the interquartile range or less than the lower quartile minus 1.5 times the interquartile range, the apple is marked as a potential outlier. The setting of the threshold parameter needs to be adjusted according to the specific application scenario and data distribution. Too high a threshold value will cause some real outliers to be missed, while too low a threshold value will misjudge some normal data points as outliers. For example, for the weight data of apples, if the threshold value is set too low, some small or large apples may be misjudged as outliers. Traverse each data point in the agricultural product data set and calculate its anomaly score. The anomaly score can be calculated according to the selected outlier detection algorithm. For example, using the 3σ criterion, the anomaly score can be defined as the ratio of the deviation of the data point from the mean to the standard deviation. Assuming that the weight of an apple is 280 grams, the average weight is 200 grams, and the standard deviation is 20 grams, the anomaly score of the apple is (280-200) / 20=4. Mark the data points whose anomaly score exceeds the preset threshold as potential outliers. For example, if the threshold value is set to 3, the apple weighing 280 grams in the above example will be marked as a potential outlier because its anomaly score of 4 is greater than the threshold value of 3. Eliminate all data points marked as potential outliers to obtain the subset of agricultural product data after eliminating outliers. Doing so can effectively remove noise from the data and improve the accuracy of subsequent analysis. For example, by removing all apples weighing less than 140 grams and more than 260 grams, the remaining apple data subset will be more representative and better reflect the weight distribution of normal apples. Perform a quality assessment on the agricultural product data subset after removing outliers. The purpose of quality assessment is to ensure the high quality of data samples and provide a reliable data basis for subsequent analysis. Quality assessment can be performed from multiple dimensions, such as: data integrity, consistency, accuracy, etc. For example, check whether there are missing values ​​in the data subset, whether the data types of each attribute are consistent, whether the value range of the data is reasonable, etc. Suppose that after the outliers are removed, it is found that the value range of the sweetness data in the apple data subset is between 0 and 15, while the normal sweetness of apples should be between 5 and 12, which means that there may be problems with the data and further investigation is needed.Finally, a high-quality agricultural product data set is obtained, which can be used for subsequent analysis. This data set has been processed through multiple steps such as data cleaning, missing value processing, and outlier detection. The quality of the data has been effectively guaranteed and can be used for subsequent analysis such as building an agricultural product quality assessment model.

[0052] 3. Define unified specifications and interface standards for high-quality agricultural product data samples and convert data formats to obtain preprocessed data.

[0053] Specifically, in the processing of agricultural product data, the first challenge is inconsistent data formats. For example, suppose data on apples and oranges are collected from different origins, where the data on apples is in kilograms and the data on oranges is in pounds, and the data are recorded in different ways. To solve this problem, data cleaning technology can be used to convert formats and unify units to ensure that all data has the same format and metric before entering the analysis process. The accuracy and availability of data can be further improved through denoising and deduplication processing. For example, by identifying duplicate records and deleting them, the uniqueness of each piece of data can be guaranteed. Next, it is necessary to formulate unified data specifications and interface standards based on the characteristics of agricultural product data. This includes determining the unit, precision, and encoding method of the data. For example, for temperature data, it can be stipulated that Celsius should be used uniformly and the precision should be kept to two decimal places to avoid data errors caused by inconsistent units or precision problems. Using data mining technology to extract features and analyze attributes of agricultural product data can effectively identify which attributes have the greatest impact on the quality of agricultural products. For example, by analyzing factors such as soil type, moisture content, and light duration, the key factors affecting crop growth can be found, thus providing a basis for standardized processing. The application of data mapping technology, such as mapping data in different formats into a unified data model, is a key step in achieving data standardization. In this process, middleware tools can be used to automatically identify and convert data formats, such as converting XML format to JSON format, to ensure the consistency and operability of all data in the database. In terms of data quality control, machine learning algorithms such as decision trees and support vector machines can be used for quality grading and anomaly detection of agricultural product data. For example, by training a support vector machine model to identify substandard agricultural products, such as fruits damaged by pests and diseases, it is ensured that only healthy and mature agricultural products enter the market. Finally, the development of data conversion tools based on unified data specifications can realize the automatic conversion and integration of agricultural product data from different batches and origins. This not only improves the efficiency of data processing, but also ensures the accuracy and consistency of data processing. By statistically analyzing and visually displaying standardized high-quality agricultural product data, we can better understand the laws and trends behind the data and provide scientific decision support for the production and sales of agricultural products.

[0054] This embodiment can also use machine learning algorithms to perform quality grading and anomaly detection on standardized agricultural product data, identify and eliminate data of unqualified quality, and improve data quality.

[0055] Obtain the agricultural product data after standardization, select appropriate machine learning algorithms such as support vector machines and random forests according to the characteristic attributes of the data, and establish a quality grading model. According to the established quality grading model, the agricultural product data is graded into different quality levels, such as high-quality, good, qualified, and unqualified. For the agricultural product data after quality grading, anomaly detection algorithms such as isolated forests and local anomaly factors are used to identify abnormal data in each quality level. For the detected abnormal data, it is judged according to the preset abnormal threshold. If the degree of abnormality exceeds the threshold, the data is marked as unqualified quality data. The agricultural product data marked as unqualified quality is removed from the data set to obtain the agricultural product data set after the abnormal and unqualified data are removed. The data quality assessment indicators such as completeness, accuracy, consistency, etc. are used to assess the quality of the agricultural product data set after the abnormal and unqualified data are removed, and the data quality score is calculated. The data quality assessment results are compared with the original data quality score to determine whether the data quality has been improved. If the improvement reaches the expected goal, the improved high-quality agricultural product data set is output as the input for subsequent analysis.

[0056] Furthermore, the agricultural product quality characteristics extracted from the preprocessed data include:

[0057] A deep learning model is constructed based on the convolutional neural network to automatically learn and extract high-level semantic features from the preprocessed data. The weights of semantic features are dynamically adjusted in combination with the attention mechanism to find out the feature subset that plays a key role in predicting the quality of agricultural products.

[0058] Specifically, a convolutional neural network model is constructed based on the preprocessed agricultural product data, and the multi-scale and multi-level semantic features of the agricultural product data are automatically extracted through the convolution layer and the pooling layer. On the basis of the convolutional neural network model, the attention mechanism is introduced, and the weights of different semantic features are dynamically adjusted through the attention layer to highlight the features that play a key role in the prediction of agricultural product quality. According to the semantic feature weights adjusted by the attention mechanism, a subset of key features with larger weights are screened out as the input of the agricultural product quality prediction model to simplify the model input dimension.

[0059] In this embodiment, a convolutional neural network model is constructed based on the preprocessed agricultural product data. Convolutional neural networks are good at capturing local features and can automatically extract features of data such as images and audio. In this example, the various index data of agricultural products can be regarded as two-dimensional data similar to images, and the multi-scale and multi-level semantic features of agricultural product data can be extracted using convolutional layers. For example, the first convolutional layer can extract the underlying features such as color and shape of agricultural product data, and the second convolutional layer can extract more advanced features such as texture and contour based on the first layer, and so on. The role of the pooling layer is to reduce the feature dimension, reduce the amount of calculation, and improve the robustness of the model. On the basis of the convolutional neural network model, an attention mechanism is introduced. The attention mechanism can dynamically adjust the weights of different features according to the importance of different features, highlighting the features that play a key role in predicting the quality of agricultural products. For example, for apples, features such as color, size, and sweetness have a greater impact on quality prediction, while features such as origin and planting time have a relatively small impact, so the attention mechanism can give features such as color, size, and sweetness a higher weight. According to the semantic feature weights adjusted by the attention mechanism, a subset of key features with larger weights is screened out as the input of the agricultural product quality prediction model, which simplifies the model input dimension, avoids model overfitting, and improves the generalization ability of the model.

[0060] Further, updating the quality prediction model includes:

[0061] An incremental learning strategy is used to dynamically update the quality prediction model; when the prediction error exceeds the preset threshold, the quality characteristics of agricultural products are used to optimize the model parameters.

[0062] Specifically, the feature set is expanded through the quality features of agricultural products. Based on the expanded feature set, a machine learning algorithm is used to construct an agricultural product quality prediction model. Assuming that the support vector machine (SVM) algorithm is selected, the model is trained using the feature values ​​and corresponding quality labels (such as high-quality, qualified, and unqualified) in the sample data. Through cross-validation and parameter tuning, the model performance is optimized so that it can accurately predict the quality of new samples. In the agricultural production process, new apple sample data is continuously collected. For example, a batch of new data is collected from different farms every quarter. After preprocessing, the quality is predicted using the trained quality prediction model. Assuming that the predicted quality of a batch of apples is high-quality, but after actual testing, some apples are found to have quality problems, the error between the predicted result and the actual quality is calculated. It is determined whether the prediction error exceeds a preset threshold, such as setting a threshold of 5%. If the error exceeds the threshold, the model update mechanism is triggered. The newly collected sample data is added to the original sample data set, and an incremental learning strategy is adopted to optimize the parameters of the quality prediction model using the updated sample data set and feature set. For example, through incremental learning, the model can learn new quality feature change rules in the new data, thereby improving the prediction accuracy. Repeating the above process continuously enables the quality prediction model to continuously learn and evolve. Assume that in consecutive quarters, the model gradually adapts to the impact of climate change on apple quality through incremental learning, and the prediction accuracy increases from the initial 80% to more than 90%. Dynamically adapting to changes in the agricultural production environment not only improves the accuracy of agricultural product quality prediction, but also enhances the robustness of the model. In this way, agricultural product producers and operators can grasp the product quality situation in real time and adjust production strategies in time. For example, if it is found that the sugar content of a batch of apples is generally low, it may be due to improper fertilization. The fertilization plan can be adjusted in time to improve product quality. At the same time, based on the prediction results, more targeted market positioning and sales strategy formulation can be carried out, such as giving priority to high-quality apples to the high-end market to increase product added value. The meticulous processing of data cleaning and feature extraction ensures the data quality of the input model and lays a solid foundation for subsequent model training and prediction. The application of association rule mining and decision tree algorithms reveals the key quality features hidden in the data and improves the prediction ability of the model. The selection of machine learning algorithms such as support vector machines and random forests makes full use of the characteristic information of sample data to build an efficient quality prediction model. The application of incremental learning strategy enables the model to continuously absorb new data, continuously optimize its own performance, and adapt to environmental changes. This dynamic update mechanism not only improves the prediction accuracy of the model, but also enhances its stability and reliability in practical applications.

[0063] In this embodiment, incremental learning can be performed directly on the basis of the original model, and the parameters of the model can be adjusted to adapt to the new data. Incremental learning can save a lot of computing resources and time, and can adapt the model to the ever-changing data environment. Obtain the quality prediction model after incremental learning, optimize and adjust the parameters of the quality prediction model according to the feature set, and obtain the optimized quality prediction model parameters. Parameter optimization is a crucial step in machine learning. Different parameters will affect the performance of the model, and it is necessary to find the best parameter combination to make the model have the best prediction effect. For example, there is an important parameter C in the SVM model, which controls the complexity of the model. The best C value and other model parameters can be found through grid search or other optimization algorithms to improve the prediction accuracy of the model. The optimized quality prediction model parameters are applied to the quality prediction model after incremental learning to obtain the optimized quality prediction model. Apply the optimized parameters to the model to obtain the final optimized quality prediction model. This model has learned new data and the parameters have been adjusted. In theory, it should have better prediction performance than the previous model.

[0064] Furthermore, the workflow engine technology is used to globally optimize the agricultural product quality inspection process. By defining the data input and output of each link, a workflow model for agricultural product quality inspection is constructed. The workflow model is used to automatically schedule and coordinate tasks in each link, achieving seamless connection and parallel processing from sample collection to quality determination.

[0065] Specifically, according to the pre-established agricultural product quality inspection knowledge base, each link of the agricultural product quality inspection process is obtained, and for each link, its data input and data output format is determined to construct an agricultural product quality inspection workflow model. The agricultural product quality inspection workflow model is imported into the workflow engine, and the tasks of each link of agricultural product quality inspection are automatically scheduled through the workflow engine. If the data input of the current link is ready, the execution of the task of this link is triggered. In the process of executing tasks in each link of agricultural product quality inspection, multi-threaded parallel processing technology is adopted to collect and analyze test data for different inspection items at the same time, thereby shortening the time of agricultural product quality inspection. In the process of agricultural product quality inspection, the task coordination function of the workflow engine is used to judge the completion status of tasks in each link. According to the data input and output relationship defined by the agricultural product quality inspection workflow model, the output data of the previous link task is automatically passed to the next link task, so as to realize the seamless connection of each link of agricultural product quality inspection. The execution log data of tasks in each link of agricultural product quality inspection is obtained, and the log data is analyzed by machine learning algorithm to obtain the execution efficiency and resource consumption of tasks in each link, automatically adjust the task execution parameters, and dynamically optimize the agricultural product quality inspection process. The test data collected from each link of agricultural product quality testing are summarized and comprehensively analyzed using statistical learning algorithms. According to the preset agricultural product quality judgment rules, the agricultural product quality grade is automatically given and the agricultural product quality inspection report is generated. The incremental learning algorithm is used to continuously optimize the agricultural product quality judgment rules. According to the newly collected agricultural product quality inspection data, the judgment rule threshold is dynamically updated to improve the accuracy of agricultural product quality judgment and realize the global dynamic optimization of the agricultural product quality inspection process.

[0066] In this embodiment, the agricultural product quality inspection knowledge base can be understood as a database containing knowledge related to agricultural product quality inspection. It stores information such as the inspection process, inspection indicators, standard values, and inspection methods corresponding to various agricultural products. For example, for apples, the knowledge base will store the inspection process of apples, including appearance inspection, sugar content inspection, pesticide residue inspection and other links; the specific operation method of each link, such as the appearance inspection needs to observe the color, size, and presence of lesions of apples; and the standard value of each indicator, such as the sugar content of apples must reach a certain value to be qualified. The establishment of an agricultural product quality inspection workflow model can determine the input and output data format of each inspection link according to the quality inspection knowledge base information of different agricultural products. For example, the input data of the apple appearance inspection link is the image of the apple, and the output data is the size, color, and presence of lesions of the apple, and is stored in the form of structured data, such as JSON or XML format. The input data of the sugar content detection is the apple pulp sample, and the output data is the sugar content value of the apple. The workflow engine is like an automated command center, which can automatically schedule tasks in each link according to a pre-defined workflow model. For example, when the image data of apples is ready, the workflow engine will automatically trigger the execution of the task in the appearance inspection link. Multithreaded parallel processing technology can improve the inspection efficiency. For example, when inspecting apples, appearance inspection, sugar content inspection and pesticide residue inspection can be carried out simultaneously, rather than serially one by one. This can significantly shorten the inspection time. The task coordination function of the workflow engine can ensure the data transmission and seamless connection between each link. For example, after the appearance inspection link is completed, the output information such as the size and color of the apple will be automatically passed to the next link, such as the grading link, as the basis for grading. By analyzing the task execution log data, the execution efficiency and resource consumption of each link can be understood. For example, if the sugar content inspection link takes too long, it can be considered to increase the inspection equipment or optimize the inspection method to improve the overall inspection efficiency. After the inspection data of all links are summarized, a statistical learning algorithm is used for comprehensive analysis. For example, the appearance, sugar content, pesticide residue and other indicator data of apples are comprehensively analyzed, and according to the preset judgment rules, such as the size, color, sugar content, and pesticide residue indicators of apples must reach a specific level, and finally the quality level of the apples is determined to be level one. The rules for determining the quality of agricultural products are not static. By using an incremental learning algorithm, the judgment rules can be dynamically updated based on new test data. For example, as consumers' requirements for the sweetness of apples become higher and higher, the threshold of the sugar content index can be dynamically adjusted based on the newly collected apple sugar content data, so that the quality judgment rules are more in line with market demand and the accuracy of the judgment is improved. This realizes the global dynamic optimization of the agricultural product quality inspection process, enabling it to adapt to the ever-changing market demand and agricultural production environment. For example, suppose there is a batch of newly picked strawberries that need to be quality inspected.First, according to the strawberry quality detection knowledge base, the strawberry detection process is obtained, including appearance detection, sweetness detection, hardness detection and other links. Then, the image data of the strawberry is input into the appearance detection link, the strawberry pulp sample is input into the sweetness detection link, and the hardness of the strawberry is measured using a hardness tester in the hardness detection link. Multi-threaded parallel processing technology allows these three links to be carried out simultaneously. The workflow engine monitors the task execution of each link and passes the output data of the previous link to the next link. For example, the strawberry size, color and other information output by the appearance detection link will be passed to the grading link. Finally, the detection data of all links are summarized, and according to the preset strawberry quality judgment rules, such as the size, color, sweetness and hardness of the strawberry must reach a specific level, the quality grade of the strawberry is automatically given, and a strawberry quality detection report is generated. At the same time, the system will use the incremental learning algorithm to dynamically update the strawberry quality judgment rules based on the current detection data. For example, according to the feedback of consumers on the sweetness of the strawberry, the threshold of the sweetness index is dynamically adjusted, thereby improving the accuracy of the strawberry quality judgment.

[0067] Embodiment 2

[0068] This embodiment also provides an agricultural product quality detection optimization system based on big data, including:

[0069] A preprocessing module, used for preprocessing agricultural product data to obtain preprocessed data, wherein the agricultural product data is collected by sensors;

[0070] An updating module, used for extracting agricultural product quality features of preprocessed data and updating the quality prediction model through the agricultural product quality features;

[0071] The scheduling module is used to construct an agricultural product quality inspection workflow model using workflow engine technology, and automatically schedule work through the agricultural product quality inspection workflow model.

[0072] Furthermore, the preprocessing module includes:

[0073] The first preprocessing unit is used to remove obvious outliers in the agricultural product data and smooth the noise data to obtain preliminary purified data;

[0074] The second preprocessing unit automatically identifies and removes abnormal data in the preliminary cleansed data through the box plot method to obtain high-quality agricultural product data samples;

[0075] The third preprocessing unit defines unified specifications and interface standards for high-quality agricultural product data samples and performs data format conversion to obtain preprocessed data.

[0076] Furthermore, the update module includes:

[0077] A feature extraction unit, which is used to build a deep learning model based on a convolutional neural network to automatically learn and extract high-level semantic features from preprocessed data;

[0078] The weighting unit is used to dynamically adjust the weights of semantic features in combination with the attention mechanism to find out the features that play a key role in predicting the quality of agricultural products;

[0079] The updating unit is used to update the quality prediction model through the key features.

[0080] Furthermore, the updating unit comprises:

[0081] A dynamic updating unit, used for dynamically updating the quality prediction model by adopting an incremental learning strategy;

[0082] The optimization and adjustment unit is used to optimize and adjust the model parameters using the quality characteristics of agricultural products when the prediction error exceeds a preset threshold.

[0083] Embodiment 3

[0084] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0085] Embodiment 4

[0086] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0087] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for optimizing agricultural product quality detection based on big data, characterized in that: The following steps are involved: Preprocessing the agricultural product data to obtain preprocessed data, wherein the agricultural product data is collected through a sensor; Extracting agricultural product quality features from the preprocessed data, and updating a quality prediction model based on the agricultural product quality features; The workflow engine technology is adopted to construct an agricultural product quality detection workflow model, and the work is automatically scheduled through the agricultural product quality detection workflow model.

2. The method according to claim 1, characterized in that: The preprocessing of agricultural product data includes: Removing obvious outliers from the agricultural product data and smoothing noise data to obtain preliminary purified data; The box plot method is used to automatically identify and remove abnormal data in the preliminary cleansed data to obtain high-quality agricultural product data samples; Define unified specifications and interface standards for the high-quality agricultural product data samples and perform data format conversion to obtain preprocessed data.

3. The method according to claim 1, characterized in that: The agricultural product quality characteristics extracted from the preprocessed data include: A deep learning model is constructed based on a convolutional neural network to automatically learn and extract high-level semantic features from the preprocessed data; the weights of the semantic features are dynamically adjusted in combination with an attention mechanism to find out a feature subset that plays a key role in predicting the quality of agricultural products.

4. The method according to claim 1, characterized in that: The update quality prediction model includes: An incremental learning strategy is used to dynamically update the quality prediction model; when the prediction error exceeds a preset threshold, the agricultural product quality characteristics are used to optimize and adjust the model parameters.

5. A big data-based agricultural product quality detection and optimization system, characterized in that: include: A preprocessing module, used for preprocessing agricultural product data to obtain preprocessed data, wherein the agricultural product data is collected by sensors; An updating module, used for extracting the agricultural product quality characteristics of the preprocessed data, and updating the quality prediction model according to the agricultural product quality characteristics; The scheduling module is used to construct an agricultural product quality detection workflow model using workflow engine technology, and automatically schedule work through the agricultural product quality detection workflow model.

6. The system according to claim 5, characterized in that The preprocessing module comprises: A first preprocessing unit, used to remove obvious outliers in the agricultural product data and smooth noise data to obtain preliminary purified data; The second preprocessing unit automatically identifies and removes abnormal data in the preliminary cleansed data through the box plot method to obtain high-quality agricultural product data samples; The third preprocessing unit defines unified specifications and interface standards for the high-quality agricultural product data samples and performs data format conversion to obtain preprocessed data.

7. The system according to claim 5, characterized in that The update module includes: A feature extraction unit, used to construct a deep learning model based on a convolutional neural network, and automatically learn and extract high-level semantic features from the preprocessed data; A weighting unit, used to dynamically adjust the weights of the semantic features in combination with an attention mechanism, and find out the features that play a key role in predicting the quality of agricultural products; The updating unit is used to update the quality prediction model through the key features.

8. The system according to claim 5, characterized in that The updating unit comprises: A dynamic updating unit, used for dynamically updating the quality prediction model by adopting an incremental learning strategy; The optimization and adjustment unit is used to optimize and adjust the model parameters using the agricultural product quality characteristics when the prediction error exceeds a preset threshold.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claims 1-4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claims 1 to 4 are implemented.

Images