A tea picking time prediction method combining tea images and environmental data

Abstract

The present invention provides a tea picking time prediction method combining tea images and environmental data in the field of tea planting technology, including: step S1, collecting a large number of tea images and environmental data to construct a data set; step S2, creating a picking time prediction model based on the YOLOv8n network, setting the loss function of the picking time prediction model; the YOLOv8n network is constructed based on the receptive field convolution attention module, the reverse residual attention module, and the slice-assisted hyper-inference algorithm; step S3, training the picking time prediction model through the data set, and continuously optimizing the hyperparameters during the training process; step S4, evaluating the performance of the picking time prediction model based on F1 score, AUC, mean square error, root mean square error, and mean absolute error; step S5, predicting the tea picking time through the picking time prediction model. The advantages of the present invention are: greatly improving the accuracy and efficiency of tea picking time prediction.

Description

Technical Field

[0001] The invention relates to the technical field of tea planting, and in particular to a method for predicting tea picking time by combining tea images and environmental data. Background Art

[0002] After the tea leaves of the tea trees are ripe, they need to be picked. Since the tea leaves of large-leaf tea trees may have a single bud, one bud and one leaf, or one bud and two leaves, and the maturity time of single bud, one bud and one leaf, and one bud and two leaves is different, in the past, tea farmers only made rough judgments based on their planting experience, resulting in a large amount of tea being wasted every year due to missed picking. Therefore, there is a need to predict the tea picking time.

[0003] There are currently the following methods for predicting tea picking time: (1) ZhiyongGui et al. used Ghost_conv to replace the convolution in the original Yolov5 to reduce the computational complexity and size of the model; added BAM (Bottleneck Attention Module) to the backbone network to suppress invalid information; and used weighted feature fusion in the Neck layer to improve the model detection accuracy. The results show that compared with the original YOLOv5, the improved model has an improved recognition accuracy of 9.66%, a reduced floating point operation of 52.402G, and a reduced parameter volume of 22.71M. It can effectively realize tea bud detection and thus predict the tea picking time. However, the improved YOLOv5 algorithm still cannot perform statistical classification on the recognition results, and the recognition accuracy of long-distance buds is still low, which affects the prediction accuracy of tea picking time. (2) Shuang Xie et al. constructed the Tea-YOLOv8s network for the complex background of the tea garden, adopted a variety of data enhancement calculations to increase the amount of information in the image and improve its quality, added deformable convolution and attention mechanism, and improved the spatial pyramid pooling structure (SPP, Spatial PyramidPooling), thereby enhancing the model's recognition ability for complex objects, reducing the interference of irrelevant factors, and improving the model's detection accuracy. Finally, the Tea-YOLOv8s network achieved an accuracy of 88.27% for tea bud recognition, and the number of parameters and floating-point operations increased by 15.4M and 17.5G respectively; however, the Tea-YOLOv8s network still could not perform statistical classification on the recognition results, and the FPS of the model was only 37.1ms, which was slow, thus affecting the accuracy and efficiency of tea picking time prediction. (3) The YOLOv4-lighted+ICBAM network developed by Shudan Guo et al. uses SENet (Squeeze-and-Excitation Networks), ECA (Efficient Channel Attention), CBAM (Convolutional Block Attention Module), and ICBAM (Improved Convolutional Block Attention Module) to improve the YOLOv4 network to improve the detection accuracy of tea leaves; PANet (Path Aggregation Network) and FPN (Feature Pyramid Network) are added to the Neck layer. The accuracy, recall rate, and average precision of the data collected by RGB-D are 94.19%, 93.50%, and 97.29%, respectively.However, the accuracy of the YOLOv4-lighted+ICBAM network was only 87.10% in external verification, and the latency rate was high, making it impossible to quickly achieve tea grading, identification, positioning and result statistics, which in turn affected the accuracy and efficiency of tea picking time prediction.

[0004] Therefore, how to provide a tea picking time prediction method that combines tea images and environmental data to improve the accuracy and efficiency of tea picking time prediction has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for predicting tea picking time by combining tea images and environmental data, so as to improve the accuracy and efficiency of tea picking time prediction.

[0006] The present invention is implemented as follows: a method for predicting tea picking time by combining tea images and environmental data, comprising the following steps:

[0007] Step S1, collecting a large number of tea images and environmental data, pre-processing each of the tea images and environmental data, and then associating each of the tea images and environmental data to construct a data set;

[0008] Step S2, creating a picking time prediction model based on the YOLOv8n network, and setting a loss function of the picking time prediction model that introduces an auxiliary bounding box; the YOLOv8n network is constructed based on a receptive field convolution attention module, a reverse residual attention module, and a slice-assisted super-inference algorithm;

[0009] Step S3, training the picking time prediction model using the data set, and continuously optimizing hyperparameters including at least batch size, number of training rounds, random inactivation rate, weight decay rate, and learning rate during the training process;

[0010] Step S4, evaluating the performance of the picking time prediction model based on F1 score, AUC, mean square error, root mean square error and mean absolute error;

[0011] Step S5: predicting the tea picking time by using the picking time prediction model to generate a prediction result.

[0012] Furthermore, in step S1, the tea image includes different shooting angles, light intensities and tea types; the shooting angle is front, overhead, overhead, side or overall; the light intensity is low illumination, medium illumination or high illumination; the tea type is single bud, one bud and one leaf or one bud and two leaves;

[0013] The environmental data includes soil moisture collected by a soil moisture sensor, light intensity collected by a light intensity sensor, air temperature collected by an air temperature sensor, air humidity collected by an air humidity sensor, carbon dioxide concentration collected by a carbon dioxide sensor, radiation value collected by a TBQ total radiation sensor, and corresponding data collection time.

[0014] Furthermore, in step S1, the preprocessing of each tea image and environmental data, and then associating each tea image and environmental data to construct a data set are specifically as follows:

[0015] By using Make Sense, each of the tea images is labeled with tea area, tea type, tea maturity, and picking time, and brightness, contrast, and zoom image expansion operations are performed on each of the labeled tea images, thereby completing the preprocessing of each of the tea images;

[0016] Performing data cleaning on each of the environmental data, including at least null value processing, missing value processing, key value processing, and merging data, and then performing standardized processing such as format conversion, sorting, and aggregation on each of the environmental data, thereby completing the preprocessing of each of the environmental data;

[0017] Based on the shooting time of each tea image and the collection time of each environmental data, each tea image and environmental data are associated and a data set is constructed.

[0018] Furthermore, in step S2, the formula of the loss function is:

[0019]

[0020] union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter;

[0021]

[0022]

[0023] Among them, L Inner-CIoU Represents the loss value of the loss function; ρ 2 (c,c gt) represents the Euclidean distance between the center points of the predicted box and the true box; d represents the diagonal distance of the minimum closed bounding box containing the predicted box and the true box; α represents the balance parameter; v represents the aspect ratio consistency measurement coefficient; inter represents the overlapping area of ​​the predicted box and the true box; union represents the union area of ​​the predicted box and the true box; represents the scale-optimized width of the real box inside the auxiliary bounding box; c r Indicates the width of the real box inside the auxiliary bounding box; represents the scale-optimized height of the real box inside the auxiliary bounding box; c l Indicates the height of the real box inside the auxiliary bounding box; represents the scale-optimized width of the prediction box inside the auxiliary bounding box; c b Indicates the width of the prediction box inside the auxiliary bounding box; represents the scale-optimized height of the prediction box inside the auxiliary bounding box; c t Indicates the height of the prediction box inside the auxiliary bounding box; w gt Indicates the width of the real box; h gt Represents the height of the real box; ratio represents the scaling factor; w represents the width of the predicted box; h represents the height of the predicted box; Indicates the center point of the real box; (x d ,y d ) represents the center point of the prediction box.

[0024] Furthermore, in step S2, the receptive field convolution attention module is used to improve the attention of the picking time prediction model to fresh tea leaves, suppress irrelevant background noise, and enhance recognition accuracy and generalization ability;

[0025] The reverse residual attention module is used to improve the receptive field and downstream task processing capabilities of the picking time prediction model;

[0026] The slice-assisted hyper-inference algorithm is used to improve the recognition capability of the picking time prediction model for long-distance targets and small targets.

[0027] Furthermore, in step S3, the training of the picking time prediction model using the data set is specifically as follows:

[0028] The data set is divided into a training set, a test set, and a validation set in a ratio of 7:2:1;

[0029] The picking time prediction model is trained by using the training set until the loss value of the loss function is less than a preset loss threshold;

[0030] The trained picking time prediction model is tested by the test set to determine whether the prediction accuracy is greater than a preset accuracy threshold. If not, the training set is expanded to continue training; if yes, then:

[0031] The tested picking time prediction model is verified by the verification set to determine whether the confidence is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the training is terminated.

[0032] Furthermore, in step S3, the hyperparameters are optimized by a grid search method, a random search method or a Bayesian optimization method.

[0033] Furthermore, the step S4 is specifically as follows:

[0034] A preset number of samples are randomly selected from the data set to calculate the F1 score, AUC, mean square error, root mean square error and mean absolute error of the picking time prediction model, and the F1 score, AUC, mean square error, root mean square error and mean absolute error are weighted averaged to obtain a performance evaluation value, and the performance of the picking time prediction model is evaluated based on the performance evaluation value.

[0035] Furthermore, the step S5 is specifically as follows:

[0036] A user interface for calling the verified picking time prediction model is created through Gradio, and the input tea image to be predicted and the environmental data to be predicted are obtained through the user interface. After preprocessing the tea image to be predicted and the environmental data to be predicted, the tea picking time prediction model is input to predict the tea picking time, and the prediction result is generated and displayed through the user interface.

[0037] Furthermore, the step S5 further includes:

[0038] The picking time prediction model is further optimized based on the prediction results.

[0039] The advantages of the present invention are:

[0040] 1. By collecting a large number of tea images and environmental data, each tea image and environmental data is preprocessed and a data set is constructed; then a picking time prediction model is created based on the YOLOv8n network, and the loss function of the picking time prediction model introducing the auxiliary bounding box is set. The YOLOv8n network is constructed based on the receptive field convolution attention module, the reverse residual attention module, and the slice-assisted hyper-inference algorithm; then the picking time prediction model is trained through the data set, and the hyperparameters including at least batch size, number of training rounds, random inactivation rate, weight decay rate and learning rate are continuously optimized during the training process, and the performance of the picking time prediction model is evaluated based on the F1 score, AUC, mean square error, root mean square error and mean absolute error. Finally, the picking time prediction model is trained through the data set. The picking time prediction model predicts the tea picking time and generates prediction results; the receptive field convolution attention module is used to improve the attention of the picking time prediction model to fresh tea leaves, suppress irrelevant background noise, and enhance recognition accuracy and generalization ability; the reverse residual attention module is used to improve the receptive field and downstream task processing capabilities of the picking time prediction model; the slice-assisted super-inference algorithm is used to improve the recognition ability of the picking time prediction model for long-distance targets and small targets; that is, the traditional YOLOv8n network is improved, and an auxiliary bounding box is introduced into the loss function to improve the convergence speed, recognition accuracy, generalization ability, long-distance target recognition ability, and small target recognition ability of the picking time prediction model, thereby greatly improving the accuracy and efficiency of tea picking time prediction.

[0041] 2. By collecting tea images with different shooting angles, lighting intensities and tea types, and performing brightness adjustment, contrast adjustment and scaling operations on each tea image, the sample size of tea images is greatly expanded, thereby greatly improving the generalization ability of the picking time prediction model.

[0042] 3. By combining tea images with environmental data including soil moisture, light intensity, air temperature, air humidity, carbon dioxide concentration, and radiation value to train the picking time prediction model, the factors affecting the tea picking time are fully considered, thereby greatly improving the accuracy of tea picking time prediction.

[0043] 4. By continuously optimizing the hyperparameters including at least batch size, number of training rounds, random inactivation rate, weight decay rate and learning rate during the training process of the picking time prediction model, the picking time prediction model is continuously optimized based on the prediction results, thereby further improving the accuracy of tea picking time prediction.

[0044] 5. By calculating the F1 score, AUC, mean square error, root mean square error and mean absolute error of the picking time prediction model, the weighted average of the F1 score, AUC, mean square error, root mean square error and mean absolute error is obtained. The performance of the picking time prediction model is evaluated based on the performance evaluation value. The combination of five indicators effectively improves the rationality of the performance evaluation of the picking time prediction model, and provides a basis for the subsequent optimization of the picking time prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0046] Figure 1 It is a flow chart of a method for predicting tea picking time by combining tea images and environmental data according to the present invention. DETAILED DESCRIPTION

[0047] The technical solution in the embodiment of the present application has the following overall idea: the picking time of tea is predicted based on a picking time prediction model constructed based on an improved YOLOv8n network, and the YOLOv8n network is constructed based on a receptive field convolutional attention module, a reverse residual attention module, and a slice-assisted super-inference algorithm; the receptive field convolutional attention module is used to improve the attention of the picking time prediction model to fresh tea leaves, suppress irrelevant background noise, and enhance recognition accuracy and generalization ability; the reverse residual attention module is used to improve the receptive field and downstream task processing capabilities of the picking time prediction model; the slice-assisted super-inference algorithm is used to improve the recognition ability of the picking time prediction model for long-distance targets and small targets; and an auxiliary bounding box is introduced into the loss function of the picking time prediction model to improve the convergence speed, recognition accuracy, generalization ability, long-distance target recognition ability, and small target recognition ability of the picking time prediction model, thereby improving the accuracy and efficiency of tea picking time prediction.

[0048] Please refer to Figure 1 As shown, a preferred embodiment of a method for predicting tea picking time by combining tea images and environmental data of the present invention comprises the following steps:

[0049] Step S1, collecting a large number of tea images and environmental data, pre-processing each of the tea images and environmental data, and then associating each of the tea images and environmental data to construct a data set;

[0050] In the specific implementation, in order to enhance the prediction effect of the picking time prediction model in practical applications, during the tea image acquisition process, the distance between the tea garden patrol robot and the tea trees can be imitated, and the shooting distance is set to 20-100cm, the distance from the ground is 100-160cm, the OS is set to 200, the locked aperture is 4.0, and the locked shutter speed is 1 / 300.

[0051] Step S2, creating a picking time prediction model based on the YOLOv8n network, and setting the loss function of the picking time prediction model for introducing an auxiliary bounding box; the YOLOv8n network is constructed based on a receptive-field convolution attention module (RFCBAMConv, Rceptive-Field Convolution Block Attention Module Conv), an inverted residual attention module (iRMB, Inverted Residual Mobile Block), and a slice-assisted super-inference algorithm; the picking time prediction model is used to identify tea regions, tea types, and tea maturity, and predict the optimal picking time of tea;

[0052] According to the depth (Depth), width (Width) and maximum number of channels (Max Channels) of the model, the YOLOv8 network is divided into five types: n, s, m, l and x.

[0053] Since the YOLOv8n network has the advantages of small floating-point operations, relatively high precision and recall rate, and fast detection speed, the YOLOv8n network is selected as the basic network to create a picking time prediction model; however, the YOLOv8n network still has a lot of room for improvement in recognition accuracy, slow convergence speed, poor adaptability to complex environments, and difficulty in recognizing distant targets. Therefore, the Backbone layer and Neck layer of the original YOLOv8n network are improved (receptive field convolution attention module, reverse residual attention module), an auxiliary bounding box is introduced in the loss function calculation, and the slice-assisted hyper-inference algorithm is integrated to form an improved YOLOv8n network;

[0054] Step S3, training the picking time prediction model using the data set, and continuously optimizing hyperparameters including at least batch size, number of training rounds, random inactivation rate, weight decay rate, and learning rate during the training process;

[0055] Step S4, evaluating the performance of the picking time prediction model based on F1 score, AUC, mean square error, root mean square error and mean absolute error;

[0056] Step S5: predicting the tea picking time by using the picking time prediction model to generate a prediction result.

[0057] In step S1, the tea image includes different shooting angles, light intensities and tea types; the shooting angle is front, overhead, overhead, side or overall; the light intensity is low, medium or high; the tea type is single bud, one bud and one leaf or one bud and two leaves;

[0058] In order to further ensure that the picking time prediction model has a strong recognition effect on tea leaves under different lighting, different slopes, and different angles, and has stronger versatility in field applications, tea images are collected in multiple time periods (different light intensities) and multiple angles (different shooting angles).

[0059] The environmental data includes soil moisture collected by a soil moisture sensor, light intensity collected by a light intensity sensor, air temperature collected by an air temperature sensor, air humidity collected by an air humidity sensor, carbon dioxide concentration collected by a carbon dioxide sensor, radiation value collected by a TBQ total radiation sensor, and corresponding data collection time.

[0060] Due to the poor network quality in the tea garden, when environmental data is transmitted through the wireless network, when the network quality fluctuates greatly or the signal is weak, it is very easy to cause data packet loss. In this case, the stage average value can be used to process the collected environmental data.

[0061] In step S1, the preprocessing of the tea leaves images and the environmental data, and then associating the tea leaves images and the environmental data to construct a data set are specifically as follows:

[0062] By using Make Sense, each of the tea images is labeled with tea area, tea type, tea maturity, and picking time, and brightness, contrast, and zoom image expansion operations are performed on each of the labeled tea images, thereby completing the preprocessing of each of the tea images;

[0063] In order to further enhance the generalization ability of the picking time prediction model in complex environments and low-light environments, and to better extract the features of tea leaves at different distances and different fields of view, image expansion operations are performed.

[0064] Performing data cleaning on each of the environmental data, including at least null value processing, missing value processing, key value processing, and merging data, and then performing standardized processing such as format conversion, sorting, and aggregation on each of the environmental data, thereby completing the preprocessing of each of the environmental data;

[0065] Based on the shooting time of each tea image and the collection time of each environmental data, each tea image and environmental data are associated and a data set is constructed.

[0066] In step S2, the formula of the loss function is:

[0067]

[0068] union=(w gt *h gt )*(ratio)2 +(w*h)*(ratio) 2 -inter;

[0069]

[0070] Among them, L Inner-CIoU Represents the loss value of the loss function; ρ 2 (c,c gt ) represents the Euclidean distance between the center points of the predicted box and the true box; d represents the diagonal distance of the minimum closed bounding box containing the predicted box and the true box; α represents the balance parameter; v represents the aspect ratio consistency measurement coefficient; inter represents the overlapping area of ​​the predicted box and the true box; union represents the union area of ​​the predicted box and the true box; represents the scale-optimized width of the real box inside the auxiliary bounding box; c r Indicates the width of the real box inside the auxiliary bounding box; represents the scale-optimized height of the real box inside the auxiliary bounding box; c l Indicates the height of the real box inside the auxiliary bounding box; represents the scale-optimized width of the prediction box inside the auxiliary bounding box; c b Indicates the width of the prediction box inside the auxiliary bounding box; represents the scale-optimized height of the prediction box inside the auxiliary bounding box; c t Indicates the height of the prediction box inside the auxiliary bounding box; w gt Indicates the width of the real box; h gt Indicates the height of the real box; ratio indicates the scaling factor, which is used to control the size of the auxiliary bounding box and calculate the loss. The value range is 0.5 to 1.5. When the ratio is less than 1, the auxiliary bounding box is smaller than the real box. Although the effective range of regression is smaller than IoU, the absolute value of the gradient is higher than that of IoU, which can accelerate the convergence of the model. When the ratio is greater than 1, the auxiliary bounding box can expand the effective range of regression and enhance the regression effect; w indicates the width of the prediction box; h indicates the height of the prediction box; Indicates the center point of the real box; (x d ,y d ) represents the center point of the prediction box.

[0071] The CIoU loss function used by the YOLOv8n network in the prediction box regression can describe the regression state of the prediction box, but it lacks adaptive adjustment function for different detection tasks and has relatively weak generalization ability. In order to enhance the detection performance of the YOLOv8n network, accelerate the prediction box regression, and improve the generalization ability, auxiliary bounding boxes are further introduced for loss calculation. That is, for high IoU samples, small auxiliary bounding boxes are used for loss calculation to improve the convergence speed. For low IoU samples, large auxiliary bounding boxes are used for loss calculation.

[0072] In step S2, the receptive field convolution attention module is used to improve the attention of the picking time prediction model to fresh tea leaves, suppress irrelevant background noise, and enhance recognition accuracy and generalization ability;

[0073] The receptive field convolutional attention module is the basis of RFCBAMConv. As a lightweight attention module, the receptive field convolutional attention module is mainly composed of independently running channel attention modules (CAM, Channel Attention Module) and spatial attention modules (SAM, Spatial Attention Module); among them, the channel attention module is responsible for paying attention to the key information in the image, and the spatial attention module is responsible for paying attention to the position information of fresh tea leaves. In the CAM stage, the convolutional block attention module calculates the maximum eigenvalue and average eigenvalue of the input feature map through the parallel MaxPool layer and AvgPool layer, uses the Share MLP module to compress and expand the number of channels, and uses the ReLU activation function to obtain two activated results. Finally, the activation results are added element by element, processed by the Sigmoid activation function, and then multiplied with the original image to ensure that the channel dimension remains unchanged. In the SAM stage, CBAM generates two feature maps through the MaxPool layer and the AvgPool layer respectively, which are concatenated and processed through the convolution layer to generate spatial attention weights. After being processed by the Sigmoid activation function, they are multiplied with the original image to restore (C: number of channels, H: height, W: width).

[0074] In specific implementation, SE (Squeeze-and-Excitation Networks) can be used to replace the channel attention module in the receptive field convolution attention module to reduce the amount of calculation; K with a step size of K 2 The convolution operation extracts feature information and performs weighted calculations on channel and spatial attention to further enhance RFCBAMConv’s attention to the spatial features of the receptive field.

[0075] The reverse residual attention module is used to improve the receptive field and downstream task processing capabilities of the picking time prediction model;

[0076] The reverse residual attention module only contains basic deep convolution and expanded window multi-head self-attention (EW-MHSA); the reverse residual attention module not only has the static short-range modeling capability of CNN and the dynamic long-range feature interaction capability of Transformer, but also can enhance the receptive field of the YOLOv8n network and improve the ability to handle downstream tasks.

[0077] Compared with the Meta Mobile Block (MMB), the reverse residual attention module is changed to a serial extended window multi-head self-attention and deep convolution (DW-Conv, Depth-Wise Convolution) operation. Compared with the original multi-head self-attention (MHSA, Multi-Head Self-Attention), the parameters Q and K in the extended window multi-head self-attention both use the image input X, and the parameter V uses the expanded Xe. This serial connection method can enhance the receptive field expansion speed of the YOLOv8n network to a certain extent.

[0078] The slice-assisted hyper-inference algorithm is used to improve the recognition capability of the picking time prediction model for long-distance targets and small targets.

[0079] In the process of fresh tea grading and identification, it is difficult to accurately identify and locate long-distance targets and small targets only by improving the YOLOv8n network, so the slice-assisted hyper-inference algorithm is further integrated. Compared with the direct recognition of the YOLOv8n network, the slice-assisted hyper-inference algorithm first processes the tea image into multiple slices (the number of slices can be set), performs target detection on each slice separately, and then restores the slice; for overlapping detection frames, the slice-assisted hyper-inference algorithm can directly perform intelligent merging during the splicing process to ensure the accuracy of the recognition results; in addition, after integrating the slice-assisted hyper-inference algorithm, the model can achieve memory optimization through image decomposition, reducing the hardware requirements of the model.

[0080] In step S3, the training of the picking time prediction model by using the data set is specifically as follows:

[0081] The data set is divided into a training set, a test set, and a validation set in a ratio of 7:2:1;

[0082] The picking time prediction model is trained by using the training set until the loss value of the loss function is less than a preset loss threshold;

[0083] The trained picking time prediction model is tested by the test set to determine whether the prediction accuracy is greater than a preset accuracy threshold. If not, the training set is expanded to continue training; if yes, then:

[0084] The tested picking time prediction model is verified by the verification set to determine whether the confidence is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the training is terminated.

[0085] In step S3, the hyperparameters are optimized by a grid search method, a random search method or a Bayesian optimization method.

[0086] The step S4 is specifically as follows:

[0087] A preset number of samples are randomly selected from the data set to calculate the F1 score, AUC, mean square error, root mean square error and mean absolute error of the picking time prediction model, and the F1 score, AUC, mean square error, root mean square error and mean absolute error are weighted averaged to obtain a performance evaluation value, and the performance of the picking time prediction model is evaluated based on the performance evaluation value.

[0088] The F1 score is the harmonic mean of precision and recall, which strikes a balance between the two and is an indicator that comprehensively considers precision and recall. AUC (Area Under the ROC Curve) is the area under the ROC curve (Receiver Operating Characteristic Curve), which provides a single indicator to evaluate the performance of a classifier. The ROC curve is a graph of the true positive rate (TPR) and the false positive rate (FPR) drawn at different thresholds. The mean squared error (MSE) and root mean squared error (RMSE) are used to measure the average squared difference between the predicted value and the actual value. The mean absolute error (MAE) is a measure of the average absolute difference between the predicted value and the actual value.

[0089] The step S5 is specifically as follows:

[0090] A user interface for calling the verified picking time prediction model is created through Gradio, and the input tea image to be predicted and the environmental data to be predicted are obtained through the user interface. After preprocessing the tea image to be predicted and the environmental data to be predicted, the tea picking time prediction model is input to predict the tea picking time, and the prediction result is generated and displayed through the user interface.

[0091] In order to reduce the difficulty of operating the picking time prediction model and allow users to easily input data, select models, and view the prediction results of the picking time prediction model, Gradio is used to build and deploy the visual recognition system, that is, to create a user interface for calling the picking time prediction model.

[0092] The step S5 further comprises:

[0093] The picking time prediction model is further optimized based on the prediction results.

[0094] In summary, the advantages of the present invention are:

[0095] 1. By collecting a large number of tea images and environmental data, each tea image and environmental data is preprocessed and a data set is constructed; then a picking time prediction model is created based on the YOLOv8n network, and the loss function of the picking time prediction model introducing the auxiliary bounding box is set. The YOLOv8n network is constructed based on the receptive field convolution attention module, the reverse residual attention module, and the slice-assisted hyper-inference algorithm; then the picking time prediction model is trained through the data set, and the hyperparameters including at least batch size, number of training rounds, random inactivation rate, weight decay rate and learning rate are continuously optimized during the training process, and the performance of the picking time prediction model is evaluated based on the F1 score, AUC, mean square error, root mean square error and mean absolute error. Finally, the picking time prediction model is trained through the data set. The picking time prediction model predicts the tea picking time and generates prediction results; the receptive field convolution attention module is used to improve the attention of the picking time prediction model to fresh tea leaves, suppress irrelevant background noise, and enhance recognition accuracy and generalization ability; the reverse residual attention module is used to improve the receptive field and downstream task processing capabilities of the picking time prediction model; the slice-assisted super-inference algorithm is used to improve the recognition ability of the picking time prediction model for long-distance targets and small targets; that is, the traditional YOLOv8n network is improved, and an auxiliary bounding box is introduced into the loss function to improve the convergence speed, recognition accuracy, generalization ability, long-distance target recognition ability, and small target recognition ability of the picking time prediction model, thereby greatly improving the accuracy and efficiency of tea picking time prediction.

[0096] 2. By collecting tea images with different shooting angles, lighting intensities and tea types, and performing brightness adjustment, contrast adjustment and scaling operations on each tea image, the sample size of tea images is greatly expanded, thereby greatly improving the generalization ability of the picking time prediction model.

[0097] 3. By combining tea images with environmental data including soil moisture, light intensity, air temperature, air humidity, carbon dioxide concentration, and radiation value to train the picking time prediction model, the factors affecting the tea picking time are fully considered, thereby greatly improving the accuracy of tea picking time prediction.

[0098] 4. By continuously optimizing the hyperparameters including at least batch size, number of training rounds, random inactivation rate, weight decay rate and learning rate during the training process of the picking time prediction model, the picking time prediction model is continuously optimized based on the prediction results, thereby further improving the accuracy of tea picking time prediction.

[0099] 5. By calculating the F1 score, AUC, mean square error, root mean square error and mean absolute error of the picking time prediction model, the weighted average of the F1 score, AUC, mean square error, root mean square error and mean absolute error is obtained. The performance of the picking time prediction model is evaluated based on the performance evaluation value. The combination of five indicators effectively improves the rationality of the performance evaluation of the picking time prediction model, and provides a basis for the subsequent optimization of the picking time prediction model.

[0100] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for predicting tea picking time by combining tea images and environmental data, characterized in that: The steps include: Step S1, collecting a large number of tea images and environmental data, pre-processing each of the tea images and environmental data, and then associating each of the tea images and environmental data to construct a data set; Step S2: Create a picking time prediction model based on the YOLOv8n network, and set the loss function of the picking time prediction model by introducing the auxiliary bounding box; the YOLOv8n network is constructed based on the receptive field convolution attention module, the reverse residual attention module, and the slice-assisted super-inference algorithm. In step S2, the formula of the loss function is: union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter; Among them, L Inner-CIoU Represents the loss value of the loss function; ρ 2 (c,c gt ) represents the Euclidean distance between the center points of the predicted box and the true box; d represents the diagonal distance of the minimum closed bounding box containing the predicted box and the true box; α represents the balance parameter; v represents the aspect ratio consistency measurement coefficient; inter represents the overlapping area of ​​the predicted box and the true box; union represents the union area of ​​the predicted box and the true box; represents the scale-optimized width of the real box inside the auxiliary bounding box; c r Indicates the width of the real box inside the auxiliary bounding box; represents the scale-optimized height of the real box inside the auxiliary bounding box; c l Indicates the height of the real box inside the auxiliary bounding box; represents the scale-optimized width of the prediction box inside the auxiliary bounding box; c b Indicates the width of the prediction box inside the auxiliary bounding box; represents the scale-optimized height of the prediction box inside the auxiliary bounding box; c t Indicates the height of the prediction box inside the auxiliary bounding box; w gt Indicates the width of the real box; h gt Represents the height of the real box; ratio represents the scaling factor; w represents the width of the predicted box; h represents the height of the predicted box; Indicates the center point of the real box; (x d ,y d ) represents the center point of the prediction box; Step S3, training the picking time prediction model using the data set, and continuously optimizing hyperparameters including at least batch size, number of training rounds, random inactivation rate, weight decay rate, and learning rate during the training process; Step S4, evaluating the performance of the picking time prediction model based on F1 score, AUC, mean square error, root mean square error and mean absolute error; Step S5: predicting the tea picking time by using the picking time prediction model to generate a prediction result.

2. A method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: In step S1, the tea image includes different shooting angles, light intensities and tea types; the shooting angle is front, overhead, overhead, side or overall; the light intensity is low, medium or high; the tea type is single bud, one bud and one leaf or one bud and two leaves; The environmental data includes soil moisture collected by a soil moisture sensor, light intensity collected by a light intensity sensor, air temperature collected by an air temperature sensor, air humidity collected by an air humidity sensor, carbon dioxide concentration collected by a carbon dioxide sensor, radiation value collected by a TBQ total radiation sensor, and corresponding data collection time.

3. The method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: In step S1, the preprocessing of the tea leaves images and the environmental data, and then associating the tea leaves images and the environmental data to construct a data set are specifically as follows: By using Make Sense, each of the tea images is labeled with tea area, tea type, tea maturity, and picking time, and brightness, contrast, and zoom image expansion operations are performed on each of the labeled tea images, thereby completing the preprocessing of each of the tea images; Performing data cleaning on each of the environmental data, including at least null value processing, missing value processing, key value processing, and merging data, and then performing standardized processing such as format conversion, sorting, and aggregation on each of the environmental data, thereby completing the preprocessing of each of the environmental data; Based on the shooting time of each tea image and the collection time of each environmental data, each tea image and environmental data are associated and a data set is constructed.

4. The method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: In step S2, the receptive field convolution attention module is used to improve the attention of the picking time prediction model to fresh tea leaves, suppress irrelevant background noise, and enhance recognition accuracy and generalization ability; The reverse residual attention module is used to improve the receptive field and downstream task processing capabilities of the picking time prediction model; The slice-assisted hyper-inference algorithm is used to improve the recognition capability of the picking time prediction model for long-distance targets and small targets.

5. The method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: In step S3, the training of the picking time prediction model using the data set is specifically as follows: The data set is divided into a training set, a test set, and a validation set in a ratio of 7:2:1; The picking time prediction model is trained by using the training set until the loss value of the loss function is less than a preset loss threshold; The trained picking time prediction model is tested by the test set to determine whether the prediction accuracy is greater than a preset accuracy threshold. If not, the training set is expanded to continue training; if yes, then: The tested picking time prediction model is verified by the verification set to determine whether the confidence is greater than a preset confidence threshold. If not, the training set is expanded to continue training; if yes, the training is terminated.

6. The method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: In step S3, the hyperparameters are optimized by a grid search method, a random search method or a Bayesian optimization method.

7. The method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: The step S4 is specifically as follows: A preset number of samples are randomly selected from the data set to calculate the F1 score, AUC, mean square error, root mean square error and mean absolute error of the picking time prediction model, and the F1 score, AUC, mean square error, root mean square error and mean absolute error are weighted averaged to obtain a performance evaluation value, and the performance of the picking time prediction model is evaluated based on the performance evaluation value.

8. The method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: The step S5 is specifically as follows: A user interface for calling the verified picking time prediction model is created through Gradio, and the input tea image to be predicted and the environmental data to be predicted are obtained through the user interface. After preprocessing the tea image to be predicted and the environmental data to be predicted, the tea picking time prediction model is input to predict the tea picking time, and the prediction result is generated and displayed through the user interface.

9. The method for predicting tea picking time by combining tea images and environmental data as claimed in claim 1, characterized in that: The step S5 further comprises: The picking time prediction model is further optimized based on the prediction results.

Images