Agricultural product detection method and system based on lightweight deployment model and application

Abstract

The invention discloses an agricultural product detection method and system based on a lightweight deployment model and application. The agricultural product detection model training method comprises the steps of providing a training set; training the neural network model to obtain an agricultural product detection model; the neural network model is obtained by adjusting a basic Yolov5s model; the basic model comprises a backbone network, a neck network and a head network; compared with the basic model, the backbone network is integrally replaced by a MobilenetV3 network, and a plurality of C3 modules in the neck network are all replaced by C3Ghost modules. According to the technical scheme provided by the invention, the basic model is improved, the improved Yolov5 model is used for detecting different qualities, including whether diseases exist or not and the types of the diseases, of the agricultural products, and the detection precision can be ensured to reach 95% or above while the light weight of the model is realized, so that the detection requirements of industrial scenes on the agricultural products are met.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural product image detection, and in particular to an agricultural product detection method, system and application based on a lightweight deployment model. Background Art

[0002] Detection of agricultural products, especially rapid and accurate disease detection, is crucial to agricultural operation and development.

[0003] For example, wheat is the staple food of mankind and is vital to global food security and human health and nutrition. Affected by weather, environment and other factors, wheat will have quality and safety problems such as fusarium wilt, mildew, scab, sprouting, etc. during storage, resulting in a large number of imperfect wheat grains.

[0004] Specifically, wheat is rich in carbohydrates, proteins, fats and inorganic salts, and is prone to toxins and mold, which seriously affect human health. Spectrometers can be used to identify wheat mold. Wheat fusarium causes wheat grains to shrink and dry, resulting in poor quality, reduced yield, and low germination rate. These wheat grains contain emetic and estrogenic toxins, which can easily lead to poisoning in humans and animals. In addition, wheat seed germination causes the total starch content of flour to decrease, while the content of damaged starch increases. In the early stage, flour mill workers conducted random inspections of incoming wheat, which was time-consuming and relied entirely on subjective evaluation. The final sorting accuracy was only about 80%.

[0005] Therefore, a fast and accurate detection algorithm that can replace manual work was developed and applied in the wheat quality detection system.

[0006] Specifically, in order to solve the above problems of manual detection, several detection methods have been developed, which will be beneficial to the analysis and application of wheat grain images. Including traditional machine learning-based methods. The defects of imperfect wheat kernels were detected using a visible near-infrared (VNIR) hyperspectral imaging system, and severely germinated kernels were classified using a classification procedure that combined spectral and spatial features. Image features were also distinguished by using a classification model. The morphological features used for classification include area, projected volume, perimeter, ellipse eccentricity, and major axis length.

[0007] In addition, traditional methods cannot make full use of image information. With the development of research, a crop disease recognition method based on convolutional neural network is proposed to try to solve the problem of early detection of wheat and rice. However, due to the imperfect model structure, the method has a large amount of computation, which limits its application. In addition, improved model structure-based methods and descriptor-based methods are introduced to detect and classify imperfect wheat grains. However, few people have combined machine vision with deep learning object detection methods to detect wheat grain quality. The target detection algorithm based on deep learning can solve the problems of multi-category target detection and multi-target connection in the same field, which can significantly improve the speed and accuracy of wheat ear detection and avoid the uncertainty of manual detection. Therefore, it is an open challenge to study wheat quality detection methods based on deep learning object detection methods, trying to solve the problem of early detection of wheat powdery mildew and yellow rust. Wheat grains are significantly different from the conveyor belt background, which reduces the difficulty of the target detection task. In order to facilitate the deployment of the algorithm, it is necessary to select a lightweight model with high classification accuracy and fast inference speed.

[0008] However, the models currently used in this field find it difficult to achieve a balance or achieve both detection accuracy and model lightweightness. Either the detection accuracy is low and it is difficult to meet the needs, or the insufficient lightweightness leads to high hardware requirements, making the model difficult to apply. Summary of the invention

[0009] In view of the deficiencies in the prior art, the purpose of the present invention is to provide an agricultural product detection method, system and application based on a lightweight deployment model, so as to achieve high-precision and lightweight agricultural product detection application.

[0010] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention includes: In a first aspect, the present invention provides a method for training an agricultural product detection model based on a lightweight deployment model, comprising: Providing a training set, the training set comprising a plurality of images of agricultural products and corresponding quality status labels; Training the neural network model based on the training set to obtain an agricultural product detection model; The neural network model is obtained by adjusting the basic Yolov5s model; the basic Yolov5s model includes a backbone network, a neck network and a head network; Compared with the basic Yolov5s model, the backbone network of the neural network model is completely replaced by the MobilenetV3 network, and multiple C3 modules in the neck network are replaced by C3_Ghost modules.

[0011] In a second aspect, the present invention further provides an agricultural product detection method based on a lightweight deployment model, which comprises: Providing an agricultural product detection model trained by the agricultural product detection model training method; The image of the target agricultural product is input into the agricultural product detection model to obtain the quality status corresponding to the target agricultural product.

[0012] In a third aspect, the present invention further provides an agricultural product detection model training system based on a lightweight deployment model, which includes: A training set module, used to provide a training set, wherein the training set includes images of a plurality of agricultural products and corresponding quality status labels; A model training module, used to train a neural network model based on the training set to obtain an agricultural product detection model; The neural network model is obtained by adjusting the basic Yolov5s model; the basic Yolov5s model includes a backbone network, a neck network and a head network; Compared with the basic Yolov5s model, the backbone network of the neural network model is completely replaced by the MobilenetV3 network, and multiple C3 modules in the neck network are replaced by C3_Ghost modules.

[0013] In a fourth aspect, the present invention further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed, the steps of the above-mentioned agricultural product detection model training method are executed.

[0014] Based on the above technical solution, compared with the prior art, the beneficial effects of the present invention at least include: The technical solution provided by the present invention improves the basic Yolov5s model and utilizes the improved Yolov5 model to detect the different qualities of agricultural products, including whether there are diseases and the types of diseases. While achieving lightweight model, it can ensure that the detection accuracy reaches more than 95%, meeting the detection needs of agricultural products in industrial scenarios.

[0015] The above description is only an overview of the technical solution of the present invention. In order to enable those skilled in the art to more clearly understand the technical means of the present application and implement them according to the contents of the specification, the following is a description of the preferred embodiments of the present invention in conjunction with detailed drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a method for training a model for detecting agricultural products provided by a typical implementation case of the present invention; Figure 2 It is a structural schematic diagram of an image acquisition device for agricultural products provided in a typical implementation case of the present invention; Figure 3is a sample image of a plurality of agricultural products provided by a typical implementation case of the present invention; Figure 4 It is a schematic diagram of a process of expanding an image provided by a typical implementation case of the present invention; Figure 5 It is a schematic diagram of a deep separable convolution process in a neural network model provided by a typical implementation case of the present invention; Figure 6 It is a schematic diagram of the operation process of the SE submodule in the neural network model provided by a typical implementation case of the present invention; Figure 7 It is a schematic diagram of the structure and operation process of the Ghost unit in the neural network model provided by a typical implementation case of the present invention; Figure 8 This is an example diagram of the test results of multiple agricultural products provided in a typical implementation case of the present invention. DETAILED DESCRIPTION

[0017] In view of the deficiencies in the prior art, the inventor of this case has proposed the technical solution of the present invention after long-term research and extensive practice. The technical solution, its implementation process and principle will be further explained as follows.

[0018] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0019] Moreover, relational terms such as “first” and “second” are merely used to distinguish one component or method step from another with the same name, but do not necessarily require or imply any such actual relationship or order between these components or method steps.

[0020] See also Figure 1 As shown, an embodiment of the present invention provides a method for training an agricultural product detection model based on a lightweight deployment model, which includes the following steps: Providing a training set, the training set includes multiple images of agricultural products and corresponding quality status marks; generally, the quality status mark may include whether there is a disease or abnormality and the type of the disease or abnormality, such as the fusarium scaly mentioned above, mold, scab, sprouting, etc. Of course, as the variety of agricultural products is different, the quality status category should also be adjusted accordingly; The neural network model is trained based on the training set to obtain an agricultural product detection model.

[0021] Among them, the neural network model is obtained by adjusting the basic Yolov5s model; the basic Yolov5s model includes a backbone network, a neck network and a head network; compared with the basic Yolov5s model, the backbone network of the neural network model is replaced as a whole with a MobilenetV3 network, and multiple C3 modules in the neck network are replaced with C3_Ghost modules.

[0022] The key improvement of the present invention is the improvement based on the basic architecture of the existing Yolov5s model. As the most representative network model in the Yolov5 detection model series, Yolov5s has the characteristics of minimum depth and feature map width in the network, small model size, fast inference speed, etc., which has led many scholars to further improve and study it. The Yolov5s model is mainly composed of four parts: input, backbone network, neck network, and output (also known as head network, Head).

[0023] Yolov5s has shown good performance in the field of target detection, with advantages such as fast training speed, good results, and easy deployment. It also has certain advantages in image classification. The overall network architecture can also be modified and run by functions to meet different performance requirements.

[0024] Yolov5s can simultaneously perform two tasks: target detection and image classification. The backbone network is used for feature extraction. The more layers of the backbone network, the richer the extracted features. Yolov5 contains a large number of C3 structures, which makes the feature extraction capability strong, but the number of parameters is large, the detection speed is slow, and the application is limited on some small mobile devices. The model occupies a large amount of memory. The practical scenario of quality inspection of a large number of agricultural products requires real-time accuracy and low latency. Therefore, the method of directly using Yolov5s for inspection cannot be widely popularized. Therefore, a more effective model with small model memory, high accuracy, and fast response speed has become the pursuit of current researchers. In the future, large-scale industrial equipment will also be improved more and more in the direction of being lightweight, portable and fast.

[0025] The purpose of this invention is to combine the lightweight network model with the yolov5s network model with better performance to build a new lightweight network that is lighter and more efficient, achieves a perfect fusion of memory size, accuracy and speed, and is suitable for deployment on embedded devices.

[0026] The main improvements of the Yolov5_Mobilenet_Ghost network structure after the improvement of the present invention are reflected in: 1. Replacing the backbone network of Yolov5 with MobilenetV3, 2. Using the C3_Ghost module to modify the original C3 module of the neck network to further reduce the model calculation amount, and in some more preferred implementations, it also includes 3. Pruning the neck network.

[0027] The above technical solution utilizes the characteristics of the lightweight deep learning model MobilenetV3, which has fast inference speed, small model parameters and calculation amount, and replaces the backbone structure of the Yolov5s network with MobilenetV3. It can reduce the model complexity when extracting image features and improve the efficiency and real-time performance of model detection. In order to solve the problem that the C3 module in the Yolov5s neck network structure has multiple convolutions, resulting in a large number of model parameters and high calculation cost, a lightweight deep learning module C3_Ghost module is used to modify the original C3 module of the neck network into a C3_Ghost module with a C3_Ghost network structure to reduce model parameters. In a more preferred implementation, the neck network layer of the Yolov5s network is pruned to remove the network with small channel layer connection correlation, thereby further simplifying the model complexity while ensuring the accuracy of model detection, improving the model detection efficiency, and making the model more lightweight.

[0028] Based on the above technical means, the present invention not only achieves the lightweight of the model without reducing the prediction accuracy in the application scenario of agricultural product quality detection, thereby promoting its wide application, but also achieves an improvement in prediction accuracy compared with multiple control methods; of course, in the specific experiments of the present invention, it can be found that even without pruning optimization, the combined performance of the lightweight model and detection accuracy is significantly stronger than many existing or control implementation methods.

[0029] In practical applications, both the training process and the detection process can be done by Figure 2 The image acquisition device shown collects images of agricultural products to construct a training set or perform detection; the obtained images are, for example, Figure 3 As shown, during training, the number of samples can generally be expanded by sample amplification, for example Figure 4 The image shown is a flipped amplification.

[0030] Based on the above content, it can be known that the neck network can also preferably be pruned. In some more preferred embodiments, the specific processing method is preferably structured pruning.

[0031] In some more specific implementation schemes, the process of structured pruning processing specifically includes: a model training process, a model pruning process and a model retraining process; The present invention retains the original structure of the network through structured pruning, so that the pruned network is easier to run on the existing hardware platform, significantly reduces the computational complexity and storage requirements of the model, and has a small loss of accuracy; wherein, during the pruning process, the pruning optimization coefficient β determines the intensity of pruning, that is, the pruning optimization coefficient of the model pruning process is β. During the pruning process, the pruning optimization coefficient β represents the proportion of parameters removed from the neck network. By adjusting β, the performance and efficiency of the model are balanced. The value range in the present invention is 0.3-0.5.

[0032] At present, most deep learning frameworks and hardware have no obvious acceleration effect on sparse connection structures after such pruning. Through structured pruning, redundant channels or entire network layers can be deleted, effectively reducing the network capacity and memory usage. At the same time, since the model structure is not destroyed, it can accelerate general hardware such as CPU. The present invention adopts the removal of sparse structural units and neuron channels with small weights as pruning criteria, sets fine-tuning parameters in the fine-tuning process, and loads pre-trained models to optimize the model accuracy after pruning.

[0033] Therefore, in order to further simplify the model complexity and improve the efficiency of the detection model, the preferred example of the present invention uses a structured pruning method to streamline the modified Yolov5 neck network. In Yolov5, the key role of the neck network is to further integrate the incoming feature information from the backbone layer. In the specific implementation, the neck layer is pruned by setting a coefficient β, where the reference value of β is 0.3-0.5, and then the optimal model pruning coefficient is determined by comparing the performance of the two training models.

[0034] Regarding specific improvements to the backbone network, in some implementation schemes, the MobilenetV3 network includes multiple stacked Bneck modules and an SPPF module; some of the Bneck modules and the SPPF module output multi-scale features to the neck network, and the neck network performs multiple upsampling based on the multi-scale features to achieve feature fusion.

[0035] Since its development, Mobilenet network has achieved great success in lightweight deep learning networks. MobilenetV1 introduced deep separable convolution in the network, enhanced feature extraction, improved training speed, and significantly reduced model calculation parameters and computation. Compared with MobilenetV1, MobilenetV2 adds an inverted residual structure to the network structure, first increases the dimension and then reduces the dimension, enhances the propagation of gradients, significantly reduces the memory usage required during reasoning, and changes the original activation function to a linear activation function. As the latest network in the series, MobilenetV3 not only retains the advantages of the previous series of networks, with small parameters and computation, but also adds SE (Squeeze-and-Excitation) submodules, changes the original swish function to s-swish function, and adds neural architecture search technology (NAS), with the aim of further accelerating the model's reasoning speed and network detection efficiency while maintaining high accuracy.

[0036] like Figure 5 As shown in the figure, in this network, compared with traditional convolution operations, deep separable convolution can separate channel and spatial features, reduce model calculation and parameter quantity, and is the key to improving network efficiency. The specific implementation is mainly to combine deep convolution and point-by-point convolution and then extract features. First, each channel is convolved independently to obtain partial feature maps, and then a 1x1 convolution operation is used to aggregate all channel features before output, in order to reduce parameters, reduce operations, and make the model lighter.

[0037] like Figure 6 As shown in the figure, the SE submodule of MobilenetV3 includes two parts: Squeeze and Excitation. This module first performs global average pooling on the feature map channels to reduce the computational complexity, compresses the feature map of each channel, adjusts the weight of each channel, and finally uses the fully connected layer to perform Excitation operations, which enhances the network model's ability to perceive and extract important features and improves detection efficiency.

[0038] The most important component of MobilenetV3 is the Bneck module, which includes the backbone and residual edge parts. The use of deep separable convolution can better extract features, and significantly reduces the number of parameters and operations compared to ordinary convolution. MobilenetV3 is mainly composed of stacked Bneck submodules. Each submodule has a similar structure, but the parameters such as the size and step size of the convolution kernel are different. As shown in the figure above, it mainly includes operations such as deep separable convolution, 1x1 convolution and h-swish activation function. The input feature map is first convolved by 1x1, then extracted by a 3x3 deep separable convolution layer, and finally reduced to the number of output channels by a 1x1 convolution.

[0039] As for the specific improvements of the neck network, in some implementation schemes, compared with the C3 module, the C3_Ghost module replaces multiple Bottleneck layers in the C3 module with a Ghost bottleneck layer; the Ghost bottleneck layer includes multiple Ghost units, and the Ghost unit includes a convolution operation and a linear operation. The convolution operation performs ordinary convolution on the input feature map to obtain multiple convolution feature maps, and the linear operation performs multi-scale linear transformation on the multiple convolution feature maps to obtain multiple linear feature maps. The output of the Ghost unit is the sum of the convolution feature map and the linear feature map.

[0040] like Figure 7 As shown in the figure, GhostNet is a lightweight network for deep learning, which mainly solves the problem of feature map redundancy. Ordinary convolution feature maps are transformed into similar feature maps using linear transformation, and high-dimensional convolution effects are produced with a small amount of computational cost, which reduces the number of parameters and computational complexity of the network model. The module that combines convolution operation and linear operation is called Ghostmodule (i.e., the Ghost unit).

[0041] Compared with ordinary convolution, Ghost model divides the ordinary convolution layer into two parts. The first part mainly performs ordinary convolution calculations to obtain feature maps with less channel information; the second part performs a series of simple linear operations on the feature maps generated by the first part, and then generates more feature maps. Finally, the two parts of the feature maps are connected to obtain a complete output feature map.

[0042] Among them, GhostConv (Ghost convolution) uses parallel convolution operations to extract features at different scales, so it can obtain detailed features of the detection target from different levels. Multi-scale feature extraction makes the model crucial to target perception and enhances the accuracy and efficiency of network model detection. GhostConv obtains rich input features while performing parallel convolution operations, so it uses the previous and next information of the input features to improve the accuracy of the model. GhostConv not only reduces the number of parameters of the model but also reduces the amount of calculation of the model. Its lightweight characteristics accurately achieve a balance between efficiency and accuracy. By reducing the computational complexity, it can be deployed on edge AI devices with average computing power. Compared with the traditional C3 module, the C3_Ghost module proposed in the present invention adds GhostBottleneck (i.e., the Ghost bottleneck layer) to replace the N Bottleneck modules in the C3 module. Ghost Bottleneck is similar to the basic residual block in ResNet, which can effectively reduce the number of parameters and the amount of calculation.

[0043] Regarding the specific structure of the Ghost bottleneck layer, in some embodiments, the Ghost bottleneck layer includes a first Ghost unit and a second Ghost unit stacked in series, the first Ghost unit increases the number of channels, and the second Ghost unit reduces the number of channels to the original input channel number of the first Ghost unit.

[0044] The Ghost Bottleneck module is mainly composed of two Ghost modules stacked together. The first Ghost module is used as an expansion layer to increase the number of channels, and the second Ghost module is used to reduce the number of channels of the output feature map so that it can match the original number of channels at the input end and reduce the size of the model parameters.

[0045] And in some more specific embodiments, the Ghost bottleneck layer is divided into a first part and a second part, and the outputs of the first part and the second part are in an additive relationship; in the first part, the input feature map passes through the first Ghost unit and the second Ghost unit in sequence and then is added to the input feature map as the output; in the second part, compared with the first part, a convolution layer with a step size of 2 is inserted between the first Ghost unit and the second Ghost unit.

[0046] The Ghost bottleneck layer is also divided into two parts, stride=1 and stride=2. The two parts are connected, which is equivalent to addition. When the stride is 1, the input feature map will pass through two Ghost modules, each of which will be followed by a corresponding activation function, and then added to the input feature to assist in network optimization to obtain a complete GhostNet structure. This process will not compress the height and width of the input feature layer, and its main function is to deepen the depth of the network. When the stride is 2, the input feature will first pass through a Ghost module for feature extraction, then perform height and width compression, and then pass through a Ghost module for feature extraction. Similarly, it will finally be added to the input feature to optimize the network. This process changes the shape of the input feature layer, effectively reducing the number of parameters.

[0047] A second aspect of an embodiment of the present invention further provides an agricultural product detection method based on a lightweight deployment model, which includes the following steps: Providing an agricultural product detection model trained by the agricultural product detection model training method provided in any of the above implementation modes; The image of the target agricultural product is input into the agricultural product detection model to obtain the quality status corresponding to the target agricultural product.

[0048] Furthermore, a second aspect of an embodiment of the present invention further provides an agricultural product detection model training system based on a lightweight deployment model, which includes: A training set module, used to provide a training set, wherein the training set includes images of a plurality of agricultural products and corresponding quality status labels; A model training module, used to train a neural network model based on the training set to obtain an agricultural product detection model; The neural network model is obtained by adjusting the basic Yolov5s model; the basic Yolov5s model includes a backbone network, a neck network and a head network; Compared with the basic Yolov5s model, the backbone network of the neural network model is completely replaced by the MobilenetV3 network, and multiple C3 modules in the neck network are replaced by C3_Ghost modules.

[0049] In addition, an embodiment of the present invention further provides a readable storage medium, in which a computer program is stored. When the computer program is executed, the steps of the agricultural product detection model training method provided in any of the above-mentioned embodiments are executed.

[0050] The technical solution of the present invention is further described in detail below through several embodiments and in conjunction with the accompanying drawings. However, the selected embodiments are only used to illustrate the present invention, and do not limit the scope of the present invention.

[0051] Example 1 This embodiment takes wheat as an example to train the model and detect wheat quality, as shown below: In order to collect images of the experimental objects, a visual acquisition system was established to obtain wheat grain images and establish a data set. These included six types of wheat samples: black embryo diseased grains, fusarium diseased grains, germinated grains, damaged grains, normal grains, and insect-damaged grains. LED lights were placed inside the integrating sphere to provide light for the industrial camera on the top to make the lighting more uniform. Six different types of wheat grains were placed directly under the integrating sphere for collection. Figure 2 The length, width and height of wheat grains are generally distributed around 5 mm, 3.5 mm and 3 mm. Considering the characteristics of wheat grain wrinkles, insect eyes and germination, the system resolution accuracy is required to be no more than 100 μm. Finally, an industrial camera with a pixel size of 2.4 μmx2.4 μm, a sensor size of 1 inch and a focal length of 12 mm was selected.

[0052] Images of fusarium-infected grains, worm-infested grains, germinated grains, black embryo-infected grains, damaged grains, and normal grains were collected. In order to obtain a general and reasonable algorithm model, the collected images were cut, disordered, and reorganized, and the image size was adjusted to a resolution of 640 × 640. In the process of training the neural network, the more images in the training set, the higher the accuracy and the lower the overfitting. Therefore, the data set collection requires that all wheat grain images should include all situations as much as possible. However, it is often difficult to directly collect comprehensive and rich image data in actual situations. The data with fewer wheat category images were expanded, mainly by flipping, increasing brightness, etc. to average the six types of wheat seed image data. Finally, 1,700 wheat grain images were obtained, and 1,400 were randomly selected as training sets and 300 images as test sets.

[0053] The neural network model is constructed in the manner shown above. The basic network architecture uses the Yolov5s model, the backbone network is replaced by the MobilenetV3 network, the C3 modules in the neck network are replaced by the C3_Ghost modules, and the neck structure pruning optimization is set with a coefficient of 0.3 or 0.5. The one with higher accuracy is taken as the final model.

[0054] A self-made dataset was selected for training, and the algorithm constructed in this paper was used for training. The experimental environment was a desktop computer. The CPU was i7-13700k, the memory was 24G, the GPU was RTX3090ti, the operating environment was Windows11, the CUDA version was 11.7, and the development language was python3.8. The final test results were as follows: Figure 8 shown.

[0055] Test Case Typically, in object detection tasks, if the intersection between the prediction and the label is greater than a threshold, the result is considered a positive sample. Otherwise, it is treated as a negative sample. Based on these analyses, the detection results can be divided into true positive (TP), false positive (FP), true negative (TN), and false negative (FN).

[0056] All the metrics used include precision (P), recall (R), mean average precision (mAP), and computational effort (FLOPs). Among them, precision (P) and recall (R) are defined as: ; Where precision is the percentage of true positives among correctly identified images in the validation set, and recall is the percentage of images in the validation set that are correctly detected as positive samples.

[0057] mAP (Mean average precision) refers to the average precision of multiple categories. AP can be described as: ; mAP@.5:95 represents the average mAP at different IoU thresholds (from 0.5 to 0.95 with a step size of 0.05), i.e., mean average precision.

[0058] FLOPs stands for the number of floating-point operations (the amount of computation in the model), which can be used to measure the complexity of the model and estimate the execution performance of the computer. It is often used as an indirect measure of the speed of neural network models.

[0059] In order to evaluate the effectiveness of the improved model, denoted as Yolov5s_Mobilenet_Ghost_P, the present invention also conducted an ablation experiment and gradually compared and analyzed the performance of different models. The experiment focused on evaluating the model's memory usage, model computation, precision, recall rate, and average accuracy. The ablation experiment results are shown in Table 1 below.

[0060] Table 1. Training and testing results of different models

[0061] Among them, Yolov5s represents the basic Yolov5s model, Yolov5s_Mobilenet means that only the backbone network is replaced as a whole with the MobilenetV3 network, Yolov5s_Ghost means that only multiple C3 modules in the neck network are replaced with C3_Ghost modules, Yolov5s_Mobilenet_Ghost means that the backbone network is replaced as a whole with the MobilenetV3 network, and multiple C3 modules in the neck network are replaced with C3_Ghost modules, but the pruning optimization of the neck network is not performed, and Yolov5s_Mobilenet_Ghost_P represents the network model in the embodiment.

[0062] As can be seen from Table 1, after replacing the Yolov5s backbone network with the Mobilenetv3 network, the complexity of the network model is significantly reduced, the model's memory usage is reduced by 48%, the computational workload is 6.0 GLOPs, a 54% reduction, the average accuracy is reduced by 2%, and the CPU inference computational workload is reduced by 53%. The results show that the Mobilenetv3 backbone effectively extracts and retains a large number of target features, improves the network's wheat detection performance, reduces the number of model parameters, speeds up the detection process, but reduces the accuracy. Using C3_Ghost to replace the C3 module of the Yolov5s neck network alone, it can be found that although the computational workload increased by 1.4 GFLOPs, the model memory usage was reduced by 1.2M, and the average model accuracy was improved by 1.2%, indicating that C3_Ghost can enhance feature extraction in the neck network and improve the model detection accuracy, but at the expense of training efficiency.

[0063] Compared with using the C3_Ghost module to replace the C3 structure in the neck network, the amount of calculation can be reduced to a certain extent and the detection performance can be improved. It can be found that after Yolov5s_Mobilenet_Ghost uses C3_Ghost to replace C3, the memory usage of the model is only 5.9M, a reduction of 57%, the amount of calculation is 4.1GFLOps, a reduction of 69%, and the average accuracy is 98.6%. It not only achieves significant advantages in lightweight, but also achieves a reversal in accuracy. Compared with the improvement of the backbone network alone and the improvement of the neck network alone, it has achieved obvious comprehensive advantages.

[0064] Furthermore, after pruning the neck network model, it can be found that the model complexity is further reduced, with an accuracy of 98.5%. While keeping the model accuracy basically unchanged, the model's memory usage is only 3.8M, a reduction of 72%, and the computational complexity is only 2.5GFLOPs, a reduction of 81%. This shows that structural pruning of the neck network can further improve the overall effect, but it does not mean that the implementation scope of the present invention necessarily includes pruning, because Yolov5s_Mobilenet_Ghost has shown obvious advantages.

[0065] In general, the improved model has the best comprehensive detection performance, which shows the pertinence and effectiveness of the improvement method provided by the present invention.

[0066] In addition, this test example also compares the performance of the lightweight model solution provided by the present invention with the existing different lightweight models, as shown below: In order to verify the effectiveness of this method for wheat seed detection, five target detection algorithms, including Faster R-CNN, SSD, Yolov3tiny, Yolov5s, and Yolov7tiny, were compared. Under the same conditions, the apple flower training set was trained based on these five target detection algorithms, and the number of network training epochs was 300. Then, the performance of the above five detection algorithms was evaluated using the test set. Table 2 lists the performance comparison of Faster R-CNN, SSD, Yolov3-tiny, Yolov5s, Yolov7-tiny and the method provided in this embodiment.

[0067] Table 2 Performance comparison results of different target detection algorithms

[0068] Based on the above experiments, the experimental comparison results are summarized as shown in Table 2. Although Faster R-CNN has a high accuracy, its ModelSize and GFLOPs are too large, occupying too many resources, slow detection speed, and not conducive to deployment. Compared with the traditional Yolov5s, the more complex model structure depth does not improve the detection accuracy of the model. On the contrary, the redundant calculation amount leads to a decrease in detection efficiency. It is not the most suitable model for wheat seed grain detection in this article. Compared with Faster R-CNN and SSD, Yolov5s, Yolov3-tiny, and Yolov7-tiny have significantly reduced model complexity and calculation amount, and ModelSize and GFLOPs are reduced a lot, but the detection accuracy has decreased, and the sacrifice is too large. In contrast, it can be found that the Yolov5s_Mobilenet_Ghost_P network improved based on Yolov5s can best meet the application requirements of efficient and accurate detection of agricultural products. It can not only perform well on granular agricultural products such as fruits or seeds (such as wheat shown in the above embodiment), but also perform well in plant organs, such as diseases or quality detection of flowers.

[0069] Compared with Yolov5s, the ModelSize of Yolov5s_Mobilenet_Ghost_P network is only 3.8M, GFLOPs is 2.5, and Map is 98.5%, which significantly reduces the inference time. Through these improved methods, the model complexity and size are further reduced, the model performance is improved, and it is easy to deploy on the device.

[0070] In summary, based on yolov5s, the present invention proposes a lightweight Yolo_Mobilenet_Ghost_P target detection model, which solves the problems of large size, complex calculation, and poor recognition accuracy of the Yolo detection algorithm. The model uses MobilenetV3 to replace the Yolov5s backbone network, redesigns the feature extraction, enhances the feature extraction capability, changes the C3 module in the Yolov5s neck network to the C3_Ghost module, and finally prunes the neck network to further reduce the size and calculation of the model. The improved model has an accuracy rate of up to 98.5%, which improves the detection accuracy. It exceeds the original Yolov5s performance; compared with existing models such as Faster R-CNN, SSD, Yolov3tiny, and Yolov7-tiny, the improved lightweight model not only guarantees the detection accuracy, but also reduces the complexity of the model structure. It shows the best comprehensive performance in agricultural product detection.

[0071] It should be understood that the above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for training an agricultural product detection model based on a lightweight deployment model, characterized in that: include: Providing a training set, the training set comprising a plurality of images of agricultural products and corresponding quality status labels; Training the neural network model based on the training set to obtain an agricultural product detection model; The neural network model is obtained by adjusting the basic Yolov5s model; the basic Yolov5s model includes a backbone network, a neck network and a head network; Compared with the basic Yolov5s model, the backbone network of the neural network model is replaced as a whole by the MobilenetV3 network, and multiple C3 modules in the neck network are replaced by C3_Ghost modules; wherein the MobilenetV3 network includes multiple stacked Bneck modules and an SPPF module, and some of the Bneck modules and the SPPF module output multi-scale features to the neck network, and the neck network performs multiple upsampling based on the multi-scale features to achieve feature fusion.

2. The agricultural product detection model training method according to claim 1, characterized in that: The neck network is subjected to structured pruning.

3. The agricultural product detection model training method according to claim 2, characterized in that: The structured pruning process specifically includes: a model training process, a model pruning process, and a model retraining process; The pruning optimization coefficient of the model pruning process is β. During the pruning process, the pruning optimization coefficient β represents the proportion of parameters removed from the neck network, and the value range of β is 0.3-0.

5.

4. The agricultural product detection model training method according to claim 1, characterized in that: Compared with the C3 module, the C3_Ghost module replaces multiple Bottleneck layers in the C3 module with Ghost bottleneck layers; The Ghost bottleneck layer includes multiple Ghost units, and the Ghost unit includes convolution operations and linear operations. The convolution operation performs ordinary convolution on the input feature map to obtain multiple convolution feature maps, and the linear operation performs multi-scale linear transformation on the multiple convolution feature maps to obtain multiple linear feature maps. The output of the Ghost unit is the sum of the convolution feature map and the linear feature map.

5. The agricultural product detection model training method according to claim 4, characterized in that: The Ghost bottleneck layer includes a first Ghost unit and a second Ghost unit stacked in series, the first Ghost unit increases the number of channels, and the second Ghost unit reduces the number of channels to the original input channel number of the first Ghost unit.

6. The agricultural product detection model training method according to claim 5, characterized in that: The Ghost bottleneck layer is divided into a first part and a second part, and the outputs of the first part and the second part are in an additive relationship; In the first part, the input feature map passes through the first Ghost unit and the second Ghost unit in sequence and then is added to the input feature map as an output; in the second part, compared with the first part, a convolutional layer with a step size of 2 is inserted between the first Ghost unit and the second Ghost unit.

7. A method for detecting agricultural products based on a lightweight deployment model, characterized in that: include: Providing an agricultural product detection model trained by the agricultural product detection model training method according to any one of claims 1 to 6; The image of the target agricultural product is input into the agricultural product detection model to obtain the quality status corresponding to the target agricultural product.

8. An agricultural product detection model training system based on a lightweight deployment model, characterized in that: include: A training set module, used to provide a training set, wherein the training set includes images of a plurality of agricultural products and corresponding quality status labels; A model training module, used to train a neural network model based on the training set to obtain an agricultural product detection model; The neural network model is obtained by adjusting the basic Yolov5s model; the basic Yolov5s model includes a backbone network, a neck network and a head network; Compared with the basic Yolov5s model, the backbone network of the neural network model is replaced as a whole by the MobilenetV3 network, and multiple C3 modules in the neck network are replaced by C3_Ghost modules; wherein the MobilenetV3 network includes multiple stacked Bneck modules and an SPPF module, and some of the Bneck modules and the SPPF module output multi-scale features to the neck network, and the neck network performs multiple upsampling based on the multi-scale features to achieve feature fusion.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed, the steps of the agricultural product detection model training method according to any one of claims 1 to 6 are executed.

Images