Deep learning-based wild ginseng rootlet detection and quality grade classification method and system

Abstract

The application discloses a wild ginseng stem detection and quality grade classification method and system based on deep learning, uses a target detection algorithm PP-Pico-Det V2 as a baseline model, improves the backbone network thereof by using a global attention mechanism, adds adaptive feature fusion in detection neck, realizes a model PP-Pico-Det-AG that is special for wild ginseng stem detection and quality grade classification, and improves the detection accuracy. The scheme adopts an online quantification method to quantize the model, reduces the volume of the model, deploys the model to an embedded device by using a Paddle-Lite framework, and realizes a wild ginseng stem detection and quality grade classification system that integrates image acquisition, quality grade classification, information management and printing functions of wild ginseng by using devices such as a camera, a thermal printer and a display, effectively improves the detection accuracy and efficiency, and enriches the application scene of the wild ginseng detection and classification system.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of wild mountain ginseng quality grade identification, and particularly relates to a wild mountain ginseng rootlet body detection and quality grade classification method and system based on deep learning. BACKGROUND

[0002] Wild mountain ginseng is a natural growth in deep mountain dense forest for more than 15 years. Wild mountain ginseng has a high appreciation space and collection value. Because wild mountain ginseng is precious and scarce, the quality grade identification of wild mountain ginseng is highly valued. The identification of wild mountain ginseng is divided into three aspects: one is quality grade identification, mainly according to the five shapes and six bodies of wild mountain ginseng to divide special ginseng, first-class ginseng and second-class ginseng; two is grading identification, that is, according to the weight of wild mountain ginseng to divide into eight levels; three is true and false discrimination, mainly using chemical composition analysis method to distinguish true and false, but it cannot identify the quality grade and destroys the shape of wild mountain ginseng. For wild mountain ginseng, due to its rarity, there is no public data set based on the whole wild mountain ginseng root stem at present, which leads to that the quality grade inspection and detection technology of wild mountain ginseng lags behind the target detection and identification research of other plants, and lacks large-scale labeled data as the basis for training and evaluation.

[0003] There are few researches on intelligent identification of wild mountain ginseng quality grade in the prior art. The patent with application number 202210710776.7 proposes a wild ginseng nondestructive identification method based on angle hyperspectral information, which mainly detects the year and origin data of wild ginseng, and cannot give the quality grade information of wild mountain ginseng. The patent with application number 201910223723.0 uses image recognition method to determine the number, position and relative size of the rootlet, body and stem of wild mountain ginseng, uses BP neural network to realize feature extraction of wild mountain ginseng, and prints the features in the form of text. This method has the following defects:

[0004] 1) This method needs to determine the wild mountain ginseng region based on edge detection. On the one hand, edge detection cannot distinguish whether there is wild mountain ginseng in the picture, even if there is no wild mountain ginseng in the picture, edge detection will still output the detection result; on the other hand, it cannot distinguish the objects with complex background or occlusion;

[0005] 2) Only the features of each part of wild mountain ginseng can be obtained, that is, only the positions of the rootlet, body and stem in the picture can be pointed out, and the quality grade information of wild mountain ginseng cannot be obtained from the morphological point of view;

[0006] 3) The BP neural network used is a multi-layer feedforward neural network trained according to the error backpropagation algorithm, which is easy to fall into local extremum in the learning process, resulting in low prediction accuracy; there is no unified and complete theoretical guidance for the selection of the number of layers and the number of neurons of the BP network, and good feature engineering is needed for technical support.

[0007] In addition, Cai Yong published the paper "Design of Wild Ginseng Grade Classification Auxiliary System Based on PP-YOLO-tiny-BiFPN" in May 2022, designed a wild ginseng grade classification model based on deep learning algorithm, and realized the classification of first-grade ginseng and second-grade ginseng. First, in the 20,000 wild ginseng picture samples provided by a certain supervision and inspection center, the location of ginseng and the grade information identified by artificial are marked, and 20,000 wild ginseng data sets are made, of which 10,000 each of first-grade ginseng and second-grade ginseng samples; then, based on the lightweight deep learning target detection algorithm PP-YOLO-tiny provided by Baidu PaddlePaddle, the detection neck is improved, and after the training and parameter optimization of the model, a deep neural network model suitable for the classification of first-grade and second-grade wild ginseng is obtained. The method has the following defects:

[0008] 1) It can only detect the main body position of wild ginseng in the target detection task, and the classification type is only first-grade and second-grade, and it cannot identify other ginsengs that are not second-grade;

[0009] 2) It cannot provide important feature information that is the basis for wild ginseng quality grade identification, such as the type of rhizome, the position and number of rhizome, and cannot effectively use these feature information to distinguish the quality grade of wild ginseng. The basis for the classification of the quality grade of wild ginseng is relatively monotonous. SUMMARY

[0010] The present application proposes a wild ginseng rhizome and quality grade classification method and system based on deep learning, constructs a deep learning model dedicated to wild ginseng quality grade identification, and based on image data, tries to mine potential features to realize the automation and intelligentization of wild ginseng quality grade identification.

[0011] The present application proposes a wild ginseng rhizome and quality grade classification method based on deep learning. First, the target detection confidence is used to judge whether the wild ginseng main body exists in the input image, and the position of the wild ginseng main body is obtained by screening the confidence area, so as to exclude the edge detection failure caused by potential shielding or complex background and the abnormal situation that there is no wild ginseng in the picture. Then, under the condition that the wild ginseng main body exists, the wild ginseng feature information such as the position of rhizome, the type of rhizome, the position of rhizome and the overall shape feature of wild ginseng is extracted, which is used as an important basis for wild ginseng quality grade identification, and then the wild ginseng rhizome and quality grade classification is realized, specifically:

[0012] Step A, constructing wild ginseng quality grade data set: classifying wild ginseng main body, including first-grade ginseng, second-grade ginseng and out-of-grade ginseng; marking wild ginseng feature information, including but not limited to wild ginseng main body, horse-tooth rhizome, round rhizome and rhizome features, and using a rectangular frame to represent the position of the feature information;

[0013] Step B, constructing the wild ginseng quality grade classification model PP-Pico-Det-AG: taking PP-Pico-Det V2 as the baseline model, improving the backbone network PP-LCNet and the detection neck network PP-LCPAN, and the specific improvement contents are as follows:

[0014] (1) Redirection of the input of the backbone network PP-LCNet, inputting the local feature information such as the position and category of the wild ginseng root and the global feature information mainly including the position and overall shape of the wild ginseng body, inputting the shape position information and the main body information into the backbone network at the same time;

[0015] (2) For PP-LCNet, 1 layer of depth separable convolution layer is added in Block4, and 2 layers of depth separable convolution layer are added in Block6, and the improved PP-LCNet contains 6 blocks and a total of 17 layers;

[0016] (3) For PP-LCNet, the original Block2-Block6 is Basemode structure, and the GAM attention mechanism is introduced in the last two layers of Block6, so that BasemodeG structure is used;

[0017] (4) The Basemode adopts the structure of standard convolution + batch normalization + HardSwish activation function, and the BasemodeG changes the Depthwise Separable Convolution to depthwise convolution + GAM attention mechanism + pointwise convolution structure based on the Basemode;

[0018] (5) For the detection neck LCPAN, the adaptive feature fusion algorithm ASFF is connected after the output feature maps P6, P5 and P4, to obtain the output feature maps A6, A5 and A4, then A6, A5 and A4 three feature maps are fused to obtain A7, and A7 replaces the original P7 for output;

[0019] (6) The high confidence region of the position of the root, the rootlet and the body of wild ginseng is located by using target boundary box regression, and the region of interest is obtained by cropping; the local shape feature information and the global semantic feature information of wild ginseng are extracted by inputting the region image into PP-LCNet for forward propagation; the local shape feature information includes but is not limited to the position, type and size of the rootlet, and the global semantic feature information includes but is not limited to the overall shape and texture of the main body of wild ginseng; the region of interest is aligned by using the bilinear interpolation method; they are respectively encoded into feature vectors by full connection layer and pooling operation, and the feature vectors are fused by splicing and weighted summation, and then input into PP-LCNet for back propagation, and finally the quality grade classification of wild ginseng is carried out by comprehensively considering the position information of the root, the rootlet and the body and the shape feature information.

[0020] Step C, training the constructed PP-Pico-Det-AG and deploying the model: the dynamic graph model of the PP-Pico-Det-AG derived through model training is converted into a static graph model, and is converted into an.nb file, and then is deployed by using a Paddle-Lite framework;

[0021] Step D, realizing quality grade identification and classification of wild ginseng and detection of wild ginseng feature information position based on the deployed model.

[0022] In addition, the application further provides a wild ginseng stem and leaf body detection and quality grade classification method system based on deep learning, which comprises an image acquisition module, a wild ginseng detection controller and a model updating module, and the PP-Pico-Det-AG and a WEB information management system are deployed in the wild ginseng detection controller.

[0023] The image acquisition module inputs the collected wild ginseng image to be identified into the wild ginseng detection controller for inference prediction, and stores the detection result in the WEB information management system;

[0024] The model updating module is used for iterative training of the PP-Pico-Det-AG, and the result detected by the system is manually adjusted and added to the wild ginseng data set, so as to expand the data required during model updating and training;

[0025] The wild ginseng detection controller adopts a Raspberry Pi 4B+ development board, and the Paddle-Lite framework and the wild ginseng quality grade classification model PP-Pico-Det-AG are transplanted into the wild ginseng detection controller, so that the wild ginseng quality grade classification device is miniaturized and intelligentized.

[0026] The WEB information management system provides a graphical interface for users, and completes wild ginseng information management, uploads wild ginseng images, and photographs measured wild ginseng images, integrates the functions of each module and provides a visual interface.

[0027] Compared with the prior art, the application has the advantages and positive effects that:

[0028] (1) According to the wild ginseng image data provided by a certain inspection and detection center and the results identified by manual method, the positions and quality grade information of Ma Ya Lu, garden Lu, and body of wild ginseng sample pictures and other characteristics are marked to make a wild ginseng data set; a wild ginseng quality grade classification model based on a deep learning algorithm is designed, which meets the specifications of transfer learning and provides the possibility for future model transfer learning; wild ginseng pictures are collected by a camera, and the existence of the main body of wild ginseng is judged according to the confidence of the wild ginseng quality grade classification model, and the feature information of the position detection and grade classification of each feature part of wild ginseng is extracted in the case of existence, the position information and confidence of the main body of wild ginseng and Ma Ya Lu, garden Lu and body are marked in the picture, and the quality grade information of wild ginseng is judged according to the local and global morphological feature information and position information of wild ginseng, including first-class ginseng, second-class ginseng and out-of-specification ginseng, which effectively improves the efficiency and accuracy of wild ginseng quality grade identification; a wild ginseng information management system is designed and realized, which saves the sample and result of each detection for future backtracking and transfer learning;

[0029] (2) The PP-Pico-Det-AG proposed in the present application is quantized online, realizes the characteristics of lightweight deployment, high accuracy, small model capacity and fast reasoning, can be deployed on resource-limited embedded identification devices, and can also be deployed on machines integrated in wild ginseng identification centers. The quality grade identification technology of wild ginseng is realized to be automatic and intelligent, so that the identification work is more efficient, fair and just. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The PP-LCNet structure schematic diagram of the embodiment of the present application is shown in the figure;

[0031] Figure 2 The Basemode structure schematic diagram of the embodiment of the present application is shown in the figure;

[0032] Figure 3 The detection neck PP-LCPAN structure schematic diagram of the embodiment of the present application is shown in the figure;

[0033] Figure 4 The improved detection neck PP-LCPAN structure schematic diagram of the embodiment of the present application is shown in the figure;

[0034] Figure 5 The improved PP-LCNet structure schematic diagram of the embodiment of the present application is shown in the figure;

[0035] Figure 6 The improved PP-LCNet detailed structure schematic diagram of the embodiment of the present application is shown in the figure;

[0036] Figure 7 The BasemodeG structure schematic diagram of the embodiment of the present application is shown in the figure;

[0037] Figure 8 The flowchart of the method applied in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0038] In order to enable persons skilled in the art to more clearly understand the above-mentioned objects, features and advantages of the present application, the present application will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict. In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, and therefore, the present application is not limited to the specific embodiments disclosed below.

[0039] Embodiment 1, the embodiment discloses a wild ginseng stem and root detection and quality grade classification method based on deep learning, a wild ginseng quality grade dataset is created using a dataset annotation tool, a position detection and classification model is constructed, and model training and parameter optimization are performed, thereby effectively realizing wild ginseng main body, stem and root position detection, stem type classification and wild ginseng quality grade classification.

[0040] Specifically, the following steps are included:

[0041] 1. Constructing a wild ginseng quality grade dataset

[0042] Generally, a target detection algorithm needs to be trained on a dataset, and the quality of the dataset has an important influence on the performance of the final model. In the embodiment, in order to classify the quality grade of wild ginseng, a dataset containing annotation information needs to be created. The original samples used in the wild ginseng dataset are provided by a certain detection center, and there are a total of 30629 samples. The open source annotation tool Labelme is used to create a wild ginseng quality grade dataset, and the wild ginseng main body is classified and labeled, including first-class wild ginseng, second-class wild ginseng and substandard ginseng. The positions of the main body, Ma Ya stem, round stem and root of wild ginseng are also labeled. After labeling, the constructed dataset contains 16392 first-class wild ginseng, 12389 second-class wild ginseng and 1848 substandard ginseng. The dataset is divided into training set, validation set and test set in the ratio of 8:1:1 to ensure that the model is evaluated and verified at different stages.

[0043] During model training and inference, the samples in the dataset need to be preprocessed and data augmented. The preprocessing operations include adjusting image size, standardization, normalization and other operations. Data augmentation includes random cropping, random flipping and random distortion. By randomly changing the brightness, contrast and color of the dataset, as well as randomly padding and randomly scaling, the generalization ability of the model is enhanced.

[0044] 2. Design of wild ginseng quality grade classification model

[0045] The PP-Pico-Det-AG wild ginseng quality grade classification model is obtained by improving the backbone network and the detection neck network of the PP-Pico-Det V2 selected as the baseline model, and strengthening the network feature extraction capability. The picture is input into the improved backbone network PP-LCNet for feature extraction, and then input into the improved detection neck LCPAN for multi-scale feature fusion. Then the feature map is transmitted to the classification head for positioning and classification, and finally the detection result is obtained.

[0046] The backbone network PP-LCNet of the PP-Pico-Det V2 is divided according to the level of output, and mainly composed of 6 blocks, including 14 basic modules, as shown in Figure 1 The first basic module in Block1 is a 3x3 standard convolution, and the remaining each basic module is composed of a depth separable convolution, and the structure Basemode is as shown in Figure 2 The output feature maps of Block4, Block5 and Block6 in the network are sent to the detection neck network as the output of the backbone network. The detection neck PP-LCPAN plays a key role in the target detection algorithm, which receives the output feature maps from the backbone network, performs some processing and outputs to the subsequent classification head, as shown in Figure 3 The classification head of the PP-Pico-Det V2 is PicoHead, which uses depth separable convolution and 5x5 convolution to expand the receptive field.

[0047] The PP-Pico-Det-AG model capable of accurately classifying wild ginseng quality grade and realizing position detection of main body, cane and root and type classification of cane is designed, which takes PP-Pico-Det V2 as the baseline model, and the specific implementation is as follows:

[0048] 1) Detection process design

[0049] The input end of the backbone network PP-LCNet of the PP-Pico-Det V2 model is redirected, and local feature information such as the position and category of the root and cane of wild ginseng and global feature information mainly including the position of the main body of wild ginseng and the overall shape characteristics are required to be input, so as to fully utilize the shape feature information of the root and cane, and judge the quality grade of wild ginseng.

[0050] To achieve the above purpose, the target detection algorithm is used to detect and locate the position information of the stem, leaf and body of wild ginseng, and then output the bounding box and confidence of each target. The regions with confidence higher than 50% are selected, and the stem, leaf and body regions are selected for cropping to obtain the region of interest (ROI). The divided ROI region is input into PP-LCNet for forward propagation to obtain the morphological feature information of each region at different layers. The bilinear interpolation method is used for ROI alignment (ROI Align) operation. Through full connection layer and pooling operation, they are respectively encoded into feature vectors and spliced and weighted summed. The fused feature vector contains the information of the stem and leaf region and the main body position information of wild ginseng. Finally, they are input into PP-LCNet for back propagation to enable it to classify the quality grade according to the main body and morphological feature information of wild ginseng.

[0051] 2) Improvement of attention mechanism in backbone network

[0052] In the fourth block of the backbone network, one layer of depth separable convolution layer is added to increase the ability to capture the diversity and complexity of features. In the sixth block, two layers of depth separable convolution layer are added to further improve the expression ability and discrimination ability of features. The improved backbone network PP-LCNet has 6 blocks and 17 layers, as shown in Figure 5 , and the detailed structure is shown in Figure 6 . Block1 is called STEM block, which is composed of one layer of ConvBNLayer. Block2-Block6 are composed of different numbers of depth separable convolution layers to enhance the importance and relevance of features.

[0053] The attention mechanism of the PP-Pico-Det V2 original model is the SE attention mechanism, but the SE attention mechanism only considers the channel information of the feature map and ignores the spatial information. Moreover, when the SE attention mechanism suppresses unimportant pixels in the feature map, it has the problem of low efficiency. The GAM attention mechanism is composed of a channel attention module (CAM) and a spatial attention module (SAM). Compared with the SE attention module, it increases the extraction of spatial features. The GAM attention mechanism pays attention to the interaction between channels and space, enhances cross-dimensional information interaction, and can capture important features in three dimensions.

[0054] Because different wild ginseng individuals have different sizes, different sizes of each wild ginseng bowl, different sizes and numbers of the bowl, there is a problem of multi-scale detection. By connecting the adaptive feature fusion algorithm ASFF after detecting the output feature map P6, P5 and P4 of the neck, introducing the fusion operation of channel attention weight and spatial feature, effectively fusing the multi-scale feature map, improving the perception ability of the target detection algorithm to multi-scale target, and further enhancing the feature extraction ability of the detection neck.

[0055] In order to further improve the performance of the backbone network, GAM attention mechanism is introduced in the last two layers of Block4, Block5 and Block6 respectively for experiment, and the average precision, accuracy, reasoning delay and volume of the model are evaluated respectively. After analysis, the effect of introducing GAM attention mechanism in the last two layers of Block6 is the best, which can enhance the importance of key features. Therefore, the scheme of introducing GAM attention mechanism in Block6 is adopted in the present application, and the BasemodeG structure of Block6 is as shown in Figure 7 After the above improvement, the average precision (Mean Average Precision, mAP for short) and accuracy (Accuracy, Acc for short) of the model reach the highest value in the acceptable range.

[0056] 3) Enhance the feature extraction ability of the detection neck network

[0057] PP-LCPAN uses 1*1 convolution to unify the channel number of the feature with the minimum channel number output by the backbone network, thereby reducing the calculation amount and ensuring that the feature fusion performance is not affected. In order to enhance the feature extraction ability of the detection neck PP-LCPAN and improve the detection effect of the model, the adaptive feature fusion ASFF is connected after the output feature maps P6, P5 and P4 of the PP-LCPAN, to obtain the output feature maps A6, A5 and A4, and then the three feature maps are fused to obtain A7, and A7 is output instead of the original P7, as shown in Figure 4 .

[0058] When PP-LCNet is trained for target detection and classification tasks, the model first obtains the ability to judge the target position through target bounding box regression in the detection process, and then obtains the classification ability through the classification head in the classification process. Overall, the detection process includes steps such as convolution, pooling, anchor box, target bounding box regression; the classification process includes steps such as convolution, pooling, anchor box, classification head, non-maximum value prediction, wherein the boundary position obtained by the classification head in the classification process is obtained by target bounding box regression in the detection process.

[0059] Specifically, the data input image is a three-channel (RGB) input image with an input size of 320x320 pixels, and high-level semantic features in the image are gradually extracted through an EfficientNet convolutional feature extraction network. First, different scale features are fused in the feature pyramid network to generate multi-layer feature representations, and the feature fusion operations are repeatedly performed up and down and left and right to effectively pass information and improve the expression ability of the features. The Compound Scaling method is used to balance the accuracy and computational efficiency of the model, and the depth, width and resolution of the network are scaled to achieve the best performance under different resource budgets.

[0060] Then scale division is performed on the image, the image is divided into multiple grids, and multiple candidate boxes are predicted for each grid. In this embodiment, the image is divided into 13x13, 26x26 and 52x52 grids, and each grid is responsible for detecting objects of a specific size. Each candidate box is represented by five basic attributes: the center coordinates of the bounding box (offset value relative to the grid), the width and height of the bounding box, and the predicted probability of the target class. Candidate boxes (anchor boxes) are generated at different positions and scales to locate and identify targets. In target bounding box regression, the offsets between the target box and the anchor box are learned to accurately locate the target position. The model applies a classifier to each candidate box through convolution and fully connected layers to determine whether it contains a target and the class to which the target belongs. The classifier outputs a probability distribution for each class, indicating the likelihood that the candidate box belongs to a certain class.

[0061] Next, the classification head is used to make target classification predictions on each anchor box, receiving the feature map extracted by the target detection network as input, and converting the features into final class prediction results through further convolution and fully connected operations. In addition to using a deep learning model to identify the main candidate box of an object, a watershed algorithm is also used for verification, which treats the image as a terrain based on its gray level, color or gradient information, and separates the foreground and background in the image by constructing a water flow model.

[0062] Since one target may be detected by multiple candidate boxes, in order to remove redundant candidate boxes, a non-maximum suppression (NMS) algorithm is used for screening. The NMS algorithm retains the most representative candidate box based on the confidence and overlap of the candidate boxes. In this way, candidate boxes with high overlap can be excluded, and only the most relevant candidate boxes are retained. After NMS screening, the final output result includes the retained candidate boxes and their classification results. For each candidate box, the output result usually includes the class label of the target, the confidence score, and the coordinates of the bounding box (represented by the pixel positions of the top-left corner and the bottom-right corner). These classification results can locate target objects in the image, identify the type of objects, and perform further analysis and application.

[0063] Finally, the model combines the outputs of the detection process and the classification process to obtain the complete target detection result, i.e., the bounding box information and the confidence of the class of each target. Through the above process, the model not only realizes the ability to train the bounding box regression parameters through the position information and then obtain the position information prediction, but also achieves the purpose of helping to better classify the quality grades by giving various feature information containing the shape class and position information.

[0064] After the model is built, the model needs to be trained. In this embodiment, a server provided by Baidu is used to train the model, which has a memory of 100G, a display memory of 32G, a display card model of Nvidia A100, and an operating system of Ubuntu. In terms of data set, the training set of the Yeshan ginseng detection data set prepared above is used for training, and the performance test is carried out on the test set thereof.

[0065] The training parameters of the model are set as follows: in terms of the selection of batch size, generally, the larger the batch size, the more accurate the gradient descent direction and the smaller the shock, but if the batch size is too large, local optimum may occur. Therefore, after considering gradient descent, memory utilization, and sample balance, the size of the batch size is selected to be 64; the training round epoch is 300 rounds, the model performance is evaluated every 20 rounds, and 388 iterations are performed in each training round; in terms of learning rate, since the weights of the model are randomly initialized, using a large learning rate at the beginning of training may cause oscillation of the model. In order to slow down the early overfitting phenomenon of the model in the initial stage and maintain the stability of the distribution, this embodiment uses the LinearWarmup method to linearly increase the learning rate from low to the set value, and combines the CosineDecay method to improve the convergence effect of the model, so the initial value of the learning rate is set to 0.08; the momentum optimization algorithm is used as the optimizer, the momentum factor is set to the default value of 0.9, the regularization method is L2Decay, and the coefficient is set to 0.00004.

[0066] Through the above design and improvement, a model suitable for quality grade classification of Yeshan ginseng is constructed. First, the existence of the main part of Yeshan ginseng is verified according to the confidence, and then the main position of Yeshan ginseng, the type and position of the reed bowl, and the position of the stem are provided under the condition that the main part of Yeshan ginseng exists, which provides a reliable reference basis for the quality grade identification result of Yeshan ginseng. Then, the model gives the quality grade and confidence of Yeshan ginseng according to these bases. The model has the characteristics of high accuracy, small capacity, and fast reasoning speed. The model is designed for resource-limited embedded identification devices, providing automatic and intelligent technical support for the quality grade identification of Yeshan ginseng, making the identification work more efficient, fair, and just.

[0067] In order to realize the identification and classification of wild ginseng quality grades and the detection of wild ginseng feature information positions on resource-limited embedded devices, the embodiment operates according to the following steps:

[0068] 1) Model quantization processing: In the model training process, floating-point numbers need to be used to enhance the training effect of the model. However, in the model inference process, most of the time does not require such high calculation precision, that is, low-precision operation will not have too much impact on the model inference result. Therefore, model quantization can achieve a substantial reduction in model size, reduce computational complexity, reduce the space occupied by the model deployed on embedded devices, and improve the prediction speed of the model by mapping full-precision parameters in the model to limited integer spaces such as INT8 at the cost of a small amount of precision loss. Using the idea of analog quantization, the online quantization method of PaddlePaddle is used to update the weights during model training, which realizes fitting and reduces quantization error.

[0069] 2) Framework deployment: cross-compile Paddle-Lite using a cross-compilation tool, then deploy it in the wild ginseng detection controller, and install other dependent packages related to model conversion and deployment.

[0070] 3) Model export: In the model export process, the model structure and weight parameters obtained by final training are stored locally. The stored model file contains the dynamic graph model of the forward, backward and optimizer information of the model, mainly the.pdparams and.pdopt files. The.pdparams file saves the weight parameters of the model, and the.pdopt file saves the optimizer state information of the model. Because only the forward structure and forward weight parameters in the model are needed when deploying the model, static graph models.pdmodel and.pdiparams files need to be exported. The.pdmodel file saves the structure information of the model, and the.pdiparams file saves the weight parameters of the model. Through the above conversion, the size of the model file is effectively reduced, and the loading performance of the model in the production environment is improved, realizing more efficient deployment and inference.

[0071] 4) Model conversion: convert the static graph model obtained in the above steps into.nb file in naive_buffer format for deployment and calling in the embedded controller.

[0072] 5) Model deployment: deploy the.nb file to the embedded controller-raspberry pi 4B, call the.nb file for inference, realize the identification and classification of wild ginseng quality grades and the detection of wild ginseng feature information positions.

[0073] 3、Model application

[0074] The specific application process is as shown in the following table: Figure 8 To obtain the wild ginseng image sample, the CSI-2 protocol is used to drive the camera to ensure that the camera can communicate with the embedded device. Initialize and read the image: instantiate the camera as an object and perform initialization configuration, then open the camera and read the wild ginseng image in real time, save the image to a designated location for subsequent processing and analysis.

[0075] The wild ginseng sample pictures collected by the camera in real time are standardized and normalized, and the normalized pictures are input into the model for forward propagation. The model classifies and detects the wild ginseng in the image and outputs the quality grade, target position coordinates and confidence score. The detection results (including quality grade, position information, etc.) and detection date information are stored in the database for subsequent query and analysis.

[0076] According to the size of the collected wild ginseng image, the relative position coordinate information needs to be transformed, and the specific transformation is as follows:

[0077] Because the input image size does not strictly match the size required by the network, the normalization process mainly performs padding or cropping, scaling and offsetting on the collected pictures, maps the pixel values to the [-1, 1] interval, and adjusts the image size to 320x320 or other sizes suitable for model input. The specific process is as follows.

[0078] 1) Padding or cropping transformation: add boundary pixels around the image or crop boundary pixels to transform the size of the image, control the size of the input image to be in a 1:1 ratio;

[0079] 2) Scaling and offsetting transformation: when the input image needs to be scaled to the input size of the network, multiply the coordinates of the bounding box by the scaling factor to obtain the coordinate position on the original image according to the scaling ratio of the image; if there is an offset between the original image and the input image, the coordinates also need to be translated accordingly.

[0080] 3) Scale transformation: coordinate transformation on different levels of feature maps. Since the target detection network uses multiple layers of feature maps to predict targets of different sizes and proportions, it is necessary to map the relative coordinates of the target to the absolute coordinates corresponding to the different scale feature maps. This can be achieved by scaling and mapping the coordinate information of the target according to the pyramid structure of the network.

[0081] 4) bounding box regression and point alignment transformation: bounding box regression is a method of learning the positional offset of the target through training. The deep neural network outputs the positional offset value of the target, and then applies these offset values to the coordinates of the candidate box to obtain the final target bounding box. Bounding box regression corrects the positioning error introduced by network prediction, thereby improving the accuracy of target detection; point alignment transformation is a transformation of the coordinates of the target key points, to detect and locate the key points of the target. Through point alignment transformation, the key point coordinates of the target on different scale feature maps are mapped to the absolute coordinate position in the original image.

[0082] Through the above transformation, the absolute coordinate information of the target to be detected in the original picture is obtained, i.e. the position information of the main body, the stem, the toothed reed and the round reed of the wild ginseng in the original picture. The quality grade category information is processed by non-maximum suppression (NMS) to complete the quality grade recognition of the wild ginseng.

[0083] Example 2, based on the wild ginseng quality grade classification method proposed in Example 1, this embodiment proposes a wild ginseng stem and reed body detection and quality grade classification system based on deep learning, which includes an image acquisition module, a wild ginseng detection controller, a human-computer interaction device module and a model updating module. The wild ginseng stem and reed body detection and quality grade classification model and the WEB information management system are deployed in the wild ginseng detection controller.

[0084] The wild ginseng stem and reed body detection and quality grade classification dataset is made in combination with Example 1, then the network structure of the target detection algorithm is designed and improved, and the model is trained to obtain the wild ginseng stem and reed body detection and quality grade classification model, and the trained model is deployed to the wild ginseng detection controller; through the image acquisition module, the collected wild ginseng image is input to the wild ginseng detection controller for inference prediction, and the detection result is transmitted to the WEB information management system, if the user selects to save the detection result, it is saved to the database for the user to view the wild ginseng detection result, and the human-computer interaction device module can also be called to print the result; after each detection, the prediction result can also be provided as a training sample to the system, combined with the model updating module, after adjustment and confirmation by the administrator, it is added to the wild ginseng training dataset, providing more training data for future model updating.

[0085] Image acquisition module: this module uses Raspberry Pi Camera v2.1 sensor to collect wild ginseng images, and the photosensitive chip model is selected as Sony IMX219, and finally the collected images are transmitted to the wild ginseng detection module for stem and reed body detection and quality grade classification.

[0086] Wild mountain ginseng detection controller: the controller is the carrier of the wild mountain ginseng body detection and quality grade classification model, and needs to have certain computing power. Therefore, the specific model of the controller of the system is Raspberry Pi 4B+, and the operating system is Ubuntu. The wild mountain ginseng body detection and quality grade classification model needs to select a lightweight neural network model, and needs to consider the detection accuracy and inference delay as well as the model size. In this embodiment, PP-Pico-Det-AG designed in embodiment 1 is used as the system model.

[0087] WEB information management system: the WEB information management system provides a graphical interface for users, integrates the functions of each module and provides a visual interface, mainly completes wild mountain ginseng information management, real-time shooting of measured wild mountain ginseng images, local uploading of wild mountain ginseng images, display of inference results and the like. The WEB system is deployed on the controller, so that the controller can not only realize wild mountain ginseng body detection and quality grade classification, but also realize information management in the detection and grade classification process.

[0088] Model updating module: the model updating module refers to iterative training of the wild mountain ginseng body detection and quality grade classification model, and adding wild mountain ginseng data set after manual adjustment of the system detection result, so as to expand the data required for model updating training, improve the generalization ability of the wild mountain ginseng body detection and quality grade classification model, and maintain the long-term vitality of the model.

[0089] Human-computer interaction device module: the human-computer interaction device module mainly includes a display and a thermal printer. The display provides the function of displaying and operating the WEB system for users. The thermal printer provides the function of printing wild mountain ginseng inference result label for users. The label content includes wild mountain ginseng number, position coordinate information of wild mountain ginseng body, wild mountain ginseng quality grade classification information and inference date and the like.

[0090] In this embodiment, the relational database management system MySQL is used to store related data. The main database tables include a user information table, a role permission table, a wild mountain ginseng information table and a wild mountain ginseng detection result information table. The user information table mainly stores the basic information of users. The role permission table mainly stores the relationship between roles and permissions. The roles include super administrator, administrator and ordinary user and the like. The wild mountain ginseng information table mainly stores the information of wild mountain ginseng owner. The wild mountain ginseng detection result information table stores the position and confidence information of the body, the main body, the wild mountain ginseng body and the like after each detection, the quality grade classification information and its confidence, the detection time, the detection personnel and the like. One wild mountain ginseng can have multiple inference results.

[0091] Experimental verification

[0092] The PP-Pico-Det-AG designed in the application is compared with commonly used lightweight models such as SSDlite-MobilenetV3, PP-YOLO-BiFPN-tiny, YOLOv5NANO and the like in performance, mainly comparing the mAP, Accuracy, F1-score and Kappa coefficient of the models, and the experimental results are shown in Table 1, wherein P represents Precision and R represents Recall.

[0093] Table 1: Performance comparison of different models

[0094]

[0095] From the data in the above table, it can be seen that the application performs well in various indicators, and can achieve an average accuracy of 99.25% in the wild ginseng quality grade classification test sample. In the wild ginseng quality grade classification and the body target detection task of the wild ginseng, using mAP as an evaluation index, when IoU is 0.5, the mAP value reaches 0.791, which can accurately detect the position of the main body, the body and the body of the wild ginseng and perform quality grade classification. This shows that the model of the application has high accuracy and effect in the wild ginseng quality grade classification and target detection task.

[0096] The above is only a preferred embodiment of the application, and is not intended to limit the application in other forms. Any skilled person in the art can modify or change the above disclosed technical content to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made on the basis of the technical essence of the application to the above embodiments without departing from the technical scheme of the application still falls within the protection scope of the application.

Claims

1. A method for detecting the rhizome and classifying the quality grade of wild ginseng based on deep learning, characterized in that, Includes the following steps: Step A: Construct a wild ginseng quality grade dataset: Label the wild ginseng body into categories, including first-class, second-class, and unequal-class parameters; identify and label the wild ginseng feature information, which includes but is not limited to the wild ginseng body, horse tooth rhizome, round rhizome, and stem features, and use rectangles to indicate the location of the feature information; Step B: Construct the wild ginseng quality grade classification model PP-Pico-Det-AG: Use PP-Pico-Det V2 as the baseline model, and input the local and global feature information of wild ginseng; The structures of its backbone network PP-LCNet and detection neck network PP-LCPAN are improved to enhance feature fusion capabilities and acquire more feature information; In step B, the specific design and improvement of the model's structure include: (1) The baseline model PP-Pico-Det V2 includes the backbone network PP-LCNet, the detection neck LCPAN and the classification head. The backbone network PP-LCNet contains 6 blocks with a total of 14 layers. Blocks 2 to 6 are composed of the Basemode structure. One depthwise separable convolutional layer is added to Block 4 of the backbone network PP-LCNet, and two depthwise separable convolutional layers are added to Block 6. The GAM attention mechanism is introduced in the last two layers of Block 6 to form the BasemodeG structure. After the improvement, PP-LCNet contains 6 blocks with a total of 17 layers. (2) For the detection bottleneck LCPAN of the baseline model PP-Pico-Det V2, the adaptive feature fusion algorithm ASFF is connected after the output feature maps P6, P5 and P4 to obtain the output feature maps A6, A5 and A4. Then, the three feature maps A6, A5 and A4 are fused to obtain A7, which replaces the original P7 for output. Step C: Train and deploy the constructed wild ginseng quality grade classification model: After model training, convert the dynamic graph model of the exported wild ginseng quality grade classification model into a static graph model, convert it into a .nb file, and then deploy it using the Paddle-Lite framework. Step D: Based on the deployed model, identify and classify the quality grade of wild ginseng and detect the location of wild ginseng feature information.

2. The method for detecting and classifying the quality grade of wild ginseng rhizome based on deep learning according to claim 1, characterized in that: In step B, the input of the backbone network PP-LCNet of the PP-Pico-Det V2 model is redirected to include local feature information on the location and category of the ginseng rootstock and global feature information mainly on the location and overall morphological features of the ginseng body. This is to fully utilize the morphological feature information of the rootstock to determine the quality grade of the wild ginseng. Specifically: (1) After detecting and locating the morphological categories and morphological location information of the ginseng's stem, rhizome, and body, the bounding box and confidence score of each target are output. Based on the confidence score, the morphological regions of the stem and rhizome with high confidence scores and the main body location region of the wild ginseng are selected and cropped to obtain the image of the region of interest. (2) Input the above results into PP-LCNet for forward propagation to extract the local morphological features of the location, type, and size of the wild ginseng's rhizome and the overall morphology and texture of the wild ginseng body, and display them in the feature maps of different layers. (3) Use bilinear interpolation to align the region of interest and encode it into a feature vector through a fully connected layer and pooling operation respectively; (4) The feature vectors are fused by splicing and weighted summation, and then backpropagated through PP-LCNet for classification, so as to classify the quality grade by combining the positional information and morphological features of the ginseng.

3. The method for detecting and classifying the quality grade of wild ginseng rhizome based on deep learning according to claim 1, characterized in that: In step (1), the Basemode of the baseline model adopts the structure of standard convolution + batch normalization + HardSwish activation function. The BasemodeG of Block6 changes the Depthwise Separable Convolution to the depthwise convolution + GAM attention mechanism + pointwise convolution structure based on the Basemode.

4. The method for detecting and classifying the quality grade of wild ginseng rhizome based on deep learning according to claim 1, characterized in that: In step D, in practical applications, the collected wild ginseng sample images are standardized and normalized, the pixel values ​​are mapped to the range of [-1, 1], the image size is unified, and the trained model is called to perform inference on the preprocessed image; the normalized image is input into PP-Pico-Det-AG for forward propagation, the model classifies the wild ginseng in the image and outputs the quality grade, the coordinates of the target location and the confidence score.

5. The method for detecting and classifying the quality grade of wild ginseng rhizome based on deep learning according to claim 1, characterized in that: In step A, when constructing the dataset, the samples in the dataset are preprocessed and data augmented. The preprocessing operations include adjusting image size, standardization, and normalization. The data augmentation includes random cropping, random flipping, and random distortion. By randomly changing the brightness, contrast, and color of the dataset, as well as random filling and random scaling operations, the generalization ability of the model is enhanced.

6. A system for detecting and classifying the quality grade of wild ginseng based on deep learning as described in any one of claims 1-5, characterized in that, The system includes an image acquisition module, a wild ginseng detection controller, and a model update module. The wild ginseng detection controller deploys PP-Pico-Det-AG and a WEB information management system. The image acquisition module inputs the acquired wild ginseng image to be identified into the wild ginseng detection controller for inference and prediction, and stores the detection result in the WEB information management system; The model update module is used to iteratively train PP-Pico-Det-AG, and the results detected by the system are manually adjusted and added to the wild ginseng dataset in order to expand the data required for model update training. The wild ginseng detection controller uses a Raspberry Pi 4B+ development board. The Paddle-Lite framework and the wild ginseng quality grade classification model PP-Pico-Det-AG are ported to the wild ginseng detection controller to realize the miniaturization and intelligence of the wild ginseng quality grade classification device. The WEB information management system provides users with a graphical interface to manage wild ginseng information, upload wild ginseng images, take photos of the wild ginseng being tested, integrate the functions of each module, and provide a visual interface.

7. The system according to claim 6, characterized in that, The system also includes a human-computer interaction device module, which includes a display and a thermal printer. The display provides users with the function of displaying and operating the WEB information management system; the thermal printer provides users with the function of printing wild ginseng reasoning result labels. The label content includes the wild ginseng number, the location coordinates of the wild ginseng root and the main body, the wild ginseng grade classification information, and the reasoning date.

Images