Construction method and application of Fritillaria thunbergii target detection method based on improved YOLO-X

Abstract

The present invention discloses a construction method and application of a target detection method for Fritillaria thunbergii based on improved YOLO-X, which belongs to the field of agricultural product detection. The present invention is an algorithm improvement based on YOLO-X as a baseline model, mainly embedding a layer of padding with a dilation coefficient of 4 ( padding ) expands the receptive field of the target detection algorithm and increases its sensitivity to scale features without increasing the number of parameters or computational complexity. This effectively reduces the probability of misjudgment of certain grades of Fritillaria thunbergii by the YOLO-X target detection algorithm, improving the accuracy and speed of target detection. Furthermore, this invention centrally captures, processes, and accurately grades multiple materials to be tested, resolving the issues of high image frequency and low detection efficiency in existing technologies. This has promoted their development in the Traditional Chinese Medicine market and provided new insights for nondestructive testing of other similar products.

Description

Technical Field

[0001] The present invention relates to the technical field of non-destructive detection and sorting of the appearance quality and size of agricultural products, and more specifically to a construction method and application of a Fritillaria thunbergii target detection method based on an improved YOLO-X. Background Art

[0002] Fritillaria thunbergii, a herbaceous plant of the genus Fritillaria in the Liliaceae family, is a traditional Chinese medicinal material. Its main medicinal components are alkaloids and saponins, which have antitussive, expectorant, anti-inflammatory, antibacterial, and anti-tumor properties. According to statistics, China's total production of Fritillaria thunbergii exceeded 4,000 tons between 2017 and 2018, with the cultivation industry generating approximately 6 billion yuan. According to "TCACM 1021.24-2018 Commercial Specifications and Grades of Fritillaria thunbergii in Traditional Chinese Medicine," Fritillaria thunbergii are classified by diameter as special grade, first grade, and second grade, provided they are free of mold, insect damage, or breakage. Fritillaria thunbergii that are moldy, insect-infested, or broken cannot be used medicinally and must be discarded. However, sizing Fritillaria thunbergii is currently typically performed using sieves or cylindrical rollers. These methods, however, are not ideal due to their irregular shape, and the screening process can easily cause mechanical damage to the Fritillaria, making it difficult to screen out moldy, insect-infested, or broken Fritillaria. Therefore, it is crucial to develop a method for accurately measuring the appearance quality and size of Fritillaria thunbergii and to develop a rapid, non-destructive grading production line.

[0003] Currently, the main technologies used for non-destructive online inspection of agricultural products include electronic noses and computer vision. Computer vision uses cameras and computers to identify, track, and measure targets instead of the human eye, and is characterized by accuracy, speed, and cost-effectiveness. Deep learning is an important tool for solving computer vision tasks. It is an algorithm that uses artificial neural networks as a framework to perform high-level data abstraction. Compared with traditional machine learning, deep learning is easier to model and can automatically learn deeper features from data, thereby achieving efficient data utilization. Currently, the YOLO (You Only Look Once) series has become a mainstream deep learning-based object detection algorithm and has been widely used in the field of agricultural product inspection, achieving good results.

[0004] However, there are currently no studies reporting the application of target detection algorithms to the detection and grading of the appearance quality and size of Fritillaria thunbergii. Fritillaria thunbergii is different from other material objects in that it has many grades. Special-grade, first-grade and second-grade Fritillaria thunbergii are similar in color, shape and texture, and can only be distinguished by size. The region of interest in the Fritillaria thunbergii dataset accounts for a large proportion, which results in the receptive field of the currently popular target detection algorithm becoming relatively small, making it insensitive to scale features, and ultimately resulting in a weak detection capability for Fritillaria thunbergii. Therefore, achieving accurate detection of Fritillaria thunbergii is quite difficult. In addition, most of the current online detection technologies based on computer vision technology adopt a single-shot shooting and processing method for a single material, so a large number of photos need to be taken in actual production, which places high demands on the use intensity and lifespan of the image acquisition equipment, and also limits its detection efficiency. Therefore, it is of great significance to centrally shoot, process and accurately grade multiple materials to be detected. Summary of the Invention

[0005] To overcome the problems of the prior art, the present invention aims to provide a method for constructing and applying a Fritillaria thunbergii target detection method based on an improved YOLO-X algorithm. This invention improves the backbone feature extraction network of the latest YOLO target detection algorithm, thereby enhancing the model's effectiveness without increasing the number of parameters, computational complexity, or significantly modifying the original model. This invention is expected to address the shortcomings of existing Fritillaria thunbergii detection and grading technologies, while also providing new ideas for nondestructive testing of other similar products.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for constructing a multi-target detection and sorting method for Fritillaria thunbergii based on improved YOLO-X, comprising the following steps:

[0008] Step (1): Place Fritillaria thunbergii of different grades and different postures on a white board at the bottom of a black box with a reflective layer. Under diffuse lighting conditions, use a camera to capture the same number of single-target images and multi-target images. The number of each type of Fritillaria thunbergii in the multi-target image is the same, and multiple Fritillaria thunbergii grains are arranged in a single row and located at the center line of the camera field of view to simulate an online detection scenario. To avoid losing or distorting target features, the pixels of the captured RGB image are adjusted to 3200*3200. As the Fritillaria thunbergii dataset, the dataset is divided into training set, validation set, and test set in a ratio of 6:2:2, and the YOLO-X target detection algorithm is trained, validated, and tested.

[0009] Step (2): Based on the test results in step (1), add a layer of feature extraction network at the end of the YOLO-X target detection algorithm. paddingAfter optimization, the dilation coefficient of the dilated convolution structure is adjusted to 4;

[0010] Step (3): Based on the training set and validation set of the Fritillaria thunbergii dataset in step (1), the improved YOLO-X target detection algorithm is trained and a model is obtained;

[0011] Step (4): Test the model obtained in step (3) based on the test set of the Fritillaria thunbergii dataset in step (1). AP 、 mAP 、 F 1 as well as FPS As the evaluation index of the test results, it is compared with other target detection algorithms based on this index. By comparing the inference accuracy and fit degree of the prediction box of the detection effect diagram of the model before and after improvement, the effectiveness of the improved YOLO-X is verified;

[0012] Step (5): Based on the target detection model that has been tested and meets the test index requirements, the materials are sorted using the terminal execution structure of the detection platform.

[0013] In step (3), a fine-tuning training strategy is adopted, and the weights of the model yolo_x.pth file pre-trained on the open source dataset COCO dataset are used as the initial weights of the improved YOLO-X; the frozen training rounds are set to 50, the unfrozen training rounds are set to 150, the learning rate is set to 0.001, the frozen batch size is set to 8, and the non-frozen batch size is set to 4; based on the above training strategy, transfer training is performed on the Fritillaria thunbergii training set and test set. After 200 rounds of training, the model with the smallest training loss function value is selected as the optimal model obtained by the improved YOLO-X based on the Fritillaria thunbergii training set.

[0014] In step (5), the materials to be inspected carried on the conveyor belt of the inspection platform sequentially trigger the photoelectric sensor located on one side of the conveyor belt. When the number of times the photoelectric sensor is triggered reaches a preset value N, and there are N materials in the photo, the camera takes a snapshot of the materials that have triggered the photoelectric sensor. The photo is processed by the target detection algorithm, and the activation value of the photoelectric sensor located at the end of the conveyor belt and above the air nozzle is updated according to the processing result. When the number of triggers reaches the value determined by the target detection algorithm, the host computer sends a blowing signal to the air nozzle, and the air nozzle blows the materials with the same code as the materials into the collection box on the other side of the conveyor belt. i Activation value of a photoelectric sensor A i As shown in formula (1),

[0015] (1)

[0016] In the formulaI is the serial number of the material in the photo, τ i In the direction of conveyor belt movement, smaller than the material I The number of materials in the category number.

[0017] In the described construction method, the distance between the two materials on the conveyor belt is greater than the distance between the black box and the last air nozzle at the end of the conveyor belt, so as to ensure that after the last material in the previous photo is blown off to the corresponding collection box, the activation value of the photoelectric sensor above the air nozzle corresponding to the first material in the next photo is updated.

[0018] A method for detecting Fritillaria thunbergii targets based on improved YOLO-X, comprising the following steps:

[0019] Step (1): Use a camera to capture an image of the Fritillaria thunbergii to be detected;

[0020] Step (2): Input the Fritillaria thunbergii image into the Fritillaria thunbergii target based on the improved YOLO-X

[0021] Detect the optimal model, use the model to detect the Fritillaria thunbergii image, and output the result.

[0022] A Fritillaria thunbergii online detection and sorting device is disclosed. The Fritillaria thunbergii online detection device is constructed using an optimal model for Fritillaria thunbergii target detection based on improved YOLO-X obtained by the construction method and used for online detection and sorting of Fritillaria thunbergii.

[0023] Beneficial effects of the present invention:

[0024] A layer of padding with a dilation factor of 4 is embedded at the output of the effective feature layer of the YOLO-X backbone feature extraction network (CSPDarkNet-53). padding ) with a dilated convolutional structure, while not increasing the number of parameters or computational complexity, expands the receptive field of the target detection algorithm and increases its sensitivity to scale features. This effectively reduces the probability of misjudging certain grades of Fritillaria thunbergii by the YOLO-X target detection algorithm, improving the accuracy and speed of Fritillaria thunbergii target detection. Furthermore, by centrally photographing, processing, and sequentially and accurately grading multiple materials to be tested, the present invention effectively reduces the frequency of image acquisition equipment on the detection line, lowers the investment cost of the detection line, increases detection efficiency, addresses the shortcomings of existing technologies, promotes their development in the traditional Chinese medicine market, and provides new ideas for non-destructive testing of other similar products. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of the Fritillaria thunbergii target detection method provided by the present invention.

[0026] Figure 2 This is the image acquisition device and partial images of the data set provided by the present invention.

[0027] Figure 3 This is a schematic diagram of the network structure based on the improved YOLO-X backbone feature extraction network (CSPDarkNet53-DC) and its corresponding target detection algorithm (YOLOX-DC) provided by the present invention.

[0028] Figure 4 The YOLOX-DC provided by the present invention is the model with the smallest validation loss function value on the validation set after 155 training generations. AP value, mAP value.

[0029] Figure 5 The YOLOX-DC provided by the present invention is the model with the smallest validation loss function value on the validation set after 155 training generations. F1 value.

[0030] Figure 6 This is the partial detection visualization result of YOLO-X provided by the present invention on the Fritillaria thunbergii test set.

[0031] Figure 7 This is the partial detection visualization result of YOLOX-DC provided by the present invention on the Fritillaria thunbergii test set.

[0032] Figure 8 This is the graphical control interface of the Fritillaria thunbergii detection and sorting platform provided by the present invention.

[0033] Figure 9 Schematic diagram of the multi-target sorting method provided by the present invention. DETAILED DESCRIPTION

[0034] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0035] See also Figure 1 The embodiment of the present invention provides a multi-target detection and sorting method of Fritillaria thunbergii based on improved YOLOX-DC, which specifically includes the following steps:

[0036] Step (1): Image acquisition and dataset creation.

[0037] This example refers to the TCACM 1021.24—2018 Standard for the Specification and Grading of Traditional Chinese Medicine Commodities, which classifies Fritillaria thunbergii into six grades: special grade, first grade, second grade, insect-damaged, moldy, and broken. RGB optical images of Fritillaria thunbergii from Pan'an County, Jinhua City, Zhejiang Province (29°3'N, 120°25'48'E, 319 meters above sea level) were collected as datasets for training, validating, and testing the object detection algorithm.

[0038] This example uses an electronic eye (VerVide, UK) as the RGB optical image acquisition device. Figure 2 (Part a) shows the hardware, which consists of a SLR camera (Nikon D7000, Japan), a black box with a reflective coating, a light source, and a base plate. Using an electronic eye, the grayscale histogram of Fritillaria thunbergii against five background colors (blue, green, black, red, and white) was measured. Compared to other colors, the grayscale histogram distribution of the image against a white background was wider, with more pronounced bimodal peaks in the R, G, and B channels. Therefore, a white plate was chosen as the base plate for the image acquisition device.

[0039] In order to restore the true color, shape, texture and other characteristics of Fritillaria thunbergii to the greatest extent possible and reduce the highlight area on the surface of the material caused by the light source, after repeated experiments, the light source adopts diffuse reflection mode and the shooting distance is 45cm for the best effect. Based on the above conditions, 200 images of each of the six types of Fritillaria thunbergii in two different positions were taken, and a total of 2400 single target images were taken. Figure 2 (B1-B6). To enrich the dataset and improve the generalization of the target detection model, the experiment also captured multi-target images. To balance the number of dataset categories, the multi-target images contained the same number of Fritillaria thunbergii in different positions. To simulate online detection scenarios, multiple Fritillaria thunbergii were arranged in a single row. A total of 400 multi-target images were captured, as shown in Figure 1. Figure 2 As shown in (part b7-b8), the Fritillaria thunbergii dataset has a total of 2,800 RGB optical images.

[0040] Object detection algorithms have a fixed input size, typically a multiple of 32. If the input image doesn't meet the required size, it will be resized to meet it. To accommodate the neural network input, the images in the dataset were cropped from 4928×3264 to 3200×3200, allowing the object detection algorithm to scale them proportionally. This avoids stretching or distortion of the ROI in the image, preserving its original shape and size.

[0041] The dataset was annotated using Labelimg software, and the annotated label file was saved with an XML extension. The dataset contains six labels: "super," "one level," "second level," "motheaten," "broken," and "mildew," corresponding to super-grade, first-grade, second-grade, insect-infested, moldy, and broken Fritillaria thunbergii. Each category has 800 labels, resulting in a total of 4,800 labels in the dataset. This dataset has a large ROI ratio, but the dataset size is relatively small. Therefore, the training, validation, and test sets are divided into a 6:2:2 ratio (training: 1,680, validation: 560, test: 560).

[0042] Step (2): Optimize the YOLO-X backbone feature extraction network.

[0043] According to the characteristics of YOLO-X and Fritillaria thunbergii datasets, a dilated convolution structure with a dilation factor of 4 is added to the end of the YOLO-X backbone feature extraction network (Dark5 output). In order to make the output tensor after adding the dilated convolution structure match the input and output tensor size of the corresponding position of the original model and facilitate subsequent feature fusion and reuse, the dilated convolution structure is selected with padding The dilated convolution is used as the embedding structure.

[0044] Dilated convolution is a convolution method proposed to address the image semantic segmentation problem of reduced image resolution and information loss caused by downsampling. It can provide a larger receptive field without increasing the number of convolution kernels or increasing their size, enabling the output of a wider range of information. It also avoids the loss of small object information caused by pooling. F For belt padding The receptive field size of the dilated convolution is as shown in formula (3), where rate is the expansion coefficient. The larger the value, the larger the receptive field of the model. k is the convolution kernel size; padding is the edge padding value. padding The height and width of the dilated convolution output are shown in formulas (4) and (5), where H in and W in are the height and width of the input image, H out and W out are the height and width of the output image respectively, stride is the step length.

[0045]

[0046] The backbone feature extraction network extracts image features and inputs them into the subsequent feature pyramid at a specific location for further feature extraction and fusion. The number of channels of its output tensor is shown in formula (6). s is the number of grids into which the candidate region is divided, n is the number of candidate boxes on each grid, p is the confidence of belonging to a certain category of labels, x_offset, y_offset Adjust the parameters for the grid center point, h 、 w The output of the feature pyramid is fed into the YOLO-Head to integrate features and adjust the number of channels to obtain the final prediction result.

[0047]

[0048] The output tensor size at Dark5 of the YOLO-X backbone feature extraction network is 20×20×1024. After the dilated convolution processing, according to formulas (3), (4), (5), and (6), its tensor size is still 20×20×1024, which is consistent with the tensor size of the FPN corresponding receiver, facilitating subsequent feature fusion and reuse. The improved backbone feature extraction network is named CSPDarkNet53-DC, and its corresponding target detection algorithm is named YOLOX-DC, as shown in Figure 3 shown.

[0049] Step (3): Train YOLOX-DC.

[0050] The computer configuration and deep learning environment used for this training example are described below. The computer's operating system is Windows 2019; the CPU is an Intel 9900k; the GPU is an NVIDIA Tesla V100-SXM2 (32GB of DDR4 RAM); the CUDA version is 11.2, and the cudnn version is 8.1.1. The programming language is Python 3.7, and the deep learning framework is PyTorch v1.7.

[0051] Based on the above configuration, this example uses a fine-tuning training strategy. The weights of a model pre-trained on an open-source dataset (COCO dataset) (yolox_x.pth file) are used as the initial weights for YOLO-DC. Transfer training is then performed on the Fritillaria thunbergii training and test sets. An ideal model is one whose training loss curve exhibits no significant fluctuations or lies precisely between underfitting and overfitting. To find this boundary, after multiple experiments, we found that when the total number of training epochs reaches 200, the training loss curve exhibits no significant fluctuations and approaches a flattened state, indicating that the model is nearing or has converged. To optimize the training strategy, after multiple experiments, we found that the training loss curve decreases most rapidly when the number of frozen training epochs is set to 50, the number of unfrozen training (fine-tuning) epochs to 150, the learning rate to 0.001, the frozen batch size to 8, and the unfrozen batch size to 4. In order to find the optimal model obtained through training, the experiment saved the weight files of the models obtained through each round of training. After 200 rounds of training, the model with the smallest training loss function value was selected as the optimal model obtained through training of YOLOX-DC based on the Fritillaria thunbergii training set.

[0052] Step (4): Test YOLOX-DC.

[0053] The experiment divides all samples into four types according to whether the true value and the predicted value are correctly divided into positive and negative examples, namely true positive, TP ), True Negative, TN ), False Positive, FP ), False Negative, FN ).in TP It represents the positive samples that are correctly classified, that is, the number of correctly classified Fritillaria thunbergii of a certain type; TN It represents the negative samples that are correctly classified, that is, the number of Fritillaria thunbergii of other categories that are correctly classified; FN It represents the number of misclassified positive samples, that is, the number of misclassified Fritillaria thunbergii of a certain type; FP represents the number of misclassified negative samples, that is, the number of misclassified Fritillaria thunbergii in other categories. Precision ) and recall ( Recall ) is defined using the above four types of data. Precision The specific meaning is: the proportion of model predictions among all samples with positive prediction values ​​(Formula (7)). Recall The specific meaning is: the proportion of model predictions among all samples whose true values ​​are Positive (Formula (8)).

[0054]

[0055] Sort all samples by their classification confidence and calculate the confidence corresponding to each sample as the threshold for positive and negative sample division. Precision and Recall , the different confidence levels Precision and Recall Drawing can be obtained PR curve, PR The area enclosed by the curve and the coordinate axes is AP (Formula (9)).

[0056]

[0057] mA P is all categories AP The average value of (Formula (10)), where C is the number of sample types.

[0058]

[0059] F measure yes Precision and Recall The weighted harmonic mean of Recall and Precision Indicator, when α When 1 is taken (Formula (11)).

[0060]

[0061] FPS reflects the speed at which the model processes images (Formula (12)), where N and T n They respectively represent the number of images processed by the model in a period of time and the time taken to process the images of that number of frames.

[0062]

[0063] This example uses AP 、 mAP 、 F1 as well as FPS As the evaluation index of the test results of YOLOX-DC on the Fritillaria thunbergii test set, such as formulas (9), (10), (11), and (12). In view of the characteristics of the Fritillaria thunbergii dataset, the experiment added a layer of expansion coefficients of 2, 3, 4, 5, and 6 at the end of the YOLO-X backbone feature extraction network. padding The test results show that the model with a dilation coefficient of 4 has the best detection effect, and its mean average precision (mean AveragePrecision, mAP ) is 98.95%, and the average precision rate for special grade, first grade, second grade, insect-infested, broken and moldy Fritillaria thunbergii is 98.95%. AP ) values ​​are 99.97%, 98.33%, 98.47%, 98.71%, 98.85%, and 99.73%, respectively. Figure 4 As shown; F 1 The values ​​are 0.99, 0.92, 0.94, 0.97, 0.97, 0.99, and the frame rate (Frames Per Second, FPS ) is 29.18, such as Figure 5 shown.

[0064] This example is based on the Fritillaria thunbergii dataset and compares YOLOX-DC with other object detection algorithms. This example uses the same dataset, training equipment, training environment, training strategy and evaluation indicators to train and test the YOLO-V3, YOLO-V4, YOLO-V5, YOLO-X and Faster R-CNN object detection algorithms. The results are shown in Table 1. The models trained with the above targets generally have weak detection capabilities for special-grade, first-grade and second-grade Fritillaria thunbergii that differ only in diameter. AP and F 1 The results are all lower than the corresponding indicators of insect damage, breakage and mildew. YOLO-X’s detection effect on special, first-class and second-class Fritillaria thunbergii is better than that of other algorithms. AP The values ​​were 98.39%, 72.22% and 96.59% respectively. F 1 The values ​​are 0.90, 0.63, and 0.84 respectively, but the detection effect is still not as good as YOLOX-DC. Compared with the YOLO-X model, YOLOX-DC mAP The value increased by 4.89%, and the special, first and second grade Fritillaria thunbergii AP The values ​​increased by 0.98%, 26.11%, and 1.88% respectively. F1 The values ​​are increased by 0.09, 0.33 and 0.10 respectively. The detection effects of the original model and the improved model are as follows Figure 6-7 As shown in the figure, the former's detection errors or missed detections mostly occurred on special-grade, first-grade and second-grade Fritillaria thunbergii; the latter correctly identified all categories of Fritillaria thunbergii with a high confidence level. The prediction box fits closely with the Fritillaria thunbergii and presents an circumscribed rectangle, indicating that the model's regression of its position is also relatively accurate.

[0065] Table 1 Test results of models trained with other target detection algorithms on the Fritillaria thunbergii test set

[0066]

[0067] Step (5) The terminal of the detection platform performs the structure sorting of materials.

[0068] Use Pycharm Community Edition 2020 software, use Python language to write the host computer software under Windows system, and use Pyqt5 to encapsulate the model that meets the test indicators as an executable program (.exe) in the visual graphical operation interface of the host computer, such as Figure 8 The core part is the behavior class that plays a key role in each module, including the communication class between the host and the slave (CnCommDlg), image creation, video display class (Clistenl), image processing class (Image_handling), image training class (training), and main window class (SerialDlg).

[0069] like Figure 9 As shown, the Fritillaria thunbergii carried on the conveyor belt of the detection platform sequentially triggers the photoelectric sensor located on one side of the conveyor belt. When the number of times the photoelectric sensor is triggered reaches a preset value (N, there are N materials in the photo), the camera captures the materials that have triggered the photoelectric sensor and sorts and codes them in the order in which they trigger the photoelectric sensor (1-N). The photos are processed by the target detection algorithm, and the activation value of the photoelectric sensor at the end of the conveyor belt and above the air nozzle is updated according to the information output by the target detection algorithm. The specific algorithm is as follows: the categories of the materials to be detected are encoded as 1-n categories (n is the number of categories of the materials to be detected), and the air nozzle at the end of the conveyor belt and the photoelectric sensor above it are encoded as 1-n in sequence according to the direction of movement of the conveyor belt. The materials in the photo will move with the conveyor belt and trigger the photoelectric sensors above each air nozzle in turn. These photoelectric sensors record the number of times they are triggered. When the number of triggers reaches the value given by the target detection algorithm, the host computer sends a blowing signal to the air nozzle, and the air nozzle blows the materials with the same code into the collection box on the other side of the conveyor belt. Among them, the first i Activation value of a photoelectric sensor A i As shown in formula (1), the photoelectric sensor above the air nozzle will set different activation values ​​for the detection results of each photo according to the target detection algorithm.

[0070] (1)

[0071] In the formula I is the serial number of the material in the photo, τ i In the direction of conveyor belt movement, smaller than the material I The number of materials in the category number.

[0072] like Figure 9 As shown, the size of the collection box opening (L2) must be larger than the size of the largest material to ensure that the material can enter the collection box smoothly; the distance between the two materials must be larger than the distance between the black box and the last air nozzle at the end of the conveyor belt, that is, L1>L3, to ensure that after the last material in the previous photo is blown into the corresponding collection box, the activation value of the photoelectric sensor above the air nozzle corresponding to the first material in the next photo is updated.

[0073] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for constructing a multi-target detection and sorting method for Fritillaria thunbergii based on improved YOLO-X, characterized in that: The following steps are involved: Step (1): Place Fritillaria thunbergii of different grades and different postures on a white board at the bottom of a black box with a reflective layer. Under diffuse lighting conditions, use a camera to capture the same number of single-target images and multi-target images. The number of each type of Fritillaria thunbergii in the multi-target image is the same, and multiple Fritillaria thunbergii grains are arranged in a single row and located at the center line of the camera field of view to simulate an online detection scenario. To avoid losing or distorting target features, the pixels of the captured RGB image are adjusted to 3200*3200. As the Fritillaria thunbergii dataset, the dataset is divided into training set, validation set, and test set in a ratio of 6:2:2, and the YOLO-X target detection algorithm is trained, validated, and tested. Step (2): Based on the test results in step (1), add a layer of feature extraction network at the end of the YOLO-X target detection algorithm. padding After optimization, the dilation coefficient of the dilated convolution structure is adjusted to 4; Step (3): Based on the training set and validation set of the Fritillaria thunbergii dataset in step (1), the improved YOLO-X target detection algorithm is trained and a model is obtained; Step (4): Test the model obtained in step (3) based on the test set of the Fritillaria thunbergii dataset in step (1). AP 、 mAP 、 F 1 as well as FPS As the evaluation index of the test results, it is compared with other target detection algorithms based on this index. By comparing the inference accuracy and fit degree of the prediction box of the detection effect diagram of the model before and after improvement, the effectiveness of the improved YOLO-X is verified; Step (5): Based on the target detection model that has been tested and meets the test index requirements, the materials are sorted using the terminal execution structure of the detection platform; In step (3), a fine-tuning training strategy is adopted, and the weights of the model yolo_x.pth file pre-trained on the open source dataset COCO dataset are used as the initial weights of the improved YOLO-X; the frozen training rounds are set to 50, the unfrozen training rounds are set to 150, the learning rate is set to 0.001, the frozen batch size is set to 8, and the non-frozen batch size is set to 4; based on the above training strategy, transfer training is performed on the Fritillaria thunbergii training set and test set. After 200 rounds of training, the model with the smallest training loss function value is selected as the optimal model obtained by the improved YOLO-X based on the Fritillaria thunbergii training set.

2. The construction method according to claim 1, wherein In step (5), the materials to be inspected carried on the conveyor belt of the inspection platform sequentially trigger the photoelectric sensor located on one side of the conveyor belt. When the number of times the photoelectric sensor is triggered reaches a preset value N, and there are N materials in the photo, the camera takes a snapshot of the materials that have triggered the photoelectric sensor. The photo is processed by the target detection algorithm, and the activation value of the photoelectric sensor located at the end of the conveyor belt and above the air nozzle is updated according to the processing result. When the number of triggers reaches the value determined by the target detection algorithm, the host computer sends a blowing signal to the air nozzle, and the air nozzle blows the materials with the same code as the materials into the collection box on the other side of the conveyor belt. i Activation value of a photoelectric sensor A i As shown in formula (1), (1) In the formula I is the serial number of the material in the photo, τ i In the direction of conveyor belt movement, smaller than the material I The number of materials in the category number.

3. The construction method according to claim 1, characterized in that The distance between the two materials on the conveyor belt is greater than the distance between the black box and the last air nozzle at the end of the conveyor belt, so as to ensure that after the last material in the previous photo is blown down to the corresponding collection box, the activation value of the photoelectric sensor above the air nozzle corresponding to the first material in the next photo is updated.

4. A target detection method for Fritillaria thunbergii based on improved YOLO-X, characterized in that: The following steps are involved: Step (1): Use a camera to capture an image of the Fritillaria thunbergii to be detected; Step (2): inputting the Fritillaria thunbergii image into the Fritillaria thunbergii target detection optimal model based on improved YOLO-X obtained according to the construction method of claim 1, and outputting the result after the Fritillaria thunbergii image is detected by the model.

5. An online detection and sorting device for Fritillaria thunbergii, characterized in that: Based on the optimal model for thunbergii target detection based on improved YOLO-X obtained by the construction method according to claim 1, an online detection device for thunbergii is built and used for online detection and sorting of thunbergii.

Images