Chinese herbal medicine and foreign matter detection method and system thereof, storage medium and product

Abstract

The invention relates to the technical field of target detection, and discloses a Chinese herbal medicine and foreign matter detection method and system, a storage medium and a product. Based on a Chinese herbal medicine and foreign matter detection model, whether foreign matter exists in the to-be-detected image or not is judged according to the to-be-detected image, and the Chinese herbal medicine and foreign matter detection model is an improved YOLOv8 model. According to the Chinese herbal medicine and foreign matter detection method provided by the invention, the adopted model is an improved YOLOv8 model, and the feature extraction capability, the detection precision and the detection speed of the model are improved while the light weight of the model is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of object detection, and particularly relates to a traditional Chinese medicine and its foreign object detection method, system, storage medium and product. Background Art

[0002] In the field of traditional Chinese medicine, the quality of traditional Chinese medicine directly relates to the curative effect and safety. The traditional detection method of traditional Chinese medicine relies on manual visual inspection, which is not only inefficient but also easily affected by subjective judgment. With the development of computer vision technology, automated traditional Chinese medicine detection systems have gradually become a research hotspot.

[0003] In recent years, deep learning methods have made remarkable progress in the field of object detection. Common deep learning models include Faster R-CNN, YOLO, and SSD, etc. The object detection algorithms based on deep learning mainly have two major branches: one is the two-stage detection algorithm, such as algorithms like R-CNN, SPP-Net, Fast R-CNN, Faster R-CNN, and R-FCN; the other is the one-stage detection algorithm, such as algorithms like SSD, YOLO series, etc. The two-stage detection algorithm follows the principle of selecting candidate boxes and classification. Training two models will inevitably affect the detection speed. As the mainstream two-stage detection algorithm, Faster R-CNN has a detection speed of 5f / s, which obviously cannot meet the requirements of detection tasks with high real-time performance. Therefore, when the two-stage detection algorithm encounters bottlenecks, the one-stage detection algorithm stands out with its excellent performance. Based on the regression characteristics, the one-stage detection algorithm directly regresses on the original image, showing the target bounding box and target category. The emergence of the one-stage detection algorithm has qualitatively improved the detection speed while maintaining high accuracy. The current mainstream one-stage detection algorithm is the YOLO series.

[0004] As a latest object detection algorithm, YOLOv8 provides the possibility for the automatic recognition of traditional Chinese medicine and its foreign objects with its fast and accurate performance. The YOLOv8 system can, through a deep learning model, detect and identify in real time the types of traditional Chinese medicine and possible foreign objects mixed in, such as stones, plastic sheets, etc. The application of the YOLOv8 system can not only improve the efficiency and accuracy of traditional Chinese medicine detection, but also provide technical support for the standardized production and quality control of traditional Chinese medicine, ensuring the curative effect and safety of traditional Chinese medicine.

[0005] However, in the detection of traditional Chinese medicine and its foreign objects, small object detection has always been a technical difficulty due to characteristics such as small size, low contrast, and being easily interfered by the background. Traditional detection methods often have difficulty accurately identifying these small objects, resulting in a relatively high missed detection rate. Therefore, in complex scenarios, a traditional Chinese medicine and its foreign object detection method and system with high accuracy, low false alarm rate, and high speed are needed. Summary of the Invention

[0006] In view of this, the present invention provides a Chinese herbal medicine and its foreign object detection method, system, storage medium and product to solve the problem of low accuracy and easy omission in the automatic detection of Chinese herbal medicine in the prior art.

[0007] In a first aspect, the present invention provides a Chinese herbal medicine and its foreign object detection method, and the method includes:

[0008] Obtain a to-be-detected image containing Chinese herbal medicine;

[0009] Based on the Chinese herbal medicine and its foreign object detection model, determine whether there is a foreign object in the to-be-detected image according to the to-be-detected image, wherein the Chinese herbal medicine and its foreign object detection model is an improved YOLOv8 model.

[0010] In the Chinese herbal medicine and its foreign object detection method provided in this embodiment, the model adopted is an improved YOLOv8 model, which not only realizes the lightweight of the model, but also improves the feature extraction ability, detection accuracy and detection speed of the model.

[0011] In an optional implementation manner, the Chinese herbal medicine and its foreign object detection model is established through the following steps:

[0012] Obtain a training data set, which includes Chinese herbal medicine images and foreign object images;

[0013] On the basis of the YOLOv8 model, improve the YOLOv8 model; wherein, replace the detection head of the YOLOv8 model with a reparameterized detection head RepHead; replace the C2f module of the YOLOv8 model with a UIB module; replace the loss function in the YOLOv8 model with an InnerCIOU loss function;

[0014] Input the images in the training data set into the improved YOLOv8 model for training to obtain a trained Chinese herbal medicine and its foreign object detection model.

[0015] The present invention introduces a reparameterized detection head RepHead. In the training stage, the idea of multi-branch auxiliary training is used to improve the feature extraction ability of the model. In the inference stage, it is transformed into a serial structure, which increases the speed of inference. To meet the lightweight requirements of edge devices, the UIB module of MobileNetV4 is used to replace C2f to lightweight the network structure, which can optimize the performance of edge mobile devices. Aiming at the problems of slow network convergence and low regression accuracy when identifying small-sample foreign objects in Chinese herbal medicines, the present invention introduces the InnerCIOU loss function. Compared with loss functions such as CIoU and EIOU, the InnerCIOU loss function introduces auxiliary bounding boxes of different scales to calculate the loss, optimizes the tracking of overlapping complex backgrounds or multiple targets, and shows good performance in dealing with the detection tasks of small targets. In summary, the detection method provided by the present invention effectively improves the accuracy of identifying Chinese herbal medicines and foreign objects.

[0016] In an alternative embodiment, during the training stage, the reparameterized detection head RepHead includes:

[0017] A plurality of first convolutional modules cascaded in sequence; the first convolutional module includes: a first-size convolutional layer, a first activation layer, and a second-size convolutional layer. Among them, in the first convolutional module, the output result of the first-size convolutional layer is merged with the output result of the second-size convolutional layer and the input result of the first-size convolutional layer, and the merged result is input into the first activation layer;

[0018] During the prediction stage, the reparameterized detection head RepHead includes: a plurality of second convolutional modules cascaded in sequence; the second convolutional module includes: a first-size convolutional layer and a second activation layer connected in sequence.

[0019] In this embodiment, RepHead uses the idea of multi-branch auxiliary training in the training stage to improve the feature extraction ability of the model. In the inference stage, it is transformed into a serial structure, which increases the speed of inference and solves the problem of difficult-to-identify samples in the dataset.

[0020] In an alternative embodiment, the UIB module includes two optional depth convolutional modules; the depth convolutional module obtains multiple variants according to a preset selection method, where the variants include: inverted bottleneck, ConvNext, feed-forward network, and ExtraDW.

[0021] Replacing C2f with the UIB module to lightweight the network structure can optimize the performance on edge devices.

[0022] In an alternative embodiment, the InnerCIOU loss function includes:

[0023]

[0024] Among them, b represents the center point of the prediction box, and b gt represents the center point of the target box, ρ represents the Euclidean distance between the two center point coordinates, C represents the diagonal distance of the smallest box, and α represents the orthogonal balance parameter: v measures the consistency of the aspect ratio: w gt represents the width of the target box, h gt represents the height of the target box, w represents the width of the prediction box, and h represents the height of the prediction box.

[0025] In this embodiment, the InnerCIOU loss function is introduced, which can effectively optimize the tracking of overlapping complex backgrounds or multiple targets, and at the same time optimize the detection of small targets, improving the detection accuracy.

[0026] In an alternative embodiment, after obtaining the trained Chinese herbal medicine and foreign object detection model, it further includes:

[0027] Determine multiple preset IOU thresholds;

[0028] The Chinese herbal medicine and foreign object detection model outputs detection results corresponding one-to-one to the preset IOU thresholds based on the preset IOU thresholds;

[0029] Determine the TP value and FP value based on the detection results;

[0030] Determine the average precision at multiple preset IOU thresholds based on the TP value and FP value.

[0031] In this embodiment, based on the improved YOLOv8 model, a Chinese herbal medicine and foreign object detection model is obtained, the test image to be detected is detected, and the detection results are evaluated, which can effectively verify the effectiveness and reliability of the model in practical applications and improve the credibility of model prediction.

[0032] In an alternative embodiment, the training dataset is established through the following steps:

[0033] Collect original Chinese herbal medicine images and original foreign object images;

[0034] Perform label annotation on the original Chinese herbal medicine images and original foreign object images respectively to obtain the labeled annotation dataset;

[0035] Perform image enhancement processing on the images in the annotation dataset using at least one of the following processing methods: rotation, translation, mirroring, Gaussian blur, color gamut transformation, scaling and splicing;

[0036] Use the dataset after image enhancement processing as the training dataset.

[0037] In this embodiment, data augmentation is performed through processing methods such as rotation, translation, mirroring, image brightness transformation, and Gaussian blur, which can effectively increase the number of training samples and thus improve the model performance.

[0038] In a second aspect, the present invention provides a Chinese herbal medicine and its foreign object detection system, which includes:

[0039] An acquisition module for acquiring a to-be-detected image containing Chinese herbal medicine;

[0040] A prediction module for judging whether there is a foreign object in the to-be-detected image based on the Chinese herbal medicine and its foreign object detection model according to the to-be-detected image.

[0041] In a third aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the Chinese herbal medicine and its foreign object detection method of the first aspect or any corresponding embodiment thereof.

[0042] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the Chinese herbal medicine and its foreign object detection method of the first aspect or any corresponding embodiment thereof.

[0043] It should be noted that since the Chinese herbal medicine and its foreign object detection system, computer device, and computer-readable storage medium provided by the present invention correspond to the above-mentioned Chinese herbal medicine and its foreign object detection method. Therefore, for the beneficial effects of the Chinese herbal medicine and its foreign object detection system, computer device, and computer-readable storage medium, please refer to the description of the corresponding beneficial effects of the Chinese herbal medicine and its foreign object detection method above, and will not be elaborated here. Description of the Drawings

[0044] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0045] Figure 1 is a flowchart of the Chinese herbal medicine and its foreign object detection method according to an embodiment of the present invention;

[0046] Figure 2 is a schematic diagram of the initial network structure of YOLOv8 according to an embodiment of the present invention;

[0047] Figure 3Schematic diagram of the improved YOLOv8 network structure according to an embodiment of the present invention;

[0048] Figure 4 Schematic diagram of the RepHead structure according to an embodiment of the present invention;

[0049] Figure 5 Schematic diagram of the UIB structure according to an embodiment of the present invention;

[0050] Figure 6 Block diagram of the structure of the Chinese herbal medicine and its foreign object detection system according to an embodiment of the present invention;

[0051] Figure 7 Schematic diagram of the hardware structure of the computer device according to an embodiment of the present invention. Detailed implementation manners

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] According to an embodiment of the present invention, an embodiment of a method for detecting Chinese herbal medicine and its foreign objects is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0054] In this embodiment, a method for detecting Chinese herbal medicine and its foreign objects is provided, which can be executed by devices such as servers, terminals, and mobile terminals. Figure 1 Flowchart of the method for detecting Chinese herbal medicine and its foreign objects according to an embodiment of the present invention, as Figure 1 shown, the process includes the following steps:

[0055] Step S101, obtain a to-be-detected image containing Chinese herbal medicine. For example, it can be a Chinese herbal medicine image containing honeysuckle, artemisia annua, gardenia, etc. Further, the to-be-detected image needs to be processed to meet the model input requirements.

[0056] Step S102: Based on the Chinese herbal medicine and its foreign object detection model, determine whether there are foreign objects in the image to be detected according to the image to be detected. In this embodiment, the detection of Chinese herbal medicine and its foreign objects is an improved YOLOv8 model. Loading the improved YOLOv8 to predict the image to be detected in the input network can obtain the class names, confidence scores, and corresponding target box coordinate information of each detection target, and display them on the picture, and finally confirm whether there are foreign objects in the image to be detected.

[0057] The method for detecting Chinese herbal medicine and its foreign objects provided in the present invention can be merely a drug detection method, that is, a method for detecting Chinese herbal medicine, or a method for detecting foreign objects in Chinese herbal medicine.

[0058] In this embodiment, the trained Chinese herbal medicine and its foreign object detection model can be embedded into the network architecture of the Chinese herbal medicine and foreign object monitoring system to effectively identify and locate Chinese herbal medicine and foreign objects in the real-time monitoring screen, and achieve the detection goals of Chinese herbal medicine and foreign objects designed by the system.

[0059] The model adopted in the method for detecting Chinese herbal medicine and its foreign objects provided in this embodiment is an improved YOLOv8 model. While achieving model lightweight, it also improves the model's feature extraction ability, detection accuracy, and detection speed.

[0060] In some alternative embodiments, the Chinese herbal medicine and its foreign object detection model is established through the following steps:

[0061] Step S1011: Obtain a training data set, which includes Chinese herbal medicine images and foreign object images.

[0062] Step S1012: Based on the YOLOv8 model, improve the YOLOv8 model; wherein, replace the detection head of the YOLOv8 model with a reparameterized detection head RepHead; replace the C2f module of the YOLOv8 model with a UIB module; replace the loss function in the YOLOv8 model with an InnerCIOU loss function. Refer to Figure 2 The structure diagram of the original YOLOv8 model is shown in Figure 3 As shown, it is the improved YOLOv8 model in this embodiment.

[0063] Step S1013: Input the images in the training dataset into the improved YOLOv8 model for training to obtain a trained Chinese herbal medicine and foreign object detection model. Specifically, the environment variables can be configured, the image size imagesize input to the network is set to 640×640; an initial model model_body is created; the number of epochs for batch training is 200, the initial learning rate lr is 0.01, and the batch size is 16; the network model is trained and the network parameters are updated, and best.pt is retained; when epoch = 200, the final model weights last.pt are saved, and the model training ends.

[0064] The present invention introduces a reparameterized detection head RepHead, which uses the idea of multi-branch auxiliary training in the training stage to improve the feature extraction ability of the model, and is transformed into a serial structure in the inference stage, increasing the speed of inference. For the lightweight requirement at the edge, the UIB module of MobileNetV4 is used to replace C2f to lightweight the network structure, which can optimize the performance of mobile devices at the edge. Aiming at the problem that the network converges slowly and the regression accuracy is low when identifying small-sample foreign objects in Chinese herbal medicines, the present invention introduces an InnerCIOU loss function. Compared with loss functions such as CIoU and EIOU, different-scale auxiliary bounding boxes are introduced to calculate the loss, optimizing the tracking of overlapping complex backgrounds or multi-targets, and showing good performance in dealing with small-target detection tasks. In summary, the detection method provided by the present invention effectively improves the accuracy of identifying Chinese herbal medicines and foreign objects.

[0065] Refer to Figure 4 As shown, in some optional embodiments, in the training stage, the reparameterized detection head RepHead includes:

[0066] A plurality of first convolutional modules cascaded in sequence; the first convolutional module includes: a first-size convolutional layer conv, a first activation layer ReLU, and a second-size convolutional layer. Among them, in the first convolutional module, the output result of the first-size convolutional layer is combined with the output result of the second-size convolutional layer and the input result of the first-size convolutional layer, and the combined result is input to the first activation layer. Among them, identity is also an activation function, indicating that the input of the node is equal to the output, that is, identity.

[0067] In the prediction stage, the reparameterized detection head RepHead includes: a plurality of second convolutional modules cascaded in sequence; the second convolutional module includes: a first-size convolutional layer and a second activation layer ReLU connected in sequence.

[0068] The first size can be 3×3, and the second size can be 1×1.

[0069] That is, during training, use identities similar to ResNet (when dimensions match) and 1×1 branches. The information flow during training for such a building block is as follows:

[0070] y = x + g(x) + f(x)

[0071] where g(x) is a convolutional shortcut connection implemented by a 1×1 convolution, and f(x) is a convolutional shortcut connection implemented by a 3×3 convolution.

[0072] During the prediction phase, the trained block needs to be converted into a single 3×3 convolutional layer for inference. The process is as follows:

[0073] Convert the BN layer and the convolutional layer in front of it into a convolutional layer with a bias vector. Here, the BN layer is the batch normalization layer, which normalizes each batch of data so that the input data of each layer of the network has a similar distribution, accelerating convergence and improving stability.

[0074] First, convert each BN layer and the convolutional layer in front of it into a convolutional layer with a bias vector. Let the kernel and bias converted from W, μ, σ, γ, β be W ′ , b ′ , then we have After this conversion, bn(M * W, μ, σ, γ, β) :,i,:,: =(M * W ′ ) :,i,:,: + b i ′ . This conversion also applies to the identity branch because the identity can be regarded as a convolution with an identity matrix as the kernel.

[0075] Combine the parameters to obtain the final convolutional layer:

[0076] After the above conversion, a 3×3 kernel, two 1×1 kernels, and three bias vectors will be obtained. Add the three bias vectors to get the final bias, and zero-pad the two 1×1 kernels to 3×3 first and then add them to the center point of the 3×3 kernel to get the final 3×3 kernel (this operation requires that the 3×3 and 1×1 layers have the same stride, and the padding configuration of the 1×1 layer is one pixel less than that of the 3×3 layer. For example, when the 3×3 layer pads one pixel for the input, the 1×1 layer padding should be 0).

[0077] where, is the kernel of an n×n convolutional layer with C 1 input channels and C 2 output channels. μ (n) , σ (n) , γ (n) , β (n)Denote the cumulative mean, standard deviation, learned scaling factor, and bias of the BN layer after an n×n convolution, which are the parameters of the identity branch when n = 0. and are the input and output respectively, and * is the convolution operator. When C 1 = C 2 , H 1 = H 2 , W 1 = W 2 holds, we have M (2) = bn(M (1) * W (3) , μ (3) , σ (3) , γ (3) , β (3) ) + bn(M (1) * W1, μ1, σ1, γ1, β1 + bnM1 * W0, μ0, σ0, γ0, β0. Otherwise, only the first two terms are used (in the case without the identity branch), where bn is the BN function during inference, defined as bn(M * W, μ, σ, γ, β[:, i, :, :] = (M[:, i, :, :] - μi) / σi * γi + βi.

[0078] The above explains how the RepHead structure in the training stage is converted into the structure in the prediction stage.

[0079] Referring to Figure 4 the network in the training stage, use which is also the kernel of a 3×3 convolutional layer with C 1 input channels and C 2 output channels; use the kernel of a 1×1 convolutional layer with C 1 input channels and C 2 output channels. Then, for the input M (1) and the output M (2) , there is the following relationship:

[0080] M (2) = bn(M (1) * W (3) , μ (3) , σ (3) , γ (3) , β (3) )

[0081] + bn(M (1) * W (1) , μ (1) , σ (1) , γ (1) , β (1) )

[0082] +bn(M (1) *W (0) ,μ (0) ,σ (0) ,γ (0) ,β (0) )

[0083] If the shapes of the input and output are different, i.e., any one of the width, height, and number of channels is different, then they are considered different. In this case, the first two items are used:

[0084]

[0085] That is, each BN layer and the convolutional layer in front of it are converted into a convolutional layer with a bias vector. The calculation of the convolutional kernel and bias is as above. The bias is used to perform a translation operation on the output without changing the weights of the convolutional kernel. Then bn(M*W, μ, σ, γ, β) :,i,:,: =(M*W ′ ) :,i,:,: +b i ′ , where the ':' in :,i,:,: means selecting all elements in that dimension. Therefore, (M*W ′ ) :,i,:,: means selecting all the data in the height and width of the i-th channel of all samples in the tensor M*W ′ .

[0086] Therefore, after such a conversion, a 3×3 kernel, two 1×1 kernels, and three bias vectors are obtained. Since the identity can also be regarded as a 1×1 convolution with an identity matrix as the kernel. The three bias vectors are added to obtain the final bias, and the two 1×1 kernels are first zero-padded to 3×3 and then added to the center point of the 3×3 kernel to obtain the final 3×3 kernel. Zero-padding first means setting the two grids around the 1×1 convolution to 0, regarding it as a 3×3 convolution, and completing the conversion from RepHead training to RepHead inference.

[0087] The reparameterized detection head RepHead sacrifices a small amount of GFLOPs and improves the feature extraction ability of the model, which is very helpful for samples that are difficult to identify in the Chinese herbal medicine and foreign object dataset.

[0088] The reparameterized detection head RepHead introduced in the present invention uses the idea of multi-branch auxiliary training in the training stage to improve the feature extraction ability of the model, and is converted into a serial structure in the prediction stage, increasing the speed of inference.

[0089] Refer to Figure 5As shown, in some alternative embodiments, the UIB module includes two alternative depth convolution modules; the depth convolution modules obtain multiple variants according to a preset selection method, where the variants include: inverted bottleneck, ConvNext, feed-forward network, and ExtraDW.

[0090] In this embodiment, there are four possible instantiation methods for the two alternative depth convolutions of the UIB module, including inverted bottleneck (IB), ConvNext, feed-forward network (FFN), and a novel extra depth (ExtraDW) variant. Each instantiation has different trade-offs in terms of spatial mixing, receptive field, and computational utilization. For example, IB performs spatial mixing on the expanded feature activations to provide a larger model capacity at the cost of increased cost; ConvNext allows for cheaper spatial mixing with a larger kernel size by performing spatial mixing before expansion; ExtraDW combines the advantages of ConvNext and IB; FFN is a stack of two 1x1 pointwise convolutions with activation and normalization layers in the middle, and the PW operation is beneficial for accelerators and works well in cooperation with other modules. This structure allows for adaptively and effectively expanding the model to various platforms without overly complicating the architecture search process. Replacing C2f with the UIB module can lighten the network structure and optimize the performance on edge devices.

[0091] In some alternative embodiments, the InnerCIOU loss function includes:

[0092]

[0093] where b represents the center point of the predicted bounding box, b gt represents the center point of the target bounding box, ρ represents the Euclidean distance between the two center point coordinates, C represents the diagonal distance of the smallest bounding box, and α represents the orthogonal balance parameter: v measures the consistency of the aspect ratio: w gt represents the width of the target bounding box, h gt represents the height of the target bounding box, w represents the width of the predicted bounding box, and h represents the height of the predicted bounding box.

[0094] The InnerCIOU loss function in this embodiment combines the idea of Inner and the CIOU loss function. The main idea of Inner is that for high IOU samples, using a smaller auxiliary bounding box to calculate the loss can accelerate convergence, while a larger auxiliary bounding box is suitable for low IOU samples. For different datasets and detectors, a scaling factor ratio is introduced to control the scale size of the auxiliary bounding box for calculating the loss.

[0095] The IOU loss can accurately describe the degree of matching between the predicted bounding box and the GT box (ground truth box, which refers to the position and size of the annotated real object in the image), ensuring that the model can learn the position information of the target during training. As a basic part of the existing mainstream bounding box regression loss function, the IOU is defined as follows:

[0096]

[0097] where B and B gt represent the predicted box and the GT box respectively. After defining the IOU, the corresponding loss can be defined as: L IOU = 1 - IOU.

[0098] The CIOU in this embodiment adds a new distance loss term on the basis of the IOU, mainly achieving faster convergence and better performance by minimizing the normalized distance between the centers of the two bounding boxes, and at the same time considering the shape loss.

[0099] In this embodiment, the InnerCIOU loss function is introduced, which can effectively optimize the tracking of overlapping complex backgrounds or multiple targets, and at the same time optimize the detection of small targets, improving the detection accuracy.

[0100] In some alternative embodiments, after obtaining the trained Chinese herbal medicine and foreign object detection model, it further includes:

[0101] Determine multiple preset IOU thresholds. For example: 0.5, 0.75, etc., and each IOU threshold will be used to evaluate the overlapping degree between the predicted box and the real box.

[0102] The Chinese herbal medicine and foreign object detection model outputs detection results corresponding one-to-one to the preset IOU thresholds based on the preset IOU thresholds. Using the improved YOLOv8 model, different IOU thresholds are used for object detection to generate a series of predicted detection results, and the detection results can include the bounding box coordinates, category, and confidence score of each detected target prediction box, etc.

[0103] Determine the TP value and the FP value based on the detection results. That is, judge which predicted boxes are considered correct (TP) and which predicted boxes are considered wrong (FP).

[0104] Determine the average precision at multiple preset IOU thresholds based on the TP value and the FP value.

[0105] For each IOU threshold, calculate the precision and recall rate at that threshold respectively:

[0106]

[0107] Among them, P1 represents precision, R represents recall, TP represents the number of samples that are actually positive and predicted as positive, FP represents the number of samples that are actually negative but predicted as positive, and FN represents the number of samples that are actually positive but predicted as negative.

[0108] Then, the evaluation metric mAP is obtained by calculating the average precision of the model at the IOU thresholds:

[0109]

[0110] Among them, mAP represents the mean average precision, and N represents the total number of the preset IOU thresholds.

[0111] In this embodiment, based on the improved YOLOv8 model, a Chinese herbal medicine and foreign object detection model is obtained. The test image to be detected is detected, and the detection results are evaluated, which can effectively verify the effectiveness and reliability of the model in practical applications and improve the credibility of model prediction.

[0112] The following is a comparative example provided by the present invention. To verify the effectiveness of an automatic detection method for Chinese medicinal materials and foreign objects assisted by deep learning provided by the present invention, the comparison results of two schemes are provided.

[0113] First, the experimental environment and parameter settings. The hardware configuration of this experiment is NVIDIA RTX3060TI-G6X GPU and INTEL i5-12400 2.70GHz CPU, and the software environment is the Pytorch deep learning framework under the Windows 10 system. The input image size is 640×640, with a total of 200 training epochs, and the initial learning rate for network training is set to 0.01.

[0114] Second, prepare the experimental dataset. According to the needs of the experiment, three types of Chinese medicinal materials and foreign object pictures taken are collected and processed to simulate the detection environment in the real world. In the experiment, Chinese medicinal materials such as honeysuckle, artemisia annua, and gardenia, as well as foreign objects such as stones, glass, and fibers in them, are selected as the research objects. These datasets will be used to train and test the Chinese herbal medicine and foreign object detection model. The dataset is divided into 2762 training sets, 200 validation sets, and 620 test sets for experiments to evaluate the performance of each strategy.

[0115] The performance of the model is evaluated according to the following evaluation metrics: mean average precision (mAP), frames per second (FPS), and computational complexity (GLOPs).

[0116] Finally, the detection experiment results are obtained. To verify the effectiveness and superiority of this algorithm, on the premise of the same training conditions and data, the method proposed in this invention is experimentally compared with related methods. Several models are tested and compared for the detection results of average precision and processing speed on the Chinese herbal medicine and its foreign object dataset. The experimental results are shown in Table 1.

[0117] Table 1: Comparative experiment results

[0118]

[0119] From the experimental results of various object detection algorithms in Table 1, it can be clearly seen that the algorithm of this invention is on par with other improved algorithms in terms of the average precision mAP50 index, and is superior to other improved algorithms in the mAP50-95 index. At the same time, it is significantly superior to other improved algorithms in GFLOPs (billions of floating-point operations per second). In addition, in terms of inference speed, compared with AFPN-YOLOv8 and MoblieV3-YOLOv8, it has increased by 33% and 22.4% respectively.

[0120] In summary, while ensuring the accuracy, this invention also lightweightens the model, which better meets the requirements of Chinese herbal medicine and its foreign object recognition.

[0121] In some optional embodiments, the training dataset is established through the following steps:

[0122] Collect the original Chinese herbal medicine images and original foreign object images.

[0123] Perform label annotation on the original Chinese herbal medicine images and original foreign object images respectively to obtain the labeled annotation dataset. Specifically, the annotation tool Labelimg can be used to perform label annotation on the original Chinese herbal medicine images and original foreign object images, and automatically generate the corresponding TXT format annotation files.

[0124] Perform image enhancement processing on the images in the annotation dataset using at least one of the following processing methods: rotation, translation, mirroring, Gaussian blur, color gamut transformation, resizing and stitching.

[0125] Use the dataset after image enhancement processing as the training dataset.

[0126] Specifically, the formula used for image rotation in this embodiment can be:

[0127]

[0128] where (x, y) represents the pixel coordinates in the original image, (x ′ , y ′ ) represents the pixel coordinates of the output image corresponding to the pixel point (x, y) after rotation transformation, and θ is the rotation angle.

[0129] The formula for image translation can be:

[0130]

[0131] where (x 0 , y 0 ) is the translation amount of the image along the x-axis and y-axis.

[0132] The formula for vertical mirror transformation of the image can be:

[0133]

[0134] The formula for horizontal mirror transformation of the image can be:

[0135]

[0136] where N and M are the length and width of the image respectively.

[0137] The formula for light and dark transformation of the image can be:

[0138] g(x ′ , y ′ ) = αf(x, y) + β

[0139] where f(x, y) is the original image pixel, g(x ′ , y ′ ) is the output image pixel, α > 0 is the gain, and β is the bias parameter, which control the contrast and brightness respectively.

[0140] The formula for Gaussian blur transformation can be:

[0141]

[0142] where r is the blur radius and σ is the standard deviation of the normal distribution.

[0143] Furthermore, the Chinese medicinal materials and their foreign object datasets can be labeled and divided into picture dataset files and picture corresponding label files;

[0144] The labeled datasets are classified. For example, the classification can include Chinese medicinal materials such as honeysuckle, artemisia annua, and gardenia, as well as foreign objects such as stones, glass, and fibers therein.

[0145] Then, the processed datasets are preprocessed, and the training datasets can be divided into a training set, a validation set, and a test set in a ratio of 7:1:2.

[0146] In this embodiment, data augmentation is performed through processing methods such as rotation, translation, mirroring, image brightness transformation, and Gaussian blur, which can effectively increase the number of training samples, thereby improving the model performance.

[0147] In this embodiment, a Chinese herbal medicine and its foreign object detection system are also provided. This system is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0148] This embodiment provides a Chinese herbal medicine and its foreign object detection system, as Figure 6 shown, the system includes:

[0149] An acquisition module 201, configured to acquire a to-be-detected image containing Chinese herbal medicine;

[0150] A prediction module 202, configured to determine whether there is a foreign object in the to-be-detected image based on the Chinese herbal medicine and its foreign object detection model according to the to-be-detected image.

[0151] In some alternative implementation manners, the system further includes:

[0152] A training module, configured to acquire a training data set, where the data set includes Chinese herbal medicine images and foreign object images; based on the YOLOv8 model, improve the YOLOv8 model; wherein, replace the detection head of the YOLOv8 model with a reparameterized detection head RepHead; replace the C2f module of the YOLOv8 model with a UIB module; replace the loss function in the YOLOv8 model with an InnerCIOU loss function; input the images in the training data set into the improved YOLOv8 model for training to obtain a trained Chinese herbal medicine and its foreign object detection model.

[0153] An evaluation module, configured to determine multiple preset IOU thresholds; the Chinese herbal medicine and its foreign object detection model outputs detection results corresponding to the preset IOU thresholds based on the preset IOU thresholds; determine the TP value and the FP value based on the detection results; determine the average precision at multiple preset IOU thresholds based on the TP value and the FP value.

[0154] The Chinese herbal medicine and its foreign object detection system in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0155] The further function descriptions of the above-mentioned various modules and units are the same as those in the corresponding embodiments above, and will not be repeated here.

[0156] An embodiment of the present invention further provides a computer device having the above-mentioned Figure 6 Chinese herbal medicine and foreign object detection system shown.

[0157] Please refer to Figure 7 , Figure 7 FIG. is a schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. As Figure 7 shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output system (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 7 In FIG., one processor 10 is taken as an example.

[0158] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.

[0159] Among them, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above-mentioned embodiment.

[0160] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memory.

[0162] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0163] An embodiment of the present invention further provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented by downloading through a network and originally stored in a remote storage medium or a non-transitory machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may further include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiment is implemented.

[0164] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for detecting Chinese herbal medicine and foreign matter therein, characterized in that: The method comprises: Acquire an image to be detected containing Chinese herbal medicine; Based on a Chinese herbal medicine and foreign body detection model, it is determined whether there are foreign bodies in the image to be detected according to the image to be detected, wherein the Chinese herbal medicine and foreign body detection model is an improved YOLOv8 model.

2. The method according to claim 1, characterized in that The Chinese herbal medicine and foreign body detection model is established by the following steps: Acquire a training data set, wherein the data set includes Chinese herbal medicine images and foreign body images; Based on the YOLOv8 model, the YOLOv8 model is improved; wherein the detection head of the YOLOv8 model is replaced with a re-parameterized detection head RepHead; the C2f module of the YOLOv8 model is replaced with a UIB module; and the loss function in the YOLOv8 model is replaced with an InnerCIOU loss function; The images in the training data set are input into the improved YOLOv8 model for training to obtain the trained Chinese herbal medicine and foreign body detection model.

3. The method according to claim 2, characterized in that During the training phase, the re-parameterized detection head RepHead includes: A plurality of first convolution modules cascaded in sequence; the first convolution module comprises: a first-size convolution layer, a first activation layer and a second-size convolution layer, wherein in the first convolution module, an output result of the first-size convolution layer is merged with an output result of the second-size convolution layer and an input result of the first-size convolution layer, and the merged result is input into the first activation layer; In the prediction stage, the re-parameterized detection head RepHead includes: a plurality of second convolution modules cascaded in sequence; the second convolution module includes: a first-size convolution layer and a second activation layer connected in sequence.

4. The method according to claim 2, characterized in that: The UIB module includes two optional deep convolution modules; the deep convolution module obtains multiple variants according to a preset selection method, wherein the variants include: reverse bottleneck, ConvNext, feedforward network and ExtraDW.

5. The method according to claim 2, characterized in that: The InnerCIOU loss function includes: Among them, b represents the center point of the prediction box, b gt represents the center point of the target box, ρ represents the Euclidean distance between the coordinates of the two center points, C represents the diagonal distance of the minimum box, and α represents the orthogonal balance parameter: v measures the consistency of aspect ratio: w gt Indicates the width of the target box, h gt represents the height of the target box, w represents the width of the prediction box, and h represents the height of the prediction box.

6. The method according to claim 1, characterized in that After obtaining the trained Chinese herbal medicine and foreign body detection model, the method further includes: Determine multiple preset IOU thresholds; The Chinese herbal medicine and foreign body detection model outputs a detection result corresponding to the preset IOU threshold value based on the preset IOU threshold value; Determine the TP value and the FP value based on the detection result; Based on the TP value and the FP value, the average precision under multiple preset IOU thresholds is determined.

7. The method according to claim 2, characterized in that The training data set is established through the following steps: Collect original Chinese herbal medicine images and original foreign body images; Labeling the original Chinese herbal medicine image and the original foreign body image respectively to obtain a labeled data set; Performing image enhancement processing on the images in the annotated data set by using at least one of the following processing methods: rotation, translation, mirroring, Gaussian blur, color gamut transformation, scaling and splicing; The data set after image enhancement processing is used as the training data set.

8. A Chinese herbal medicine and foreign matter detection system, characterized in that: The system comprises: An acquisition module, used for acquiring an image to be detected containing Chinese herbal medicine; The prediction module is used to determine whether there is a foreign body in the image to be detected based on the Chinese herbal medicine and foreign body detection model and the image to be detected.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for detecting Chinese herbal medicine and foreign matter thereof as described in any one of claims 1-7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method for detecting Chinese herbal medicines and foreign matter thereof according to any one of claims 1 to 7.

Images