Traditional Chinese medicinal material identification method based on image identification

Abstract

The invention discloses a traditional Chinese medicinal material identification method based on image identification, and the method mainly comprises the following steps: firstly collecting an original image, and carrying out the preprocessing of the original image, and obtaining first image data; then, establishing an initial model by taking YOLOv8 as a basic model architecture, and adjusting to obtain a first recognition model; inputting the first image data into a first recognition model for training to obtain a trained second recognition model; image data to be recognized are input into a second recognition model, the model obtains high-dimensional feature representation of the image data through multi-layer convolution and pooling operation, a prediction layer outputs category probability and position information, low-confidence recognition results are filtered out, and a final recognition result is obtained. The method is characterized in that weight parameters are designed according to the characteristics of the traditional Chinese medicinal materials, the network structure and parameters are adjusted, and the recognition accuracy and reliability are improved. In addition, an anchor frame calculation method and a loss function design method are further provided, and the recognition efficiency is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of medicinal material identification, and in particular relates to a Chinese medicinal material identification method based on image recognition. Background Art

[0002] There are many types of Chinese medicinal materials, and efficient and accurate identification methods are needed to identify their authenticity and determine their medicinal value. Traditional identification methods mainly rely on manual visual observation, which is time-consuming, labor-intensive and prone to errors. With the continuous development of computer vision technology, image recognition technology has gradually become an important means of identifying Chinese medicine. However, existing image recognition technology still has certain problems in the identification of Chinese medicinal materials, such as low recognition accuracy and slow recognition speed.

[0003] The application number is 2017109396722, and the name of the invention is: A method for identifying Chinese herbal medicines based on deep neural networks, which uses web crawlers and manual photography to collect pictures of Chinese herbal medicines as input of the data set and performs preprocessing; the training and prediction process uses the Bagging method of ensemble learning, and uses the classic convolutional neural network model to fine-tune the training of each sub-training set to generate multiple weak classifiers, and finally cooperates with the Softmax classification algorithm, and uses the ensemble learning combination strategy to obtain a strong classifier to obtain the classification result. Although this patent can realize the identification of Chinese herbal medicines, it may be difficult to obtain enough sample data for training AI models for some rare or special Chinese herbal medicines. This will lead to limited identification capabilities of the model for these herbal medicines. For example, due to the special growth environment and rare yield of some wild Chinese herbal medicines, it is difficult to obtain samples, and the amount of data is difficult to meet the requirements of model training. Data from different sources may have quality differences, such as inconsistencies in image clarity and annotation accuracy, which affect the training effect of the model. For example, some early collected images of Chinese herbal medicines may have low resolution, or the annotation information may not be detailed and accurate enough, which will interfere with the learning of the model.

[0004] To this end, we propose a Chinese medicinal material identification method based on image recognition to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to solve the problem of inaccurate and slow identification of some Chinese medicinal materials in the prior art, and to propose a Chinese medicinal material identification method based on image recognition.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A method for identifying Chinese medicinal materials based on image recognition, comprising: S10: collecting original images, preprocessing the original images, and obtaining first image data; the original images include images of Chinese medicinal materials at different growth stages, different origins, and different collection environments; the preprocessing includes normalizing the original images and image enhancement; S20: Establish an initial model based on YOLOv8 as the basic model architecture, and adjust the initial model, including adjusting the depth and width of the network, and adding a combination of convolutional layers and pooling layers. In the loss function of the model, introduce weight parameters for the characteristics of Chinese medicinal materials to obtain the first recognition model; S30: inputting the first image data into the first recognition model for training, fine-tuning the first image data during the training, freezing the backbone part, training at a learning rate of 0.001 in the final head part, and using an early stopping method to prevent overfitting, thereby obtaining a trained second recognition model; S40: The image data to be identified is input into the second recognition model. The second recognition model extracts features from the image data to be identified, obtains high-dimensional feature representation of the image data to be identified through multi-layer convolution and pooling operations, and the prediction layer outputs the category probability and position information of the image data to be identified, filters out recognition results with low confidence, and obtains the final recognition result.

[0007] Preferably, in step S20, the depth of the network is adjusted by deepening the FPN structure of the initial model to a 4-layer structure, while adding a detection layer of the DETECT part, and using a clustering algorithm to add four types of preset anchor frames.

[0008] Preferably, the anchor box calculation method is as follows: S21: Extract all bounding box coordinates and convert the coordinate data into the width and height of the box; S22: Initialize k anchor boxes by randomly selecting k values ​​from all bounding boxes as the initial values ​​of k anchor boxes; S23: Calculate the iou value of each bounding box and each anchor box, calculate the error d(n,k) of each bounding box for each anchor box, and compare the error size of each bounding box for each anchor box {d(i,1), d(i,2), …, d(i,k)}, select the anchor box with the smallest error, and classify the bounding box to it; S24: Calculate the width, height and median size of the bounding box in each anchor box, and use the width, height and median size as the new size of the anchor box; S25: Repeat S23-S24 until the anchor box class to which all bounding boxes belong is consistent with the anchor box class to which they previously belonged; S26: Use the final anchor boxes and each bounding box to calculate their IOU values. For each bounding box, select the highest IOU value, which means it belongs to a certain anchor box class. Calculate the average IOU value of all bounding boxes to get the final accuracy value.

[0009] Preferably, in step S20, the width of the network is adjusted by introducing a combination of a semantic attention mechanism of the channel layer and a decoupled attention mechanism of the spatiotemporal layer.

[0010] Preferably, the loss function in step S20 includes classification loss, positioning loss, and confidence loss. The positioning loss represents the error between the predicted box and the real box, and the confidence loss represents the target of the box. The classification loss and confidence loss are calculated by the binary cross entropy loss function, and the bounding box regression loss is calculated by CIoU.

[0011] Preferably, EIoU Loss is used based on CIoU. The loss function consists of three parts: overlap loss, center distance loss, and width and height loss. The first two parts continue the method in CIOU, but the width and height loss directly minimizes the difference between the width and height of the target box and the anchor box.

[0012] Preferably, Focal EIOU Loss is used in combination with Focal Loss on the basis of EIOU to separate high-quality anchor boxes from low-quality anchor boxes. The penalty term formula is as follows: , Among them, IOU = |A∩B| / |A∪B|, γ is a parameter that controls the degree of outlier suppression.

[0013] Preferably, during the training process of step S30, a validation set is used to perform periodic performance evaluation, and the evaluation indicators include accuracy, precision, recall and F1 score.

[0014] In summary, the technical effects and advantages of the present invention are: 1. Through deep learning and image recognition technology, artificial intelligence can quickly and accurately identify the types of Chinese medicinal materials. Compared with traditional manual identification methods, artificial intelligence reduces human errors and improves the accuracy and reliability of identification; 2. Through data preprocessing and model optimization technology, the generalization ability of the model is improved, so that it can adapt to the images of Chinese medicinal materials collected in different environments; 3. By using transfer learning and optimized training strategies, the model training time and computing resource consumption are reduced, and the training efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the present invention; Figure 2 This is a diagram of the improved FPN structure in Example 1; Figure 3 is the attention module diagram in Embodiment 1; Figure 4 This is a diagram for replacing the attention module in Embodiment 1; Figure 5 It is the topological diagram of the algorithm solution in the second embodiment; Figure 6 This is the distribution diagram of the tangerine peel and hawthorn detection data in Example 2; Figure 7 This is a schematic diagram of the detection accuracy of dried tangerine peel and hawthorn in Example 2; Figure 8 This is an example diagram of the training data of dried tangerine peel and hawthorn in Example 2; Fig. 9 The training loss function and accuracy graph of the hawthorn authenticity detection algorithm model in Example 2; Fig.10 This is an example diagram of the training data for detecting the authenticity of hawthorn in Example 2; Fig.11 The loss function and accuracy rate diagram of the tangerine peel origin detection algorithm model training process in Example 2; Fig.12 This is an example diagram of the tangerine peel origin detection training data in Example 2; Fig.13 The loss function and accuracy rate diagram of the Xinhui tangerine peel year detection algorithm model training process in Example 2; Fig.14 This is an example diagram of the Xinhui dried tangerine peel year detection training data in Example 2. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0017] As shown in the figure, a method for identifying Chinese medicinal materials based on image recognition includes: S10: collecting original images, preprocessing the original images, and obtaining first image data; the original images include images of Chinese medicinal materials at different growth stages, different origins, and different collection environments; the preprocessing includes normalizing the original images and image enhancement; S20: Establish an initial model based on YOLOv8 as the basic model architecture, and adjust the initial model, including adjusting the depth and width of the network, and adding a combination of convolutional layers and pooling layers. In the loss function of the model, introduce weight parameters for the characteristics of Chinese medicinal materials to obtain the first recognition model; S30: inputting the first image data into the first recognition model for training, fine-tuning the first image data during the training, freezing the backbone part, training at a learning rate of 0.001 in the final head part, and using an early stopping method to prevent overfitting, thereby obtaining a trained second recognition model; S40: The image data to be identified is input into the second recognition model. The second recognition model extracts features from the image data to be identified, obtains high-dimensional feature representation of the image data to be identified through multi-layer convolution and pooling operations, and the prediction layer outputs the category probability and position information of the image data to be identified, filters out recognition results with low confidence, and obtains the final recognition result.

[0018] The technical solutions in the above-mentioned embodiments of the present application have at least the following technical effects or advantages: the algorithm of the present invention improves the accuracy of Chinese medicinal materials identification, can effectively distinguish similar Chinese medicinal materials, reduce misjudgment, and provide strong support for Chinese medicinal materials quality control and authenticity identification. Through data preprocessing and model optimization technology, the generalization ability of the model is improved, so that it can adapt to Chinese medicinal materials images collected in different environments. By using transfer learning and optimized training strategies, the model training time and computing resource consumption are reduced, and the training efficiency is improved.

[0019] Embodiment 1 The embodiment of the present application also provides a method for identifying Chinese medicinal materials based on image recognition.

[0020] Collect image data of various Chinese medicinal materials, including images of Chinese medicinal materials at different growth stages, different origins, and different collection environments. Image sources include but are not limited to field photography, images from Chinese medicinal materials specimen libraries, etc.

[0021] Perform preprocessing operations on the collected images. First, perform image normalization to normalize the pixel values ​​of the image to a specific range of 0-1 to improve the stability of model training. Then perform image enhancement operations such as rotation, flipping, scaling, adding noise, etc. to expand the data set and improve the generalization ability of the model.

[0022] YOLOv8 is selected as the basic model architecture. YOLOv8 has the advantages of fast and accurate object detection.

[0023] According to the characteristics of Chinese herbal medicine recognition, the YOLOv8 model is improved. The depth and width of the network are adjusted to adapt to the complexity of Chinese herbal medicine images. Specific convolutional layers and pooling layers are added to improve the model's ability to extract local and overall features of Chinese herbal medicines.

[0024] In view of the characteristics of Chinese medicinal materials identification, since image details are very important in medicinal materials identification, and there are problems such as a large proportion of pictures and unbalanced samples, the FPN of YOLOv8 is deepened from 3 layers to 4 layers. In addition, in order to match the depth of FPN, the detection layer of the Detect part is added, and four types of preset anchor frames are added using the clustering algorithm. This anchor frame is obtained based on the analysis of Chinese medicinal materials data, and has one more P6 layer than the original Yolo network. Afterwards, target detection is performed on the feature maps output by P3, P4, P5, and P6 in turn. After adding the detection layer, the laying interval of the anchor frame becomes more reasonable, and the stability of training as well as the convergence speed and accuracy of the model are effectively improved. The simplified version of the FPN structure used is as follows: Figure 2 As shown in Figure 2, the improved model has an object recognition accuracy improvement of 2.1% compared to the original model.

[0025] The specific method for calculating the anchor box is as follows: Step 1: First, you need to extract all the bounding box coordinates. Maybe a picture has one rectangular box, or maybe there are multiple rectangular boxes, but you need to extract all the rectangular boxes of all pictures without distinction and put them together.

[0026] Step 2: Data processing to obtain the width and height data of all training data bounding boxes. The given training data is often the 4 coordinates of its bounding box, but the subsequent clustering analysis requires the width and height of the bounding box, so the coordinate data needs to be converted into the width and height of the box. The calculation method is very simple: length = lower right corner horizontal coordinate - upper left corner horizontal coordinate, width = lower right corner vertical coordinate - upper left corner vertical coordinate.

[0027] Step 3: Initialize k anchor boxes by randomly selecting k values ​​from all bounding boxes as the initial values ​​of k anchor boxes.

[0028] Step 4: Calculate the iou value of each bounding box and each anchor box. The traditional clustering method uses Euclidean distance to measure the difference. That is to say, if the traditional k-means clustering algorithm is used, the width and height of the bounding box can be directly clustered to generate k anchor boxes with different widths and heights. However, this method has a greater error when the box size is larger, so the iou value is introduced to avoid this problem.

[0029] Step 5: Classification. After the calculation in the previous step, we can get the error d(n,k) of each bounding box for each anchor box. By comparing the error size of each bounding box for each anchor box {d(i,1), d(i,2), ..., d(i,k)}, select the anchor box with the smallest error and classify it. Do this operation for each bounding box, and finally record which bounding boxes belong to each anchor box.

[0030] Step 6: Update anchor box. After the previous step, we know which bounding boxes belong to each anchor box. Then, for the bounding boxes in each anchor box, we calculate the width and height median of these bounding boxes and use them as the new size of the anchor box.

[0031] Step 7: Repeat steps 4 to 6 until it is found in step 5 that the anchor box class to which all bounding boxes belong is exactly the same as the anchor box class to which they belonged before.

[0032] Step 8: Calculate the accuracy of anchor boxes. In step 7, the anchor boxes have been calculated by the k-means algorithm. Use the final anchor boxes to calculate the IOU value of each bounding box. For each bounding box, select the highest IOU value (which means it belongs to a certain anchor box class), and then calculate the average of the IOU values ​​of all bounding boxes, which is the final accuracy value.

[0033] In addition, in order to improve the detection accuracy of multi-scale target objects in images, a combination of the semantic attention mechanism of the channel layer and the decoupled attention mechanism of the spatiotemporal layer is introduced to explicitly model the similarity of network extracted features at the spatial and semantic levels, thereby realizing saliency detection of multi-scale target object features.

[0034] First, the spatial and channel attention features are extracted from the feature map at the same time, and then weighted fusion is performed. The CBAM attention structure in parallel mode is a plug-and-play module that can be inserted into each convolution module in Backbone to improve the network's convergence speed, accuracy, and regression ability to the target boundary. For different feature map data, since it is impossible to predict the degree of attention of the feature map to the channel and space, the channel attention module and the spatial attention module are first given the same priority, and then the mixed domain feature information is extracted in a learnable weighted fusion method to exchange information between the feature map space and channels, making full use of the attention information in the channel and spatial dimensions. The structure is as follows Figure 3 Figure 4 As shown in the experimental test, in the original Neck structure of Yolov8, Figure 4 The Conv layer in the red circle is partially replaced with the attention module, and the other modules are retained, so the accuracy can be maximized. The improved model improves the accuracy of object recognition by 1.7% compared with the original model.

[0035] In the loss function of the model, weight parameters for the characteristics of Chinese medicinal materials are introduced. For easily confused Chinese medicinal materials, their weights in the loss function are increased, which prompts the model to pay more attention to these differences during training and improve the accuracy of classification.

[0036] Yolov8 loss functions include classification loss, localization loss, and confidence loss. The localization loss represents the error between the predicted box and the true box, and the confidence loss represents the target of the box. Yolov8 uses the binary cross entropy loss function to calculate the classification loss and confidence loss, and uses CIoU to calculate the bounding box regression loss. CIoU adds the loss of the detection box scale on the basis of GIoU and DIoU, and adds the loss of length and width, so that the predicted box will be more consistent with the true box. During the training process, the design of the loss function greatly affects the final accuracy and precision.

[0037] Therefore, the more accurate EIoU loss is used.

[0038] , Among them, the three items of CIoU used by Yolov8 correspond to IoU, center point distance, and aspect ratio calculation. α and v are aspect ratios, and w, h, w(gt), and h(gt) represent the height and width of the predicted box and the height and width of the real box respectively.

[0039] Although CIoU takes into account the overlapping area, center point distance, and aspect ratio of bounding box regression, the difference in aspect ratio reflected by v in its formula is not the actual difference between width and height and their confidence, which sometimes hinders the model from effectively optimizing similarity. In order to increase the detection accuracy of the bounding box, EIoU Loss is used here, and the Focal high-quality anchor box is added.

[0040] The penalty term of EIOU is based on the penalty term of CIOU, which separates the influencing factors of aspect ratio and calculates the length and width of the target box and anchor box respectively. The loss function contains three parts: overlap loss, center distance loss, and width and height loss. The first two parts continue the method in CIOU, but the width and height loss directly minimizes the difference between the width and height of the target box and the anchor box, making the convergence speed faster. The penalty term formula is as follows: , Among them, Cw and Ch are the width and height of the smallest bounding box covering the two Boxes.

[0041] Considering that there is also an imbalanced training sample problem in BBox regression, that is, the number of high-quality anchor boxes with small regression errors in an image is far less than the number of low-quality samples with large errors, and the poor-quality samples will produce too large gradients that affect the training process. We use Focal EIOU Loss in combination with Focal Loss on the basis of EIOU, and separate high-quality anchor boxes from low-quality anchor boxes from the perspective of gradient. The penalty term formula is as follows: , Where IOU = |A∩B| / |A∪B|, γ is a parameter that controls the degree of suppression of outliers. The Focal in this loss is somewhat different from the traditional Focal Loss. The traditional Focal Loss has a greater loss for more difficult samples, which plays the role of mining difficult samples; while according to the above formula: the higher the IOU, the greater the loss, which is equivalent to a weighted effect, giving a greater loss to a better regression target, which helps to improve the regression accuracy.

[0042] For model training and optimization The improved YOLOv8 model was trained using the preprocessed Chinese medicinal materials image dataset. Batch training was adopted, with the batch size set to 512, the initial learning rate to 0.01, and a 10% decrease every 10 epochs.

[0043] During the training process, transfer learning technology is used. The model is first initialized with weights pre-trained on a large-scale natural image dataset, and then fine-tuned on a Chinese herbal medicine image dataset, using a frozen backbone part and a learning rate of 0.001 for training in the final head part. This speeds up the model convergence.

[0044] Early stopping and other strategies are used to prevent overfitting. At the same time, the validation set is used to evaluate the performance of the model after each training epoch. The evaluation indicators include accuracy, precision, recall, and F1 score. After each evaluation, the changes in these indicators are recorded and analyzed to track the improvement of the model.

[0045] The image of the Chinese herbal medicine to be identified is input into the trained YOLOv8-based model. The model first extracts features from the image and obtains the high-dimensional feature representation of the image through multi-layer convolution and pooling operations. Then, the prediction layer is used to output the category probability, location information, etc. of each Chinese herbal medicine target. According to the set threshold, the prediction results with low confidence are filtered out, and finally an accurate Chinese herbal medicine recognition result is obtained.

[0046] Embodiment 2 like Figure 5-Figure 14 As shown, taking the identification of tangerine peel and hawthorn as an example like Figure 5 As shown, for the detected medicinal material frame, the image is cropped and sent to the corresponding image classifier network through the image preprocessing algorithm, among which the hawthorn is sent to the hawthorn authenticity classifier network to realize the identification of hawthorn and crabapple; the tangerine peel is first sent to the origin classifier network and divided into four origins; then the Xinhui tangerine peel is further sent to the year classifier network to realize the classification of the year of Xinhui tangerine peel.

[0047] This training process uses a 4-card 2080Ti server and the pytorch deep learning training framework.

[0048] In summary, the invention designs weight parameters for the characteristics of Chinese medicinal materials, adjusts the network structure and parameters, and improves recognition accuracy and reliability. In addition, an anchor frame calculation method and a loss function design method are proposed to improve recognition efficiency.

[0049] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for identifying Chinese medicinal materials based on image recognition, characterized in that: The steps include: S10: collecting original images, preprocessing the original images, and obtaining first image data; the original images include images of Chinese medicinal materials at different growth stages, different origins, and different collection environments; the preprocessing includes normalizing the original images and image enhancement; S20: Establish an initial model based on YOLOv8 as the basic model architecture, and adjust the initial model, including adjusting the depth and width of the network, and adding a combination of convolutional layers and pooling layers. In the loss function of the model, introduce weight parameters for the characteristics of Chinese medicinal materials to obtain the first recognition model; S30: inputting the first image data into the first recognition model for training, fine-tuning the first image data during the training, freezing the backbone part, training at a learning rate of 0.001 in the final head part, and using an early stopping method to prevent overfitting, thereby obtaining a trained second recognition model; S40: The image data to be identified is input into the second recognition model. The second recognition model extracts features from the image data to be identified, obtains high-dimensional feature representation of the image data to be identified through multi-layer convolution and pooling operations, and the prediction layer outputs the category probability and position information of the image data to be identified, filters out recognition results with low confidence, and obtains the final recognition result.

2. The method for identifying Chinese medicinal materials based on image recognition according to claim 1, characterized in that: In the step S20, the depth of the network is adjusted by deepening the FPN structure of the initial model to a 4-layer structure, while adding a detection layer of the DETECT part, and adding four types of preset anchor frames through a clustering algorithm.

3. The method for identifying Chinese medicinal materials based on image recognition according to claim 2, characterized in that: The anchor frame calculation method is as follows: S21: Extract all bounding box coordinates and convert the coordinate data into the width and height of the box; S22: Initialize k anchor boxes by randomly selecting k values ​​from all bounding boxes as the initial values ​​of k anchor boxes; S23: Calculate the iou value of each bounding box and each anchor box, calculate the error d(n,k) of each bounding box for each anchor box, and compare the error size of each bounding box for each anchor box {d(i,1), d(i,2), …, d(i,k)}, select the anchor box with the smallest error, and classify the bounding box to it; S24: Calculate the width, height and median size of the bounding box in each anchor box, and use the width, height and median size as the new size of the anchor box; S25: Repeat S23-S24 until the anchor box class to which all bounding boxes belong is consistent with the anchor box class to which they previously belonged; S26: Use the final anchor boxes and each bounding box to calculate their IOU values. For each bounding box, select the highest IOU value, which means it belongs to a certain anchor box class. Calculate the average IOU value of all bounding boxes to get the final accuracy value.

4. The method for identifying Chinese medicinal materials based on image recognition according to claim 1, characterized in that: In step S20, the width of the network is adjusted by introducing a combination of a semantic attention mechanism of the channel layer and a decoupled attention mechanism of the spatiotemporal layer.

5. The method for identifying Chinese medicinal materials based on image recognition according to claim 1, characterized in that: The loss function in step S20 includes classification loss, positioning loss, and confidence loss. The positioning loss represents the error between the predicted box and the real box, and the confidence loss represents the target of the box. The classification loss and confidence loss are calculated by the binary cross entropy loss function, and the bounding box regression loss is calculated by CIoU.

6. The method for identifying Chinese medicinal materials based on image recognition according to claim 5, characterized in that: EIoU Loss is used based on CIoU. The loss function consists of three parts: overlap loss, center distance loss, and width and height loss. The first two parts continue the method in CIOU. The width and height loss is obtained by minimizing the difference between the width and height of the target box and the anchor box.

7. The method for identifying Chinese medicinal materials based on image recognition according to claim 6, characterized in that: Based on EIOU, Focal EIOU Loss is used in combination with Focal Loss to separate high-quality anchor boxes from low-quality anchor boxes. The penalty term formula is as follows: ， Among them, IOU = |A∩B| / |A∪B|, γ is a parameter that controls the degree of outlier suppression.

8. The method for identifying Chinese medicinal materials based on image recognition according to claim 1, characterized in that: During the training process of step S30, a validation set is used to perform periodic performance evaluation, and the evaluation indicators include accuracy, precision, recall and F1 score.

Images