An image-based angelica dahurica quality detection method

Abstract

The present application belongs to the cross field of computer vision, deep learning and medicinal material quality detection, and provides a white peony root quality detection method based on images, so as to realize automatic and intelligent Chinese medicinal material quality detection and meet the requirements of fast, convenient and accurate white peony root quality identification; firstly, an original white peony root image is acquired, and sample data sets are formed through data enhancement, and then training data sets are formed; then, a white peony root quality detection model is constructed and trained, the backbone network adopts resnet50, the auxiliary network adopts EllipseNet, and an ellipse-like regression frame is proposed, so that the network is more accurate in predicting each part of the white peony root, and can better extract the features of each region of the white peony root slice; finally, the white peony root quality grade prediction result is completed through the white peony root quality detection model. In summary, the present application models according to the specific characteristics of white peony root, has excellent effect on white peony root quality identification, and solves the problem that the existing technology cannot capture the characteristics of white peony root slices well.

Description

Technical Field

[0001] The present invention belongs to the intersection of computer vision, deep learning and medicinal material quality detection, relates to medicinal material quality identification technology, and specifically provides an image-based quality detection method for Angelica dahurica. Background Art

[0002] The quality of medicinal materials has a profound impact on the development of the Traditional Chinese Medicine (TCM) industry and the health of patients. "Distinguishing appearance from quality" is the essence of traditional Chinese medicine quality appraisal. This method, which rapidly determines the authenticity and quality of TCM from multiple dimensions—"shape, color, aroma, and flavor"—is irreplaceable to modern physical and chemical evaluation. However, its accuracy is affected by the appraiser's professional experience and subjective awareness. However, the current shortage of "veteran pharmacists" cannot meet the needs of the industry, and modern education models make it difficult to quickly pass on and promote the skills and experience of "veteran pharmacists." Angelica dahurica, a type of TCM, possesses significant medical value. According to historical herbal texts such as the Compendium of Materia Medica and the Shennong Herbal Classic, angelica dahurica has the effects of dispelling exterior pathogens and cold, dispelling wind and relieving pain, clearing the nasal passages, drying dampness and stopping leukorrhea, reducing swelling and draining pus, and dispelling wind and relieving itching. It also inhibits tyrosinase activity, preventing melanin production, achieving a whitening effect. It can also treat freckles. Its main chemical components also have anti-tumor and liver-protecting properties.

[0003] The use of machine learning technology can quickly and accurately complete the quality testing of medicinal materials, improving the efficiency and reliability of quality testing; at the same time, machine learning allows the automation of the quality testing process of traditional Chinese medicines, can process large-scale traditional Chinese medicine samples, reduce manual labor costs, and improve the scalability of testing; in addition, the application of machine learning technology to traditional Chinese medicine quality testing methods can also achieve real-time monitoring and early warning, reduce errors caused by human subjective factors and other advantages.

[0004] For example, a Chinese patent document with publication number CN106769927A discloses a method for detecting the quality of Astragalus membranaceus medicinal materials, which detects the quality of Astragalus membranaceus medicinal materials by measuring the ultraviolet-visible light-shortwave near-infrared diffuse reflectance spectrum of the medicinal materials. The method mainly includes the following steps: 1) using an Astragalus membranaceus medicinal material sample with known quality information as a modeling sample, measuring its ultraviolet-visible light-shortwave near-infrared diffuse reflectance spectrum, and collecting its spectral data; 2) performing spectral preprocessing and wavelength screening on the spectral data obtained in step 1), and using a principal component analysis discriminant model to detect the quality of the Astragalus membranaceus medicinal materials (when the quality detection is sulfur content detection, the quality detection model is a linear regression analysis prediction model); 3) detecting the quality of the Astragalus membranaceus medicinal material to be tested using the Astragalus membranaceus quality detection model in step 2);

[0005] For example, the Chinese patent document with publication number CN112434646A discloses a finished tea quality identification method based on transfer learning and computer vision technology. The main steps of the method are as follows: using a shooting device to collect tea leaf images and performing preprocessing; standardizing, enhancing, and dividing the data set for the preprocessed images; fine-tuning the pre-trained convolutional neural network, constructing a transfer learning model, and retraining using the finished tea data set to obtain a tea leaf image classification model; testing the finished tea images that are not used for training on the trained model to evaluate the classification effect.

[0006] However, the patent document CN106769927A uses reflectance spectroscopy to detect the quality of Radix Astragali, and uses a linear regression classification model for prediction. However, the quality identification of Angelica dahurica needs to consider multiple features, and the linear regression model is difficult to extract and consider deep features. In the patent document CN112434646A, tea leaf images are used to identify the quality of finished tea, and a pre-trained ResNet50 is used for quality identification. However, the model does not make targeted improvements for specific identification objects. SUMMARY

[0007] The purpose of the present application is to provide an image-based Angelica dahurica quality detection method to realize automatic and intelligent Chinese herbal medicine quality detection and meet the needs of fast, convenient, and accurate Angelica dahurica quality identification.

[0008] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0009] An image-based Angelica dahurica quality detection method, characterized by the following steps:

[0010] Step 1. Slice Angelica dahurica and use a shooting device to take images of the cut surface to obtain original Angelica dahurica images.

[0011] Step 2. Perform data enhancement on the original Angelica dahurica images to form a sample data set.

[0012] Step 3. Label each Angelica dahurica image in the sample data set to form a training set and a test set.

[0013] Step 4. Construct an Angelica dahurica quality detection model, wherein the main network uses resnet50 to perform initial feature extraction on the Angelica dahurica images and output multi-scale initial feature maps P i , i = 1, 2, 3; the auxiliary network uses EllipseNet, and the feature maps P i are input into EllipseNet respectively, and the corresponding region segmentation maps are output by EllipseNet and pixel-mapped to the original feature dimension. The region segmentation maps include: Angelica dahurica boundary xylem boundary Cambium region The feature map P i and Weighted feature map F i After upsampling, feature maps F2 and F3 maintain the same dimension as feature map F1. After weighting the three, feature map F is obtained. Feature map F is sequentially subjected to dropout operation and BN operation and then input into the fully connected layer, which outputs the classification prediction result.

[0014] Step 5. Construct a loss function and use the training set to complete the training of the Angelica dahurica quality detection model;

[0015] Step 6. Input the image of Angelica dahurica to be tested into the trained Angelica dahurica quality detection model, and the model outputs the Angelica dahurica quality grade prediction result.

[0016] Furthermore, in step 2, the data enhancement includes random rotation, mirroring and / or brightness adjustment operations, wherein the rotation is represented as:

[0017]

[0018] Among them, (x, y) represents the coordinates of the original pixel point, (x′, y′) represents the coordinates of the enhanced pixel point, θ r Indicates the rotation angle;

[0019] Horizontal mirroring and vertical mirroring are expressed as:

[0020]

[0021] The original and enhanced Angelica dahurica images were cropped using image processing software. The cropped images contained complete Angelica dahurica slices. The cropped original and enhanced Angelica dahurica images together constituted a sample dataset.

[0022] Furthermore, in step 3, use labelm e Each Angelica dahurica image in the sample dataset is annotated with slice information and quality grade information. The slice information includes the boundary outline, cambium outline, and xylem outline of the Angelica dahurica slice; the quality grade information is divided into special grade, first grade, second grade, third grade, or fourth grade.

[0023] Furthermore, in step 4, EllipseNet uses an improved FCOS network. The FCOS network outputs the position prediction information (o, t, b, l, r) of the Angelica dahurica slice, where o represents the center point, t, b, l, and r represent the distance from the center point to the upper, lower, left, and right boundaries, respectively. FCOS predicts a rectangular regression box based on the position prediction information, and EllipseNet improves FCOS based on the unique shape of Angelica dahurica. Specifically, EllipseNet defines an ellipse-like regression box based on the position prediction information (o, t, b, l, r). The ellipse regression box is expressed as:

[0024] E(x,y)=Concat(Ellipse(t,l),Ellipse(l,b),Ellipse(b,r),Ellipse(r,t))

[0025] Where E(x, y) represents the predicted ellipse regression box of the pixel point with coordinates (x, y), Concat represents the splicing operation, and Ellipse(A, B) represents a quarter ellipse formed with A and B as the semi-major axis and semi-minor axis;

[0026] According to the ellipse-like regression frame, the region segmentation map is obtained, including: Angelica dahurica boundary E t 、Cambium boundary E c 、Xylem boundary E w and cambium region E c-w , E c-w =E c -E w .

[0027] Furthermore, in step 4, the region segmentation map is mapped to the original feature map through pixel mapping; the pixel mapping is specifically expressed as:

[0028]

[0029] Among them, s represents the scaling factor from the original feature map to the scaled feature map, (x, y) represents the pixel point of the scaled feature map, Represents the pixel point of the original feature map, Indicates floor operation.

[0030] Furthermore, in step 5, the loss function L is expressed as:

[0031] L=αL gui +βL ratio +L cls

[0032] Among them, α and β are learnable hyperparameters; L gui represents the EllipseNet loss, L ratiodenotes an auxiliary loss, L cls denotes a classification loss;

[0033] EllipseNet loss is represented as:

[0034]

[0035] wherein, and E j denote the annotated contour and the predicted contour respectively, and N denotes the total number of pixel points;

[0036] The auxiliary loss is represented as:

[0037]

[0038] wherein, τ is a temperature factor, δ is a proportion confidence, S t and S w denote the area of the complete Angelica sinensis slice and the xylem part respectively;

[0039] The classification loss is represented as:

[0040] L cls = -∑y k log(p k )

[0041] wherein, p k denotes the predicted probability that the Angelica sinensis slice belongs to the kth quality grade, and y k denotes the true label of the kth class.

[0042] Based on the above technical solutions, the beneficial effects of the present application are:

[0043] The present application provides an image-based Angelica sinensis quality detection method, aiming at the shape characteristics of the Angelica sinensis slice, an improved FCOS network EllipseNet is proposed, because the rectangular regression frame may produce a larger blank area or miss the detail features at the edge of the slice, the present application uses an elliptical-like regression frame to replace the original rectangular regression frame of the FCOS network, compared with the rectangle, the elliptical-like shape is more in line with the original natural shape of the Angelica sinensis slice, and the elliptical-like regression frame can more accurately capture the edge features of the slice, especially when the slice shape is irregular or close to a circle, thus making the network more accurate in predicting the regions of each part of the Angelica sinensis, and better extracting the features of each region of the Angelica sinensis slice; at the same time, aiming at the influence factor of the proportion of xylem on the quality of Angelica sinensis in the Angelica sinensis quality identification system, the present application designs a corresponding auxiliary loss for enhancing the network, so as to make more accurate quality detection of the Angelica sinensis slice; compared with the existing medicinal material quality identification technology, the present application constructs a quality identification model for Angelica sinensis, models according to the specific characteristics of Angelica sinensis, and has excellent effect on Angelica sinensis quality identification, solving the problem that the existing technology cannot capture the characteristics of the Angelica sinensis slice well. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The present invention provides a flow chart of the image-based quality detection method for Angelica dahurica.

[0045] Figure 2 It is a schematic diagram of the cross section of the Angelica dahurica slices in the present invention.

[0046] Figure 3 Schematic diagram of the structure of the Angelica dahurica quality detection model in the present invention.

[0047] Figure 4 This is a schematic diagram of the ellipse-like regression frame output by the EllipseNet network of the Angelica dahurica quality detection model in the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0049] This embodiment provides an image-based quality detection method for Angelica dahurica, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0050] Step 1. Angelica dahurica image acquisition;

[0051] Slice the Angelica dahurica and use a photographing device (such as a mobile phone or a camera) to capture the slice to obtain an original Angelica dahurica image;

[0052] Step 2. Perform data augmentation on the original Angelica dahurica image to form a sample dataset;

[0053] To further optimize the training data for Angelica dahurica images, image data augmentation techniques were employed to improve the performance and generalization capabilities of the deep learning model. Data augmentation included random rotation, mirroring, and / or brightness adjustment to generate more diverse and rich training samples. Random rotation and / or mirroring simulated Angelica dahurica images taken from different angles, increasing the model's adaptability to different viewpoints, while brightness adjustment helped the model better handle Angelica dahurica images under varying lighting conditions.

[0054] Specifically, the rotation operation is expressed as:

[0055]

[0056] Among them, (x, y) represents the coordinates of the original pixel point, (x′, y′) represents the coordinates of the enhanced pixel point, θ r Indicates the rotation angle;

[0057] The horizontal mirroring operation and the vertical mirroring operation are expressed as:

[0058]

[0059] Data augmentation was performed on the original Angelica dahurica image. The original and enhanced Angelica dahurica images were cropped using image processing software (Photoshop). The cropped images contained complete Angelica dahurica slices with a resolution of 512×512. The cropped original and enhanced Angelica dahurica images together constituted the sample dataset.

[0060] Step 3. Label each Angelica dahurica image in the sample dataset to form a training set and a test set;

[0061] Using labelm e Each Angelica dahurica image in the sample dataset is annotated with slice information and quality grade information. The slice information includes the boundary outline, cambium outline and xylem outline of the Angelica dahurica slice, such as Figure 2 As shown; the quality grade information is divided into special grade, first grade, second grade, third grade or fourth grade;

[0062] Step 4. Construct a quality detection model for Angelica dahurica;

[0063] The quality detection model of Angelica dahurica is as follows: Figure 3 As shown, the backbone network uses resnet50 to perform initial feature extraction on the angelica image and output three initial feature maps P1, P2, and P3 with successively decreasing scales; resnet50 is a deep convolutional neural network with multiple convolutional layers and pooling layers, which is suitable for capturing details and complex structures in images. Using resnet50 to extract features from angelica images can deeply explore the visual information of angelica slices and describe the characteristics of angelica in a more comprehensive way; at the same time, resnet50 helps to capture subtle changes in plant tissues (such as the color information of angelica cross-sections), providing a powerful feature representation for accurately describing the cross-sectional information of angelica; deep neural networks also help to capture the differences between different regions of the cross-section, thereby better evaluating the quality grade of angelica; and the cambium color is an important part of the characteristics of angelica plants. Since the cambium part accounts for a small proportion of the image, and as the number of network layers deepens, the receptive field expands, which is not conducive to extracting small area features. The residual connection of resnet50 can just solve this problem;

[0064] The auxiliary network uses EllipseNet, which improves the FCOS network and is used to perform regional segmentation on the Angelica dahurica image. Figure 2As shown in the figure, the Angelica dahurica slice mainly includes the boundary (cortex), cambium, and xylem. In order to capture the different feature information of each part, the FCOS network is used to segment the Angelica dahurica image, and the rectangular regression box output by the FCOS network is improved to an ellipse-like regression box, which is renamed EllipseNet in the present invention. It should be noted that the network structure of EllipseNet in the present invention is the same as that of the FCOS network, but the output regression box structure is different; the FCOS network outputs the position prediction information (o, t, b, l, r) of the Angelica dahurica slice, where o represents the center point, t, b, l, and r represent the distance from the center point to the upper, lower, left, and right boundaries, respectively, and thus outputs a rectangular regression box, as shown in FIG. Figure 4 As shown in the middle left figure; In the present invention, considering that the slices and regions of Angelica dahurica are mostly elliptical, the elliptical regression frame is defined based on the position prediction information (o, t, b, l, r), as shown in FIG. Figure 4 As shown in the middle right figure, it is specifically expressed as:

[0065] E(x,y)=Concat(Ellipse(t,l),Ellipse(l,b),Ellipse(b,r),Ellipse(r,t))

[0066] Where E(x, y) represents the predicted ellipse regression box of the pixel point with coordinates (x, y), Concat represents the splicing operation, and Ellipse(A, B) represents a quarter ellipse formed with A and B as the semi-major axis and semi-minor axis;

[0067] Since the FCOS network first performs further feature extraction on the input original feature map to obtain a scaled feature map; and the ellipse-like regression box is annotated on the scaled feature map, it is necessary to map the ellipse-like regression box to the original feature map through pixel mapping; the pixel mapping is specifically expressed as:

[0068]

[0069] Among them, s represents the scaling factor from the original feature map to the scaled feature map, (x, y) represents the pixel point of the scaled feature map, Represents the pixel point of the original feature map, Indicates floor operation;

[0070] After region segmentation and pixel mapping, we can finally get the boundary E of Angelica dahurica. t 、Cambium boundary E c 、Xylem boundary E w , then the cambium area is represented by E c-w =E c -E w ;

[0071] On this basis, the feature map Pi , i = 1, 2, 3 are input to EllipseNet respectively, and EllipseNet outputs the corresponding region segmentation map: Angelica dahurica boundary Xylem boundary Cambium region The feature map P i and Weighted pixel-by-pixel connection to obtain the feature map F at the current scale i , specifically expressed as:

[0072]

[0073] Among them, λ, γ, and η are learnable hyperparameters;

[0074] After upsampling, feature maps F2 and F3 maintain the same dimension as feature map F1. After weighting the three, feature map F is obtained. Feature map F is sequentially subjected to dropout operation and BN operation and then input into the fully connected layer, which outputs the classification prediction result.

[0075] Step 5. Construct a loss function and use the training set to complete the training of the Angelica dahurica quality detection model;

[0076] The loss function is expressed as:

[0077] L=αL gui +βL ratio +L cls

[0078] Among them, α and β are learnable hyperparameters; L gui represents the EllipseNet loss, L ratio represents auxiliary loss, L cls represents the classification loss;

[0079] For the image area segmented by EllipseNet, the IoU loss function is used to evaluate the segmentation accuracy, which is specifically expressed as:

[0080]

[0081] in, With E j They represent the marked contour and the predicted contour respectively, and N represents the total number of pixels. For the convenience of representation, the image segmentation losses of the entire Angelica dahurica, xylem, and cambium are summed up to a total of L gui ;

[0082] Regarding auxiliary loss, in addition to considering the color and texture characteristics of each part, the proportion of xylem is also an important factor in the identification of Angelica dahurica. The present invention proposes an auxiliary loss based on the proportion of xylem to assist in the identification of the quality grade of Angelica dahurica. The auxiliary loss is expressed as:

[0083]

[0084] Among them, τ is the temperature factor, δ is the confidence factor, S t With S w Respectively represent the area of ​​the complete Angelica dahurica slice and the xylem part; the confidence level used in this embodiment is 1 / 3, which can be adjusted according to different Angelica dahurica species;

[0085] For classification loss, it is specifically expressed as:

[0086] L cls =-∑y k log(p k )

[0087] Among them, p k represents the predicted probability that the Angelica dahurica slice belongs to the kth quality level, y k Represents the true label of the kth category;

[0088] Step 6. Input the image of Angelica dahurica to be tested into the trained Angelica dahurica quality detection model, and the model outputs the Angelica dahurica quality grade prediction result.

[0089] The above description is only a specific embodiment of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes; all disclosed features, or all steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. An image-based quality detection method for Angelica dahurica, characterized in that: The following steps are involved: Step 1. Slice the Angelica dahurica and capture the slice using a camera to obtain an original Angelica dahurica image. Step 2. Perform data augmentation on the original Angelica dahurica image to form a sample dataset; Step 3. Label each Angelica dahurica image in the sample dataset to form a training set; Step 4. Construct the Angelica dahurica quality detection model, in which the backbone network uses resnet50 to extract the initial features of the Angelica dahurica image and output the multi-scale initial feature map P i ,i=1,2,3; the auxiliary network uses EllipseNet, the feature map P i They are input into EllipseNet respectively, and EllipseNet outputs the corresponding region segmentation map, and the pixels are mapped to the original feature dimension. The region segmentation map includes: Xylem boundary Cambium region The feature map P i and Weighted feature map F i After upsampling, feature maps F2 and F3 maintain the same dimension as feature map F1. After weighting the three, feature map F is obtained. Feature map F is sequentially subjected to dropout operation and BN operation and then input into the fully connected layer, which outputs the classification prediction result. EllipseNet uses an improved FCOS network. The FCOS network outputs the position prediction information (o, t, b, l, r) of the Angelica dahurica slice, where o represents the center point, t, b, l, and r represent the distance from the center point to the upper, lower, left, and right boundaries, respectively. EllipseNet defines an ellipse-like regression box based on the position prediction information (o, t, b, l, and r). The ellipse-like regression box is expressed as: E(x,y)=Concat(Ellipse(t,l),Ellipse(l,b),Ellipse(b,r),Ellipse(r,t)) Where E(x,y) represents the predicted ellipse regression box of the pixel point with coordinates (x,y), Concat represents the splicing operation, and Ellipse(A,B) represents a quarter ellipse formed with A and B as the semi-major axis and semi-minor axis; According to the ellipse-like regression frame, the region segmentation map is obtained, including: the boundary E of Angelica dahurica t , cambium boundary E c 、Xylem boundary E w and cambium region E c-w , E c-w =E c -E w ; Step 5. Construct a loss function and use the training set to complete the training of the Angelica dahurica quality detection model; the loss function L is expressed as: L=αL gui +βL ratio +L cls Among them, α and β are learnable hyperparameters; L gui represents the EllipseNet loss, L ratio represents auxiliary loss, L cls represents the classification loss; EllipseNet loss is expressed as: in, With E j Represent the marked contour and predicted contour respectively, and N represents the total number of pixels; The auxiliary loss is expressed as: Among them, τ is the temperature factor, δ is the confidence factor, S t With S w represent the area of ​​the complete Angelica dahurica slice and the xylem part, respectively; The classification loss is expressed as: L cls =-∑y k log(p k ) Among them, p k represents the predicted probability that the Angelica dahurica slice belongs to the kth quality level, y k Represents the true label of the kth category; Step 6. Input the image of Angelica dahurica to be tested into the trained Angelica dahurica quality detection model, and the model outputs the Angelica dahurica quality grade prediction result.

2. The image-based quality detection method for Angelica dahurica according to claim 1, characterized in that: In step 2, the data augmentation includes random rotation, mirroring and / or brightness adjustment operations, where rotation is expressed as: Among them, (x, y) represents the coordinates of the original pixel point, (x′, y′) represents the coordinates of the enhanced pixel point, θ r Indicates the rotation angle; Horizontal mirroring and vertical mirroring are expressed as: The original and enhanced Angelica dahurica images were cropped using image processing software. The cropped images contained complete Angelica dahurica slices. The cropped original and enhanced Angelica dahurica images together constituted a sample dataset.

3. The image-based quality detection method for Angelica dahurica according to claim 1, characterized in that: In step 3, labelme is used to annotate each Angelica dahurica image in the sample dataset. The annotation content includes slice information and quality grade information. The slice information includes the boundary outline, cambium outline, and xylem outline of the Angelica dahurica slice; the quality grade information is divided into special grade, first grade, second grade, third grade, or fourth grade.

4. The image-based quality detection method for Angelica dahurica according to claim 1, characterized in that: In step 4, the region segmentation map is mapped to the original feature map through pixel mapping; the pixel mapping is specifically expressed as: Among them, s represents the scaling factor from the original feature map to the scaled feature map, (x, y) represents the pixel point of the scaled feature map, Represents the pixel point of the original feature map, Indicates a floor operation.

Images