Angelica dahurica quality detection method based on image

Abstract

The invention belongs to the crossing field of computer vision, deep learning and medicinal material quality detection, and provides an image-based radix angelicae quality detection method, which is used for realizing automatic and intelligent quality detection of traditional Chinese medicinal materials and meeting rapid, convenient and accurate radix angelicae quality identification requirements. Firstly, an original angelica dahurica image is obtained, a sample data set is formed through data enhancement, and then a training data set is formed; then, a radix angelicae quality detection model is constructed and trained, a backbone network adopts resnet50, an auxiliary network adopts EllipseNet, and an ellipse-like regression frame is proposed, so that the network can predict regions of all parts of the radix angelicae more accurately, and the features of all regions of the radix angelicae slices can be better extracted; and finally, completing an angelica dahurica quality grade prediction result through the angelica dahurica quality detection model. In conclusion, modeling is carried out according to the specific characteristics of the radix angelicae, the method has an excellent effect on quality identification of the radix angelicae, and the problem that the characteristics of the radix angelicae slices cannot be well captured in the prior art is solved.

Description

Technical Field

[0001] The present invention belongs to the intersection of computer vision, deep learning and medicinal material quality detection, relates to medicinal material quality identification technology, and specifically provides an image-based Angelica dahurica quality detection method. Background Art

[0002] The quality of medicinal materials has a profound impact on the development of the Chinese medicine industry and the health of patients. "Distinguishing appearance and discussing quality" is the essence of traditional Chinese medicine quality identification. It can quickly judge the authenticity and quality of Chinese medicine from multiple dimensions of "shape, color, smell and taste", which is also irreplaceable by modern physical and chemical evaluation, but its accuracy is affected by the professional experience and subjective consciousness of the appraiser; however, the lack of contemporary "old pharmacists" cannot meet the needs of the industry, and the modern education model makes it difficult to inherit and promote the skills and experience of "old pharmacists" in a short period of time. As one of the Chinese medicinal materials, Angelica dahurica has great medical value; according to the records of ancient herbal works, such as "Compendium of Materia Medica" and "Shennong's Herbal Classic", Angelica dahurica has the effects of dispelling cold, dispelling wind and relieving pain, clearing the nasal passages, drying dampness and stopping leukorrhea, reducing swelling and pus, and dispelling wind and relieving itching. At the same time, Angelica dahurica can inhibit the activity of tyrosinase, prevent the formation of melanin, achieve the effect of whitening, and can also treat freckles. The main chemical components it contains also have anti-tumor and liver protection effects.

[0003] The use of machine learning technology can quickly and accurately complete the quality inspection of medicinal materials, improving the efficiency and reliability of quality inspection; at the same time, machine learning allows the automation of the quality inspection process of traditional Chinese medicine, can process large-scale traditional Chinese medicine samples, reduce labor costs, and improve the scalability of inspection; in addition, the application of machine learning technology to the quality inspection method of traditional Chinese medicine can also achieve real-time monitoring and early warning, reduce errors caused by human subjective factors and other advantages.

[0004] For example, a Chinese patent document with publication number CN106769927A discloses a method for detecting the quality of Astragalus medicinal materials, which detects the quality of Astragalus medicinal materials by measuring the ultraviolet-visible light-shortwave near-infrared diffuse reflectance spectrum of the medicinal materials. The method mainly includes the following steps: 1) using an Astragalus medicinal material sample with known quality information as a modeling sample, measuring its ultraviolet-visible light-shortwave near-infrared diffuse reflectance spectrum, and collecting its spectral data; 2) performing spectral preprocessing and wavelength screening on the spectral data obtained in step 1), and using a principal component analysis discriminant model to perform quality detection on the Astragalus medicinal materials (when the quality detection is sulfur content detection, the quality detection model is a linear regression analysis prediction model); 3) using the Astragalus quality detection model of step 2) to detect the quality of the Astragalus medicinal material to be tested;

[0005] Another example is a Chinese patent document with publication number CN112434646A that discloses a finished tea quality identification method based on transfer learning and computer vision technology. The main steps of the method are: using a camera to collect tea images and preprocess them; standardizing, enhancing and dividing the preprocessed images into data sets; fine-tuning the pre-trained convolutional neural network to build a transfer learning model, and retraining it using the finished tea data set to obtain a tea image classification model; using finished tea images that were not used for training to test the trained model and evaluate the classification effect;

[0006] However, patent document CN106769927A uses reflectance spectroscopy to detect the quality of Astragalus membranaceus and uses a linear regression classification model for prediction. However, the quality identification of Angelica dahurica requires consideration of multiple features, and the linear regression model is difficult to extract and consider deep-level features. Patent document CN112434646A uses tea leaves to identify the quality of finished tea, and uses a pre-trained ResNet50 for quality identification, but the model does not make targeted improvements for specific identification objects. Summary of the invention

[0007] The purpose of the present invention is to provide an image-based Angelica dahurica quality detection method to achieve automated and intelligent quality detection of Chinese medicinal materials and meet the needs of rapid, convenient and accurate Angelica dahurica quality identification.

[0008] To achieve the above object, the technical solution adopted by the present invention is:

[0009] An image-based method for detecting the quality of Angelica dahurica comprises the following steps:

[0010] Step 1. Slice the Angelica dahurica and use a camera to capture the slice to obtain an original Angelica dahurica image;

[0011] Step 2. Perform data enhancement on the original Angelica dahurica image to form a sample data set;

[0012] Step 3. Label each Angelica dahurica image in the sample data set to form a training set and a test set;

[0013] Step 4. Construct the Angelica dahurica quality detection model, in which the backbone network uses resnet50 to extract the initial features of the Angelica dahurica image and output the multi-scale initial feature map P i , i = 1, 2, 3; the auxiliary network uses EllipseNet, the feature map P i They are input into EllipseNet respectively, and EllipseNet outputs the corresponding region segmentation map, and maps the pixels to the original feature dimension. The region segmentation map includes: Xylem Boundary Cambium region The feature map P i and Weighted feature map F i After upsampling, feature maps F2 and F3 maintain the same dimension as feature map F1. After weighting the three, feature map F is obtained. Feature map F is input into the fully connected layer after dropout operation and BN operation in turn, and the fully connected layer outputs the classification prediction result;

[0014] Step 5. Construct a loss function and use the training set to complete the training of the Angelica dahurica quality detection model;

[0015] Step 6. Input the Angelica dahurica image to be tested into the trained Angelica dahurica quality detection model, and the model outputs the Angelica dahurica quality grade prediction result.

[0016] Furthermore, in step 2, the data enhancement includes random rotation, mirroring and / or brightness adjustment operations, wherein the rotation is expressed as:

[0017]

[0018] Among them, (x, y) represents the coordinates of the original pixel, (x′, y′) represents the coordinates of the enhanced pixel, θ r Indicates the rotation angle;

[0019] Horizontal mirroring and vertical mirroring are expressed as:

[0020]

[0021] The original Angelica dahurica image and the enhanced Angelica dahurica image were cropped by image processing software. The cropped image contained a complete Angelica dahurica slice. The cropped original Angelica dahurica image and the enhanced Angelica dahurica image together constituted a sample data set.

[0022] Furthermore, in step 3, labelm is used e Each Angelica dahurica image in the sample data set is annotated with slice information and quality grade information. The slice information includes the boundary contour, cambium contour and xylem contour of the Angelica dahurica slice; the quality grade information is divided into special grade, first grade, second grade, third grade or fourth grade.

[0023] Furthermore, in step 4, EllipseNet uses an improved FCOS network, which outputs the position prediction information (o, t, b, l, r) of the Angelica dahurica slice, where o represents the center point, t, b, l, r represent the distance from the center point to the upper, lower, left, and right boundaries, respectively. FCOS predicts a rectangular regression box through the position prediction information, and EllipseNet improves FCOS according to the unique shape of Angelica dahurica. Specifically, EllipseNet defines a class ellipse regression box based on the position prediction information (o, t, b, l, r), and the ellipse regression box is expressed as:

[0024] E(x,y)=Concat(Ellipse(t,l),Ellipse(l,b),Ellipse(b,r),Ellipse(r,t))

[0025] Wherein, E(x, y) represents the predicted ellipse regression box of the pixel point with coordinates (x, y), Concat represents the splicing operation, and Ellipse(A, B) represents a quarter ellipse formed with A and B as the semi-major axis and semi-minor axis;

[0026] According to the ellipse-like regression frame, the regional segmentation maps are obtained, including: the boundary E of Angelica dahurica t 、Cambium boundary E c 、Xylem boundary E w and cambium region E c-w , E c-w =E c -E w .

[0027] Furthermore, in step 4, the region segmentation map is mapped to the original feature map by pixel mapping; the pixel mapping is specifically expressed as:

[0028]

[0029] Among them, s represents the scaling factor from the original feature map to the scaled feature map, (x, y) represents the pixel point of the scaled feature map, Represents the pixel point of the original feature map, Indicates floor operation.

[0030] Furthermore, in step 5, the loss function L is expressed as:

[0031] L=αL gui +βL ratio +L cls

[0032] Among them, α and β are learnable hyperparameters; L gui represents the EllipseNet loss, L ratiorepresents auxiliary loss, L cls represents the classification loss;

[0033] EllipseNet loss is expressed as:

[0034]

[0035] in, With E j They represent the marked contour and the predicted contour respectively, and N represents the total number of pixels;

[0036] The auxiliary loss is expressed as:

[0037]

[0038] Among them, τ is the temperature factor, δ is the confidence factor, S t With S w Represent the area of ​​the complete Angelica dahurica slice and the xylem part, respectively;

[0039] The classification loss is expressed as:

[0040] L cls =-∑y k log(p k )

[0041] Among them, p k represents the predicted probability that the Angelica dahurica slice belongs to the kth quality level, y k Represents the true label of the kth category.

[0042] Based on the above technical solution, the beneficial effects of the present invention are:

[0043] The invention provides an image-based Angelica dahurica quality detection method. Aiming at the shape characteristics of Angelica dahurica slices, an improved FCOS network EllipseNet is proposed. Since a rectangular regression frame may generate a large blank area or miss detail features at the edge of a slice, the invention adopts an ellipse-like regression frame to replace the original rectangular regression frame of the FCOS network. Compared with a rectangle, an ellipse-like shape is more consistent with the original natural shape of the Angelica dahurica slices, and the ellipse-like regression frame can more accurately capture the edge features of the slices, especially when the slice shape is irregular or close to a circle, so that the network can more accurately predict the regions of various parts of the Angelica dahurica and can better extract the features of various regions of the Angelica dahurica slices. Meanwhile, aiming at the influence factor of the proportion of wood in the Angelica dahurica quality identification system on the quality of the Angelica dahurica, the invention designs a corresponding auxiliary loss for enhancing the network, so that the quality detection of the Angelica dahurica slices is performed more accurately. Compared with the existing medicinal material quality identification technology, the invention constructs a quality identification model for the Angelica dahurica, models the specific features of the Angelica dahurica, has excellent effect on the quality identification of the Angelica dahurica, and solves the problem that the features of the Angelica dahurica slices cannot be well captured in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The present invention provides a flow chart of the method for detecting the quality of Angelica dahurica based on images.

[0045] Figure 2 It is a schematic diagram of the section of the Angelica dahurica slices in the present invention.

[0046] Figure 3 It is a structural schematic diagram of the Angelica dahurica quality detection model in the present invention.

[0047] Figure 4 It is a schematic diagram of an ellipse-like regression frame output by the EllipseNet network of the Angelica dahurica quality detection model in the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0049] This embodiment provides an image-based quality detection method for Angelica dahurica, the process of which is as follows: Figure 1 As shown, the specific steps include:

[0050] Step 1. Angelica dahurica image acquisition;

[0051] Slice the Angelica dahurica and use a photographing device (such as a mobile phone or a camera) to take an image of the cut surface to obtain an original Angelica dahurica image;

[0052] Step 2. Perform data enhancement on the original Angelica dahurica image to form a sample data set;

[0053] In order to further optimize the training data of Angelica dahurica images, image data augmentation technology was used to improve the performance and generalization ability of the deep learning model; data augmentation includes random rotation, mirroring and / or brightness adjustment operations to generate more diverse and rich training samples; among them, random rotation and / or mirroring operations can simulate Angelica dahurica images under different shooting angles, increasing the model's adaptability to different viewing angles, while brightness adjustment helps the model better process Angelica dahurica images under different lighting conditions;

[0054] Specifically, the rotation operation is expressed as:

[0055]

[0056] Among them, (x, y) represents the coordinates of the original pixel, (x′, y′) represents the coordinates of the enhanced pixel, θ r Indicates the rotation angle;

[0057] The horizontal mirroring operation and the vertical mirroring operation are expressed as:

[0058]

[0059] Data enhancement was performed on the original Angelica dahurica image. The original Angelica dahurica image and the enhanced Angelica dahurica image were cropped using image processing software (Photoshop). The cropped image contained a complete Angelica dahurica slice with a resolution of 512×512. The cropped original Angelica dahurica image and the enhanced Angelica dahurica image together constituted a sample data set.

[0060] Step 3. Label each Angelica dahurica image in the sample data set to form a training set and a test set;

[0061] Using labelm e Each Angelica dahurica image in the sample data set is annotated with slice information and quality grade information. The slice information includes the boundary contour, cambium contour and xylem contour of the Angelica dahurica slice, such as Figure 2 As shown; the quality grade information is divided into special grade, first grade, second grade, third grade or fourth grade;

[0062] Step 4. Construct a quality detection model for Angelica dahurica;

[0063] The quality detection model of Angelica dahurica is as follows: Figure 3 As shown in the figure, the backbone network uses resnet50 to extract the initial features of the Angelica dahurica image and output three initial feature maps P1, P2, and P3 with decreasing scales in sequence; resnet50 is a deep convolutional neural network with multiple convolutional layers and pooling layers, which is suitable for capturing details and complex structures in images. Using resnet50 to extract features from the Angelica dahurica image can deeply explore the visual information of the Angelica dahurica slices and describe the characteristics of the Angelica dahurica in a more comprehensive way; at the same time, resnet50 helps to capture subtle changes in plant tissues (such as the color information of the Angelica dahurica cross section), and provides a powerful feature representation for accurately describing the cross-sectional information of the Angelica dahurica; deep neural networks also help to capture the differences between different regions of the cross section, so as to better evaluate the quality grade of the Angelica dahurica; and the cambium color is an important part of the Angelica dahurica plant characteristics. Since the cambium part accounts for a small proportion of the image, and as the number of network layers increases, the expansion of the receptive field is not conducive to extracting small area features, and the residual connection of resnet50 can just solve this problem;

[0064] The auxiliary network uses EllipseNet, which improves the FCOS network and is used to segment the Angelica dahurica image. Figure 2As shown in the figure, the Angelica dahurica slice mainly includes the boundary (cortex), cambium, and xylem. In order to capture the different feature information of each part, the FCOS network is used to segment the Angelica dahurica image, and the rectangular regression frame output by the FCOS network is improved to an ellipse-like regression frame, which is renamed as EllipseNet in the present invention. It should be noted that the network structure of EllipseNet in the present invention is the same as that of the FCOS network, but the output regression frame structure is different; the FCOS network outputs the position prediction information (o, t, b, l, r) of the Angelica dahurica slice, where o represents the center point, t, b, l, and r represent the distance from the center point to the upper, lower, left, and right boundaries, respectively, thereby outputting a rectangular regression frame, as shown in FIG. Figure 4 As shown in the left figure; in the present invention, considering that the slices and regions of Angelica dahurica are mostly elliptical, an elliptical regression frame is defined based on the position prediction information (o, t, b, l, r), as shown in FIG. Figure 4 As shown in the middle right figure, it is specifically expressed as:

[0065] E(x,y)=Concat(Ellipse(t,l),Ellipse(l,b),Ellipse(b,r),Ellipse(r,t))

[0066] Wherein, E(x, y) represents the predicted ellipse regression box of the pixel point with coordinates (x, y), Concat represents the splicing operation, and Ellipse(A, B) represents a quarter ellipse formed with A and B as the semi-major axis and semi-minor axis;

[0067] Since the FCOS network first further extracts features from the input original feature map to obtain a scaled feature map, and the ellipse-like regression box is annotated on the scaled feature map, it is necessary to map the ellipse-like regression box to the original feature map through pixel mapping. The pixel mapping is specifically expressed as:

[0068]

[0069] Among them, s represents the scaling factor from the original feature map to the scaled feature map, (x, y) represents the pixel point of the scaled feature map, Represents the pixel point of the original feature map, Indicates floor operation;

[0070] After region segmentation and pixel mapping, we finally get the boundary E of Angelica dahurica. t 、Cambium boundary E c 、Xylem boundary E w , then the cambium area is represented by E c-w =E c -E w ;

[0071] On this basis, the feature map Pi , i = 1, 2, 3 are input to EllipseNet respectively, and the corresponding region segmentation map is output by EllipseNet: Angelica dahurica boundary Xylem boundary Cambium region The feature map P i and Weighted pixel-by-pixel connection to obtain the feature map F at the current scale i , specifically expressed as:

[0072]

[0073] Among them, λ, γ, and η are learnable hyperparameters;

[0074] After upsampling, feature maps F2 and F3 maintain the same dimension as feature map F1. After weighting the three, feature map F is obtained. Feature map F is input into the fully connected layer after dropout operation and BN operation in turn, and the fully connected layer outputs the classification prediction result;

[0075] Step 5. Construct a loss function and use the training set to complete the training of the Angelica dahurica quality detection model;

[0076] The loss function is expressed as:

[0077] L=αL gui +βL ratio +L cls

[0078] Among them, α and β are learnable hyperparameters; L gui represents the EllipseNet loss, L ratio represents auxiliary loss, L cls represents the classification loss;

[0079] For the image area segmented by EllipseNet, the IoU loss function is used to evaluate the segmentation accuracy, which is specifically expressed as:

[0080]

[0081] in, With E j denote the marked contour and predicted contour respectively, and N denotes the total number of pixels. For the convenience of representation, the image segmentation losses of the entire Angelica dahurica, xylem, and cambium are summed up to a total of L gui ;

[0082] Regarding auxiliary loss, when testing the quality of Angelica dahurica, in addition to considering the color and texture characteristics of each part, the proportion of xylem is also an important consideration for identification; the present invention proposes an auxiliary loss based on the proportion of xylem to assist in distinguishing the quality grade of Angelica dahurica, and the auxiliary loss is expressed as:

[0083]

[0084] Among them, τ is the temperature factor, δ is the confidence factor, S t With S w Respectively represent the area of ​​the complete Angelica dahurica slice and the xylem part; the confidence level used in this embodiment is 1 / 3, which can be adjusted according to different Angelica dahurica species;

[0085] For classification loss, it is specifically expressed as:

[0086] L cls =-∑y k log(p k )

[0087] Among them, p k represents the predicted probability that the Angelica dahurica slice belongs to the kth quality level, y k Represents the true label of the kth category;

[0088] Step 6. Input the Angelica dahurica image to be tested into the trained Angelica dahurica quality detection model, and the model outputs the Angelica dahurica quality grade prediction result.

[0089] The above description is only a specific implementation mode of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other alternative features that are equivalent or have similar purposes; all the disclosed features, or all the steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. An image-based method for detecting the quality of Angelica dahurica, characterized in that: The following steps are involved: Step 1. Slice the Angelica dahurica and use a camera to capture the slice to obtain an original Angelica dahurica image; Step 2. Perform data enhancement on the original Angelica dahurica image to form a sample data set; Step 3. Label each Angelica dahurica image in the sample data set to form a training set; Step 4. Construct the Angelica dahurica quality detection model, in which the backbone network uses resnet50 to extract the initial features of the Angelica dahurica image and output the multi-scale initial feature map P i ,i=1,2,3; the auxiliary network uses EllipseNet, the feature map P i They are input into EllipseNet respectively, and EllipseNet outputs the corresponding region segmentation map, and maps the pixels to the original feature dimension. The region segmentation map includes: Xylem boundary Cambium region The feature map P i and Weighted feature map F i After upsampling, feature maps F2 and F3 maintain the same dimension as feature map F1. After weighting the three, feature map F is obtained. Feature map F is input into the fully connected layer after dropout operation and BN operation in turn, and the fully connected layer outputs the classification prediction result; Step 5. Construct a loss function and use the training set to complete the training of the Angelica dahurica quality detection model; Step 6. Input the Angelica dahurica image to be tested into the trained Angelica dahurica quality detection model, and the model outputs the Angelica dahurica quality grade prediction result.

2. The image-based quality detection method for Angelica dahurica according to claim 1, characterized in that: In step 2, the data enhancement includes random rotation, mirroring and / or brightness adjustment operations, where rotation is expressed as: Among them, (x, y) represents the coordinates of the original pixel, (x′, y′) represents the coordinates of the enhanced pixel, θ r Indicates the rotation angle; Horizontal mirroring and vertical mirroring are expressed as: The original Angelica dahurica image and the enhanced Angelica dahurica image were cropped by image processing software. The cropped image contained a complete Angelica dahurica slice. The cropped original Angelica dahurica image and the enhanced Angelica dahurica image together constituted a sample data set.

3. The image-based quality detection method for Angelica dahurica according to claim 1, characterized in that: In step 3, labelme is used to annotate each Angelica dahurica image in the sample dataset. The annotation content includes slice information and quality grade information. The slice information includes the boundary outline, cambium outline and xylem outline of the Angelica dahurica slice; the quality grade information is divided into special grade, first grade, second grade, third grade or fourth grade.

4. The image-based quality detection method for Angelica dahurica according to claim 1, characterized in that: In step 4, EllipseNet uses an improved FCOS network, which outputs the position prediction information (o, t, b, l, r) of the Angelica dahurica slice, where o represents the center point, t, b, l, r represent the distance from the center point to the upper, lower, left, and right boundaries, respectively. EllipseNet defines a class ellipse regression box based on the position prediction information (o, t, b, l, r), and the class ellipse regression box is expressed as: E(x,y)=Concat(Ellipse(t,l),Ellipse(l,b),Ellipse(b,r),Ellipse(r,t)) Among them, E(x,y) represents the predicted ellipse regression box of the pixel point with coordinates (x,y), Concat represents the splicing operation, and Ellipse(A,B) represents a quarter ellipse formed with A and B as the semi-major axis and semi-minor axis; According to the ellipse-like regression frame, the regional segmentation maps are obtained, including: the boundary E t 、Cambium boundary E c 、Xylem boundary E w and cambium region E c-w , E c-w =E c -E w .

5. The image-based quality detection method for Angelica dahurica according to claim 4, characterized in that: In step 4, the region segmentation map is mapped to the original feature map by pixel mapping; the pixel mapping is specifically expressed as: Among them, s represents the scaling factor from the original feature map to the scaled feature map, (x, y) represents the pixel point of the scaled feature map, Represents the pixel point of the original feature map, Indicates floor operation.

6. The image-based quality detection method for Angelica dahurica according to claim 1, characterized in that: In step 5, the loss function L is expressed as: L=αL gui +βL ratio +L cls Among them, α and β are learnable hyperparameters; L gui represents the EllipseNet loss, L ratio represents auxiliary loss, L clls represents the classification loss; EllipseNet loss is expressed as: in, With E j They represent the marked contour and the predicted contour respectively, and N represents the total number of pixels; The auxiliary loss is expressed as: Among them, τ is the temperature factor, δ is the confidence factor, S t With S w Represent the area of ​​the complete Angelica dahurica slice and the xylem part, respectively; The classification loss is expressed as: L cls =-∑y k log(pk) Among them, p k represents the predicted probability that the Angelica dahurica slice belongs to the kth quality level, y k Represents the true label of the kth category.

Images