Polymorphic Chinese herbal medicine plant identification method

Abstract

The invention belongs to the technical field of crossing of artificial intelligence and biomedical engineering, and discloses a polymorphic Chinese herbal medicine plant identification method. The method comprises the following steps: constructing a Chinese herbal medicine plant image data set, and performing data enhancement and preprocessing on the data set; the method comprises the following steps: constructing a WCMSA-Net model based on a ResNet50 architecture, introducing a wavelet convolution layer in an initial stage of a network, and introducing a WMSPA module in a residual block; training a WCMSA-Net model by using a Chinese herbal medicine plant image data set, and optimizing model parameters by adopting a label smooth focus loss function; and inputting a to-be-identified Chinese herbal medicine plant image into the trained WCMSA-Net model, and outputting an identification result. According to the method, the problem that a traditional network model is low in identification precision for polymorphic Chinese herbal medicine plants under a complex background is solved, and the model network identification accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application mainly relates to the technical field of the cross of artificial intelligence and biomedical engineering, and particularly relates to a multi-morphology Chinese herbal medicine plant identification method. BACKGROUND

[0002] Most of the Chinese herbal medicine plant identification methods rely on artificial experience or are realized based on morphological characteristics and professional biological identification technology. Chinese herbal medicine plants are not only the foundation of the earth's ecological system, but also one of the key resources of human society. As a treasure of China's traditional culture, Chinese herbal medicine has a history of thousands of years and plays a key role in the medical and health care fields in China and even the world. Chinese herbal medicine plant identification technology has gradually become a research hotspot in many fields such as medicine and botany. In recent years, the Chinese herbal medicine industry has developed rapidly and gradually entered a new stage of development. The demand for intelligent and automated technology has become increasingly large. However, the accurate identification rate of Chinese herbal medicine plants still faces many technical difficulties. Under the condition that there are many types of Chinese herbal medicine plants, the morphology is variable, and the environment is greatly influenced, it has become a key technical problem in the current research field to improve the precision and efficiency of Chinese herbal medicine image identification.

[0003] In recent years, with the rapid development of artificial intelligence and computer vision technology, Chinese herbal medicine plant identification technology based on computer vision has gradually become a key alternative solution. At present, some studies have adopted wavelet transform, convolutional neural network, attention mechanism and other technologies to improve the image classification accuracy. However, the method based on the conventional CNN architecture still has theoretical defects in cross-scale feature fusion and frequency domain information utilization. The fixed receptive field limits the joint modeling ability of multi-morphology features, and there is a lack of a systematic method to fuse the multi-scale features of Chinese herbal medicine plant images and effectively enhance the adaptability of the model to complex backgrounds. SUMMARY

[0004] The purpose of the present application is to provide a multi-morphology Chinese herbal medicine plant identification method, which aims to solve the problem of low identification accuracy of traditional network models for multi-morphology Chinese herbal medicine plants in complex backgrounds and improve the network identification accuracy of the model.

[0005] To achieve the above purpose, the present application provides a multi-morphology Chinese herbal medicine plant identification method, which comprises the following steps: Step 1: Collect multi-morphology images of Chinese plant image library and field shooting, construct Chinese herbal medicine plant dataset CHM-Morph-50, and perform data enhancement and preprocessing on the dataset; Step 2: Construct the Chinese herbal medicine plant recognition network WCMSA-Net model, the WCMSA-Net model is based on the ResNet50 architecture, introduces a wavelet convolution layer in the initial stage of the network, and introduces a wavelet multi-scale space pyramid attention WMSPA module in the residual block, the WMSPA module includes a hierarchical wavelet convolution WT-HPC module and a spatial pyramid recalibration SPR module; Step 3: Train the WCMSA-Net model using the Chinese herbal medicine plant image dataset CHM-Morph-50, and use the label smoothing focal loss function to optimize the model parameters; Step 4: Input the Chinese herbal medicine plant image to be recognized into the trained WCMSA-Net model, and after feature extraction by the wavelet convolution layer, the WMSPA module and the residual block, output the class probability distribution through the global average pooling and the fully connected layer, and take the class corresponding to the maximum probability as the final recognition result.

[0006] Further, the Chinese herbal medicine plant dataset CHM-Morph-50 in step 1 contains multi-morphology images of 50 types of Chinese herbal medicine plants, which include multi-morphology part images such as leaves, flowers, stems, fruits and roots, different growth stage images such as seedling stage and mature stage, and complex background images such as natural light, occlusion and shadow.

[0007] Further, the data augmentation method in step 1 includes: K1: Randomly crop and resize, after cropping, scale the multi-morphology image to 224x224 pixels; K2: Random horizontal flip; K3: Color enhancement, including adjusting brightness, contrast and saturation.

[0008] Further, the data preprocessing in step 1 includes the following steps: S1: Manually remove low-quality multi-morphology images including blur and overexposure; S2: Crop and adjust the multi-morphology images after manual removal to 224x224 pixels; S3: Convert the image data to a tensor using the pytorch self-contained function transforms.ToTensor(); S4: Normalize using mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225].

[0009] Further, the processing process of the wavelet convolution layer in step 2 includes the following steps: S5: First, use the wavelet convolution layer to perform two-dimensional Haar wavelet decomposition on the input multi-modal image, generate 1 low-frequency component and 3 high-frequency components, the low-frequency component is the reserved multi-modal image main structure LL, and the 3 high-frequency components are LH, HL and HH corresponding to the vertical, horizontal and diagonal edge details respectively; S6: Apply small kernel convolution to extract features from the low-frequency component and the high-frequency component in step S5 respectively; S7: Recombine the low-frequency and high-frequency features by inverse wavelet transform to generate a feature map that fuses multi-band features.

[0010] Further, the processing procedure of the hierarchical wavelet convolution WT-HPC module in step 2 includes the following steps: S8: First, perform channel segmentation operation, and the shape of the input feature map is , where is the input feature, is the feature mapping, C is the number of channels, H is the spatial height, and W is the spatial width; Through the segmentation operation, it is segmented into s sub-feature maps, and the shape of each sub-feature map is , where i ∈ {1, 2, …, s}, ω = C / s; S9: Perform hierarchical residual connection wavelet convolution operation: Perform wavelet convolution operation on each sub-feature map , and the output after convolution is processed by batch normalization. For each subsequent sub-feature map , where i > 1, the convolution result is element-wise added to the convolution result of the previous layer , and then the convolution operation is continued. This process can be represented by the expression , where represents the wavelet convolution operation, is the bias term; S10: Finally, the WT-HPC module combines all the enhanced feature maps into the final output feature map through the Concat operation. This process can be represented by the expression , and the shape of the final output feature map is .

[0011] Further, the processing procedure of the spatial pyramid recalibration SPR module in step 2 includes the following steps: S11: First, perform adaptive average pooling on the input feature map through the spatial pyramid aggregation block SPA to generate feature descriptors of different scales ; S12: The feature descriptor is further processed by a channel interaction block CI to generate channel attention weights Vi; S13: Finally, the channel attention weights are multiplied with the input feature map at the element level to realize the re-calibration of the feature map.

[0012] Further, the training configuration of the WCMSA-Net model in step 3 uses the Adam optimizer, the learning rate is initialized to 0.001, the learning rate is reduced at equal intervals, the batch size is set to 32, and the training rounds range from 50 to 100 rounds, with the validation set loss and accuracy monitored for early stopping.

[0013] Further, the label smoothing focal loss function is used to optimize the model parameters in step 3, including the following steps: S14: First, the input label is smoothed, and the formula is where ε is the smoothing factor, C is the number of classes, is the probability distribution of the label, and the true label is adjusted from 1 to 1-ε, and the values of the remaining labels except the true label are changed from 0 to ε / (C-1), to alleviate model overfitting; S15: Then, the focal loss formula after label smoothing is calculated as where is the probability of matching the model's predicted value with the true value, is the predicted value after label smoothing, γ is the adjustment factor, and α is the hyperparameter for balancing classes, by reducing the weight of easily classified samples, forcing the model to focus on difficult-to-classify samples.

[0014] The present application provides a multi-morphology Chinese herbal medicine plant identification method, compared with the prior art, has the following beneficial effects: (1) The multi-site (leaves / flowers / stems / fruits / roots), multi-growth stage (seedlings / mature / withered), multi-background (natural complex scene) images of the collected Chinese herbal medicine plant data set, construct the CHM-Morph-50 data set, ensure the diversity of data, improve the adaptability of the model to different shooting conditions, and reduce the risk of overfitting. The data set fills the gap of the existing data set, and provides more professional and targeted data support for the research of Chinese herbal medicine plant identification. (2) The WCMSA-Net network designed by the present application enhances the detail preservation ability through the frequency domain feature characteristics of wavelet convolution and high frequency / low frequency component collaborative analysis; adopts a multi-scale spatial pyramid attention mechanism to dynamically capture the key features of Chinese herbal medicine; (3) The label smoothing alleviates the overfitting problem caused by hard label, so that the model output is more robust; the focal loss mechanism dynamically adjusts the sample weight through the adjustment factor, focuses on difficult-to-classify samples (such as morphologically similar Chinese herbal medicine), and improves the classification accuracy. (4) While maintaining high accuracy, the automatic identification process and efficient algorithm design change the efficiency bottleneck of the traditional identification method, and provide a breakthrough solution for the rapid detection of Chinese herbal medicine plant industry. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 It is the overall flowchart of the multi-morphology Chinese herbal medicine plant identification method related to the embodiments of the present application; Figure 2 It is the overall structure diagram of the WCMSA-Net network model of the multi-morphology Chinese herbal medicine plant identification method related to the embodiments of the present application; Figure 3 It is the wavelet multi-scale spatial pyramid attention WMSPA module diagram of the multi-morphology Chinese herbal medicine plant identification method related to the embodiments of the present application; Figure 4 It is the hierarchical wavelet convolution WT-HPC module diagram of the multi-morphology Chinese herbal medicine plant identification method related to the embodiments of the present application; Figure 5 It is the experimental comparison result diagram of the multi-morphology Chinese herbal medicine plant identification method related to the embodiments of the present application. DETAILED DESCRIPTION

[0017] The preferred mechanism and implemented method of the present application will be further described below in combination with the drawings and specific embodiments.

[0018] Figures 1 to 5 As shown in the drawings, the embodiments of the present application disclose a technical scheme of a multi-morphology Chinese herbal medicine plant identification method. Figure 1 It is the overall flowchart of the multi-morphology Chinese herbal medicine plant identification method related to the embodiments of the present application; Figure 2is the overall structure diagram of the WCMSA-Net network model of the multi-morphology Chinese herbal medicine plant identification method involved in the embodiment of the present application. Figure 3 is a wavelet multi-scale space pyramid attention WMSPA module diagram of the multi-morphology Chinese herbal medicine plant identification method involved in the embodiment of the present application. Figure 4 is a hierarchical wavelet convolution WT-HPC module diagram of the multi-morphology Chinese herbal medicine plant identification method involved in the embodiment of the present application. Figure 5 is an experimental comparison result diagram of the multi-morphology Chinese herbal medicine plant identification method involved in the embodiment of the present application. Embodiment

[0019] As shown in Figure 1 The present application provides a multi-morphology Chinese herbal medicine plant identification method, which realizes accurate identification of multi-part and multi-growth stage images of Chinese herbal medicine plants by constructing a specific Chinese herbal medicine plant dataset and constructing and training a WCMSA-Net neural network model.

[0020] A multi-morphology Chinese herbal medicine plant identification method, the detailed implementation steps are as follows: Step 1: Construct the CHM-Morph-50 Chinese herbal medicine plant image dataset; The present application collects multi-source images from the Chinese Plant Image Library (PPBC) and field shooting, constructs the CHM-Morph-50 dataset containing 50 kinds of Chinese herbal medicine plants, covers multi-morphology part images such as leaves, flowers, stems, fruits and roots, covers different growth stages (seedling stage, mature stage, etc.) and complex backgrounds (natural light, shielding, shadow, etc.), and performs data enhancement and preprocessing on the dataset to improve the generalization ability of the model.

[0021] Data collection obtains standardized images reviewed by experts from the PPBC platform, and supplements shooting in different natural environments (such as slopes, lake sides and roadsides) through self-built devices, so as to ensure that each type of plant contains 400-600 images, and the total sample size reaches 26510.

[0022] The data enhancement method adopts random cropping and size adjustment (scaling to 224x224 pixels after cropping), random horizontal flipping (probability 50%), color enhancement (adjusting brightness, contrast and saturation), etc. to expand sample diversity.

[0023] Data preprocessing methods included manually removing low-quality images such as blurry or overexposed images; uniformly resizing images to 224×224 pixels; using the PyTorch built-in function transforms.ToTensor() to convert image data into tensors; and normalizing using ImageNet mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225].

[0024] Step 2: Construct the WCMSA-Net network model as follows Figure 2 As shown, based on the ResNet50 architecture, wavelet convolutional layers are introduced in the initial stage of the network for multi-band feature extraction; a wavelet multi-scale spatial pyramid attention (WMSPA) module is embedded in the residual block as follows: Figure 3 As shown, this module includes a hierarchical wavelet convolution (WT-HPC) module, as follows: Figure 4 The Spatial Pyramid Recalibration (SPR) module, as shown, enables multi-scale feature fusion and key region focusing; (1) The processing steps of wavelet convolutional layers include the following steps: First, a two-dimensional Haar wavelet decomposition is performed on the input image using a wavelet convolutional layer, generating one low-frequency component (LL, which preserves the main structure of the image) and three high-frequency components (LH, HL, and HH, which correspond to vertical, horizontal, and diagonal edge details, respectively). Next, small kernel convolution (such as 3×3 convolution) is applied to each frequency component to extract features. Then, the low-frequency and high-frequency features are recombined through inverse wavelet transform to generate a feature map that integrates multi-frequency band features. This process enhances the model's ability to capture texture and structural details.

[0025] The processing steps of the hierarchical wavelet convolution WT-HPC module include the following: First, channel segmentation is performed, and the shape of the input feature map is... ,in These are input features. It is a feature map, where C is the number of channels, H is the spatial height, and W is the spatial width; It is divided into s sub-feature maps through a segmentation operation, and the shape of each sub-feature map is as follows. Where i ∈ {1,2,…,s} and ω=C / s; Next, a wavelet convolution operation with hierarchical residual connections is performed on each sub-feature map. Wavelet convolution is performed, and the output after convolution is batch normalized. For each subsequent sub-feature map Fi (where i>1), its convolution result is... It will be combined with the convolution result of the previous layer The element-wise addition is performed, and then the convolution operation is continued. wherein represents the wavelet convolution operation, is a bias term, and the design enhances feature expression through cross-layer information flow; Finally, the WT-HPC module combines all the enhanced feature maps into the final output feature map through the Concat operation , and the operation process can be represented as , and the shape of the final output feature map is .

[0026] The processing process of the spatial pyramid re-calibration (SPR) module includes the following steps: First, the spatial pyramid aggregation block (SPA) performs adaptive average pooling on the input feature map to generate feature descriptors of different scales . Then, the channel interaction block (CI) processes the feature descriptors to generate channel attention weights Vi. Finally, the channel attention weights are multiplied with the input feature map at the element level to realize the re-calibration of the feature map.

[0027] Step 3: input the Chinese herbal medicine plant image processed by step 1 into the WCMSA-Net network model of step 2 for training iteration to obtain the trained model. In the embodiment of the present application, the training configuration of the WCMSA-Net network model uses the Adam optimizer, the learning rate is initialized to 0.001, the learning rate is decreased at equal intervals, the batch size is set to 32 (which can be adjusted according to hardware resources), the training rounds (Epoch) are 50-100 rounds, and the validation set loss and accuracy are monitored for early stopping.

[0028] The loss function used in the embodiment of the present application is a label smoothing focal loss function (Label Smoothing Focal Loss) designed for Chinese herbal medicine plant data sets. First, the input label is smoothed, and the formula is wherein ε is the smoothing factor (which can be 0.1), the number of classes is C, and P is the probability distribution of the label. The true label is adjusted from 1 to 1-ε, and the values of the remaining labels except the true label are changed from 0 to ε / (C-1), which relieves the model from overfitting.

[0029] The focal loss formula after label smoothing is wherein is the probability of matching the model prediction value with the true value, is the predicted value of the label smoothing, is the adjustment factor (2 can be taken), and is the hyperparameter for balancing classes (0.25 can be taken), By reducing the weight of easily classified samples, the model is forced to focus on difficult-to-classify samples.

[0030] Step 4: input the Chinese herbal medicine plant image to be identified into the trained WCMSA-Net model, after feature extraction by the wavelet convolution layer, the WMSPA module and the residual block, output the class probability distribution through the global average pooling and the full connection layer, and take the class corresponding to the maximum probability as the final identification result.

[0031] The effectiveness of the present application is verified by comparative experiments, and the four groups of models of ResNet50, ResNet50 using wavelet convolution alone, ResNet50 using attention mechanism alone, and WCMSA-Net model are trained by using the label smoothing focal loss function (LSFL) proposed in the present application and the classical cross-entropy loss function (CE), respectively, and the differences in model performance are compared, and the results are shown in Figure 5 Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 As shown in the results, from the accuracy, the WCMSA-Net model is better than the other models, and the label smoothing focal loss function brings an improvement in accuracy in all experimental groups, with an improvement of 0.1%, 0.4%, 0.8%, and 0.5%, respectively. On the loss rate level, compared with the original cross-entropy loss function, the loss function proposed in the embodiment of the present application has a lower loss rate.

[0032] The present application provides a multi-form Chinese herbal medicine plant recognition method, which solves the problem of low recognition accuracy of traditional network models for multi-form Chinese herbal medicine plants in complex backgrounds, and improves the model network recognition accuracy.

[0033] Finally, it should be noted that: the above is only the preferred embodiment of the present application, and is not used to limit the present application, although the present application is described in detail with reference to the embodiments, for those skilled in the art, the technical solutions recorded in the above embodiments can be modified, or some technical features can be replaced, but any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for identifying multimorphic Chinese herbal medicine plants, characterized in that, Includes the following steps: Step 1: Collect multimorphic images from the China Plant Image Database and field photography to construct the Chinese herbal medicine plant dataset CHM-Morph-50, and perform data augmentation and preprocessing on the dataset; Step 2: Construct the WCMSA-Net model for identifying Chinese herbal plants. The WCMSA-Net model is based on the ResNet50 architecture. Wavelet convolutional layers are introduced in the initial stage of the network, and wavelet multi-scale spatial pyramid attention (WMSPA) modules are introduced in the residual blocks. The WMSPA modules include hierarchical wavelet convolution (WT-HPC) modules and spatial pyramid recalibration (SPR) modules. Step 3: Train the WCMSA-Net model using the CHM-Morph-50 dataset of Chinese herbal plant images, and optimize the model parameters using the label smoothing focus loss function; Step 4: Input the image of the Chinese herbal medicine plant to be identified into the trained WCMSA-Net model. After feature extraction through wavelet convolutional layer, WMSPA module and residual block, the model outputs the class probability distribution through global average pooling and fully connected layer. The class corresponding to the maximum probability is taken as the final recognition result.

2. The method for identifying multimorphic Chinese herbal medicine plants according to claim 1, characterized in that, The CHM-Morph-50 dataset of Chinese medicinal plants in step 1 contains polymorphic images of 50 categories of Chinese medicinal plants. The polymorphic images cover images of various parts including leaves, flowers, stems, fruits, and roots, as well as images of different growth stages such as seedlings and mature plants, and images with complex backgrounds including natural light, occlusion, and shadows.

3. The method for identifying multimorphic Chinese herbal medicine plants according to claim 1, characterized in that, The data augmentation methods in step 1 include: K1: Random cropping and resizing, scaling multi-shape images to 224×224 pixels after cropping; K2: Random horizontal flip; K3: Color enhancement, including adjustments to brightness, contrast, and saturation.

4. The method for identifying multimorphic Chinese herbal medicine plants according to claim 1, characterized in that, The data preprocessing in step 1 includes the following steps: S1: Manually remove low-quality, multi-morphological images, including blurry and overexposed ones; S2: Crop and adjust the manually removed multi-morphic images to 224×224 pixels; S3: Use the built-in PyTorch function transforms.ToTensor() to convert image data into tensors; S4: Normalize using the mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225].

5. The method for identifying multimorphic Chinese herbal medicine plants according to claim 1, characterized in that, Step 2 Wavelet convolutional layer processing Includes the following steps: S5: First, a wavelet convolutional layer is used to perform two-dimensional Haar wavelet decomposition on the input polymorphic image, generating one low-frequency component and three high-frequency components. The low-frequency component is the main structure LL of the polymorphic image, and the three high-frequency components are LH, HL, and HH, which correspond to the vertical, horizontal, and diagonal edge details, respectively. S6: Apply small kernel convolution to extract features from the low-frequency and high-frequency components in step S5 respectively; S7: Low-frequency and high-frequency features are recombined through inverse wavelet transform to generate a feature map that integrates multi-band features.

6. The method for identifying multimorphic Chinese herbal medicine plants according to claim 5, characterized in that, The processing steps of the hierarchical wavelet convolution WT-HPC module in step 2 include the following steps: S8: First, perform channel segmentation to determine the shape of the input feature map. ,in These are input features. It is a feature map, where C is the number of channels, H is the spatial height, and W is the spatial width; It is divided into s sub-feature maps through a segmentation operation, and the shape of each sub-feature map is as follows. Where i ∈ {1,2,…,s} and ω=C / s; S9: Wavelet convolution operation for hierarchical residual concatenation: For each sub-feature map Wavelet convolution is performed, and the output after convolution is batch normalized for each subsequent sub-feature map. Where i > 1, its convolution result It will be combined with the convolution result of the previous layer Perform element-wise summation, then continue with convolution. This process can be expressed as an expression. It means that, among them This represents the wavelet convolution operation. It is a bias term; S10: Finally, the WT-HPC module will combine all the enhanced feature maps The feature maps are merged into the final output feature map through the Concat operation. This process can be expressed using an expression. This indicates that the final output feature map has the following shape: .

7. The method for identifying multimorphic Chinese herbal medicine plants according to claim 6, characterized in that, Step 2, the process of recalibrating the Space Pyramid SPR module, includes the following steps: S11: First, adaptive average pooling is performed on the input feature map using Spatial Pyramid Aggregation Blocks (SPA) to generate feature descriptors of different scales. ; S12: Then process the feature descriptor through the channel interaction block CI. Process the data to generate channel attention weights Vi; S13: Finally, the channel attention weights are multiplied element-wise with the input feature map to achieve feature map recalibration.

8. The method for identifying multimorphic Chinese herbal medicine plants according to claim 1, characterized in that, The WCMSA-Net model in step 3 is trained using the Adam optimizer, with the learning rate initialized to [value missing]. The learning rate is set to 0.001 with equal intervals, the batch size is set to 32, the training epochs range from 50 to 100, and the validation set loss and accuracy are monitored to stop early.

9. The method for identifying multimorphic Chinese herbal medicine plants according to claim 1 or 8, characterized in that, Step 3 uses the label smoothing focus loss function to optimize the model parameters, including the following steps: S14: First, smooth the input labels using the following formula: Where ε is the smoothing factor and C is the number of categories. This is the probability distribution of the labels. The true label is adjusted from 1 to 1-ε, and the values ​​of the other labels are changed from 0 to ε / (C-1) to alleviate model overfitting. S15: Next, calculate the focus loss formula after label smoothing. ,in, It is the probability that the model's predicted value matches the actual value. γ is the predicted value after label smoothing, α is the adjustment factor, and α is a hyperparameter used to balance the categories. By reducing the weight of easily classified samples, the model is forced to focus on difficult-to-classify samples.

Images