Chinese herbal medicine image recognition method based on triple receptive fields
By constructing a triple receptive field deep neural network model, the problem of insufficient capture of global structure and frequency information in the image recognition of Chinese herbal medicines is solved, the recognition accuracy and adaptability are improved, and the intelligent management of Chinese herbal medicines is supported.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- LINYI UNIVERSITY
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-12
AI Technical Summary
Existing image recognition technologies for Chinese herbal medicines are insufficient in capturing global image structure and frequency information, cross-regional dependencies, and data scarcity, resulting in limited ability to distinguish similar medicinal materials and limited improvement in model performance.
A deep neural network model based on triple receptive fields is adopted, which combines spatial and frequency domain data augmentation, dense residual blocks, covariance attention, and fast Fourier convolution to construct a multi-layer receptive field module to enhance feature learning capabilities.
It improves the accuracy and generalization performance of image recognition of Chinese herbal medicines, enhances the adaptability to changes in lighting and background, and supports the intelligent production and resource management of Chinese medicinal materials.
Smart Images
Figure CN122023908A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition technology, specifically to a method for recognizing images of traditional Chinese medicine based on a triple receptive field. Background Technology
[0002] With the continuous development of traditional Chinese medicine (TCM) and the increasing global health awareness, image recognition technology for Chinese herbal medicines is playing an increasingly important role in the modernization, quality control, and resource protection of TCM. Existing image recognition technologies mainly rely on deep learning models, especially convolutional neural networks (CNNs), which have achieved some success in recognition tasks. However, these technologies still face some challenges and limitations in the field of Chinese herbal medicine image recognition. First, traditional CNNs rely on fixed-size convolutional kernels, which, while effectively capturing local detailed features such as leaf veins and serrations, are insufficient in modeling the global structure of the image, such as the overall morphology of the plant and the distribution of petals, resulting in limited ability to distinguish similar medicinal materials. Second, the frequency domain of natural images contains rich information such as texture and periodic patterns, but most models only learn features in the spatial domain, ignoring the joint representation potential of the frequency and spatial domains, thus failing to fully utilize the frequency information in the image. Furthermore, the computational complexity of traditional attention mechanisms increases quadratically with sequence length, making it difficult to efficiently capture long-distance dependencies across regions in Chinese herbal medicine images, such as the spatial relationship between leaves and roots, limiting the model's understanding of complex structures. Furthermore, the scale of Chinese herbal medicine image datasets is limited, especially for rare species, and professional annotation requires the participation of Chinese medicine experts. This makes it difficult for training data to cover the entire life cycle and complex scenarios. Data scarcity and high annotation costs have become important factors restricting the improvement of model performance.
[0003] To address this issue, those skilled in the art have proposed a method for recognizing images of traditional Chinese medicine based on a triple receptive field. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a method for image recognition of traditional Chinese medicine based on a triple receptive field, comprising the following sequentially executed steps: S1. Data enhancement processing is performed on images of traditional Chinese medicine. A multi-level data enhancement system is constructed by combining geometric transformation, color adjustment spatial domain and frequency domain based on Fourier transform. The image is transformed from the spatial domain to the frequency domain, and its frequency and information are extracted using the following formula: (1) in, The spectrum result is the Fourier transform. Input image sequences for the spatial domain. These are the discrete pixel values of a spatial domain image. It is the frequency index in the frequency domain, and its value is... Different Corresponding to different frequency components, Total number of pixels It is the imaginary unit; S2. Construct a triple receptive field module, including: S2.1, Local receptive field based on dense residuals: (2) in, Indicates the first The output of the layer, Indicates the first The nonlinear transformation function of the layer, This represents the output of the previous layer or all preceding layers. S2.2 represents the output of the previous layer, and the global receptive field based on covariance attention: Covariance attention calculation: The covariance attention mechanism is used to process the normalized features and capture long-range information features of the image. A covariance matrix is used to construct an attention mechanism to capture the global receptive field and extract long-range information features from the image; the calculation method of covariance attention is as follows: (3) In the formula, and These represent the query matrix and the key matrix, respectively. Represents a value matrix; A Transformer block is constructed by combining covariance attention, GELU2 activation function, LayerNorm normalization, and multilayer perceptron (MLP). The structural formula of the block is as follows: (4) Here, LayerNorm represents the layer normalization operation, which normalizes the fused features, and MultiHeadAttention represents the multi-head attention mechanism. This refers to a multilayer perceptron, which further processes the features output by the covariance attention mechanism to extract higher-level feature representations. The features processed by the multilayer perceptron are then used as the output features. S2.3, Frequency domain receptive field based on fast Fourier convolution: Fast Fourier Convolution (FFC) is used to make the receptive field cover the entire image. FFC divides the channels into local and global branches. The local branch uses convolution to update the local feature map, while the global branch performs a Fourier transform on the feature map and updates it in the frequency domain. To ensure the output is real-valued, FFC applies a real-valued FFT to half the spectrum and computes the inverse real-valued FFT accordingly. S3. Construct a deep neural network model for Chinese herbal medicine recognition based on the above triple receptive field module: The described deep neural network model is constructed using the aforementioned triple receptive field module, with each layer containing one triple receptive field module. This multi-layer deep neural network model is built by stacking these layers. Unlike traditional convolutional neural network models, each layer of this deep neural network model is not a single convolutional layer, but rather a multi-receptive field module layer. It includes dense residual blocks constructed from convolutional layers and dense residual learning, Transformer blocks constructed from covariance attention mechanisms, and Fast Fourier Convolutional layers constructed from FFC. Unlike traditional neural network models where each layer only possesses the local receptive field learning capability of a convolutional layer, the deep neural network model constructed using this method possesses multiple different receptive field learning capabilities in each layer, resulting in stronger feature representation capabilities and effectively improving the performance of the neural network model.
[0005] Preferably, the data augmentation process includes: Geometric transformations, including horizontal flipping and vertical flipping, wherein the horizontal flipping mirrors the image symmetrically along the vertical central axis, and the vertical flipping mirrors the image symmetrically along the horizontal central axis; Color adjustment, which includes brightness adjustment, contrast adjustment, and saturation transformation, wherein brightness adjustment is achieved through a linear transformation formula: This is to achieve brightness enhancement or reduction, among which This is the preset brightness scaling factor. The contrast adjustment is based on the global mean of the image, where a preset brightness offset is used to enhance contrast. The formula is as follows: In the formula, The mean of the image. The saturation transformation is performed using the preset contrast coefficient and formula. The saturation channel is adjusted independently in the HSV color space to obtain the adjusted saturation value. ,in, This is the original saturation value. This is the preset saturation scaling factor; Frequency domain enhancement is based on obtaining the frequency domain information of the image through Fourier transform. The original input spatial domain image information and frequency domain information are fused to obtain fused features. The fusion is completed by splicing. Preferably, the triple receptive field module includes: The system consists of a dense residual block constructed from convolutional layers and dense residual learning, a Transformer block constructed from a covariance attention mechanism, and a fast Fourier convolutional layer constructed from FFC. The dense residual block is implemented by dense residual learning, i.e., the formula (2), and has the ability to learn local receptive fields. The core component of the Transformer block is the covariance attention mechanism, which is calculated by covariance attention, i.e., the formula (3). The multi-head attention mechanism is implemented through multiple sets of different attention, and then the overall Transformer module is constructed through LayerNorm normalization and MLP multilayer perceptron, which has the ability to learn long-distance features. The fast Fourier convolution FFC is used to enable the receptive field covering the entire image. The FFC divides the channel into local and global branches. The local branch uses traditional convolution to update the local feature map, and the global branch performs Fourier transform on the feature image and updates it in the frequency domain. In order to ensure that the output is real, a real FFT with only half the spectrum is applied, and the inverse real FFT is calculated accordingly to obtain the valuable signal. These three module components work in parallel streams, using linear projection to connect and fuse features, and employing self-learned weight parameters for fusion. TRFM is embedded in parallel with three different receptive fields, enabling the simultaneous feature extraction capabilities of multiple different receptive fields in the same layer, thus possessing better feature learning and representation capabilities.
[0006] Preferably, the method further includes the following steps: The preprocessing step involves preprocessing the images of Chinese herbal medicines to remove noise and interference and improve image quality. The training and optimization steps involve training and optimizing the model to improve its accuracy and generalization ability in recognizing images of traditional Chinese medicine herbs. The evaluation and verification steps assess and verify the identification results to ensure their reliability and accuracy.
[0007] The model can learn on diverse data, thereby improving the effectiveness of model training and recognition performance.
[0008] This invention provides a method for image recognition of traditional Chinese medicine based on a triple receptive field. It has the following beneficial effects: 1. The triple receptive field depth model employed in this invention simulates the multi-scale perception mechanism of human vision, enabling simultaneous capture of the microscopic texture and macroscopic morphology of traditional Chinese medicine (TCM) images. By combining frequency domain feature learning, dense residual block construction, and attention mechanisms, this method not only enhances the model's adaptability to scale changes but also, through feature acquisition from multiple different receptive fields, simultaneously captures the microscopic texture and macroscopic morphology of TCM images, effectively integrating low-level details and high-level semantic information, thus improving the model's adaptability to scale changes and enhancing the accuracy of TCM image recognition.
[0009] 2. This invention designs a triple receptive field module, which enhances the model's feature learning ability and effectively reduces the impact of external environmental factors such as lighting and background on the recognition of Chinese herbal medicine images. This design enables the system to maintain stable recognition performance under different lighting conditions and complex backgrounds, improving the system's adaptability to changes in practical application scenarios.
[0010] 3. Through a dynamic weight calibration mechanism and a triple receptive field learning capability, including the frequency domain, the system of this invention can capture the global semantic features of the data. This not only enhances the model's resistance to noise and interference but also effectively improves the system's generalization performance. This robustness ensures that the system can still provide reliable recognition results when faced with unknown or changing data.
[0011] 4. This invention provides key technical support for the traceability system and intelligent production of Chinese medicinal materials through automated image recognition technology for Chinese medicinal herbs. This not only improves the efficiency of sorting processed Chinese medicinal herbs and reduces the misjudgment rate, but also helps to achieve efficient identification of rare medicinal materials, promoting the rational development and sustainable utilization of Chinese medicine resources. Attached Figure Description
[0012] Figure 1 This is a flowchart of the present invention; Figure 2 This is an example diagram of image data enhancement for traditional Chinese medicine herbs according to the present invention; Figure 3 This is a schematic diagram of the frequency domain data enhancement based on Fourier transform according to the present invention; Figure 4 This is a diagram of the triple receptive field module architecture of the present invention; Figure 5 This is a block diagram of the Chinese herbal medicine recognition network based on a triple receptive field module according to the present invention; Figure 6 This is a graph showing the loss and accuracy during the model training process of this invention. Detailed Implementation
[0013] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0014] Please see the appendix Figure 1 - Appendix Figure 6 This invention provides a method for recognizing images of traditional Chinese medicine based on a triple receptive field, comprising the following steps performed in sequence: S1. Data enhancement processing is performed on images of traditional Chinese medicine. A multi-level data enhancement system is constructed by combining geometric transformation, color adjustment spatial domain and frequency domain based on Fourier transform. The image is transformed from the spatial domain to the frequency domain, and its frequency and information are extracted using the following formula: (1) in, The spectrum result is the Fourier transform. Input image sequences for the spatial domain. These are the discrete pixel values of a spatial domain image. It is the frequency index in the frequency domain, and its value is... Different Corresponding to different frequency components, Total number of pixels It is the imaginary unit; S2. Construct a triple receptive field module, including: S2.1, Local receptive field based on dense residuals: (2) in, Indicates the first The output of the layer, Indicates the first The nonlinear transformation function of the layer, This represents the output of the previous layer or all preceding layers. This indicates the output of the previous layer; S2.2 Global receptive field based on covariance attention: Covariance attention calculation: The covariance attention mechanism is used to process the normalized features and capture long-range information features of the image. A covariance matrix is used to construct an attention mechanism to capture the global receptive field and extract long-range information features from the image; the calculation method of covariance attention is as follows: (3) In the formula, and These represent the query matrix and the key matrix, respectively. Represents a value matrix; A Transformer block is constructed by combining covariance attention, GELU2 activation function, LayerNorm normalization, and multilayer perceptron (MLP). The structural formula of the block is as follows: (4) Here, LayerNorm represents the layer normalization operation, which normalizes the fused features, and MultiHeadAttention represents the multi-head attention mechanism. This refers to a multilayer perceptron, which further processes the features output by the covariance attention mechanism to extract higher-level feature representations. The features processed by the multilayer perceptron are then used as the output features. S2.3, Frequency domain receptive field based on fast Fourier convolution: Fast Fourier Convolution (FFC) is used to make the receptive field cover the entire image. FFC divides the channels into local and global branches. The local branch uses convolution to update the local feature map, while the global branch performs a Fourier transform on the feature map and updates it in the frequency domain. To ensure the output is real-valued, FFC applies a real-valued FFT to half the spectrum and computes the inverse real-valued FFT accordingly.
[0015] S3. Construct a deep neural network model for Chinese herbal medicine recognition based on the above triple receptive field module: The system consists of a dense residual block constructed from convolutional layers and dense residual learning, a Transformer block constructed from a covariance attention mechanism, and a fast Fourier convolutional layer constructed from FFC. The dense residual block is implemented by dense residual learning, i.e., the formula (2), and has the ability to learn local receptive fields. The core component of the Transformer block is the covariance attention mechanism, which is calculated by covariance attention, i.e., the formula (3). The multi-head attention mechanism is implemented through multiple sets of different attention, and then the overall Transformer module is constructed through LayerNorm normalization and MLP multilayer perceptron, which has the ability to learn long-distance features. The fast Fourier convolution FFC is used to enable the receptive field covering the entire image. The FFC divides the channel into local and global branches. The local branch uses traditional convolution to update the local feature map, and the global branch performs Fourier transform on the feature image and updates it in the frequency domain. In order to ensure that the output is real, a real FFT with only half the spectrum is applied, and the inverse real FFT is calculated accordingly to obtain the valuable signal. These three module components work in parallel streams, using linear projection to connect and fuse features, and employing self-learned weight parameters for fusion. TRFM is embedded in parallel with three different receptive fields, enabling the simultaneous feature extraction capabilities of multiple different receptive fields in the same layer, thus possessing better feature learning and representation capabilities.
[0016] The data augmentation process includes: Geometric transformations, including horizontal flipping and vertical flipping, wherein the horizontal flipping mirrors the image symmetrically along the vertical central axis, and the vertical flipping mirrors the image symmetrically along the horizontal central axis; Color adjustment, which includes brightness adjustment, contrast adjustment, and saturation transformation, wherein brightness adjustment is achieved through a linear transformation formula: This is to achieve brightness enhancement or reduction, among which This is the preset brightness scaling factor. The contrast adjustment is based on the global mean of the image, where a preset brightness offset is used to enhance contrast. The formula is as follows: In the formula, The mean of the image. The saturation transformation is performed using the preset contrast coefficient and formula. The saturation channel is adjusted independently in the HSV color space to obtain the adjusted saturation value. ,in, This is the original saturation value. This is the preset saturation scaling factor; Frequency domain enhancement is based on obtaining the frequency domain information of the image through Fourier transform. The original input spatial domain image information and frequency domain information are fused to obtain fused features. The fusion is completed by splicing. The method further includes the following steps: The training and optimization steps involve training and optimizing the model to improve its accuracy and generalization ability in recognizing images of traditional Chinese medicine herbs. The evaluation and verification steps assess and verify the identification results to ensure their reliability and accuracy.
[0017] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for image recognition of traditional Chinese medicine based on triple receptive fields, characterized in that, The steps are executed in the following order: S1. Data enhancement processing is performed on images of traditional Chinese medicine. A multi-level data enhancement system is constructed by combining geometric transformation, color adjustment spatial domain and frequency domain based on Fourier transform. The image is transformed from the spatial domain to the frequency domain, and its frequency and information are extracted using the following formula: （1） in, The spectrum result is the Fourier transform. Input image sequence for spatial domain, These are the discrete pixel values of a spatial domain image. It is the frequency index in the frequency domain, and its value is... Different Corresponding to different frequency components, Total number of pixels It is the imaginary unit; S2. Construct a triple receptive field module, including: S2.1, Local receptive field based on dense residuals: （2） in, Indicates the first The output of the layer, Indicates the first The nonlinear transformation function of the layer, This represents the output of the previous layer or all preceding layers. This indicates the output of the previous layer; S2.2 Global receptive field based on covariance attention: Covariance attention calculation: The covariance attention mechanism is used to process the normalized features and capture long-range information features of the image. A covariance matrix is used to construct an attention mechanism to capture the global receptive field and extract long-range information features from the image; the calculation method of covariance attention is as follows: （3） In the formula, and These represent the query matrix and the key matrix, respectively. Represents a value matrix; A Transformer block is constructed by combining covariance attention, GELU2 activation function, LayerNorm normalization, and multilayer perceptron (MLP). The structural formula for the block is as follows: （4） Here, LayerNorm represents the layer normalization operation, which normalizes the fused features, and MultiHeadAttention represents the multi-head attention mechanism. This refers to a multilayer perceptron, which further processes the features output by the covariance attention mechanism to extract higher-level feature representations. The features processed by the multilayer perceptron are then used as the output features. S2.3, Frequency domain receptive field based on fast Fourier convolution: Fast Fourier Convolution (FFC) is used to make the receptive field cover the entire image. FFC divides the channels into local and global branches. The local branch uses convolution to update the local feature map, while the global branch performs a Fourier transform on the feature map and updates it in the frequency domain. To ensure that the output is real, FFC applies a real-valued FFT of half the spectrum and calculates the inverse real-valued FFT accordingly. S3. Construct a deep neural network model for Chinese herbal medicine recognition based on the above triple receptive field module: A triple receptive field network integrating local, global, and frequency domain receptive fields captures multi-scale features, extracts and outputs high-level features; The described deep neural network model is constructed using the aforementioned triple receptive field module, with each layer containing one triple receptive field module. This multi-layer deep neural network model is built by stacking these layers. Unlike traditional convolutional neural network models, each layer of this deep neural network model is not a single convolutional layer, but rather a multi-receptive field module layer. It includes dense residual blocks constructed from convolutional layers and dense residual learning, Transformer blocks constructed from covariance attention mechanisms, and Fast Fourier Convolutional layers constructed from FFC. Unlike traditional neural network models where each layer only possesses the local receptive field learning capability of a convolutional layer, the deep neural network model constructed using this method possesses multiple different receptive field learning capabilities in each layer, resulting in stronger feature representation capabilities and effectively improving the performance of the neural network model.
2. The method for image recognition of traditional Chinese medicine based on triple receptive fields according to claim 1, characterized in that, The data augmentation process includes: Geometric transformations, including horizontal flipping and vertical flipping, wherein the horizontal flipping mirrors the image along the vertical central axis and the vertical flipping mirrors the image along the horizontal central axis; Color adjustment, which includes brightness adjustment, contrast adjustment, and saturation transformation, wherein brightness adjustment is achieved through a linear transformation formula: To achieve brightness enhancement or reduction, among which This is the preset brightness scaling factor. The contrast adjustment is based on the global mean of the image, where a preset brightness offset is used to enhance contrast. The formula is as follows: In the formula, The mean of the image. The saturation transformation is performed using the preset contrast coefficient and formula. The saturation channel is adjusted independently in the HSV color space to obtain the adjusted saturation value. ,in, This is the original saturation value. This is the preset saturation scaling factor; Frequency domain enhancement is achieved by obtaining the frequency domain information of the image based on Fourier transform, fusing the original input spatial domain image information and frequency domain information to obtain fused features. The fusion is accomplished by splicing to achieve frequency domain enhancement.
3. The method for image recognition of traditional Chinese medicine based on triple receptive fields according to claim 1, characterized in that, The triple receptive field module includes: The system consists of a dense residual block constructed from convolutional layers and dense residual learning, a Transformer block constructed from a covariance attention mechanism, and a fast Fourier convolutional layer constructed from FFC. The dense residual block is implemented by dense residual learning, i.e., the formula (2), and has the ability to learn local receptive fields. The core component of the Transformer block is the covariance attention mechanism, which is calculated by covariance attention, i.e., the formula (3). The multi-head attention mechanism is implemented through multiple sets of different attention, and then the overall Transformer module is constructed through LayerNorm normalization and MLP multilayer perceptron, which has the ability to learn long-distance features. The fast Fourier convolution FFC is used to enable the receptive field covering the entire image. The FFC divides the channel into local and global branches. The local branch uses traditional convolution to update the local feature map, and the global branch performs Fourier transform on the feature image and updates it in the frequency domain. In order to ensure that the output is real, a real FFT with only half the spectrum is applied, and the inverse real FFT is calculated accordingly to obtain the valuable signal. These three module components work in parallel streams, using linear projection to connect and fuse features, and employing self-learned weight parameters for fusion. TRFM is embedded in parallel with three different receptive fields, enabling the simultaneous feature extraction capabilities of multiple different receptive fields in the same layer, thus possessing better feature learning and representation capabilities.
4. The method for image recognition of traditional Chinese medicine based on triple receptive fields according to claim 1, characterized in that, The method further includes the following steps: The training and optimization steps involve training and optimizing the model to improve its accuracy and generalization ability in recognizing images of traditional Chinese medicine herbs. The evaluation and verification steps assess and verify the identification results to ensure their reliability and accuracy.