Fritillaria automatic identification system and method based on hierarchical deep learning

Abstract

The invention discloses an automatic fritillaria identification system and method based on hierarchical deep learning, and relates to the technical field of intelligent detection of traditional Chinese medicinal materials. The system comprises a hardware integration and image acquisition module, an image processing and target positioning module and a grading identification algorithm module. Wherein ConvNeXt-is adopted for the category pairs of the Songbei and the Pingbei, which are relatively low in similarity; carrying out classification and identification through a Tiny network; the method comprises the following steps of: performing classification identification by adopting a double-branch ConvNeXt model aiming at furnace shell and illite type pairs with relatively high similarity, respectively extracting local microscopic texture and overall macroscopic morphological characteristics by adopting texture branches and morphological branches which are constructed in parallel, and performing optimization by adopting a joint loss function fusing tuple marginal penalty terms so as to enhance the inter-class discrimination capability. According to the invention, full-process automation from sample feeding to identification result output is realized, the identification precision is high, the speed is high, and the method is suitable for rapid nondestructive batch detection in the circulation link of traditional Chinese medicinal materials.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and quality inspection of Chinese medicinal materials. Specifically, it relates to an automated identification system and method for Fritillaria cirrhosa that integrates automated hardware and hierarchical deep learning algorithms, which is used to achieve rapid, non-destructive, and high-precision identification of genuine and counterfeit Fritillaria cirrhosa. Background Technology

[0002] Fritillaria is a precious Chinese medicinal herb, but its genuine products (such as Songbei and Lubei) and counterfeit products (such as Pingbei and Yibei) are highly similar in appearance. Traditional identification methods (such as empirical identification, microscopic identification, and physicochemical identification) have disadvantages such as strong subjectivity, complicated operation, damage to samples or expensive equipment, which make it difficult to meet the needs of rapid, non-destructive, and large-scale screening in the market circulation process.

[0003] Deep learning-based image recognition technology offers a new approach to the identification of traditional Chinese medicinal materials. Existing technologies, such as the improved YOLO series models, can automatically detect Fritillaria cirrhosa, but their accuracy is limited in fine-grained classification of highly similar categories. Single-branch convolutional neural networks (such as CNN and ResNet) perform well when there are significant differences between classes, but their ability to distinguish between extremely similar categories like "Lu Bei-Yi Bei" is insufficient. Furthermore, existing research mostly focuses on the algorithms themselves and lacks complete system solutions integrated with automated hardware, making it difficult to directly apply them to actual production environments.

[0004] Therefore, there is an urgent need for a systematic solution that integrates automated sample processing, high-quality image acquisition, and high-precision intelligent identification to address the challenge of rapid and non-destructive identification of similar-looking Chinese medicinal materials such as Fritillaria cirrhosa. Summary of the Invention

[0005] In view of this, the present invention aims to provide an automated identification system and method for fritillaria based on hierarchical deep learning, in order to solve the technical problems of the lack of a fully automated solution and insufficient accuracy in identifying highly similar categories in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: An automated fritillary bulb identification system based on hierarchical deep learning is provided, comprising: The hardware integration and image acquisition module is used to realize the automatic feeding and conveying of fritillaria samples, and to acquire sample images under standard lighting conditions; The image processing and target localization module is used to process the acquired sample images, extract and crop out the normalized fritillary target region image; The hierarchical identification algorithm module is used to input the target region image of the fritillary bulb into the corresponding deep learning model for classification and identification based on the similarity information of the preset fritillary bulb category.

[0007] The hierarchical identification algorithm module adopts a hierarchical identification strategy; The hierarchical identification algorithm module includes: The basic identification unit is used to classify and identify the low-similar pine clam and flat clam categories using the ConvNeXt-Tiny network; An enhanced discrimination unit is used to classify and distinguish between highly similar Rube and Ibe categories using a two-branch ConvNeXt model.

[0008] The dual-branch ConvNeXt model includes: Texture branching is used to extract local micro-texture features of mother-of-pearl from the input features; Morphological branching is used to extract the overall macroscopic contour features of the mother-of-pearl from the input features; The feature fusion and classification unit is used to fuse the texture feature vector output by the texture branch and the morphological feature vector output by the morphological branch, perform classification through a fully connected network, and output the class probability.

[0009] The texture branch includes, in sequence, a 7×7 convolutional layer, a ReLU activation function, an adaptive average pooling layer, and a layer normalization layer, and outputs a 256-dimensional texture feature vector; the morphology branch includes, in sequence, a 3×3 max pooling layer, an adaptive average pooling layer, and a layer normalization layer, and outputs a 768-dimensional morphology feature vector.

[0010] The feature fusion and classification unit first concatenates the 256-dimensional texture feature vector and the 768-dimensional morphological feature vector along the channel dimension to form a 1024-dimensional fused feature, and then outputs the class probability through a classifier containing two fully connected layers.

[0011] The training of the dual-branch ConvNeXt model in the enhanced discrimination unit employs a joint loss function, which is: ,in, The standard cross-entropy loss is calculated using the following formula: ; In the formula, One-hot encoding for the real label, For the model to class The predicted probability, To define the marginal penalty term for tuples in the design for the Furnace and Ibe categories, the following definition is provided: ; In the formula, and represent the model's predicted probabilities for Lube and Ibe, respectively. For the marginal threshold, This is the balance coefficient.

[0012] An automated method for identifying mother-of-pearl based on hierarchical deep learning is provided, comprising the following steps: S1: The fritillaria samples are automatically fed and transported through the hardware integration and image acquisition module, and sample images are acquired under standard lighting conditions; S2: The sample image is processed by the image processing and target localization module to extract and crop out the normalized fritillary target region image; S3: Based on the preset similarity information of the fritillaria category, the corresponding deep learning model is selected by the hierarchical identification algorithm module to classify and identify the fritillaria target area image, and the result is output.

[0013] The selection of the corresponding deep learning model in step S3 includes: if the fritillaria to be identified belongs to the low-similar category pair of Pine Fritillaria and Flat Fritillaria, then the ConvNeXt-Tiny network is called for classification and identification; if the fritillaria to be identified belongs to the high-similar category pair of Lu Fritillaria and Yi Fritillaria, then the two-branch ConvNeXt model is called for classification and identification.

[0014] The process of calling the dual-branch ConvNeXt model for classification and identification includes: extracting deep feature maps of the input image through the backbone network to obtain input features; extracting local micro-texture features of the mother-of-pearl from the input features through parallel texture branches; extracting the overall macro-contour features of the mother-of-pearl from the input features through parallel morphology branches; fusing the texture feature vector and the morphology feature vector; classifying through a fully connected network; and outputting the class probability.

[0015] A computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of an automated identification method for mother-of-pearl based on hierarchical deep learning.

[0016] The beneficial effects of this invention are as follows: 1. High system integration: An end-to-end automated system with hardware and algorithm collaboration has been built, realizing non-destructive, rapid, and batch testing of fritillaria bulbs, which is highly practical.

[0017] 2. Hierarchical identification strategy: The model is adaptively selected according to the difficulty of the task. While ensuring overall efficiency, an enhanced model is used for high-difficulty tasks, achieving a balance between accuracy and speed.

[0018] 3. Strong algorithm discrimination capability: The dual-branch ConvNeXt model designed for extremely similar categories fully extracts discriminative features through feature decoupling and fusion, and significantly improves fine-grained discrimination accuracy by combining the tuple marginal loss function. Attached Figure Description

[0019] Figure 1 This is a flowchart of how the system of the present invention works; Figure 2 This is a schematic diagram of the dual-branch ConvNeXt model of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0021] System overall architecture: The automated fritillary identification system of this invention mainly comprises three modules: The hardware integration and image acquisition module is used to automatically feed and transport fritillaria samples and acquire sample images under standard lighting conditions. It includes an automatic feeding device, conveyor belt, Sony CMOS industrial camera, and uniform surface light source. The Sony CMOS industrial camera has a resolution of 4024×3036. An ARM Cortex-M4 control core controls the motor via PWM to achieve smooth sample transport and image acquisition.

[0022] The image processing and target localization module is used to process the acquired sample images, extract and crop out the normalized target region image of the mother-of-pearl. It performs grayscale conversion, binarization, morphological operations (opening / closing operations), and connected component analysis on the acquired sample images to accurately locate the mother-of-pearl from the black conveyor belt background and crop out the normalized region of interest (ROI).

[0023] The hierarchical identification algorithm module is the core algorithm module. It implements a hierarchical identification strategy and uses the similarity information of the target area image of the fritillary bulb according to the preset fritillary bulb category to input the image into the corresponding deep learning model for classification and identification.

[0024] like Figure 1 As shown, when the system is working, the conveyor belt is started to feed materials, and then target detection is performed to determine whether the target has entered the center of the screen. If not, the target detection is returned. If it has, the conveyor belt feeding is paused, the target sub-image is located and cut, and then input into the fritillary bulb classification and recognition network. Finally, the recognition results are statistically analyzed.

[0025] The hierarchical identification algorithm module adopts a hierarchical identification strategy. The hierarchical identification algorithm module includes: a basic identification unit, which is used to classify and identify the low-similar-similar-short-shell and flat-shell category pairs using the ConvNeXt-Tiny network; and an enhanced identification unit, which is used to classify and identify the high-similar-similar-similar-short-shell and elk category pairs using a two-branch ConvNeXt model.

[0026] Hierarchical identification strategy and algorithm implementation: The core innovation of the system lies in its hierarchical identification strategy, which determines the identification task category of the current sample based on pre-defined knowledge. For the "Pine Clam vs. Flat Clam" identification task: This category exhibits relatively distinguishable differences in appearance, such as scale cohesion and surface texture. The system invokes the basic identification unit and uses a lightweight pre-trained ConvNeXt-Tiny network to directly classify the ROI image. This network has a simple structure and fast inference speed, making it sufficient for this type of task.

[0027] For the "Lube-Ibe" discrimination task: This category is visually highly similar, making discrimination difficult. The system calls an enhanced discrimination unit, using a two-branch ConvNeXt model.

[0028] The dual-branch ConvNeXt model includes: a texture branch, used to extract local micro-texture features of the mother-of-pearl from the input features; a morphology branch, used to extract the overall macro-contour features of the mother-of-pearl from the input features; and a feature fusion and classification unit, used to fuse the texture feature vector output by the texture branch and the morphology feature vector output by the morphology branch, classify them through a fully connected network, and output the class probability.

[0029] The texture branch consists of a 7×7 convolutional layer, a ReLU activation function, an adaptive average pooling layer, and a layer normalization layer, outputting a 256-dimensional texture feature vector. The morphology branch consists of a 3×3 max pooling layer, an adaptive average pooling layer, and a layer normalization layer, outputting a 768-dimensional morphological feature vector. The feature fusion and classification unit first concatenates the 256-dimensional texture feature vector and the 768-dimensional morphological feature vector along the channel dimension to form a 1024-dimensional fused feature, and then outputs the class probability through a classifier containing two fully connected layers.

[0030] like Figure 2 As shown, the structure of the two-branch ConvNeXt model is as follows: Using ConvNeXt-Tiny as the backbone network, deep feature maps of 7×7×768 pixels are extracted from the input image. These deep feature maps are then fed in parallel into two branches: Texture branch: The texture feature vector is sequentially processed through 7×7 convolution (to expand the receptive field and capture texture continuity), ReLU activation, adaptive average pooling (compressed to 1×1×256), and layer normalization, outputting a 256-dimensional texture feature vector. .

[0031] Morphological branch: The vector sequentially undergoes 3×3 max pooling (to suppress noise and highlight structure), adaptive average pooling (compressed to 1×1×768), and layer normalization, outputting a 768-dimensional morphological feature vector. .

[0032] Feature fusion and classification: and By concatenating along the channel dimension, a 1024-dimensional fused feature vector is obtained. . The final classification result is then obtained by using a two-layer fully connected classifier: FC1: 1024 → 512 + ReLU + Dropout; FC2: 512 → 2.

[0033] The training of the two-branch ConvNeXt model in the enhanced discrimination unit uses a joint loss function, which is: ,in, The standard cross-entropy loss is used to ensure the accuracy of the model on the overall classification task, and its calculation formula is: ; In the formula, One-hot encoding for the real label, For the model to class The predicted probability, The tuple marginal penalty term designed for the Lube and Ibe category pairs aims to force the model to learn more discriminative feature representations by explicitly constraining the difference in predicted probabilities between the categories. It is defined as follows: ; In the formula, and represent the model's predicted probabilities for Lube and Ibe, respectively. The marginal threshold is set as follows in this invention: This is used to enforce a minimum difference between the two types of prediction probabilities. The balance coefficient is set as follows in this invention. This is used to adjust the weight of the marginal penalty term of the tuple in the total loss.

[0034] A method for automated identification of mother-of-pearl based on hierarchical deep learning is also provided, including the following steps: S1: Automatically feeds and transports fritillaria samples through hardware integration and image acquisition modules, and acquires sample images under standard lighting conditions; S2: The sample image is processed by the image processing and target localization module to extract and crop out the normalized fritillary target region image; S3: Based on the similarity information of the preset fritillary category, the corresponding deep learning model is selected through the hierarchical identification algorithm module to classify and identify the fritillary target area image, and the results are output.

[0035] In step S3, the selection of the corresponding deep learning model includes: if the fritillaria to be identified belongs to the low-similar category of Pine Fritillaria and Flat Fritillaria, then the ConvNeXt-Tiny network is called for classification and identification; if the fritillaria to be identified belongs to the high-similar category of Lu Fritillaria and Yi Fritillaria, then the two-branch ConvNeXt model is called for classification and identification.

[0036] The process of using the dual-branch ConvNeXt model for classification and identification includes: extracting deep feature maps of the input image through the backbone network to obtain input features; extracting local micro-texture features of the mother-of-pearl from the input features through the parallel texture branch; extracting the overall macro-contour features of the mother-of-pearl from the input features through the parallel morphology branch; fusing the texture feature vector and the morphology feature vector; classifying through a fully connected network; and outputting the class probability.

[0037] A computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of an automated identification method for mother-of-pearl based on hierarchical deep learning.

[0038] Experimental verification: This invention has achieved good results on real-world datasets: Table 1 shows the performance comparison between the dual-branch ConvNeXt model of this application and various baseline models on the Songbei-Pingbei and Lubei-Ibe subsets. It can be seen that traditional CNN and DNN models perform poorly, while ResNet-18 shows some improvement. The basic ConvNeXt-Tiny network achieves an accuracy of 85.71% in real-world testing. The dual-branch ConvNeXt model achieves an accuracy of 87.50% in real-world testing, significantly higher than the 78.57% of the basic ConvNeXt-Tiny model. Table 1 provides a performance comparison of different models, as follows: Processing speed: The system's average processing time per sample is less than 2 seconds, meeting the requirements for high-throughput detection.

[0039] Ablation experiments: On the Ruby-Ibe task, removing the texture branch resulted in a 1.71% decrease in accuracy, and removing the tuple marginal loss resulted in a 2.14% decrease in accuracy, validating the effectiveness of each module. Table 2 shows the ablation experiment results (Ruby-Ibe dataset), as follows: In summary, this invention deeply integrates automated hardware systems with hierarchical deep learning algorithms, providing a complete solution for rapid and non-destructive identification of fritillaria bulbs. This not only significantly improves identification accuracy and efficiency but also provides a referable technical framework for the quality testing of other similar agricultural products.

Claims

1. An automated identification system for mother-of-pearl based on hierarchical deep learning, characterized in that, include: The hardware integration and image acquisition module is used to realize the automatic feeding and conveying of fritillaria samples, and to acquire sample images under standard lighting conditions; The image processing and target localization module is used to process the acquired sample images, extract and crop out the normalized fritillary target region image; The hierarchical identification algorithm module is used to input the target region image of the fritillary bulb into the corresponding deep learning model for classification and identification based on the similarity information of the preset fritillary bulb category.

2. The automated fritillary bulb identification system based on hierarchical deep learning according to claim 1, characterized in that, The hierarchical identification algorithm module adopts a hierarchical identification strategy; The hierarchical identification algorithm module includes: The basic identification unit is used to classify and identify the low-similar pine clam and flat clam categories using the ConvNeXt-Tiny network; An enhanced discrimination unit is used to classify and distinguish between highly similar Rube and Ibe categories using a two-branch ConvNeXt model.

3. The automated fritillary bulb identification system based on hierarchical deep learning according to claim 2, characterized in that, The dual-branch ConvNeXt model includes: Texture branching is used to extract local micro-texture features of mother-of-pearl from the input features; Morphological branching is used to extract the overall macroscopic contour features of the mother-of-pearl from the input features; The feature fusion and classification unit is used to fuse the texture feature vector output by the texture branch and the morphological feature vector output by the morphological branch, perform classification through a fully connected network, and output the class probability.

4. The automated fritillary bulb identification system based on hierarchical deep learning according to claim 3, characterized in that, The texture branch includes, in sequence, a 7×7 convolutional layer, a ReLU activation function, an adaptive average pooling layer, and a layer normalization layer, and outputs a 256-dimensional texture feature vector; the morphology branch includes, in sequence, a 3×3 max pooling layer, an adaptive average pooling layer, and a layer normalization layer, and outputs a 768-dimensional morphology feature vector.

5. The automated fritillary bulb identification system based on hierarchical deep learning according to claim 4, characterized in that, The feature fusion and classification unit first concatenates the 256-dimensional texture feature vector and the 768-dimensional morphological feature vector along the channel dimension to form a 1024-dimensional fused feature, and then outputs the class probability through a classifier containing two fully connected layers.

6. The automated fritillary bulb identification system based on hierarchical deep learning according to any one of claims 2 to 5, characterized in that, The training of the dual-branch ConvNeXt model in the enhanced discrimination unit employs a joint loss function, which is: ,in, The standard cross-entropy loss is calculated using the following formula: ； In the formula, One-hot encoding for the real label, For the model to class The predicted probability, To define the marginal penalty term for tuples in the design for the Furnace and Ibe categories, the following definition is provided: ； In the formula, and represent the model's predicted probabilities for Lube and Ibe, respectively. For the marginal threshold, This is the balance coefficient.

7. An automated identification method for mother-of-pearl based on hierarchical deep learning, characterized in that, Using the system according to any one of claims 1 to 6, and comprising the following steps: S1: The fritillaria samples are automatically fed and transported through the hardware integration and image acquisition module, and sample images are acquired under standard lighting conditions; S2: The sample image is processed by the image processing and target localization module to extract and crop out the normalized fritillary target region image; S3: Based on the preset similarity information of the fritillaria category, the corresponding deep learning model is selected by the hierarchical identification algorithm module to classify and identify the fritillaria target area image, and the result is output.

8. The automated identification method for mother-of-pearl based on hierarchical deep learning according to claim 7, characterized in that, The selection of the corresponding deep learning model in step S3 includes: if the fritillaria to be identified belongs to the low-similar category pair of Pine Fritillaria and Flat Fritillaria, then the ConvNeXt-Tiny network is called for classification and identification; if the fritillaria to be identified belongs to the high-similar category pair of Lu Fritillaria and Yi Fritillaria, then the two-branch ConvNeXt model is called for classification and identification.

9. The automated identification method for mother-of-pearl based on hierarchical deep learning according to claim 8, characterized in that, The process of calling the dual-branch ConvNeXt model for classification and identification includes: extracting deep feature maps of the input image through the backbone network to obtain input features; extracting local micro-texture features of the mother-of-pearl from the input features through parallel texture branches; extracting the overall macro-contour features of the mother-of-pearl from the input features through parallel morphology branches; fusing the texture feature vector and the morphology feature vector; classifying through a fully connected network; and outputting the class probability.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 7 to 9.

Images