Hierarchical Herb Recognition Using Multimodal AI and EfficientNet
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods struggle to accurately identify and distinguish between the large number of Chinese herbal medicine compounds used in TCM formulations, as many herbs can look similar and vary in appearance due to factors like origin and processing methods.
Innovation Solution
A system and method utilizing a classification engine that processes input images of herbs through hierarchical classification and feature extraction, combining a Multimodal AI model with an EfficientNet model to improve recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single classification model is used to recognize herbs, then the system complexity is low, but the recognition accuracy decreases due to the large number of herb classes and visual similarities
Solution Approach 1:
The patent divides the herb recognition task into two separate classification models: a parent class classification model that first categorizes herbs into broader groups, and a sub class classification model that then identifies specific herb types within each parent class. This segmentation approach improves recognition accuracy by breaking down the complex 600+ class classification problem into more manageable stages, while each individual model maintains reasonable complexity levels.
2Measurement precision
If hierarchical classification is implemented to improve recognition accuracy, then the classification precision improves, but the processing time increases due to multiple classification stages
Solution Approach 1:
The patent implements preliminary classification by first categorizing herbs into parent classes before proceeding to sub class identification. This preliminary action organizes the classification process in a hierarchical manner, where the parent class model prepares the groundwork by narrowing down the possible herb types, thereby improving overall classification precision while managing processing time through structured progression.
3Reliability
If multiple classification models are integrated to recognize herbs with varying appearances, then the recognition reliability improves, but the device complexity increases
Solution Approach 1:
The patent merges two classification models (parent class model and sub class model) into an integrated hierarchical recognition system. The parent class model output is combined with the sub class model output to produce the final herb identification, achieving improved recognition reliability for herbs with varying appearances due to different origins and processing methods.
Solution Approach 2:
The parent class classification acts as an intermediary between the input image and the final sub class identification. This intermediary stage processes the image first to determine the broader herb category, which then guides the sub class classification process, improving overall recognition reliability while managing system complexity through structured mediation.
4Reliability
If visual features alone are used for herb identification, then the system simplicity is maintained, but the ability to distinguish similar herbs decreases
Solution Approach 1:
The patent applies local quality analysis by extracting specific visual features from different regions and aspects of the herb images. The feature extraction process focuses on identifying distinctive local characteristics that differentiate similar herbs, such as color variations, texture patterns, and morphological features, thereby improving the ability to distinguish between visually similar herb types.
Data Source
AI summary
A system for recognition of one or more herbs including: an image gateway arranged to receive an input dataset including one or more images, each image showing one or more herbs, a classification engine arranged to: process the input image by identifying at least one herb of the one or more herbs, group the identified herb into at least one predefined class, wherein the predefined class corresponds to a type of herb, perform feature extraction on the input image to extract image features, predict the type of herb based on processing the extracted image features, and; an output module arranged to output the type of herb recognised in the image based on the combination of grouping into a predefined class and predicting the type of herb from the extracted features.