Intelligent identification method and system for traditional Chinese medicinal materials based on cross-modal comparative learning and adaptive attention fusion

By integrating cross-modal contrastive learning with adaptive attention, the visual, odor, and textual information of Chinese medicinal materials are deeply integrated, solving the problems of strong subjectivity in identification results, difficulty in identifying easily confused varieties, and insufficient robustness due to modal deficiencies in existing technologies. This achieves intelligent identification of Chinese medicinal materials with high accuracy and strong adaptability.

CN121834645APending Publication Date: 2026-04-10李佳育 +1
0 Cites 0 Cited by

Patent Information

Application Number
CN202511782600.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-11-29
Publication Date
2026-04-10

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively combine visual, odor, and textual information of Chinese medicinal materials for intelligent identification, resulting in highly subjective identification results, difficulties in standardization, challenges in identifying easily confused varieties, insufficient utilization of odor information, and a lack of modality loss robustness.

Method used

We employ a cross-modal contrastive learning and adaptive attention fusion method, which maps three modalities of data—image, odor, and text—to a unified feature space through image, odor, and text encoders. We then perform adaptive weighted fusion through contrastive learning and attention mechanisms, combined with multi-task learning and modality missing robustness training, to achieve deep integration and identification of multimodal information.

Benefits of technology

It achieves high accuracy and strong generalization ability in the identification of Chinese medicinal materials, and can maintain high identification accuracy even in the absence of modalities. It is adaptable to different application scenarios and conforms to the "observation, auscultation, inquiry and palpation" concept of traditional Chinese medicine.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a traditional Chinese medicinal material intelligent identification method and system based on cross-modal contrast learning and adaptive attention fusion. The method comprises the following steps: acquiring image data, smell data and text description data of a traditional Chinese medicinal material sample, wherein the smell data can be acquired through laboratory equipment such as an electronic nose, a GC-MS (Gas Chromatography-Mass Spectrometer), a GC-IMS (Gas Chromatography-Mass Spectrometer) and the like or a low-cost sensor array; each modal feature is extracted through encoders such as a visual Transform, a time sequence Transform or a convolutional neural network, a BERT and the like; infoNCE contrast learning is adopted to realize cross-modal semantic alignment of the three-modal features; self-adaptive weighted fusion is carried out through combination of a cross-modal attention mechanism and confidence estimation of uncertainty perception; and multi-task combined learning based on prototype learning predicts the types, the production places and the quality grades of the medicinal materials at the same time. The method further introduces a modal dropout training strategy and a missing modal dynamic completion mechanism to enhance the robustness of the system. According to the method, the three heterogeneous modalities of the image, the smell and the text are deeply fused, various smell acquisition devices are supported, and the method has the advantages of high accuracy, high interpretability and modal loss robustness and is suitable for laboratory high-precision analysis and on-site rapid detection.
Need to check novelty before this filing date? Find Prior Art